You're not really writing 'Go' anymore when you're optimising it, it's defeating the point of the language as a simple but powerful interface over networked services.
What do you mean? If you don't want that level of control over e.g. memory allocation, registries, cache lines etc, there's higher level languages than Go you can pick from, e.g. Java / C# / JS.