New version of go micro

[xpunch]

Previously, we've discussed some thoughts about new version of go micro, but do not have a conclusion. I've been reading the core definition codes, and found some legacy or patched methods. Whether we should re-design the core concepts, and then migrate the implementation.

[asim]

Some of the old stuff hasn't changed since first being defined. I think now is probably the time to highlight some of those things and start thinking about improvement or replacement.

1. Should client/server contain publish/subscribe?
2. Should the transport be deprecated?
3. Should we default to grpc?

I don't think there's right or wrong answers. Just design choices. So counter arguments to the above.

1. Microservices is an event driven architecture and pubsub should be a first class citizen
2. Transport agnostic RPC is useful and many people use it
3. Grpc is just another RPC mechanism and there will be many more. Being tied to it makes it hard to move beyond it.

And then counter to those

1. Someone can import broker or events for pubsub. The client/server are for synchronous comms
2. It's more complicated to maintain a transport agnostic system and it's not as performant
3. Grpc is the defacto RPC system right now and we can swap it out in later versions if need be

I could basically go on like this forever so whatever choices we make have to be thought through before doing it.

[xpunch]

I understand what you concerned, and there already have four versions of go micro. I've received some emails asking the difference and compatibility between different versions, if the new version cannot bring improvement or new features, there is no need to do that. Some features like transport agnostic, pluggable are awesome, it make go micro very flexible, not just an rpc framework.
Once you think it's time to upgrade, I can help migrate the implementation.

[asim]

I'll give some thought to what the next phase is. I think what I'd maybe prefer to start with is a profile package much like micro which lets us define groups of plugins together for different architectures Local, Cloud, Edge and Dev.

Beyond that maybe the grpc client server can use Publish/Subscribe as async calls to an RPC endpoint that does not require waiting for the response. That will then at least eliminate the broker from it and make it pure grpc.

[xpunch]

I've created v3 branch used for go-micro/v3 bug fixing, should we create v5 branch for new features? BTW, Do we need CHANGELOG.md to describe version releases and migration documents?

[xpunch]

I'm not familiar with micro, I'll take a look at the architecture of micro recently.

[asim]

I think a changelog would be useful.

We don't need a v5 branch until we have better idea of that we're doing. Recently I just realised Google supports http2 and pubsub events to google cloud run apps. It to me validates the client/server's current model of Call/Stream/Publish, so I think we're fine from that perspective. What's important is just that we do some cleanup and create stability in the project so it can scale.

A timeline:

• v1 - This was cut after 4 years of development, the majority of which was unversioned
• v2 - This established grpc as the default client/server and embedded nats server for pubsub
• v3 - Moved everything out to plugins including grpc and nats. Focused on making go-micro lightweight
• v4 - Primarily created for vanity urls to separate go-micro from its source repo. Still requires more work for sub modules
• v5 - Should be about true stability, performance and becoming a defacto standard distributed systems framework for Go

On vanity urls, I think if we can do go.micro.dev/v4/plugins that will be useful but it will likely require moving plugins out to a separate repo again otherwise maybe finding a way to make it work on a different domain like plugins.go-micro.dev or something like that might be good.

Back to v5. It really fees like Go Micro has the opportunity to be a standard next layer ontop of Go for distributed systems development. Packages for all types of patterns. Plugins for all the use cases. Support for multiple architectures and environments. Which means we really have to do the work around the docs, benchmarks, performance, etc.

[xpunch]

Any thoughts about how to fix issues like #2140 ?
Another issue is every service have a name, but client/server do not know that, so when there is an error, it's id will be "go.micro.client" or "go.micro.server" or "go.micro.transport", it's hard to locate which service is wrong when multiple service is called and they are calling other services.

[asim]

Most client/server libraries don't have to deal with this problem. No one is asking net/http for named errors and if we didn't have error IDs no one would ask us either. When a client or transport error are thrown they are on the client side. It's up to the user to deal with this error and then respond accordingly. It's not our responsibility to name the error anything different.

Because distributed systems often result in multi service call chains people are expecting us to solve that problem. What we really need is distributed tracing and to push people to use that.

In your specific issue there you need to log more than just err if you want details. We can replace the name in the server but it doesn't fix the fact that the caller is responsible for logging and shouldn't just do log.fatal(err).

[xpunch]

OK, I get it. Services can wrap internal errors with it's own service name.
How about the #2140 ?

[asim]

This is because when you do NewService the name is set in the server. When you then specify Init and --server flag it completely replaces the server with a new grpc one. There's no way around this and there's no way to reinject the values.

[FarhanSajid1]

Could we get some distributed tracing implementation?