New ideas (top level nitro.App, remove proto dependency)

[asim]

As nitro evolves from go-micro and its focus moves beyond microservices, I think its probably a good idea to start to change a few things. One thing we did was remove the top level micro.NewService but I think nitro probably needs a nitro.App returned by nitro.NewApp().

I also think nitro should not rely on protobuf or external code generation, it should be entirely self contained with no dependency beyond Go. If it does introduce any code generation it should be defined by nitro itself as go generate commands.

[crazybber]

if nitro will focus on edge and embeded system/product ,beside less framework codes running on limit computing resource, which also should consider common devices protocol link ability、common network and data model by mqtt、websocket etc

[asim]

I think yes, this will be the focus, totally self contained binaries that work on the edge and in embedded systems using purely the Go runtime, potentially even operating in wasm. So the question is, what are the key things to support and what should be removed.

[crazybber]

some ideas ：
1、basic app framework with MQ、device protocol adapter、DB driver(sqlite、dqlite etc)、cache、basic LB support .
2、raw tcp/udp transport link ability channel to support linking with different harware .
3、mqtt/ws uplink session ability to report event/message to cloud and receive cmd/control ,download file stream from cloud etc..
4、 all other can be removed at first.

[asim]

• I think we have broker, cache, store, selector which covers 1
• We can add a "network" transport that allows us to specify Listen("udp://addr") or Listen("tcp://addr")
• mqtt/ws are not in golang itself so harder to support out of the box
• not sure what else to be removed

I think some sort of gossip networking might be useful

[crazybber]

• webrtc network model are useful for some industry video communication .
• common mqtt broker can talk with 3rd mqtt server ,which is enough ,i think.

where gossip can be used around edge/embeded ?

[asim]

Because global networks will be moving to ipv6 with public IPs it potentially means the possibility of doing direct 1:1 networking between services. I think there's the opportunity to do something p2p based that's totally async.

[crazybber]

agreed ,ipv6,that will also change relationship between cloud and edge , gossip can do more in this situation, and same time, ipv4 will also exist for a long time .

[asim]

First, I think we have to get back to a usable state. Finding the minimal path to that. I ripped out a lot of things.

1. Move service package to app
2. Define nitro.NewApp() to return a default implementation of the app
3. Include a network transport for sockets/tcp
4. Create an example of this usage
5. Strip any requirement on protobuf and opt purely for json for now

[crazybber]

according to my experience ,UDP and xml, which work for traditional indurstry communication , should be considerd firstly on edge .

[asim]

OK anyway there is some work to do to get Nitro back into a usable state. Right now as I modify examples I can feel the dev UX is subpar. I hate protobuf, the fact that I have to use any tool outside of Go is a huge problem for me. So we are using JSON only for the time being and I need to simplify the UX for calling/handling. Also without socket support this is currently only an in memory thing.

So I think we can take https://github.com/asim/nitro-plugins/blob/master/transport/tcp/tcp.go and turn it into a "net" or "socket" transport and permit specifying "tcp", "udp", "unix" as part of the addr string.

[crazybber]

• around edge, protobuf is not a necessary，which usually adapte different raw protocol ,we can totally remove it, i think.
• json is also a common serialization way which can go first .
• pluggable socket that is fine,we can follow the previous design way ..

[asim]

OK new usage https://github.com/asim/nitro#usage

This is functional but still missing pubsub. It's purely in memory until you switch out the transport. So the next step is figuring that out. I was reading about Google's Snap transport which using memory I/O rather than sockets and its 10x the performance. https://research.google/pubs/pub48630/

I think there's some potential around mmap or single socket for all comes on a machine with some sort of external hand off for multi-host networking.

[crazybber]

this is fine ,but some cases looks missed . .

considering communication between a device (e.g. an onvif camera, cmd by http ,stream by udp, or an fire controlller or an industry device) and an edge app, which is running nitro , how can we perform the arch ?

[asim]

I think this is some sort of gateway, or multi-protocol management. I don't have a solution for this. I think there needs to be separation of concerns. Your App itself should only process from a single place. So likely a Nitro Gateway provides these all in one place with the ability to create new listeners.

[asim]

Step 1. Socket Transport https://github.com/asim/nitro/tree/master/transport/socket

The socket transport defaults to tcp but will accept address in the form of network://address:port e.g udp://localhost:1234 or tcp://localhost:1234 or whatever golang supports for net.Dial and net.Listen.

Looking at a picture like so (which is how we model network programming)

It's clear at some point, we implement a "network" package that handles p2p.