How to set up round robin for microservices?

[santo74]

I'm experimenting with cluster and what I want to achieve is to be able to round robin requests between multiple instances of a particular micro service.

E.g. I have a "log" service which provides logging functionality to the cluster:

Cluster.register('log');

and a web service which connects to that log service:

var logConn = Cluster.discoverConnection('log');

As a test I start 2 instances of the log service (let's call them log1 and log2), both on a different port:

~/apps/demo-log1$ MONGO_URL=mongodb://localhost:27017/demo meteor --port 2001

~/apps/demo-log2$ MONGO_URL=mongodb://localhost:27017/demo meteor --port 2002

And 1 instance of the web service:

~/apps/demo-web$ MONGO_URL=mongodb://localhost:27017/demo meteor --port 2000

The web service can send log messages to the log service, but it's always the same instance (let's say log1) that's receiving the messages.
When I stop this instance (log1), the messages are being sent to the other one (log2).
So the failover is working, but in addition to this I also want some kind of load balancing between the two log instances so that they can both accept messages from the web service in a round robin fashion.

What am I doing wrong?

[jadsonlourenco]

Maybe you can try export CLUSTER_SELF_WEIGHT="0.6", it will balance the work better for each node of the logs service.

[rhyslbw]

Put some load on the system and you should see it being balanced properly

[santo74]

When I generate 10 messages per second they still go all to the same instance, even after specifying the cluster-weight.

[arunoda]

Okay. Load balancing is only work when we are getting connections from the outside to the client. For load balance inside the cluster. You need to create a pool of connections and send the in a round robin fashion.

See how to do it:

var loggers = [];
for(var lc=0; lc<10; lc++) {
  loggers.push(Cluster.discoverConnection('log'));
}

function sendLog(message) {
  var index = Math.floor(Math.random() * loggers.length);
  loggers[index].call('log', message);
};

for(var lc=0; lc<10000; lc++) {
  sendLog("my log messgae");
}

I'm not sure, cluster should support this by default or it's a task of a separate package.

[santo74]

Hmm, ok, thanks, but that's not the solution I was looking for.
Also, this code is expecting exactly 10 logger instances, while what I need is automatic load balancing over as many instances as there are available at any given point in time.

Therefore I would prefer the cluster package to provide such functionality out of the box, because it knows how many instances are available of any particular service.

[rhyslbw]

Hmm, yes this seems to be expected behaviour. Maybe this is more of a PR encouraged enhancement rather than separate package. @arunoda are there situations where you wouldn't want this happening?

[arunoda]

I'd love to this to be a part of cluster. Only way, I can think of this by pooling connections.

Then it's pretty straightforward to implement something for methods as I explained in the code. But the problem comes for subscriptions. I have no good solution for that.

[rhyslbw]

Is in not effective to use the same technique just with a 'publishers' pool?

[arunoda]

May be need to think that a bit.

On Fri, Mar 20, 2015 at 5:18 PM Rhys Bartels-Waller <
[email protected]> wrote:

Is in not effective to use the same technique just with a 'publishers'
pool?

—
Reply to this email directly or view it on GitHub
#32 (comment).

[santo74]

I don't know the inner details of the cluster package, nor the publish / subscribe mechanism between client and server in meteor and therefore I have no clue whether it is possible at all, but e.g. in Vert.x - which I use for another project - modules can simply publish / subscribe on the server side as well.

This means that in the example above I would start 2 instances of the "log" module and let them both subscribe to the same address, e.g. "log".
The web module can then simply publish to that address and both instances of the log module will receive the message.
If I want only one instance to handle such a request, then I can "send" the message (instead of "publish"), in which case Vert.x chooses one of the instances in a round robin fashion.

This way an application can be very easily scaled by simply adding extra instances of a module.

[rhyslbw]

@santo74

It sounds like what your describing here can be achieved using RPCs via Meteor.methods, whereas the issue @arunoda has raised relates to how internal pub/sub within the cluster would be load balanced since you can subscribe to another ddp server (or cluster service) publication from the server. The meteor docs have it listed as a 'client' feature, although another ddp server in this context is the client. I might submit a PR to address this documentation issue. The DDP protocol has a standard set of messages to manage the synchronisation of data between server/client, and it's very worthwhile to read through the docs to get a full understanding of it's potential.

Cluster refers to these as services too btw, so to keep us all on the same page, would be best to use the ubiquitous language.

[arunoda]

Yes santo. What you are referring to multi casting and publish. Yes, that's
kind true in normal pub sub Architecture.

But in meteor pub/sub refers to something else. It's for getting a subset
of data including the future changes happen to those subset.
On 2015 මාර්තු 23, සඳුදා at පෙ.ව. 5.33 Rhys Bartels-Waller <
[email protected]> wrote:

@santo74 https://github.com/santo74

It sounds like what your describing here can be achieved using RPCs via
Meteor.methods, whereas the issue @arunoda https://github.com/arunoda has
raised
#32 (comment)
relates to how internal pub/sub within the cluster would be load balanced
since you can subscribe to another ddp server (or cluster service)
publication from the server. The meteor docs
https://docs.meteor.com/#/full/meteor_subscribe have it listed as a
'client' feature, although another ddp server in this context is the
client, although it's likely more just to communicate the main intent. I
might submit a PR to address this. The DDP protocol
https://github.com/meteor/meteor/blob/devel/packages/ddp/DDP.md has a
standard set of messages to manage the synchronisation of data between
server/client, and it's very worthwhile to read through the docs to get a
full understanding of it's pote ntial.

Cluster refers to these as services too btw, so to keep us all on the
same page, would be best to use the ubiquitous language.

—
Reply to this email directly or view it on GitHub
#32 (comment).

[santo74]

Ok, I understand pub/sub is working in another way in Meteor.
What I was trying to clarify - and I must admit I probably wasn't very clear at that - is the fact that Vert.x also uses its clustering technology to be able to load balance requests between services on the server side.
In case of Vert.x this is done by using the pub/sub mechanism on the server side as well, but of course it doesn't have to be that way.

The only point I'm trying to make is that it's not uncommon to implement the loadbalancing of (micro)services on the server side on top of the clustering technology
Simply because it always knows exactly which services (and how many instances of it) are available in the system for loadbalancing the requests.

And that's what I'm currently missing in the Meteor cluster package (cfr my initial post).

[aykutyaman]

@arunoda:

Okay. Load balancing is only work when we are getting connections from the outside to the client. For load balance inside the cluster. You need to create a pool of connections and send the in a round robin fashion.

Is that true if I connect to services from the client like you did in the Microservices - Beyond Basics ?

[rhyslbw]

@santo74 Have you implemented a solution to load balance internal requests? If so, keen to know if you have iterated on arunoda's snippet

[santo74]

No, as I stated before I hope to see this kind of functionality implemented in the cluster package itself rather than having to implement it myself.

[santo74]

Update:
In the meantime I played a little bit more with the cluster package and the suggestions from Arunoda to create a pool of connections for internal load balancing of requests.

The conclusion is that there are still some important pieces missing.
First of all the connection pooling code - as suggested by Arunoda - works fine, which was to be expected.
However, it obviously stops working reliably as soon as a service leaves or joins the cluster because such an event isn't detected and therefore the pool is out of sync.
In addition to this it still only covers the load balancing of method calls and not pub/sub, as was already noticed before by Arunoda.

So yes, the cluster package offers a really neat solution for service discovery, but unfortunately the load balancing currently has some important limitations.

[rhyslbw]

@santo74 You could use the Cluster endpoints collection as a reactive data source for the pool right? As for pub/sub, unless I'm mistaking the issue, it would only impact server to server Meteor pub/sub. Is that what your doing?

[santo74]

Yes, of course I can use the cluster endpoints collection, but that's the whole point: to make the cluster package a real loadbalancing tool - as is advertised here - it should do this out of the box.
Regarding the pub/sub, yes you are correct that it's related to server to server communication.
Not that I am already extensively using that at the moment, but it would surely be a nice improvement as well.
(cfr Arunoda's comment #32 (comment))

But please don't get me wrong, I really appreciate the work being done here and I understand it is still a relatively young project.
I'm just providing my feeback here in the hope it can be made even better ;-)

[rhyslbw]

My comment was only looking at how it would be solved by the package itself. This is opensource! Submit a PR is you get a solution that makes sense :-)