Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functions worker nodes can't be accessed #7

Open
KannarFr opened this issue Jun 4, 2019 · 6 comments
Open

Functions worker nodes can't be accessed #7

KannarFr opened this issue Jun 4, 2019 · 6 comments

Comments

@KannarFr
Copy link
Contributor

KannarFr commented Jun 4, 2019

Hi,

If function worker are not in the broker but in dedicated node, they can be accessed via custom admin-url which is the url of the function worker.

So, I think pulsar-express must have functions worker list that be managed then the functions tab will display functions worker list and functions as subitems.

@bbonnin
Copy link
Owner

bbonnin commented Jun 4, 2019

Hi,
You're right that the function workers can run in dedicated nodes, but it's the role of the proxy to route the requests, isn't it ? Of course, if you don't have a proxy, we have to route by ourself :(
These features in Pulsar are a little bit under documented, I think.
Anyway, thank you for opening this issue :)

@KannarFr
Copy link
Contributor Author

KannarFr commented Jun 4, 2019

The administration of the dedicated worker nodes is currently not managed by the proxy.

Currenty to manage them, you need to pass the --admin-url which is the dedicated functions worker host.

bin/pulsar-admin --admin-url https://<dedicatedfunctionsworkerhost> --auth-params "token:<token>" functions create --jar <jarfile> --classname <classname> --tenant <tenant> --namespace functions

@bbonnin
Copy link
Owner

bbonnin commented Jun 4, 2019

In this page https://pulsar.apache.org/docs/fr/functions-worker/, there is a description of the proxy configuration that route the requests (broker and function worker), does it not fulfill the need ?

@KannarFr
Copy link
Contributor Author

KannarFr commented Jun 4, 2019

IIRC: nope, this page describes how to make dedicated functions worker communicate to brokers via proxy. But the management of the functions and get their information can't be done using proxies.

@bbonnin
Copy link
Owner

bbonnin commented Jun 4, 2019

Mmmh, are you sure ? because the last diagram in this page shows that with the proxy you will have a single endpoint for the admin URL.
And in the doc, with these sentences, we can think that the proxy will route the requests:

When you are running functions-worker in a separate cluster, the admin rest endpoints 
are split into two clusters. functions, function-worker, source and sink endpoints are 
now served by the functions-worker cluster, while all the other remaining endpoints 
are served by the broker cluster. Hence you need to configure your pulsar-admin to 
use the right service URL accordingly.

In order to address this inconvenience, you can start a proxy cluster for routing 
the admin rest requests accordingly. Hence you will have one central entry point 
for your admin service.

@bbonnin
Copy link
Owner

bbonnin commented Jun 4, 2019

I have tested a config with a proxy, a broker and a worker node: the proxy route the requests, BUT, for now, only the old API requests are routed (i.e. /admin/functions and /admin/v2/functions), not the new API requests for the functions (/admin/v3/functions used by pulsar-admin).
Here: https://github.com/apache/pulsar/blob/master/pulsar-proxy/src/main/java/org/apache/pulsar/proxy/server/AdminProxyHandler.java#L222, you will see that only some url are redirected to the function worker URL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants