grpc - added grpc transcoding feature

[zZHorizonZz]

This pull request introduces grpc transcoding functionality, extending API accessibility by allowing grpc services to be called via HTTP requests as proposed here #25837. This implementation follows grpc transcoding described by Google Cloud grpc transcoding documentation, as I was not able to find If there is currently official grpc standard for this feature.

Example Usage:

// The greeting service definition.
service Greeter {
 // Simple path
 rpc SimplePath (HelloRequest) returns (HelloReply) {
  option (google.api.http) = {
   post: "/v1/simple"
   body: "*"
  };
 }

This configuration will enable quarkus to expose a new HTTP route at /v1/simple. HTTP requests to this route will be translated into calls to the underlying SimplePath grpc method. More detailed description can be seen here.

(Sorry for the previous PR. I had some trouble squashing commits, and after my force push, the bot added almost all of the labels)

[github-actions]

🎊 PR Preview 79af515 has been successfully built and deployed to https://quarkus-pr-main-40254-preview.surge.sh/version/main/guides/

• Images of blog posts older than 3 months are not available.
• Newsletters older than 3 months are not available.

[zZHorizonZz]

I'm not sure why these changes are here. I think when I ran optimize imports on the entire gRPC folder, it might be the cause. However, the changes might improve the code by removing unnecessary spaces and tabs, so I think this could be kept since I don't think any formatting is done on js by quarkus build anyway.

[alesj]

@zZHorizonZz does plain gRPC also work along with this transcoding? Both old io.grpc and new Vert.x gRPC?
Do the tests cover this?

[zZHorizonZz]

@zZHorizonZz does plain gRPC also work along with this transcoding? Both old io.grpc and new Vert.x gRPC? Do the tests cover this?

Do you mean if a separate server is on? Honestly, I haven't tested that specifically since I forgot about that (I will try to add some tests for it). However, basic gRPC with a separate server turned off works properly you can see that in the HelloWorldTranscodingEndpointTest where it calls the endpoint, and that then calls the client. It's similar to the other gRPC integration tests.

[alesj]

If you could add test(s) for all flavors (old2new, new2old, new2new, old2old) that would be good / nice as well.
So we make sure both things can co-exist.

But on another note, I feel as having a whole new gRPC server is a bit too much ... can't this just be a route to the existing new Vert.x gRPC server?

[zZHorizonZz]

Now that I think about it, since I copied the grpc-vertx integration tests and mostly left the application.properties file the same. My question is when running JVM tests goal, does it run profiles that are specified in properties file? Are they specified somewhere? The properties file includes vertx, n2o, and o2n profiles, which (if used) should test everything with the separate server both on and off.

[alesj]

My question is when running JVM tests goal, does it run profiles that are specified in properties file? Are they specified somewhere?

The test then needs to have this profile enabled, see existing tests that have this.
You have a base test, and then separate tests with certain profiles:

@QuarkusTest
@TestProfile(N2OGRPCTestProfile.class)
class N2OLegacyHelloGrpcServiceTest extends LegacyHelloGrpcServiceTestBase {
}

[zZHorizonZz]

But on another note, I feel as having a whole new gRPC server is a bit too much ... can't this just be a route to the existing new Vert.x gRPC server?

I wanted to reuse as much as possible from the Vert.x gRPC implementation, but as I got further I started running into issues. It comes down pretty much to decoding and encoding. Basically GrpcServerImpl creates a GrpcServerRequestImpl and uses the GrpcMessageDecoder and GrpcMessageEncoder for its parameters, like this:

new MethodCallHandler<>(methodDesc, GrpcMessageDecoder.unmarshaller(methodDesc.getRequestMarshaller()), GrpcMessageEncoder.marshaller(methodDesc.getResponseMarshaller()), handler));

And then these parameters are passed to GrpcServerRequestImpl like this:

new GrpcServerRequestImpl<>(httpRequest, method.messageDecoder, method.messageEncoder, methodCall);

The problem is that I can't directly use these decoders and encoders for gRPC and decode and encode json with them. Furthermore, I need to merge route and query parameters into the payload, which is done by GrpcTranscodingRequest.

[quarkus-bot]

Status for workflow Quarkus Documentation CI

This is the status report for running Quarkus Documentation CI on commit 5c74729.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

⚠️ There are other workflow runs running, you probably need to wait for their status before merging.

[quarkus-bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 5c74729.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	MicroProfile TCKs Tests	Verify	Failures	Logs	Raw logs	🔍

Full information is available in the Build summary check run.
You can consult the Develocity build scans.

Failures

⚙️ MicroProfile TCKs Tests #

- Failing: tcks/microprofile-opentelemetry 

📦 tcks/microprofile-opentelemetry

✖ org.eclipse.microprofile.telemetry.tracing.tck.async.JaxRsClientAsyncTest.testIntegrationWithJaxRsClientAsync - History - More details - Source on GitHub

java.util.concurrent.RejectedExecutionException: event executor terminated
	at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:934)
	at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:351)
	at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:344)
	at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:836)
	at io.netty.util.concurrent.SingleThreadEventExecutor.execute0(SingleThreadEventExecutor.java:827)
	at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:817)
	at io.vertx.core.impl.EventLoopExecutor.execute(EventLoopExecutor.java:35)

[zZHorizonZz]

If you could add test(s) for all flavors (old2new, new2old, new2new, old2old) that would be good / nice as well.

@alesj I'm a bit confused about what you meant with old2new, new2old, etc as there isn't really anything that I could categorize as "old" there are only new things from my point of view.

[alesj]

If you could add test(s) for all flavors (old2new, new2old, new2new, old2old) that would be good / nice as well.

@alesj I'm a bit confused about what you meant with old2new, new2old, etc as there isn't really anything that I could categorize as "old" there are only new things from my point of view.

We have tests that check all 4 combinations - matrix between io.grpc and our Vert.x gRPC.

But this won't change the fact that we need to get this into Vert.x, and not into Quarkus gRPC,
as there really should not be 2 servers running to handle this -- only a single one, capable of handling plain gRPC and transcoding as well.

[zZHorizonZz]

But this won't change the fact that we need to get this into Vert.x, and not into Quarkus gRPC,
as there really should not be 2 servers running to handle this -- only a single one, capable of handling plain gRPC and transcoding as well.

I agree with that. I will try to open an issue on vertx-grpc and try to figure out how this could be done.

[cescoffier]

I think that most of this code should to into Vert.x gRPC server and not in Quarkus.
Duplicating the gRPC server is going to be a non-meaningless maintenance burden. We already have the legacy and old one to maintain, having a third might not be the best idea.

[cescoffier]

Marking it as on-ice. The feature is super interesting, but we need to find a proper way to implement it. We may need to tweak things in Vert.x to avoid code duplication.