Need help in designing architecture

[denyskanunnikov]

Good afternoon, I have a sincere interest in your projects. I would like to use some solutions in my product environment. Could you please tell me if it is possible to integrate into worker.php or consumer.php support for consuming events from Kafka broker via php-rdkafka client with custom logic for fixing offsets, timeouts, etc. and then design saga on temporal?

[rustatian]

Hey @denyskanunnikov 👋🏻
We have a Kafka driver for the RR. You just need to configure it and enjoy consuming messages in your worker.php from RR 😃
Config: https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml#L1199
Config2: https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml#L1643

[rustatian]

But if you need a completely custom consumer, you have 2 options here:

1. Yes, you may use any existing driver, like php-rdkafka. RR won't limiting you in writing code or using any third-party solutions.
2. You may fork our current Kafka driver written in Go and tune it to fit your needs. Then just build your custom RR with velox

EDIT: For sure you may send these messages consumed from Kafka to the Temporal or combine JOBS+Temporal. There are no limitations.

[denyskanunnikov]

@rustatian Thank you for the answer. How can I configure the enable.auto.offset.store condition to false with this driver ?

[rustatian]

RR automatically commit offset for the messages (if the message consumed w/o errors from the PHP worker). IIRC, you may set this option (enable.auto.offset.store) directly in the Kafka broker consumer options, so, no RR needed here. To consume not from the beginning of partition, you have various options here: https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml#L1780 😃