Upgrade to rdkafka 0.10.0

[malandrina]

Why?

My team maintains a stream processing application that uses Racecar. Recently we learned that a known issue with librdkafka 1.4.0 (confluentinc/librdkafka#2830) negatively impacts the performance of our application. The latest version of rdkafka, 0.10.0, upgrades the gem to use librdkafka 1.5.0. This PR proposes updating Racecar to use rdkafka 0.10.0.

References:

• Librdkafka 1.5.0 release notes: https://github.com/edenhill/librdkafka/releases?after=v1.6.0-PRE4
• rdkafka-ruby PR upgrading to librdkafka 1.5.0: Upgrade to librdkafka 1.5.0 karafka/rdkafka-ruby#165

How?

I made the following changes:

• Updated racecar.gemspec to require rdkafka 0.10.0
• Updated test setup in spec/integration/consumer_spec.rb to explicitly create topics instead of relying on Kafka to automatically create the topics. librdkafka 1.5.0+ does not support topic auto-creation.
• Committed the changes to Gemfile.lock that were generated by running bin/setup

Testing

I followed the instructions for setting up the Racecar development environment in the README and confirmed (non-dockerized) tests pass locally by running bundle exec rspec.

Considerations

• I noticed that spec/integration/consumer_spec.rb tests sometimes fail when I run bundle exec rspec but they always pass when I run bundle exec rspec spec/integration/consumer_spec.rb. I observed this on both master and my branch. I took care to stop and restart the Docker services after each test run.
• As a stopgap, my team is currently using gems built from our own forks of the racecar and rdkafka-ruby repos in production. Merging and releasing the changes in this PR will allow us to avoid using our forks.

Thanks in advance for your time! 🙇‍♀️

[dasch]

Thanks you! Can you also update the changelog?

[malandrina]

Thanks @dasch!

malandrina@bd1c2a4 malandrina@410648c updates the changelog based on prior art in #222.

[dasch]

Thanks!