About Onigmo

[sashashura]

I wonder what is your opinion about fixing fluent-bit memory access bugs when the root cause is Onigmo.
At this moment I have identified two such cases and submitted the patches: k-takata/Onigmo#164 and k-takata/Onigmo#165. However after almost two months there is no response and the last time somebody has pushed anything to the project is Feb 4, 2021.

From security point of view attackers don't really care in which library invalid memory access happens, as long as is in the program they exploit. Do you think it is a good idea to patch Onigmo locally in https://github.com/fluent/fluent-bit/tree/master/lib/onigmo until the official library provides the fix?

cc: @DavidKorczynski

[DavidKorczynski]

Do you think it is a good idea to patch Onigmo locally in https://github.com/fluent/fluent-bit/tree/master/lib/onigmo until the official library provides the fix?

Yes, can you make PRs?

The more general solution may be to switch to Oniguruma https://github.com/kkos/oniguruma

[edsiper]

thanks everybody for addressing this.

Before submitting the PR directly to the bundled library, I think it's important to understand the Onigmo situation:

• @cosmo0920 do you know if Ruby still relies on Onigmo ?, are there any downsides by switching to Oniguruma ?
• if we will maintain a stable-patched version, we can host it inside the fluent/ organization, but let's clarify the previous point first.

[cosmo0920]

Ruby still uses onigmo. They are bundled onigmo as normal sources like as other Ruby's sources:

• https://github.com/ruby/ruby/blob/master/regenc.c
• https://github.com/ruby/ruby/blob/master/regcomp.c
  And other Onigmo's header put files:

$ ag  "Onigmo \(Oniguruma-mod\) \(regular expression library\)"  
regparse.h
4:  regparse.h -  Onigmo (Oniguruma-mod) (regular expression library)

include/ruby/onigmo.h
4:  onigmo.h - Onigmo (Oniguruma-mod) (regular expression library)

regenc.h
4:  regenc.h -  Onigmo (Oniguruma-mod) (regular expression library)

regexec.c
2:  regexec.c -  Onigmo (Oniguruma-mod) (regular expression library)

regint.h
4:  regint.h -  Onigmo (Oniguruma-mod) (regular expression library)

regcomp.c
2:  regcomp.c -  Onigmo (Oniguruma-mod) (regular expression library)

regsyntax.c
2:  regsyntax.c -  Onigmo (Oniguruma-mod) (regular expression library)

enc/shift_jis.c
2:  shift_jis.c -  Onigmo (Oniguruma-mod) (regular expression library)

enc/ascii.c
2:  ascii.c -  Onigmo (Oniguruma-mod) (regular expression library)

enc/euc_jp.c
2:  euc_jp.c -  Onigmo (Oniguruma-mod) (regular expression library)

enc/shift_jis.h
2:  shift_jis.h -  Onigmo (Oniguruma-mod) (regular expression library)

enc/windows_31j.c
2:  windows_31j.c -  Onigmo (Oniguruma-mod) (regular expression library)

regparse.c
2:  regparse.c -  Onigmo (Oniguruma-mod) (regular expression library)

regenc.c
2:  regenc.c -  Onigmo (Oniguruma-mod) (regular expression library)

regerror.c
2:  regerror.c -  Onigmo (Oniguruma-mod) (regular expression library)

The main downside is: there is a possibility to change supported regular expression format. I'm not sure what is added feature in Onigmo instead of Oniguruma.

[edsiper]

got it, we cannot break regexes, I recall there was some special regex features for Ruby that also Fluentd relies on.

So I think it would be better to keep our own updated version.

@cosmo0920 do you know if the original developer can be contacted locally there in JP ?

[sashashura]

Yes, can you make PRs?

#6008
#6009

[DavidKorczynski]

Thanks @sashashura -- @edsiper will carry this forward in terms of to patch or not.

One thing we should consider is whether patching is sustainable, as the fuzzers may continue to keep finding issues in the library.

[edsiper]

@DavidKorczynski @sashashura

I've cloned the original Onigmo in our own fluent/ organization, just with the purpose to have all these patches-in.

would you please submit the PRs to the new repo instead?

https://github.com/fluent/onigmo

[cosmo0920]

@cosmo0920 do you know if the original developer can be contacted locally there in JP ?

I'm not sure where is he living and working in. But, his twitter (in Japanese) https://twitter.com/k_takata is still active.
So, we might able to reach out to him via Japanese to ask him proceeding for OSS FUZZ PRs.

[cosmo0920]

I'd sent message to him on Twitter. (In Japanese): https://twitter.com/cosmo__/status/1566973012733493249

[sashashura]

I've cloned the original Onigmo in our own fluent/ organization, just with the purpose to have all these patches-in.

would you please submit the PRs to the new repo instead?

https://github.com/fluent/onigmo

Done
fluent/onigmo#1
fluent/onigmo#2

[edsiper]

thanks everyone!

everything has been merged on https://github.com/fluent/onigmo

we also added CMake support into onigmo, merged into Fluent Bit:

• build: lib: onigmo: Bundle the patched fluent/onigmo #6097
• 1.9: build: lib: onigmo: backport for bundling the patched fluent/onigmo patch #6101