Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Brotli compression performance improvement #40

Closed
SachaMorard opened this issue Sep 4, 2024 · 1 comment · Fixed by #103
Closed

Brotli compression performance improvement #40

SachaMorard opened this issue Sep 4, 2024 · 1 comment · Fixed by #103
Labels
enhancement New feature or request

Comments

@SachaMorard
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Brotli compression is slower than other encoding algorithms

Edgee adds a header to the response which indicates the compute duration (x-edgee-compute-duration)
When an upstream server responds a gzip compressed response, the compute duration on my local env is okay (~10ms)
When an upstream server responds a brotli compressed response, the compute duration on my local env is too slow (~120ms)

The slowdown seems to originate from this part:

Some("br") => { // handle brotli encoding
    // q: quality (range: 0-11), lgwin: window size (range: 10-24)
    let mut encoder  = CompressorWriter::new(Vec::new(), 4096, 11, 24);
    encoder.write_all(new_body.as_bytes())?;
    encoder.flush()?;
    encoder.into_inner()
}

Describe the solution you'd like

  • Test if brotli compression settings can help
  • Look at the strategies and solutions used by other proxies
  • Test other libraries
@SachaMorard SachaMorard added the enhancement New feature or request label Sep 4, 2024
@SachaMorard SachaMorard added the repo/edgee label Sep 11, 2024 — with Linear
@asolimando
Copy link

Here are some figures for x-edgee-compute-duration on my machine with different setups:

  • no compression: ~3ms
  • brotli (quality 5): ~9 ms
  • brotli (quality 11): ~101 ms <-- that's the current hard-coded value
  • gzip: ~9 ms

So already by lowering the quality to 5 we have a performance closer to that of gzip, of course a couple of runs without proper benchmarking isn't conclusive, but it already suggests that there is room for improvement by changing some of the default parameters (with #97 in place it would probably be easier to have better figures).

Regarding what other reverse proxies do:
Nginx allows to tune several brotli parameters, including quality/compression level, buffer size, window size and the minimum payload size.

Additionally, not all MIME types equally benefit from compression, so we have an additional parameter for the MIME types for which one wants brotli encoding to be activated.

In summary, a reasonable starting point is to lower the compression quality parameter, and possibly add all or some of these configuration knobs to the Edgee configuration to make it tunable.

@KokaKiwi KokaKiwi linked a pull request Oct 17, 2024 that will close this issue
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants