Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ZSTD compression support to forward plugin #4162

Open
zamazan4ik opened this issue May 2, 2023 · 1 comment · May be fixed by #4657
Open

Add ZSTD compression support to forward plugin #4162

zamazan4ik opened this issue May 2, 2023 · 1 comment · May be fixed by #4657
Labels
enhancement Feature request or improve operations

Comments

@zamazan4ik
Copy link

zamazan4ik commented May 2, 2023 •
edited
Loading

Is your feature request related to a problem? Please describe.

ZSTD is a modern compression algorithm developed by Facebook. Using ZSTD has one of the best compression/resource consumption ratios in the compression area.

Describe the solution you'd like

Have a ZSTD compression support in additoin to GZip. Would be awesome, if will be a possibility to specify the compression level as well.

Describe alternatives you've considered

Left all things as is.

Additional context

Maybe this change will require changing FluentBit implementation as well.
@daipom daipom added feature request *Deprecated Label* Use enhancement label in general and removed waiting-for-triage labels May 5, 2023
@Garfield96
Copy link
Contributor

I would also like to see support for ZSTD. While the compression ratio of gzip is often decent, it is slow, which is a problem for Fluentd due to the GIL of CRuby. Apart from performance, gzip combined with TLS can be a security issue (cf. https://en.wikipedia.org/wiki/BREACH).

While there are other good compression algorithms (e.g. LZ4 or snappy), ZSTD seems to be the only one which has maintained ruby bindings (cf. https://github.com/SpringMT/zstd-ruby).

@kenhys kenhys added enhancement Feature request or improve operations and removed feature request *Deprecated Label* Use enhancement label in general labels Jul 31, 2024
@Athishpranav2003 Athishpranav2003 linked a pull request Oct 3, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Feature request or improve operations
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

compressable:Zstd comp support Athishpranav2003/fluentd
4 participants
@kenhys @zamazan4ik @daipom @Garfield96