feat: Add request body compression with optional brotli

[mixalturek]

• https://github.com/apify/apify-core/pull/28971 added support for brotli compression to BE.
• Pros: higher compression, cons: more CPU intensive.
• The brotli dependencies are defined as optional, one has to explicitly enable it and choose one. If none is available, the code falls back to gzip.
• JS client doesn't compress requests that are too small, this Python client compresses just everything. No change done, I only noticed and stating it.

Closes: #942

Related PRs

• https://github.com/apify/apify-core/pull/28971
• feat: Document support for brotli compression apify-docs#2750
• feat: Compress requests using brotli algo apify-client-js#962
• feat: Add request body compression with optional brotli #927
• feat: Compress requests using brotli algo apify-sdk-python#1031

[codecov]

Codecov Report

❌ Patch coverage is 97.11934% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.59%. Comparing base (7f61a10) to head (c810cfa).
⚠️ Report is 36 commits behind head on master.

Files with missing lines	Patch %	Lines
src/apify_client/_utils/http.py	86.36%	3 Missing ⚠️
src/apify_client/_utils/encoding.py	95.00%	2 Missing ⚠️
src/apify_client/types.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #927      +/-   ##
==========================================
+ Coverage   94.54%   94.59%   +0.04%     
==========================================
  Files          48       58      +10     
  Lines        5100     5239     +139     
==========================================
+ Hits         4822     4956     +134     
- Misses        278      283       +5     

Flag	Coverage Δ
integration	92.40% <80.65%> (-0.52%)	⬇️
unit	83.73% <97.11%> (+0.36%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[vdusek]

This has a significant impact. @Pijukatel, please check it as well.

[mixalturek]

@vdusek Nearly all comments are fixed now, thanks again for your great review.

One more thing: I forgot to mention that the JS client skips compression if the payload size is below 1024 B. Do we want to apply it to the Python client as well?

[vdusek]

One more thing: I forgot to mention that the JS client skips compression if the payload size is below 1024 B. Do we want to apply it to the Python client as well?

Thanks for noticing. I opened an issue for that: #934. Let's resolve it in a separate PR so it is documented in the changelog.

[Pijukatel]

Not sure about the non-default compression quality , but it is not blocking for me.

[vdusek]

I removed brotli from the dev dependency group. Sorry, my previous comment wasn't entirely correct. We run uv sync --all-extras everywhere in CI, so there's no need to include it there. The initial mypy/ty override ignores confused me, but they were probably added based on your local setup, where you didn't have the brotli extra installed.

Other than that, one more thing: the documentation 🙂.

Please extend the installation section in the docs here: https://docs.apify.com/api/client/python/docs#installation. You can use a similar section to the one you added to the README.

Also, please create a new concept page for compression. Something like: explaining HTTP compression, how to switch between compression methods, and the pros and cons of each approach.

[vdusek]

I did a few additional improvements. @Pijukatel do you want to re-check it?