Don't use BrotliMiddleware for already compressed data formats

[drnextgis]

When we use tile endpoint /{z}/{x}/{y}.png in any web map clients (e.g. OpenLayers but it is easy to reproduce just with a simple <img src="..." /> tag) browsers set Accept-Encoding header:

• Accept-Encoding: gzip, deflate, br (Chromium)
• Accept-Encoding: gzip, deflate (Firefox)

This header triggers BrotliMiddleware. That is, the middleware compresses already compressed files (the PNG image format already uses compression internally) increasing its size:

$ curl -X 'GET' \       
  'http://localhost:8000/cog/tiles/13/4641/3329.png...' \
  -H 'accept: image/png' -H 'accept-encoding: br, zip' > encoded.png

$ curl -X 'GET' \       
  'http://localhost:8000/cog/tiles/13/4641/3329.png...' \
  -H 'accept: image/png' > not-encoded.png

$ ls -l
total 16
-rw-rw-r-- 1 denis denis 8107 Mar 15 19:18 encoded.png
-rw-rw-r-- 1 denis denis 8103 Mar 15 19:18 not-encoded.png

[vincentsarago]

I'm a bit hesitant to remove compression for specific responses (e.g a PNG file created with ZLEVEL=1, won't be compressed).

I think it might be better to just remove any middleware https://github.com/developmentseed/titiler/blob/master/titiler/main.py#L48-L58 and let the users do what they want from here.

[kylebarron]

This header triggers BrotliMiddleware

Your analysis is mostly correct, except you miss that the response is still compressed on firefox, just with gzip encoding instead of brotli.

When br isn't in Accept-Encoding, (but gzip is) the response will be gzip encoded.

titiler/titiler/main.py

Line 57 in d93b14a

app.add_middleware(BrotliMiddleware, minimum_size=0, gzip_fallback=True)

https://github.com/fullonic/brotli-asgi/blob/117fb82081af2b7e490a914cefd80472588155b5/brotli_asgi/__init__.py#L74

[drnextgis]

My main concern is that we compress already compressed data no matter with which encoding algorithm - br or gzip.

[kylebarron]

Right but I was pointing out it's not browser dependent.

• Accept-Encoding: gzip, deflate (Firefox)

Also, you're using 5-year-old Firefox?? https://caniuse.com/?search=brotli

[drnextgis]

Not really, it seems I've stumbled upon a known issue.