Skip to content

Commit

Permalink
[DM-40656] Fix bug where "Response content shorter than Content-Length"
Browse files Browse the repository at this point in the history
When serving a file that should be cached on the other side (and
crawlspace returns a 304) we return the Content-Length of the cached
file instead of the Content-Length of the 304 message.  This makes
python emit the following error:

INFO:     68.230.27.252:40976 - "GET /api/hips/images/band_z/Norder11/Dir35540000/Npix35543833.png HTTP/1.1" 304 Not Modified
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/venv/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/fastapi/applications.py", line 270, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/venv/lib/python3.11/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/base.py", line 107, in __call__
    await response(scope, receive, send)
  File "/opt/venv/lib/python3.11/site-packages/starlette/responses.py", line 266, in __call__
    async with anyio.create_task_group() as task_group:
  File "/opt/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "/opt/venv/lib/python3.11/site-packages/starlette/responses.py", line 269, in wrap
    await func()
  File "/opt/venv/lib/python3.11/site-packages/starlette/responses.py", line 263, in stream_response
    await send({"type": "http.response.body", "body": b"", "more_body": False})
  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 159, in _send
    await send(message)
  File "/opt/venv/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 561, in send
    raise RuntimeError("Response content shorter than Content-Length")
RuntimeError: Response content shorter than Content-Length

Looking through the spec a bit more, it seems like it's okay to not
provide a content length and I think actually encouraged

https://datatracker.ietf.org/doc/html/rfc7232#section-4.1
  • Loading branch information
cbanek committed Sep 7, 2023
1 parent 1371575 commit d1ab3e9
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 1 deletion.
1 change: 1 addition & 0 deletions src/crawlspace/handlers/external.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ def get_file(

if crawlspace_file.blob.etag in etags:
logger.debug("File unchanged", path=path)
del crawlspace_file.headers["Content-Length"]
return Response(
status_code=304,
content="",
Expand Down
2 changes: 1 addition & 1 deletion tests/handlers/external_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ async def test_cache_validation(
f"{config.url_prefix}/", headers={"If-None-Match": header}
)
assert r.status_code == 304, f"If-None-Match: {header}"
assert r.headers["Content-Length"] == str(index.stat().st_size)
assert "Content-Length" not in r.headers
assert r.headers["Content-Type"] == "text/html; charset=utf-8"
assert r.headers["Etag"] == etag
mod = datetime.fromtimestamp(index.stat().st_mtime, tz=timezone.utc)
Expand Down

0 comments on commit d1ab3e9

Please sign in to comment.