Poor handling of truncated streams

[philipc]

Currently InflateStream::update nearly always returns Ok for truncated streams, and this flows through to the higher level APIs. There should be some way to determine that more data is still expected, but I don't see how the current API allows for this. The documentation says to call InflateStream::update until it returns 0 bytes, but none of the current callers do this, and I don't see how it can work. In general, the last call that passes some input data to InflateStream::update will make progress, and then there is no further input data to pass, so the caller will never get 0 bytes returned.

The one exception that I can see was introduced in #24, and that only occurs if there are less than 4 bytes currently available when in state CheckCRC. Actually, this check is wrong, because the CRC may be split over two update calls.

[eddyb]

The caller can detect truncated streams if it knows how many bytes it expects (gzip, PNG, etc.).
Or are you talking about knowing whether there are unused bits, or I guess if the current block hasn't ended yet? IIRC it might be possible to have a complete block and not be able to tell (from the stream itself) that any more data follows but I could be wrong, it's been ages.

[oyvindln]

There is a flag in the deflate block header that tells if it's the last block, so if the stream ends without hitting and ending a block with that flag, it's likely truncated. This should be pretty easy to check for.

[eddyb]

Ahh, so there's both an end of block marker and that bit. Yeah then we could have a method, e.g. is_finished or is_complete.

[philipc]

Ah, I wasn't aware that callers generally knew how many bytes to expect. In that case, this probably doesn't matter much. I just noticed it because of the test to check for missing CRC.

An is_finished method on InflateStream is simple to add.

Should InflateWriter also have this method? Another option is for InflateWriter::finish to return an error if the stream isn't finished.

Similarly, should inflate_bytes return an error? There's no other way for it to signal truncation.