Skip to content

Commit

Permalink
Fix reading binary files/streams by adjusting encoding to match mode
Browse files Browse the repository at this point in the history
When reading in binary mode, undefine `encoding`.
Otherwise, `smart_open` will return a `TextIOWrapper` in `"r"` mode.
However, reading binary streams needs a `BufferedReader`.
  • Loading branch information
amotl committed Dec 13, 2023
1 parent 9e6f857 commit 4072661
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions tap_spreadsheets_anywhere/format_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,11 @@ def get_streamreader(uri, universal_newlines=True, newline='', open_mode='r', en
SCHEME_SEP = "://"
kwargs = kwarg_dispatch.get(uri.split(SCHEME_SEP, 1)[0], lambda: {})()

# When reading in binary mode, undefine `encoding`.
# Otherwise, `smart_open` will return a `TextIOWrapper` in `"r"` mode.
# However, reading binary streams needs a `BufferedReader`.
if "b" in open_mode:
encoding = None
streamreader = smart_open.open(uri, open_mode, newline=newline, errors='surrogateescape', encoding=encoding, **kwargs)

if not universal_newlines and isinstance(streamreader, StreamReader):
Expand Down

0 comments on commit 4072661

Please sign in to comment.