Petastorm not working due to PyArrow version hell

[kiranzo]

I was trying to build something from petastorm dataset minimal example.
When I tried to launch my code, I got error:

    from pyarrow.filesystem import LocalFileSystem
ModuleNotFoundError: No module named 'pyarrow.filesystem'

Because apparently it's called pyarrow.fs now?

After downgrading my pyarrow version to 13.0, then 12.0 as suggested here:

Traceback (most recent call last):
  File "/.myenv/lib/python3.8/site-packages/petastorm/utils.py", line 21, in <module>
    import pyarrow
  File "/.myenv/lib/python3.8/site-packages/pyarrow/__init__.py", line 65, in <module>
    import pyarrow.lib as _lib
  File "pyarrow/compat.pxi", line 43, in init pyarrow.lib
  File "/.myenv/lib/python3.8/site-packages/cloudpickle/__init__.py", line 1, in <module>
    from . import cloudpickle
  File "/.myenv/lib/python3.8/site-packages/cloudpickle/cloudpickle.py", line 73, in <module>
    import uuid
  File "/usr/lib/python3.8/uuid.py", line 60, in <module>
    _platform_system = platform.system()
  File "/usr/lib/python3.8/platform.py", line 892, in system
    return uname().system
  File "/usr/lib/python3.8/platform.py", line 858, in uname
    processor = _syscmd_uname('-p', '')
  File "/usr/lib/python3.8/platform.py", line 612, in _syscmd_uname
    import subprocess
  File "/usr/lib/python3.8/subprocess.py", line 218, in <module>
    _PopenSelector = selectors.SelectSelector
AttributeError: module 'selectors' has no attribute 'SelectSelector'

What version of pyarrow is it supposed to use??

[westfly]

@kiranzo as ISSUE Validate_schema keyword not supported yet describe

 pip install pyarrow==10.0.1

[kiranzo]

@kiranzo as ISSUE Validate_schema keyword not supported yet describe

 pip install pyarrow==10.0.1

Gratz! We've already settled for Pandas with parquet format for now though 😅