Improve exclude path filesystem logic

[johnlarkin1]

• Description: Improving the use of the exclude arguments for FileSystemBlobLoader
• Issue: GenericLoader.from_filesystem "exclude" not working #13751
• Dependencies: n/a
• Tag maintainer:
• Twitter handle: this PR won't get announced, but @jlarks32

See the issue above for more detail, but I don't think that the exclude parameter is being handled really like what the API is indicating / what we were thinking (largely due to annoyances with pathlib's Path.match). Let me know if you all want me to add some more unit tests and build this out a little bit more.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Nov 27, 2023 1:53pm

[johnlarkin1]

Hey team, I wasn't able to actually run poetry install --with test so if there are issues with the formatting / builds let me know how to resolve and I definitely can.

Here was the issue i filed: #13912

[efriis]

pathlib's match sounds like a mess! However, I'm not sure the included logic is the best way of handling it, because it's a bit too custom.

I haven't looked too deeply into this, but given the python bug is tagged only up to python3.9, does the issue resolve on newer python versions?

If possible, I'd love to resolve this without any custom path matching logic in the langchain library to overcome issues in python.

[ccurme]

Closing as author is unresponsive, will re-open if needed.