Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Implement exists api #19

Merged
merged 1 commit into from
Aug 26, 2024
Merged

Core: Implement exists api #19

merged 1 commit into from
Aug 26, 2024

Conversation

yanghua
Copy link
Collaborator

@yanghua yanghua commented Aug 18, 2024

Summary 📝

Write an overview about it.

Details

Describe more what you did on changes.

  1. (...)
  2. (...)

Bugfixes 🐛 (delete if dind't have any)

Checks

  • Closed #798
  • Tested Changes
  • Stakeholder Approval

@yanghua yanghua requested a review from xianyinxin August 18, 2024 01:02
@yanghua yanghua self-assigned this Aug 18, 2024
@yanghua yanghua changed the title Override fssepc#exists default implementation to optimize performance [WIP] Override fssepc#exists default implementation to optimize performance Aug 19, 2024
@yanghua yanghua marked this pull request as draft August 19, 2024 11:41
@yanghua yanghua changed the title [WIP] Override fssepc#exists default implementation to optimize performance Core: Implement exists api Aug 21, 2024
@yanghua yanghua marked this pull request as ready for review August 21, 2024 09:11
tosfs/core.py Outdated Show resolved Hide resolved
tosfs/core.py Outdated
@@ -413,6 +416,200 @@ def _try_dir_info(self, bucket: str, key: str, path: str, fullpath: str) -> dict
except Exception as e:
raise TosfsError(f"Tosfs failed with unknown error: {e}") from e

def exists(self, path: str, **kwargs: Union[str, bool, float, None]) -> bool:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this **kwargs: Union[str, bool, float, None] mean ? What's the value that people should pass ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Souds like it's not quite clear for me to construct the argumenst when i call this method....

Copy link
Collaborator Author

@yanghua yanghua Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

**kwargs is the dynamic parameter mechanism in Python. It can be used for future expansion without changing the method signature. Add it's type Union[str, bool, float, None] is for code style check rule. The Union[str, bool, float, None] means: it can be str, bool, number or object type.

Adding this to align with fsspec#exists(**kwargs), it has this parameter due to it calls fsspe#info(**kwargs).   

If removed will cause code style warnning.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually **kwargs is a dict. You could also set **kwargs: Any.

tosfs/core.py Outdated
Comment on lines 426 to 427
**kwargs : dict, optional
Additional arguments.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pydoc says it will be a dict, while the declared arg is a Union ? Is this correct ?

Copy link
Collaborator Author

@yanghua yanghua Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, it should be cleaner in pydoc, will update.

tosfs/core.py Outdated Show resolved Hide resolved
tosfs/core.py Outdated Show resolved Hide resolved
@yanghua yanghua force-pushed the 18-exists branch 2 times, most recently from 295af56 to 1c6db07 Compare August 21, 2024 12:01
@yanghua yanghua force-pushed the 18-exists branch 3 times, most recently from 1840c03 to 5a482dd Compare August 24, 2024 03:03
@openinx openinx merged commit f1eec7f into main Aug 26, 2024
8 checks passed
@yanghua yanghua deleted the 18-exists branch October 15, 2024 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants