Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset/dataloader API #174

Open
vvmnnnkv opened this issue Aug 10, 2020 · 0 comments
Open

Dataset/dataloader API #174

vvmnnnkv opened this issue Aug 10, 2020 · 0 comments
Assignees
Labels
Type: Improvement 📈 Performance improvement not introducing a new feature or requiring a major refactor Type: New Feature ➕ Introduction of a completely new addition to the codebase
Milestone

Comments

@vvmnnnkv
Copy link
Member

Feature Description

Add helper class that to make easier data loading, batching, shuffling, etc.
I.e. dataset and/or dataloader class API that would help user to prepare data for training.

Use cases:

  • Easify and standardize data preparation on the client side, across all libraries.
  • Supply dataset into Job.train(..., {data: dataset, ...}) and update Job.train to work with the dataset API.

Design: TBD

Is your feature request related to a problem?

Standardize the way data is loaded into training loop across workers.

What alternatives have you considered?

Discussed with FL team.

Additional Context

See also #172

@vvmnnnkv vvmnnnkv added Type: New Feature ➕ Introduction of a completely new addition to the codebase Type: Improvement 📈 Performance improvement not introducing a new feature or requiring a major refactor labels Aug 10, 2020
@vvmnnnkv vvmnnnkv added this to the 0.4.0 milestone Aug 10, 2020
@cereallarceny cereallarceny modified the milestones: 0.4.0, 0.3.0 Aug 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Improvement 📈 Performance improvement not introducing a new feature or requiring a major refactor Type: New Feature ➕ Introduction of a completely new addition to the codebase
Projects
None yet
Development

No branches or pull requests

3 participants