Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check directory existence and properly deal with patterned directories #153

Closed
wants to merge 13 commits into from
Closed

Check directory existence and properly deal with patterned directories #153

wants to merge 13 commits into from

Conversation

cpcloud
Copy link
Member

@cpcloud cpcloud commented Mar 26, 2015

closes #151

  • asserts that the directory of files must exist when calling resource on it
  • patterns like test/*.csv don't iterate over everything in the directory
  • provides way to convert a Directory of CSVs into chunks(DataFrame)
  • testing on hdfs

@cpcloud
Copy link
Member Author

cpcloud commented Mar 26, 2015

Alternatively, we could make everything that contains a glob pattern mean chunks

@cpcloud cpcloud self-assigned this Mar 26, 2015
@cpcloud cpcloud added this to the 0.3.2 milestone Mar 26, 2015
@cpcloud
Copy link
Member Author

cpcloud commented Apr 26, 2015

pushing to 0.3.3

@cpcloud cpcloud modified the milestones: 0.3.3, 0.3.2 Apr 26, 2015
@cpcloud cpcloud modified the milestones: 0.3.3, 0.3.4 Jul 7, 2015
@cpcloud cpcloud modified the milestones: 0.3.4, 0.4.0 Sep 15, 2015
@cpcloud cpcloud removed this from the 0.4.0 milestone Oct 5, 2015
@cpcloud cpcloud modified the milestones: 0.3.5, 0.4.0 Oct 5, 2015
@cpcloud cpcloud modified the milestones: 0.4.0, 0.4.1 Dec 4, 2015
@kwmsmith kwmsmith modified the milestones: 0.4.1, 0.4.2, 0.5.0 Feb 2, 2016
@llllllllll
Copy link
Member

If someone wants to own this please reopen.

@llllllllll llllllllll closed this Nov 16, 2016
@ethanluoyc
Copy link

This is still occurring on version 0.5.0 (the latest development version?)
It can be reproduced with the same procedure

@ethanluoyc ethanluoyc mentioned this pull request Dec 16, 2016
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@cpcloud cpcloud

Labels
bug enhancement
Projects
None yet
Milestone
0.5.0
Development

Successfully merging this pull request may close these issues.

Directory of CSVs to DataFrame is broken
4 participants
@cpcloud @llllllllll @ethanluoyc @kwmsmith