Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Write directly to HDFS #24

Open
mcdan opened this issue Aug 29, 2017 · 2 comments
Open

Write directly to HDFS #24

mcdan opened this issue Aug 29, 2017 · 2 comments

Comments

@mcdan
Copy link

mcdan commented Aug 29, 2017

Is there any interest in teaching this library to also write directly to HDFS connected storage? I have a patch that handles it already but before I go to clean up the code and make it presentable I wanted to see if you would be at all interested in a feature like that?
@rschlussel-zz
Copy link
Contributor

To clarify, is your patch to expand the generator to be able to write to an hdfs file instead of a regular file? That seems reasonable. We'd be happy to take a PR.

@mcdan
Copy link
Author

mcdan commented Aug 29, 2017 via email

mcdan added a commit to mcdan/tpcds that referenced this issue Aug 30, 2017
@mcdan
Fixes Teradata#24 - Teach Table Generator how to write to a HDFS file.
bccf2d8 
 * Make session & scaling serializable so it can be passed around
 * Teach session factory pattern to take in a hadoop config object
 * Make a serializable hadoop config object to ensure Session is
   serializable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mcdan @rschlussel-zz