Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding custom blocks to pipeline #71

Closed
kkarrancsu opened this issue Jan 25, 2018 · 3 comments
Closed

Adding custom blocks to pipeline #71

kkarrancsu opened this issue Jan 25, 2018 · 3 comments

Comments

@kkarrancsu
Copy link
Contributor

It would be nice to be able to add additional blocks to the ML pipeline, both static and dynamic (see #70 ).

@bcyphers
Copy link
Contributor

bcyphers commented Jan 31, 2018

The way I see it, there are two ways this could be done:

  1. By adding dynamic steps before and after the actual model-training step which are subject to hyperparameters. This is already done to a certain extent with special hyperparameter keys like _PCA and _WHITEN. See the Model.make_pipeline() function in model.py. We could formalize and generalize this process and make it easier for users to apply other transformations to the data. Transformations applied in this manner would be computed once per classifier, and transformation hyperparameters could be optimized right alongside method hyperparameters.

  2. By adding static transformations at the Dataset level. For example, we could add options for dimensionality reduction or deep feature synthesis to the create_dataset function in enter_data.py. These transformations would be applied once at the time of dataset creation, the transformed data would be saved, and any dataruns which referenced the dataset would use the transformed data from the word go.

These two features could be built out separately, and I think in combination they'd give us most of what you are talking about.

@micahjsmith
Copy link
Member

Relates to #113

@micahjsmith
Copy link
Member

Closing in favor of #113 issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants