Releases: amundsen-io/amundsendatabuilder
Add a user node model
Merge pull request #12 from lyft/tfeng_update_user_model Add User node with different user attributes
Avoid using ConfigTree.with_fallback method in job level
v1.0.4 Avoid using ConfigTree.with_fallback method in job level
Adding timeout on ColumnProvider
Adding timeout on ColumnProvider.
New task to remove stale nodes and relations
New task to remove stale nodes and relations
Sample job config:
task = Neo4jStalenessRemovalTask()
job_config = ConfigFactory.from_dict({
'job.identifier': 'remove_stale_data_job',
'{}.{}'.format(task.get_scope(), neo4j_staleness_removal_task.NEO4J_END_POINT_KEY):
neo4j_endpoint,
'{}.{}'.format(task.get_scope(), neo4j_staleness_removal_task.NEO4J_USER):
neo4j_user,
'{}.{}'.format(task.get_scope(), neo4j_staleness_removal_task.NEO4J_PASSWORD):
neo4j_password,
'{}.{}'.format(task.get_scope(), neo4j_staleness_removal_task.STALENESS_MAX_PCT):
95,
'{}.{}'.format(task.get_scope(), neo4j_staleness_removal_task.TARGET_NODES):
['Table','Application', 'Column', 'Source', 'Stat', 'Timestamp', 'Watermark'],
'{}.{}'.format(task.get_scope(), neo4j_staleness_removal_task.TARGET_RELATIONS):
['READ', 'READ_BY'],
'{}.{}'.format(task.get_scope(), neo4j_csv_publisher.JOB_PUBLISH_TAG):
kwargs['templates_dict'].get(neo4j_csv_publisher.JOB_PUBLISH_TAG)
})
job = DefaultJob(conf=job_config,
task=task)
job.launch()
Add shutdown hook for Closer
Add shutdown hook for Closer
Initial release
Initial release