This repository provides a collection of example projects demonstrating various use cases and best practices for Databricks Asset Bundles (DABs). Each folder in this repo contains a self-contained project showcasing different aspects of DAB functionality and best practices for developer workflow and CI/CD (Continuous Integration / Continuous Deployment).
The repository is organized into multiple folders, each representing a distinct project:
flights-simple: Simple end to end project with workflows & DLT, source code (simple library), testsflights-bundle-template: Bundle template to deploy a version offlights-simplewith different options:- classic or serverless workflows
- library packaged as wheel or using relative imports
dais-2024-dab-mod-orch-template: Simple template for DABsts-dabs-demoDABs simple project that includes many CICD pipeline definitions
The structure of each of the folders varies according to the purpose of the project, but will include:
- A
README.mdwith specific instructions - Configuration files (such as a
databricks.ymlfile defining the bundle, other YAMLs for resources) - Source and test code
- In some instances, GH actions YAMLs
To use a project in a folder:
- Navigate to the desired folder
- Review the README for instructions
- Examine the configuration and other to understand the bundle structure
- Follow the instructions to deploy and test the bundle
The installation of the Databricks CLI is a pre-requisite for running any of the projects.
-
Install the Databricks CLI from https://docs.databricks.com/dev-tools/cli/databricks-cli.html
-
Authenticate to your Databricks workspace:
$ databricks configure -
Optionally, install developer tools such as the Databricks extension for Visual Studio Code from https://docs.databricks.com/dev-tools/vscode-ext.html. For PyCharm professional, you may use https://www.jetbrains.com/help/pycharm/databricks.html#connect-via-databricks-cli. Or read the "getting started" documentation for Databricks Connect for instructions on running the included Python code from a different IDE.
- Databricks Asset Bundles Documentation
- Databricks CLI Documentation
- Databricks Connect Documentation
Please note the code in this project is provided for your exploration only, and are not formally supported by Databricks with Service Level Agreements (SLAs). They are provided AS-IS and we do not make any guarantees of any kind. Please do not submit a support ticket relating to any issues arising from the use of these projects. The source in this project is provided subject to the Databricks License. All included or referenced third party libraries are subject to the licenses set forth below.
Any issues discovered through the use of this project should be filed as GitHub Issues on the Repo. They will be reviewed as time permits, but there are no formal SLAs for support.