DBLab Engine v3.4.0
New name: DBLab Engine
The new name for the Database Lab Engine is "DBLab Engine". Updates are currently underway across our materials to reflect this change. To align with this change, we have introduced specific domains for the product: dblab.dev
and dblab.sh
. For ease of access, we have established the following short URLs:
- https://cli.dblab.dev/ and https://dblab.sh – CLI setup script (works on macOS/Linux/Windows:
curl -sSL dblab.sh | bash
) - https://demo.dblab.dev/ – DBLab 3.4 demo (token:
demo-token
) - https://branching.dblab.dev/ – DBLab 4.0-alpha demo, implementing full-fledged DB branching and snapshots on demand (token:
demo-token
) - https://aws.dblab.dev/ – DBLab SE page in AWS Marketplace
- https://docs.dblab.dev/ – the docs
- https://api.dblab.dev/ – interactive API reference (powered by readme.io)
New contributors
🙇 Many thanks to our first-time contributors: @mbobin, @TheLiamGuy, @gromsterus, @ibash, @byx01, @davidsvenson.
By the numbers
- DBLab GitHub repository has crossed the 1700+ stars mark
- X/Twitter account @Database_Lab has more than 1300 followers
- Our LinkedIn page has amassed 1500+ followers
Thank you for the support!
New DBLab SE installer
We've expanded the installation options for DBLab SE. Besides its presence in the AWS Marketplace, you can now seamlessly install DBLab SE directly from the Postgres.ai Console.
This setup is entirely automated and can be used anywhere:
- For those who have existing machines, we support the "BYOM" (Bring Your Own Machine) method
- If you're utilizing AWS, GCP, DigitalOcean, or Hetzner Cloud, the installer handles resource provisioning—including VMs, disks, and more—automatically.
Check out the updated tutorial.
Migration notes
For those considering an upgrade, we recommend executing a fresh installation of DBLab. At present, we don't have automation in place for upgrades. If you need to update an existing DBLab installation, it's essential to stop and remove the current containers. Then, initiate a fresh start of the DBLab container using the 3.4.0
tag. More details can be found in the docs. Before initiation, ensure you've adjusted the DBLab configuration as outlined below.
Postgres images
There is a new method for running clones with customized port and socket directories. Please note that this is NOT backward compatible with previous versions of the Postgres images that DBLab utilized. It is crucial to ensure the dockerImage
options utilize tags with the -0.3.0
suffix or later, such as postgresai/extended-postgres:15-0.3.0
(details can be found in this MR).
Global telemetry
The global.telemetry
subsection has been deprecated and removed. The control to activate/deactivate telemetry has been relocated to the platform
section:
platform:
url: "https://postgres.ai/api/general"
enableTelemetry: true
Platform
From version v3.4.0
onward, the platform
section is enabled by default.
If either the accessToken
or orgKey
keys aren't empty, ensure the url
parameter is specified; otherwise, the DBLab instance will not initialize. By default, neither accessToken
nor orgKey
are defined when in case of self-managed installations.
platform:
url: "https://postgres.ai/api/general"
enableTelemetry: true
# orgKey: "org_key"
# accessToken: "platform_access_token"
Docker mounts
Please omit the following Docker mount directives during container startup, as they are no longer necessary:
--volume /sys/kernel/debug:/sys/kernel/debug:rw \
--volume /lib/modules:/lib/modules:ro \
--volume /proc:/host_proc:ro \
Changelog
New features
- Configuration: new option
ignoreErrors
in subsectionslogicalDump
andlogicalRestore
to allow not to interrupt the process of dump/restore in case of errors - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/699, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/700, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/730 - Configuration: new option
skipPolicies
in subsectionlogicalRestore
to allow to skip policies (CREATE POLICY
) during restore process, to avoid errors likepolicy "xxx" for table "yyy" already exists
: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/769 - Configuration: new option
cloneAccessAddresses
in sectionprovision
to control port publishing for clone containers; now limited to127.0.0.1
by default; support multiple addresses and IPv6: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/786 - UI: Improved Logs view with log filters - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/672, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/702
- CLI: ARM / Apple silicon (M1, M2) support for DBLab CLI binaries - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/719, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/733
- API: DBLab v3.4.0 API specification updated and is now published on api.dblab.dev: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/747
- Numerous updates to Postgres images
- supported Postgres major versions: 9.6-15
- image tags:
9.6-0.3.0
...15-0.3.0
;⚠️ older tags won't work with DBLab 3.4.x pgvector
extension added to- existing extension upgraded (docs)
- special images for DBLab SE/EE (with some extension being mocked): RDS Postgres, Aurora, GCP Cloud SQL, Supabase, Timescale, Heroku, PostGIS
- New installer for DBLab SE, available at Console.Postgres.ai, and simplified configuration for various managed PostgreSQL services such as AWS RDS, GCP Cloud SQL, Supabase, Heroku - various MRs
Improvements and fixes
- Enable debug mode by default - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/687
- UI: Instant view of configuration, not waiting for the end of retrieval (physical mode) - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/674
- UI: Collapse/expand the right-side panel - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/677, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/681
- UI: Prevent crash when project ID doesn't exist - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/684
- Add system stats (number of CPUs and size of RAM) to the EngineStarted telemetry event - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/688
- Add hint about Postgres logs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/685
- UI: "Sign out" button - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/678
- Correctly detect clone activity / idleness for timezones different from UTC - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/692, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/734
- If full refresh fails, report it in
/status
with proper messages - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/696 - Mask sensitive data in logs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/705
- UI: Extend and improve the Configuration tab - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/698
- Check in UI if retrieval mode is unknown - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/669
- Clean up data during full data refresh in logical mode - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/704
- Choose the last snapshot when reset to the latest snapshot is requested - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/708
- Remove timing estimator (experimental) - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/716
- Avoid panic with concurrent map read and map write: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/726
- Exclude timescaledb internal scheme from the restricted script: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/738
- UI: show Configuration tab if mode is unknown: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/735
- Change error message from JSON format to string: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/739
- UI: Various fixes: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/752, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/759
- Remove too detailed error message from the logs: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/777
- Exclude dump location volumes nested in the mount directory: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/778
- Ignore STDERR buffer of the process while reading exec command response (previously, DBLab considered a task failed if exit code was 0 but STDERR had messages; now only the exit code defines success/failure) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/782
- Run clones with custom port and socket directory to be able to restart or upgrade Postgres instance without losing container (useful in various cases such as testing Postgres major upgrades on clones): https://gitlab.com/postgres-ai/database-lab/-/merge_requests/780
- Issue a checkpoint after successful execution of query preprocessing scripts to mitigate possible heavy writes and avoid cloning timeouts: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/784
- Configuration: make upper boundary in
portPool
inclusive (previously, it was mistakenly exclusive, leading to unexpected behavior): https://gitlab.com/postgres-ai/database-lab/-/merge_requests/785 - Avoid panics when collecting info about sync container and no data pool is available: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/789
- Trim slashes before comparing mount points to avoid errors like
Error response from daemon: Duplicate mount point: /var/lib/dblab/dblab_pool/dataset_1/dump/
: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/791 - UI: Fix active tab detection: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/794
Internal changes
- CI/CD: Rework UI pipeline rules - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/676
- CI/CD: Manage keyring files in
trusted.gpg.d
becauseapt-key
is deprecated - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/701 - Various libraries updated - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/721, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/722
- Remove hard-coded ZFS version in Dockerfile to use default one: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/724
- CLI: in the installer script, support versions with and without prefix "v", print installed version: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/745
- CLI: Add CLI setup snippet to the CLI setup script itself: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/749
- Update linter and fix new warnings: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/768
- Improve SSH port forwarding snippet – use the actual port: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/764
- Update building tools versions and dependency packages: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/788
- DBLab ansible installer: Use Git tag for ansible repository: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/761
- Adjust ansible snippet style: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/793
Full diff
Diff between versions v3.3.0 and v3.4.0: v3.3.0...v3.4.0
Where to start / get help
- Tutorial
- DBLab 3.4 demo instance (token:
demo-token
) - Docs
- Postgres.ai Console
- Contact us
Provide feedback, contribute
We greatly value your feedback. Connect with us via:
- X/Twitter @Database_Lab
- DBLab Community Slack
Additional resources where you can get insights about DBLab and Postgres:
Interested in giving back to the project? Here's how you can make an impact:
- Give a star to our GitHub repository
- Help us reach more enthusiasts. Share about Database Lab on Twitter (don't forget to tag @Database_Lab) or any other platform you fancy
- Multilingual? Consider translating our README.md to share the knowledge in your language
- Are you a developer? Dive in and enhance the Database Lab Engine (DLE) experience; check out our CONTRIBUTING guidelines and explore the "good first issues" list on GitLab