- [Datalog] Additional arity to
update-schemato allow renaming attributes. #131 - [Search]
clear-docsfunction to wipe out search index, as it might be faster to rebuild search index than updating individual documents sometimes. #132 datalevin.constants/*data-serializable-classes*dynamic var, which can be used forbindingif additional Java classes are to be serialized as part of the default:datadata type. #134
- [Datalog] Allow passing option map as
:kv-optsto underlying KV store whencreate-conn - bump deps
- [Datalog]
clearfunction on server. #133
- [Datalog] Changed
:search-engineoption map key to:search-optsfor consistency [Breaking]
- [Search] Handle empty documents
- [Datalog] Handle safe schema migration, #1, #128
- bump deps
- [KV] visitor function for
visitcan return a special value:datalevin/terminate-visitto stop the visit.
- Fixed adding created-at schema item for upgrading Datalog DB from prior 0.6.4 (thx @jdf-id-au)
- [breaking] Simplified
open-dbisignature to take an option map instead
:validate-data?option foropen-dbi,create-connetc., #121
- Schema update regression. #124
:domainoption tonew-search-engine, so multiple search engines can coexist in the samedir, each with its own domain, a string. #112
- Server failure to update max-eid regression, #123
- Added an arity to
update-schemato allow removal of attributes if they are not associated with any datoms, #99
- Search add-doc error when alter existing docs
- Persistent server session that survives restarts without affecting clients, #119
- More robust server error handling
- Query cache memory leak, #118 (thx @panterarocks49)
- Entity retraction not removing
:db/updated-atdatom, #113
datalevin.search-utilsnamespace with some utility functions to customize search, #105 (thx @ngrunwald)
- Add
visitKV function tocorename space - Handle concurrent
add-docfor the same doc ref
- Bump deps
- Handle Datalog float data type, #88
- Allow to use all classes in Babashka pods
- Dump and load Datalog option map
- Dump and load
inter-fn - Dump and load regex
- Pass search engine option map to Datalog store
- Make configurable analyzer available to client/server
- Dot form Java interop regression in query, #103
- Option to pass an analyzer to search engine, #102
:auto-entity-time?Datalog DB creation option, so entities can optionally have:db/created-atand:db/updated-atvalues added and maintained automatically by the system during transaction, #86
- Dependency bump
doc-countfunction returns the number of documents in the search indexdoc-refsfunction returns a seq ofdoc-refin the search index
datalevin.core/copyfunction can copy Datalog database directly.
doc-indexed?function
add-doccan update existing docopen-kvfunction allows LMDB flags, #100
- Built-in full-text search engine, #27
- Key-value database
visitfunction to do arbitrary things upon seeing a value in a range
- [breaking]
:instanthandles dates before 1970 correctly, #94. The storage format of:instanttype has been changed. For existing Datalog DB containing:db.type/instant, dumping as a Datalog DB using the old version of dtlv, then loading the data is required; For existing key-value DB containing:instanttype, specify:instant-pre-06instead to read the data back in, then write them out as:instantto upgrade to the current format.
- Improve read performance by adding a cursor pool and switch to a more lightweight transaction pool
- Dependency bump
- Create pod client side
defpodfnso it works in non-JVM.
load-ednfor dtlv, useful for e.g. loading schema from a file, #101
- Serialized writes for concurrent transactions, #83
defpodfnmacro to define a query function that can be used in babashka pod, #85
- Update
max-aidafter schema update (thx @den1k)
- Updated dependencies, particularly fixed sci version (thx @borkdude)
- occasional LMDB crash during multiple threads write or abrupt exit
- Update graalvm version
- Exception handling in copy
- Handle scalar result in remote queries
- Server asks client to reconnect if the server is restarted and client reconnects automatically when doing next request
- Bump versions of all dependency
- More robust handling of abrupt network disconnections
- Automatically maintain the required number of open connections, #68
- Options to specify the number of connections in the client connection pool and to set the time-out for server requests
- Backport the dump/load fix from 0.5.20
- Dumping/loading Datalog store handles raw bytes correctly
- Remove client immediately when
disconnectmessage is received, clean up resources afterwards, so a logically correct number of clients can be obtained in the next API call on slow machines.
- Occasional server message write corruptions in busy network traffic on Linux.
- JVM uberjar release for download
- JVM library is now Java 8 compatible, #69
- Auto switch to local transaction preparation if something is wrong with remote preparation (e.g. problem with serialization)
- Do most of transaction data preparation remotely to reduce traffic
- Handle entity serialization, fix #66
- Allow a single client to have multiple open databases at the same time
- Client does not open db implicitly, user needs to open db explicitly
- New
create-connshould override the old, fix #65
DTLV_LIB_EXTRACT_DIRenvironment variable to allow customization of native libraries extraction location.
- Use clj-easy/graal-build-time, in anticipation of GraalVM 22.
- Better robust jar layout for
org.clojars.huahaiy/datalevin-native
- Release artifact
org.clojars.huahaiy/datalevin-nativeon clojars, for depending on Datalevin while compiling GraalVM native image. User no longer needs to manually compile Datalevin C libraries.
- Only check to refersh db cache at user facing namespaces, so internal db calls work with a consistent db view
- Replace unnecessary expensive calls such as
db/-searchordb/-datomswith cheaper calls to improve remote store access speed. - documentation
- More robust build
- Wrap all LMDB flags as keywords
- Don't do AOT in library, to avoid deps error due to exclusion of graal
- Expose all LMDB flags in JVM version of kv store
- Transparent networked client/server mode with role based access control. #46 and #61
dtlv exectakes input from stdin when no argument is given.
- When open db, throw exception when lacking proper file permission
- Transactable entity [Thanks @den1k, #48]
clearfunction to clear Datalog db
- Native uses the same version of LMDB as JVM, #58
- Remove GraalVM and dtlv specific deps from JVM library jar
- Update deps
- More robust dependency management
- Replacing giant values, this requires Java 11 [#56]
- Transaction of multiple instances of bytes [#52, Thanks @den1k]
- More reflection config in dtlv
- Benchmark deps
- Correct handling of rule clauses in dtlv
- Documentation clarification that we do not support "db as a value"
- Datafy/nav for entity [Thanks @den1k]
- Some datom convenience functions, e.g.
datom-eav,datom-e, etc.
- Talk to Babashka pods client in transit+json
- Exposed more functions to Babashka pod
- Native Datalevin can now work as a Babashka pod
- Compile to native on Windows and handle Windows path correctly
close-dbconvenience function to close a Datalog db
- Compile to Java 8 bytecode instead of 11 to have wider compatibility
- Use UTF-8 throughout for character encoding
- Improve dtlv REPL (doc f) display
- Provide Datalevin C source as a zip to help compiling native Datalevin dependency
- Minor improvement on the command line tool
- Native image now bundles LMDB
- Handle list form in query properly in command line shell [#42]
- Consolidated all user facing functions to
datalevin.core, so users don't have to understand and require different namespaces in order to use all features.
- [Breaking] Removed AEV index, as it is not used in query. This reduces storage and improves write speed.
- [Breaking] Change VAE index to VEA, in preparation for new query engine. Now all indices have the same order, just rotated, so merge join is more likely.
- [Breaking] Change
open-lmdbandclose-lmdbtoopen-kvandclose-kv,lmdb/transacttolmdb/transact-kv, so they are consistent, easier to remember, and distinct from functions indatalevin.core.
- GraalVM native image specific LMDB wrapper. This wrapper allocates buffer memory in C and uses our own C comparator instead of doing these work in Java, so it is faster.
- Native command line shell,
dtlv
- Improve Java interop call performance
- Allow Java interop calls in where clauses, e.g.
[(.getTime ?date) ?timestamp],[(.after ?date1 ?date2)], where the date variables are:db.type/instance. [#32]
- Changed default LMDB write behavior to use writable memory map and asynchronous msync, significantly improved write speed for small transactions (240X improvement for writing one datom at a time).
- Read
:db.type/instantvalue asjava.util.Date, not aslong[#30]
- Fixed error when transacting different data types for an untyped attribute [#28, thx @den1k]
- proper exception handling in
lmdb/open-lmdb
- Fixed schema update when reloading data from disk
- Fixed
core/get-connschema update
- Remove unnecessary locks in read transaction
- Improved error message and documentation for managing LMDB connection
core/get-connandcore/with-conn
- Correctly handle
init-max-eidfor large values as well.
- Fixed regression introduced by 0.3.6, where :ignore value was not considered [#25]
- Add headers to key-value store keys, so that range queries work in mixed data tables
- Expose all data types to key-value store API [#24]
- thaw error for large values of
:datatype. [#23]
- portable temporary directory. [#20, thx @joinr]
- Properly initialize max-eid in
core/empty-db
- Add value type for
:db/identin implicit schema
- [Breaking] Change argument order of
core/create-conn,db/empty-dbetc., and putdirin front, since it is more likely to be specified thanschemain real use, so users don't have to putnilforschema.
- correct
core/update-schema
- correctly handle
falsevalue as:data - always clear buffer before putting data in
- thaw exception when fetching large values
- clearer error messages for byte buffer overflow
- correct schema update
core/schemaandcore/update-schema
core/closed?
db/entidallows 0 as eid
- fix test
- correct results when there are more than 8 clauses
- correct query result size
- automatically re-order simple where clauses according to the sizes of result sets
- change system dbi names to avoid potential collisions
- miss function keywords in cache keys
- hash-join optimization submitted PR #362 to Datascript
- caching DB query results, significant query speed improvement
- fix invalid reuse of reader locktable slot #7
- remove MDB_NOTLS flag to gain significant small writes speed
- update existing schema instead of creating new ones
- Reset transaction after getting entries
- Only use 24 reader slots
- avoid locking primitive #5
- create all parent directories if necessary
- long out of range error during native compile
- apply query/join-tuples optimization
- use array get wherenever we can in query, saw significant improvement in some queries.
- use
db/-firstinstead of(first (db/-datom ..)),db/-populated?instead of(not-empty (db/-datoms ..), as they do not realize the results hence faster. - storage test improvements
- use only half of the reader slots, so other processes may read
- add an arity for
bits/read-bufferandbits/put-buffer - add
lmdb/closed?,lmdb/clear-dbi, andlmdb/drop-dbi
- code samples
- API doc
core/close
- Port datascript 0.18.13