Code

[feat]Rocksdb backend#121

Code

Open

bashimao wants to merge 67 commits intotensorflow:mastertensorflow/recommenders-addons:masterfrom bashimao:rocksdb-backendbashimao/recommenders-addons:rocksdb-backendCopy head branch name to clipboard

+4,157-5

Commits

Commits on Jul 17, 2021

• RocksDB hashtable backend for Tensorflow Recommenders Addons.

  bashimao

  committed
  d6a137c

  Copy full SHA for d6a137c

Commits on Jul 18, 2021

• Fix major bugs and establish proper compiler toolchain.

  bashimao

  committed
  0b48160

  Copy full SHA for 0b48160

• Only create if not in read_only mode.

  bashimao

  committed
  6eb84c4

  Copy full SHA for 6eb84c4

• Decouple cleaning from creating a column family.

  bashimao

  committed
  e8cbf37

  Copy full SHA for e8cbf37

Commits on Jul 20, 2021

• Correct pending linker issues.

  bashimao

  committed
  9cdaec7

  Copy full SHA for 9cdaec7

• Add op definitions for RocksDB.

  bashimao

  committed
  42aff13

  Copy full SHA for 42aff13

• Need to include op-definitions in compiler directives.

  bashimao

  committed
  222fe26

  Copy full SHA for 222fe26

• Automatically import RocksDB table object upon module load.

  bashimao

  committed
  052c2e2

  Copy full SHA for 052c2e2

• Add interface code to properly transfer arguments to the C++ implementation.

  bashimao

  committed
  96856d7

  Copy full SHA for 96856d7

• Typo...

  bashimao

  committed
  06738d2

  Copy full SHA for 06738d2

• Fix opening non-existent database bug.

  bashimao

  committed
  23a0b0e

  Copy full SHA for 23a0b0e

• Merge branch 'tensorflow:master' into rocksdb-backend

  bashimao

  authored
  b55ac03

  Copy full SHA for b55ac03

• Fix typo.

  bashimao

  committed
  c2ae3c3

  Copy full SHA for c2ae3c3

Commits on Jul 22, 2021

• Add connection pool functionality to allow access multiple tables in the same database simultaneously.

  bashimao

  committed
  159c4c3

  Copy full SHA for 159c4c3

• Fix insert bug. Add configurable size function, two modes: 1) through RocksDB estimation; 2) through iteration method.

  bashimao

  committed
  51e8389

  Copy full SHA for 51e8389

• Merge branch 'rocksdb-backend' of github.com:bashimao/recommenders-addons into rocksdb-backend

  bashimao

  committed
  e8558e1

  Copy full SHA for e8558e1

Commits on Jul 23, 2021

• Fix up all known issues, add registry functionalities.

  bashimao

  committed
  4c0d09f

  Copy full SHA for 4c0d09f

• Make DBWrapper thread-safe.

  bashimao

  committed
  6287344

  Copy full SHA for 6287344

• Switch to reader/writer lock for performance.

  bashimao

  committed
  101275c

  Copy full SHA for 101275c

• Allow settign export_path from Python.

  bashimao

  committed
  ccc7b58

  Copy full SHA for ccc7b58

• Extract the default value's shape.

  bashimao

  committed
  d9e8680

  Copy full SHA for d9e8680

Commits on Jul 24, 2021

• Improved support for multi-dimensional tensors.

  bashimao

  committed
  ed8f58a

  Copy full SHA for ed8f58a

• Improve multi-threading stability. Added limit for export to pass tests. (THIS MIGHT BE WRONG!)

  bashimao

  committed
  15c7953

  Copy full SHA for 15c7953

• Fix typo.

  bashimao

  committed
  2fb2dec

  Copy full SHA for 2fb2dec

• Now passing all but 2 tests.

  bashimao

  committed
  2dff086

  Copy full SHA for 2dff086

• Loop condition was wrong.

  bashimao

  committed
  c7828ee

  Copy full SHA for c7828ee

• Pointer value instead of pointed to value written to file.

  bashimao

  committed
  43fe98e

  Copy full SHA for 43fe98e

• Protect against divide by zero.

  bashimao

  committed
  1de6a0a

  Copy full SHA for 1de6a0a

• Add Import/Export lock to avoid overlapping transactions - despite reader-writer lock.

  bashimao

  committed
  626ce3a

  Copy full SHA for 626ce3a

• Remove export lock idea. Instead make the mutex absolute. We have a potential race-condition otherwise.

  bashimao

  committed
  972c4f9

  Copy full SHA for 972c4f9

• Passing all relevant checks.

  bashimao

  committed
  e6d52d9

  Copy full SHA for e6d52d9

Commits on Aug 10, 2021

• Reformatted cc-/h-files to conform to Google-style.

  bashimao

  committed
  c770dc0

  Copy full SHA for c770dc0

• Tick up copyright disclaimer version number.

  bashimao

  committed
  62600bd

  Copy full SHA for 62600bd

Commits on Aug 20, 2021

• Reformat with yapf.

  bashimao

  committed
  c8ce053

  Copy full SHA for c8ce053

Commits on Aug 22, 2021

• Tick down rules_foreign_cc package version according to Heka's recommendation.

  bashimao

  committed
  ff725b2

  Copy full SHA for ff725b2

Commits on Aug 30, 2021

• Merge branch 'master' into rocksdb-backend

  Show description for eff7a39
  bashimao

  committed
  eff7a39

  Copy full SHA for eff7a39

• Reformat python code.

  bashimao

  committed
  de7fb53

  Copy full SHA for de7fb53

Commits on Oct 17, 2021

• Merge branch 'master' into rocksdb-backend

  bashimao

  committed
  82816b0

  Copy full SHA for 82816b0

Commits on Oct 19, 2021

• Code cosmetics

  bashimao

  committed
  804efb9

  Copy full SHA for 804efb9

Commits on Oct 24, 2021

• Fix Python format according to proposed coding standards.

  bashimao

  committed
  e116424

  Copy full SHA for e116424

• Manually ran clang format. Not sure why it did not work before.

  bashimao

  committed
  36bd30a

  Copy full SHA for 36bd30a

Commits on Jan 5, 2022

• Merge branch 'master' into rocksdb-backend

  rhdong

  authored
  bbeca4a

  Copy full SHA for bbeca4a

Commits on Jan 23, 2022

• Merge branch 'master' into rocksdb-backend

  bashimao

  committed
  05d6d2a

  Copy full SHA for 05d6d2a

• Fix up Bazel rules for 0.6 foreign_cc.

  bashimao

  committed
  45a748b

  Copy full SHA for 45a748b

Commits on Jan 26, 2022

• Somehow these libs are no longer needed?!

  bashimao

  committed
  567e92d

  Copy full SHA for 567e92d

• Resolve most wiggles that prevent compiling ant testing.

  bashimao

  committed
  dbcd36b

  Copy full SHA for dbcd36b

• Temporary fix for library search path.

  bashimao

  committed
  ddbb499

  Copy full SHA for ddbb499

Commits on Jan 27, 2022

• Apply yapf.

  bashimao

  committed
  bff04bb

  Copy full SHA for bff04bb

• Apply Clang Format.

  bashimao

  committed
  08c83e4

  Copy full SHA for 08c83e4

• Change to a more commonly used path that has symlinks.

  bashimao

  committed
  fc7ac16

  Copy full SHA for fc7ac16

• Add dependencies to ci images.

  bashimao

  committed
  f6bd10d

  Copy full SHA for f6bd10d

• Fix syntax error. ;-)

  bashimao

  committed
  46be6bd

  Copy full SHA for 46be6bd

Commits on Mar 27, 2022

• Merge branch 'master' into rocksdb-backend

  bashimao

  committed
  ecdc998

  Copy full SHA for ecdc998

• Now included in base image. tensorflow#121 (comment)

  bashimao

  committed
  0011199

  Copy full SHA for 0011199

• Simplify a couple of things.

  bashimao

  committed
  fbd18bc

  Copy full SHA for fbd18bc

• Can process true/false. So why is it 0/1?

  bashimao

  committed
  b32139d

  Copy full SHA for b32139d

• Update C++ portion of RocksDB implementation to accommodate interface adjustments.

  bashimao

  committed
  15714b3

  Copy full SHA for 15714b3

Commits on May 26, 2022

• Merge branch 'tensorflow:master' into rocksdb-backend

  bashimao

  authored
  9811e82

  Copy full SHA for 9811e82

Commits on Jun 12, 2022

• Merge branch 'master' into rocksdb-backend

  bashimao

  committed
  db166a3

  Copy full SHA for db166a3

• Allow using pre-built rocksdb version.

  bashimao

  committed
  366638d

  Copy full SHA for 366638d

Commits on Aug 15, 2022

• Merge branch 'master' into rocksdb-backend

  bashimao

  committed
  a1bb85f

  Copy full SHA for a1bb85f

• Fix up bazel environment to use pre-compiled rocksdb.

  bashimao

  committed
  778f244

  Copy full SHA for 778f244

• Implement MemoryUsed API.

  bashimao

  committed
  f86ec70

  Copy full SHA for f86ec70

Commits on Aug 16, 2022

• Bugfixes and implement FindWithExists API.

  bashimao

  committed
  eaa6219

  Copy full SHA for eaa6219

• Minor bugfix.

  bashimao

  committed
  4ebd8ac

  Copy full SHA for 4ebd8ac

• Add minor error check.

  bashimao

  committed
  4a86ac2

  Copy full SHA for 4a86ac2

• Accumulation API. Dummy implementation.

  bashimao

  committed
  53b732d

  Copy full SHA for 53b732d