19 Nov 20:57

rao-vasireddy

67110c8

v2.0.5 (November 15, 2019)

YSQL changes

[YSQL] Change HINT text for unsupported "alter procedure/function". #2718
[YSQL] Support client-server TLS encryption #2471
[YSQL] Fix Master crash when deleting YSQL database #2665
[YSQL] Fix read point for row lock stmt. #2523
[YSQL] Transparent read restarts for aggregate queries. #2216
[YSQL] Show oid for YSQL tables in master UI. #1525

YCQL changes

[YCQL] Server crashes on WHERE condition for key_column and collection_column. #2638
[YCQL] NULL value for clustering PK column causes driver to keep retrying. #2737
[YCQL] Support CREATE TABLE ... WITH NUM TABLETS. #2448
[YCQL] Support nested user-defined types #2729

System improvements

Pipe cmk policy into EncryptionAtRestService #2714
[CDC] Include CDC consumer op ID while determining op ID to evict from cache. #2317
[docdb] Roll forward master tablet deletion on fault during DeleteTable #2721
Add optional placement_uuid to yb-admin. #2589
Truncate fails after add_node, problem on bootstraping #2804
Update glog to fix issue with stack trace symbolization. #2800
Implement new write throttling algorithm based on number of SST files in leader and followers. #2750
Improve sending CQL result from TServer. #1450
Fix non transactional conflict resolution. #2823
Do not flush rocksdb memtable when user drops table. #2719
Add and update placement ID for read replicas. #2876
[CDC] Use new GetChanges API on CDC Consumer. #2863
Fix DeleteTable concurrency issues. #2799
[2DC] Don't log CDC record data. #2841
Add support for in memory universe keys #2682
Lower consensus_max_batch_size_bytes default to 4MB. #2739
Inbound RPC calls are not destroyed until timeout or service shutdown #2887
Don't reset cdc_state checkpoint to 0.0. #2897
Change failure handling in ProcessRpcStatus. #2879
Do smart consumer producer tablet mapping. #2679
Add Java Client Support for Encryption at Rest #2906
Generate new rejection score on retry. #2750

Platform

[Platform] Add ability to turn off encryption at rest for an encrypted at rest universe #2730
[Platform] Cache universe key registry in memory on YW side #2770
[Platform] Add ability to launch aws instances with encryption enabled storage #2759
[Platform] Scheduled backups for Single table and Full universe flows.
[Platform] Use a YW default CMK policy instead of the AWS default policy #2896
[Platform] Enable support for Rotate Key/Disable Key/Enable Key flows. #2927
[Platform] Integrate YW KMS with in-memory encryption at rest #2915
[Platform] Rotate key with service should not rely on alias/cmk already existing #2943
[Platform] Periodic job to set latest universe key into master from YW #2728

Note on upgrades from pre 2.0 releases

Version 2.0 release includes a backward incompatible file format change for YSQL. Note: Prior to 2.0, YSQL was still in beta. This means that if you have an existing cluster running releases older than 2.0 with YSQL enabled, then you will not be able to upgrade to version 2.0 or higher release.

There are two ways to address the above note.

If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
If you want to continue using your existing data, then please save your data,
delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

Assets 2

28 Oct 22:48

rao-vasireddy

v2.0.3

60dac9a

v2.0.3 (October 28, 2019)

YSQL changes

[YSQL] Build YBTupleID value with respect to DocDB column order. #2438
[YSQL] Support TLS Server-Server Encryption through YSQL API. #1845
[YSQL] Check constraints when doing UPDATE. #2361
[YSQL] Add default password to yugabyte user for ysqlsh. #2594
[YSQL] authentication and authorization. #2610
[YSQL] client authentication update. #2614
[YSQL] Enable FOR SHARE and FOR KEY SHARE row locking in YSQL SELECT statements. DMLs on tables with foreign key references work regardless of isolation level (previously only worked in SERIALIZABLE isolation)#1199
[YSQL] Avoid namespace id resolving for import_namespace yb-admin command. #2582
[YSQL] SERIALIZABLE READ ONLY DEFERRABLE. #2161

YCQL changes

[YCQL] Fix for SIGBUS TS crash on invalid statement. #2476
[YCQL] A role shouldn't be able to modify the superuser status of itself or of a role granted to it. #2505
[YCQL] Fix issue with permissions inheritance. #2550
[YCQL] Mangling the column name in IndexTable for index selecting processing. #741
[YCQL] Wrong port sent for NEW_NODE topology change events. #2607
[YCQL] Fixed: Cannot CREATE TYPE when user has CREATE permissions for keyspace. #2556
[YCQL] Add support for IF clause in SELECT statement. #2640
[YCQL] Add description for JSONB INDEX and IF clause in SELECT. #2651
[YCQL] Not reserving keyword USER and WHEN. #1661
[YCQL] EXPLAIN SELECT causes tserver to crash. #2616

System improvements

Added missing extractor checks into version_edit, so ldb doesn't crash. #2510
Fix race in Batcher. #2501
[2DC] Update checkpoint for replicated records, ignore tablet status while looking up peer. #2487
Reduce log level while skipping over display of stats for tablets with state RaftGroupStatePB::SHUTDOWN. #1488
[2DC] Send from_checkpoint only when consumer knows the last checkpoint. #2522
Remove may exist state from YBTransaction. #2501
[CDC] Setup Java Console Connector for CDC. #2081
Fix operation memory double tracking. #1436
Reinit total replicas to move when master leader fails during tserver blacklisting. #2453
Display num sst files for tablets in yb_admin list_tablets_for_tablet_server. #2536
Use boost::amotic for op_id_copy_ member (128 bits structure). #2545
Derive proxy rpc addresses. #1023
Update consensus meta term during local bootstrap. #2570
Add metrics for thread count in each category. #1785
Cleanup queue on timeout. #2592
Do not call LogReader::Open() twice when skip_wal_rewrite is enabled. #2124
Display num log segments for tablets in yb_admin list_tablets_for_tablet_server. #2555
Fix Bug in YEDIS Keys Command. #2381
Fix num tablets per table. #2630
modify_placement_info clears read replica config. #2628
Fix use of load balancer concurrent action limits. #2624
Node should be able to transition from master to shell mode and back to master. #2007
Use new linuxbrew (20191015T202549). #2608
Remove in_memory_erase option and use SingleDelete instead of Delete. #2569
Fixed invalid read-time for RF==1 case. #2394
Move Rpcs to CDC consumer from CDC poller. #2549
[CDC] Make redirection to tablet leaders for GetChanges asynchronous. #2503

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
If you want to continue using your existing data, then please save your data,
delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

Assets 2

09 Oct 01:23

rao-vasireddy

v2.0.1

6607435

v2.0.1 (October 08, 2019)

YSQL changes

[YSQL] Allow ALTER COLUMN add/set/drop identity. #2226
[YSQL] PostgreSQL error code handling to avoid string-based error comparisons. #1289
[YSQL] Fix duplicate key error being suppressed in COPY. #2289
[YSQL] VERIFY_RESULT creates undesirable copy of Result<T&>'s value. #2298
[YSQL] Warn on non-txn COPY with incompatible relation. #2278
[YSQL] Allow ALTER COLUMN add/set/drop identity. #2226
[YSQL] PostgreSQL error code handling to avoid string-based error comparisons. #1289
[YSQL] Aggregate pushdowns in YSQL. #1851
[YSQL] Skip dropped columns when iterating through the columns for UPDATE statement. #1969
[YSQL] Use keyspace id instead of name in requests from yb-admin to catalog_manager. #1476
[YSQL] Fix insert into temp table in function body assertion error. #1541
[YSQL] Fix memory leak in psql. #2388
[YSQL] Add default password for yugabyte user. #2465

YCQL changes

[YCQL] Fixed permission check for DROP TYPE. #2373
[YCQL] Fix for 'value too long' error code mapping. #2116
[YCQL] 'expression not yet supported' error for old-format index update. #2470

System improvements

Do not retry flush/compact async tasks. #2229
Make yb-admin log an error message instead of fatal-ing on failed init. #2141
Propagate timeout to transaction. #2334
Warn on non-txn COPY with incompatible relation. #2278
[2DC] Add Exponential Backoff to CDC Consumer. #2362
Add support for cotable id in EnumerateIntents (needed for transactional DDL). #1404
TServer blacklisting for leadership role. #1748
Avoid unnecessary RocksDB lookup when we have multiple ops to one tablet. #2186
[2DC] Reduce checkpoint frequency, avoid self looping in CDC producer, apply polling delay correctly. #2364
Request intents cleanup even on failed replicas. #2411
Add number of sst files to tables and tablets pages for tserver. #2230
Fix role and term cache. #2267
yb-admin add RF to add_read_replicas. #2395
[2DC] Fix transaction status and record handling in CDC producer/consumer. #2386
Handle concurrent shutdown and remote bootstrap. #2428
[CDC] Fix bug in condition check for tablet leader. #2449
Fix remote bootstrap for encryption at rest. #2462
Implement dynamic priorities for compaction tasks. #2004
[2DC] Update checkpoint for replicated records, ignore tablet status while looking up peer. #2487
[2DC] yb-admin command to pause/resume replication. #2367

Note on upgrades from pre 2.0 releases

There are two ways to address the above note.

If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
If you want to continue using your existing data, then please save your data,
delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

Assets 2

17 Sep 01:45

rao-vasireddy

v2.0.0

0b543e8

v2.0.0 (September 17, 2019)

File format change

There are two ways to address the above note.

If you don't care about preserving your existing data, then delete your cluster, and create a new cluster with 2.0.x.x
If you want to continue using your existing data, then please save your data,
delete the cluster, create a new cluster with 2.0.x.x and then load the previously saved data.

YSQL Changes

[YSQL] Add CREATE TABLE tablet splits to grammar.#1486
[YSQL] Add catch-all ysql configuration gflags.#1140
[YSQL] Allow ALTER TABLE ALTER COLUMN SET|DROP NOT NULL. #1200
[YSQL] Add retry logic to pgbench. #2058
[YSQL] Optimize single-row UPDATE/DELETE statements. #1028
[YSQL] IN predicate pushdown. #1554
[YSQL] Add top-level ysql configuration gflags. #1140
[YSQL] Add support for various ALTER TABLE ADD COLUMN variants. #2055
[YSQL] Support UDTs, records. #1676
[YSQL] Order range columns with respect to primary key descripttion. #1250
[YSQL] Support ALTER TABLE ADD COLUMN with default value. #2055
[YSQL] Enable ANALYZE, VACUUM in grammar. #2217
[YSQL] Non-transactional COPY in YSQL. #1926
[YSQL] CREATE/DROP AGGREGATE, CAST, OPERATOR, OPERATOR CLASS. #1981
[YSQL] Create yugabyte, system_platform as default database. #2277

YCQL Changes

[YCQL] Fixed TS crash if a built-in function argument is a column. #1944

System Improvements

Propagate intents apply failure to client.#1974
Check shutdown state while scheduling compaction.#2100
Add leader check on master TS Write interface. #1995
Correctly handle duplicate key in postgres layer. #2012
Remove StopWatch from RocksDB Seek. #2136
Avoid picking read time for the serializable transactions. #2125
CDC: Producer and consumer master handshake to setup 2DC. #1977
Optimized Drop Table Locking. #2056
Reduce number of sys_catalog writes: skip no change heartbeats. #1966
CDC: Consumer changes. #1481
CDC: Add master master support. #1824
Increase WAL retention time when CDC stream is setup on table. #2050
Apply changes received from producer universe. #2153
CDC: Delete for DROP DATABASE. #2114
Show Load Balancer state in Master UI. #646
Add support for error code categories to Status. #2200
[2DC] Handle rpc controller errors and CDC consumer shutdown. #2231
Don't apply transactions on tablets without intents. #2070
Allow dynamic priorities in PriorityThreadPool. #2004
Added rocksdb_universal_compaction_always_include_size_threshold. #2085
Do full table name match while finding matching tables in 2DC. #2250
Fix cleaning Raft log after transactional writes are stopped. #2221
Fix handling of calls expired in queue. #2269
Load balancer should not be considered idle when tablet remote bootstrap. #2193
[YSQ]: Fix number of tablets created for system tables. #2293
Reduce fan-out and use tablet invoker for writing data.
SerializableReadWriteConflict needs better success probability.

Assets 3

08 Aug 17:32

rao-vasireddy

v1.3.1

f10e23d

v1.3.1 (July 31, 2019)

Overview of Changes

[IMPORTANT] This version includes a backward incompatible file format change for YSQL. This means that if you have an existing cluster with YSQL schema, then you will not be able to connect to the YSQL database after upgrading to v1.3.1. There are two ways to work around this problem:

If you don't care about preserving your existing data, then you can delete your older cluster, upgrade to v1.3.1 and create a new cluster.
If you want to continue using your existing data, then please save your older data (in the form of SQL statements), delete the older cluster, upgrade to v1.3.1, create a new cluster and then load the previously saved data.

YSQL Changes

[YSQL] Fix error in nextval_internal retry.#1783
[YSQL] Support bulk inserts for COPY FROM command.#1590
[YSQL] Support for Triggers and Extensions.#1156
[YSQL] Simplify gflags enabling YSQL.#1691
[YSQL] Support NULLS LAST for SELECT.#1734

System Improvements

Extra cleanup of intents of aborted transactions to prevent repeated log messages.#1722
Cleanup checkpoints on startup.#498
Unified yb::RandomAccessFile and rocksdb::RandomAccessFile.#1661
Clear Pending Delete State on RPC Failure.#424
Do not crash yb-master process if we fail to find a namespace when listing tables.#1716
Allow consecutive master restarts to succeed.#1074
Change soft memory limit handling.#1437
Use log prefix in MvccManager.#1835
Fix handling timeout of a local call in the service.#1866
Fixed TS crash due to NULL name in the PTBindVar object.#1821
Add /tables view to tablet server ui.#1830
Add ability to avoid using the compaction thread pool for flushes.#1874
Tablet leader tracking at catalog manager.#1881
Allow in memory erase of intents.#1240
Fix assignment order of propagated_safe_time and propagated.#1889
Memory monitor should not flush tablet that is already flushing.#1880
Fixed potential race condition in memory-based flush.#1859

Assets 2

16 Jul 03:29

rao-vasireddy

v1.3.0

928766c

v1.3.0 (Jul 15, 2019)

Overview of Changes

[IMPORTANT] This version includes a backward incompatible file format change for YSQL. This means that if you have an existing cluster with
YSQL schema, then you will not be able to connect to the YSQL database after upgrading to v1.3.0. There are two ways to work around this problem:

If you don't care about preserving your existing data, then you can delete your older cluster, upgrade to v1.3.0 and create a new cluster.
If you want to continue using your existing data, then please save your older data (in the form of SQL statements), delete the older cluster, upgrade to v1.3.0, create a new cluster and then load the previously saved data.

YSQL Changes

[YSQL] Enable DO statements (a.k.a anonymous blocks).#1656
[YSQL] Fix transaction read restart logic causing stale reads and lost writes.#1646
[YSQL] Enable beta-features by default.#1639
[YSQL] Fix value overwrite check in REPEATABLE READ isolation mode in YSQL.#1577
[YSQL] Allow SELECT from sequences.#999
[YSQL] Avoid auto-retrying non-DML requests.#1333
[YSQL] Set the read point correctly at REPEATABLE READ isolation when turning a read-only transaction into a read-write one.
[YSQL] Add new key column ybindexkeysuffix to YSQL index table [format change].#1058
[YSQL] Add support for setval().#1003
[YSQL] YSQL catalog cache consistency checking via tablet server shared memory.#1457
[YSQL] Fix persistence issue with catalog version.#1629

YCQL Changes

Added built-in function toJson() for YCQL SELECT statement.#992

System Improvements

Add Master UI page for tracking async tasks.#1145
Use rw_spinlock in meta cache to decrease lock contention.#1667
Change default maximum per-tablet RPC batch size to 32mb.#1440
Fix for flushing old tablets based on memory usage.#1672
Replaced OS_LINUX with linux for ifdefs.#1711
Unified yb::SequentialFile and rocksdb::SequentialFile.#1661
Add a compact_table command to yb-admin.#1735
Automatically filter out YSQL sys catalog tables from YBClient calls.#1537
Add TLS support to java yb-client.
Allow pausing a compaction when another compaction with a higher priority is scheduled.#1314
Fixed Tablet::Truncate to re-enable auto compactions.1765
Fix race condition in PriorityThreadPool.#1775

Assets 2

10 Jul 18:17

rao-vasireddy

v1.2.12

5adb348

v1.2.12 (Jul 03, 2019)

Overview of Changes

[IMPORTANT] This version includes a backward incompatible file format change for YSQL. This means that if you have an existing cluster with
YSQL schema, then you will not be able to connect to the YSQL database after upgrading to v1.2.12. There are two ways to work around this problem:

If you don't care about preserving your existing data, then you can delete your older cluster, upgrade to v1.2.12 and create a new cluster.
If you want to continue using your existing data, then please save your older data (in the form of SQL statements), delete the older cluster, upgrade to v1.2.12, create a new cluster and then load the previously saved data.

YSQL Changes

[YSQL] Fix scan operator when range key is not fully-specified.#1576
[YSQL] Fix ORDER BY range column of a secondary index.#1591
[YSQL] Add YSQL RPCs to tserver utilz page.#1446

System Improvements

Resolve ABA problem in LockFreeStackResolve ABA problem in LockFreeStack.#1561
Fix recursive locking in Batcher.#1584
Use boost::atomic for 16-bytes structures.#1189
Fix integer overflow during Raft log read path.#1606
Fix concurrent access to primary table info in RaftGroupMetadata.#1631
A new priority-aware thread pool implementation for compactions and flushes.#1314
Create mem trackers that reflects TC Malloc stats.#1633
Make sure we don't read an invalid replicate index from the footer.#1625

Performance Benchmarks

This section gives an overview of the performance of this release. The benchmarks were done on the following setup:

3 nodes with 16-vCPUs
Replication factor = 3
Each key-value is ~64 bytes (combined)
See setup details for more info

Summary

API	Workload	Type	Throughput	Latency	Avg CPU on DB node
YCQL	KeyValue	100% Reads	218K	1.17ms	48%
YCQL	KeyValue	100% Writes	73K	3.5ms	44%
YCQL	Batched Writes	100% Writes	247K	26ms	52%
YCQL	Secondary Index	100% Reads	187K	1.4ms	44%
YCQL	Secondary Index	100% Writes	5K	12.9ms	51%

Details

YCQL KeyValue

	Workload CassandraKeyValue with 256 writers and 0 readers: 
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric                         | mean      | min       | max       | std.dev   | median    | 1%        | 5%        | 10%       | 90%       | 95%       | 99%       |
| Read throughput, ops/sec       | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         |
| Read latency, ms/op            | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         | 0         |
| Write throughput, ops/sec      | 73037.05  | 68884.95  | 74102.43  | 1211.67   | 73480.55  | 68884.95  | 68884.95  | 71993.35  | 74051.57  | 74102.43  | 74102.43  |
| Write latency, ms/op           | 3.50      | 3.45      | 3.72      | 0.06      | 3.48      | 3.45      | 3.45      | 3.45      | 3.55      | 3.72      | 3.72      |
| Load tester CPU, user, %       | 16.91     | 14.90     | 18.10     | 0.88      | 17.30     | 14.90     | 14.91     | 15.18     | 17.70     | 18.08     | 18.10     |
| Load tester CPU, system, %     | 9.20      | 7.60      | 10.10     | 0.83      | 9.65      | 7.60      | 7.60      | 7.72      | 9.90      | 10.09     | 10.10     |
| Cluster node CPU, user, %      | 32.93     | 23.80     | 38        | 3.45      | 32.45     | 23.80     | 24.31     | 30.13     | 37.20     | 37.49     | 38        |
| Cluster node CPU, system, %    | 15.43     | 12.70     | 16.60     | 0.82      | 15.40     | 12.70     | 13.22     | 14.43     | 16.30     | 16.50     | 16.60     |
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+

	Workload CassandraKeyValue with 192 writers and 16 readers: 
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric                         | mean      | min       | max       | std.dev   | median    | 1%        | 5%        | 10%       | 90%       | 95%       | 99%       |
| Read throughput, ops/sec       | 18493.93  | 17156.62  | 19190.41  | 587.15    | 18606.10  | 17156.62  | 17156.62  | 17440.30  | 19151.33  | 19190.41  | 19190.41  |
| Read latency, ms/op            | 0.87      | 0.83      | 0.93      | 0.03      | 0.86      | 0.83      | 0.83      | 0.83      | 0.92      | 0.93      | 0.93      |
| Write throughput, ops/sec      | 60571.82  | 58544.09  | 62509.31  | 983.23    | 60550.72  | 58544.09  | 58544.09  | 59035.93  | 62380.73  | 62509.31  | 62509.31  |
| Write latency, ms/op           | 3.17      | 3.07      | 3.28      | 0.05      | 3.17      | 3.07      | 3.07      | 3.08      | 3.25      | 3.28      | 3.28      |
| Load tester CPU, user, %       | 17.68     | 16        | 20.70     | 0.87      | 17.55     | 16        | 16.06     | 17.20     | 18.39     | 20.58     | 20.70     |
| Load tester CPU, system, %     | 8.73      | 8.10      | 10.40     | 0.54      | 8.50      | 8.10      | 8.11      | 8.22      | 9.58      | 10.36     | 10.40     |
| Cluster node CPU, user, %      | 35.62     | 32.90     | 38.80     | 1.71      | 34.95     | 32.90     | 33.41     | 33.72     | 38.10     | 38.39     | 38.80     |
| Cluster node CPU, system, %    | 15.38     | 14.90     | 15.80     | 0.19      | 15.40     | 14.90     | 15.01     | 15.10     | 15.60     | 15.70     | 15.80     |
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+

	Workload CassandraKeyValue with 128 writers and 64 readers: 
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric                         | mean      | min       | max       | std.dev   | median    | 1%        | 5%        | 10%       | 90%       | 95%       | 99%       |
| Read throughput, ops/sec       | 71301.76  | 64468.86  | 78149.80  | 6074.55   | 69364.13  | 64468.86  | 64468.86  | 64563.72  | 78080.94  | 78149.80  | 78149.80  |
| Read latency, ms/op            | 0.90      | 0.82      | 0.99      | 0.08      | 0.92      | 0.82      | 0.82      | 0.82      | 0.99      | 0.99      | 0.99      |
| Write throughput, ops/sec      | 42974.61  | 40464.12  | 45473.25  | 1883.46   | 42860     | 40464.12  | 40464.12  | 40927.76  | 45053.88  | 45473.25  | 45473.25  |
| Write latency, ms/op           | 2.98      | 2.82      | 3.16      | 0.13      | 2.98      | 2.82      | 2.82      | 2.84      | 3.13      | 3.16      | 3.16      |
| Load tester CPU, user, %       | 21.46     | 19.10     | 23.30     | 1.28      | 21.60     | 19.10     | 19.15     | 20        | 23.08     | 23.29     | 23.30     |
| Load tester CPU, system, %     | 10.17     | 9.20      | 10.90     | 0.56      | 10.30     | 9.20      | 9.21      | 9.31      | 10.80     | 10.89     | 10.90     |
| Cluster node CPU, user, %      | 38.31     | 28.30     | 41.60     | 3.01      | 39.80     | 28.30     | 34.10     | 34.34     | 41.09     | 41.20     | 41.60     |
| Cluster node CPU, system, %    | 13.11     | 11.10     | 14        | 0.51      | 13.15     | 11.10     | 11.66     | 12.80     | 13.50     | 13.60     | 14        |
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+

	Workload CassandraKeyValue with 64 writers and 128 readers: 
+--------------------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric                         | mean      | min       | max       | std.dev   | median    | 1%        | 5%        | 10%       | 90%       | 95%       | 99%       |
| Read throughput, ops/sec       | 113115.50 | 111032.59 | 114690.37 | 1211.91   | 113593.04 | 111032.59 | 111032.59 | 111280.88 | 114639.78 | 114690.37 | 114690.37 |
| Read latency, ms/op            | 1.13      | 1.11      | 1.15      | 0.01      | 1.13      | 1.11      | 1.11      | 1.12      | 1.15      | 1.15      | 1.15      |
| Write throughput, ops/sec      | 21349.23  | 20717.49  | 21753.92  | 264.27    | 21335.10  | 20717.49  | 20717.49  | 20915.45  | 21699.52  | 21753.92  | 21753.92  |
| Write latency, ms/op           | 3.00      | 2.94      | 3.09      | 0.04      | 3         | 2.94      | 2.94      | 2.95      | 3.06      | 3.09      | 3.09      |
| Load tester CPU, user, %       | 23.54     | 21.80     | 27.90     | 1.24      | 23.50     | 21.80     | 21.81     | 22.03     | 24.36     | 27.7...

Assets 2

17 Apr 21:48

rkarthik007

v1.2.4

f2ecaa1

v1.2.4 (Apr 02, 2019)

Overview of Changes

[IMPORTANT] This version includes a backward incompatible file format change for YSQL. This means that if you have an existing cluster with
YSQL schema, then you will not be able to connect to the YSQL database after upgrading to v1.2.4. There are two ways to work around this problem:

If you don't care about preserving your existing data, then you can delete your older cluster, upgrade to v1.2.4 and create a new cluster.
If you want to continue using your existing data, then please save your older data (in the form of SQL statements), delete the older cluster, upgrade to v1.2.4, create a new cluster and then load the previously saved data.

YSQL Changes

Added support for primary keys. It is now possible to create a table with a primary key. #955
Added support for temporary tables. #805
Added support for CYCLE option in CREATE SEQUENCE statement. #993
Enabled SET TRANSACTION READ WRITE statement. #1081
Support RelationType in TableInfo to distinguish between system tables, user tables and user indexes. #906

Bug Fixes

Fixed a crash in tserver that is observed while inserting null value into an indexed column. #945
Fixed a bug where indexes on system catalog tables were not being updated during DDL. #1083
Fixed detection of single row transactions on tables that have primary keys.
Fixed MiniMaster startup race condition in test infrastructure. #1040

System Improvements

Reduce per-connection overhead for cache preloading. #1090 #1091
Faster release of sending buffers to reduce memory consumption when leader sends large updates to followers who are lagging behind. #1079
Parallelize DNS resolution while filling system.peers and system.local tables #1000
Script for post-processing test results and generating JSON-based result files.
Improve stability of QLTransactionTest.IntentsCleanupAfterRestart. #1102

Key Commits

[f2ecaa1] [YSQL] Fix single-row-txn detection after primary key change
[90680ca] [#1102]: Improve stability of QLTransactionTest.IntentsCleanupAfterRestart
[a4c614f] JSON-based test reports
[33835b0] [#1079]: Release sending buffers as soon as possible
[1bc77da] ENG-4682 (#906): Support RelationType (user table, index, system table) in TableInfo
[1dedcd1] ENG-4943 #945 [YSQL] Correct handling for NULL value in indexed column
[465b6d6] [YSQL] #1090 #1091 Reduce per-connection overhead for cache preloading
[382b89c] [YSQL] ENG-4601 (#805): Add support for temporary tables
[5165f01] #1081 [YSQL] Enable READ WRITE transaction
[9f31c41] ENG-4927: #1040: Fix MiniMaster startup race condition
[be0908b] [YSQL] #955: Handle primary key as an index
[4b48e25] ENG-4872: #1000: Parallel DNS resolution while filling system.peers and system.local tables
[4096f59] ENG-4900: Add external mini cluster test with TLS support
[34ee0fc] [YSQL] #1083: Handle sys catalog index updates
[ac52116] #993: [YSQL] Add support for CYCLE option in CREATE SEQUENCE
[3149f36] ENG-4363, #668: Support JSON operator on INSERT (Part 1 - Refactoring)
[a3ea483]
[b3a60ee] [yb-docker-ctl] Save initdb log (#1087)
[b131fbc] ENG-4888: #1031: Make YCQL index name optional
[9d333f7] #1064: Fix data race with YQL permissions cache
[4a56fe1] #1038: Change FLAGS_remote_boostrap_rate_limit_bytes_per_sec to FLAGS_remote_bootstrap_rate_limit_bytes_per_sec
[56a6029] #752: Add TLS encryption support to yb-admin.
[d9b6385] [YSQL] #1066: Fix memory leak in ResourceOwner
[fb4c1a2] Added yb-docker-ctl to CE (#1077)
[046b4b6] ENG-4855: #977: Handle restart read in YSQL
[30725bf] ENG-4343: #655: Pick transaction read time at tablet server when possible
[0341b29] [YSQL] #1067: Buffer create index write operations only for initdb
[78ca75a] [#1050]: Add Tablet Server capabilities
[9d724d5] ENG-4790: Removing most of yb-loadtester from repo due to migration to standalone yb-sample-apps repo
[bbcc67b] ENG-4891: Removing the SQL RPCs button.
[14279cc] [YCQL] #1042: Fix ASAN issue in YCQL executor restart
[cc3584f] ENG-4800: #952: Use call buffer for receive from socket
[8715be3] [YSQL] #1051: Bulk insert in initdb
[987abc6] ENG-4877 (#1009): yb-ctl enhancements
[0f261a9] [YSQL] 1049 ASAN reported memory leaks in Postgres
[9eaf15b] Add a new pattern for NFS directories: /Volumes/net
[3a61698] [#1041]: Skip dumping of finished call
[a322d95] [YSQL] #711 Add support for all datatypes
[7e2c89c] [#1030]: Add transactional retry writes tests 2) Second
[request]licated. Fixed by returning AlreadyPresent status and handling it in
[callbac] not handled correctly. For instance when request cannot be processed because of service shutdown, we could stop retrying to apply the operation. Fixed by correct handling of such errors.
[509c5a2] #921: Fixed: [YCQL] improve error messages related to index DDL
[e48b7da] ENG-4468: Always perform only one server rpc call for readiness check.
[0f8fdd9] [YSQL] ENG-4831 (#969): Enable drop cascade
[85836b5] Allow sp...

Assets 2

13 Mar 16:33

rkarthik007

v1.2.0

9093416

v1.2.0 (Mar 12, 2019)

Overview of Changes

Theme of this release: PostgreSQL, PostgreSQL, PostgreSQL! YSQL is a distributed PostgreSQL that can scale-out, tolerate failures and has no single point of failure.

Support has been added for a ton of Postgres functionality. The following is a list of some of the functionality:

DDL statements

CREATE, DROP and TRUNCATE tables

Data types

All primitive types including numeric types (integers and floats), text data types, byte arrays, date-time types
Support for UUID and SERIAL
Support for JSONB

DML statements

INSERT, UPDATE, SELECT and DELETE statements
JOIN, WHERE clauses, GROUP BY, ORDER BY, LIMIT, OFFSET and SEQUENCES

Transactions

ABORT, ROLLBACK, BEGIN, END, and COMMIT

Miscellaneous

Expressions: Built-in functions and operators
VIEWs
EXPLAIN PLAN
PREPARE-BIND-EXECUTE
JDBC Support

Key Commits

[aef1a1f] ENG-4683 (#874): Fix user table UI to exclude transactions and sequences tables
[c60921f] ENG-4681 [YSQL] Support INTERVAL for date/time datatypes
[4927d38] [YSQL] ENG-4807 (#957) Fix logs wrongly reporting that YugaByte transactions are not enabled
[4e69172] ENG-4780 (#930): Fix postgres server crash while creating index on existing table with data
[f004732] #972: ENG-4808: Fixed TabletInvoker::SelectTabletServer
[652bb5e] [#959]: Groups threads by stack trace in /threadz
[d33eaa9] #665: [YSQL] Enable support for sequences
[53a1331] ENG-4801 (#944): yb-ctl improvements for easier onboarding
[dce09da] ENG-4585: #796: Created test that covers case when thread pool is not yet ready for intents cleanup
[bb40833] ENG-4468: (#871) Add support for checking if tserver is ready for IO requests.
[e570c1c] [ENG-3928] [#967] Wait for NFS-based paths to exist in run_tests_on_spark.py
[3399b88] [YSQL] #255: Separate YCQL, YSQL and YEDIS namespaces
[064d366] ENG-4749: #887: Handle failure of delete at start of iteration in PgLibPqTest.SerializableColoring test
[88bb343] #961 Fix instantiation of round-robin policy
[1037c47] ENG-4577: Log Replay Count Bugfix and Test #782
[57ca114] ENG-4725: #932: Write RPC should take into account retryable_rpc_single_call_timeout_ms
[e9424ab] [YSQL] #943 Optimize single-row write transactions to avoid distributed txn path.
[c545fb5] ENG-4783: Misc fixes in shared lock manager
[45bc122] ENG-4783: Fixed LockBatch move ctor/assignment
[3db5d55] Cleanup of stray processes, NFS issues workaround, DXR integration, adjusting RPC timeout and using random PG ports in YSQL
[36431db] Update PostgreSQL code as of the following commit:
[3952394] [YSQL] ENG-4757: #902: Restart transaction when it was first executed operation and read restart is required
[d563e50] ENG-4577: Log Replay Count Bugfix and Test #782
[1ed9e95] ENG-3558: #920: Replace our own implementation of VarInt with BIGNUM from OpenSSL
[2f6dedc] [#836][ENG-4638] Fix KEYS to not Surface Deleted Keys
[7cf6ccd] [#918]: Add a pool of pre-created transaction records to avoid a Raft round-trip for transaction creation on the write path
[7b59cc8] #897: Fixed yb-tserver failure on SELECT COUNT(column) where column contains null value
[b0314eb] [YCQL] #916 Making STATIC an unreserved-keyword. Removing REFERENCES from keyword list.
[c886b86] [YSQL] #917: Support DROP INDEX
[81a80ab] [YCQL] (#878) : Add uuid() builtin function support.
[99cc736] ENG-4672: Provide more information (version,
[a101e74] ENG-4656 (#852): Enabled pg_depend
[c839f8c] Avoid using NFS-shared local Maven repository directory for running Java tests on macOS
[396ff88] [#914]: Split doc_operation into multiple files
[78776a5] [YCQL] #835 Allow OFFSET to be used as column names and identifiers
[362502d] ENG-4636 (#834): Make transaction failure messages more user friendly
[a746743] #904: ENG-4735: Fixed race between deleting obsolete files and creating new output SST file
[845cdb4] ENG-4717: #891: Not submitted write operation should block tablet peer shutdown
[1125d1e] #771 [YEDIS] Close connection on pub/sub limit
[ae1e30c] #768 Ping in subscribe mode should return a multi-bulk string
[dd97847] [YSQL] ENG-4734 Fix data race with ysql_catalog_version #903
[08c9bbf] ENG-4410 (#696): Retry resolving master addresses instead of erroring out immediately
[3ce7386] ENG-4680 (#873): Save postgres logs in tserver/logs directory
[1dd8e4b] ENG-4692 (#868): Enable access share mode for table lock
[63a0d61] ENG-4692 (#868): Enable support for read only transactions
[3779a86] #862: ENG-4728: Use libbacktrace only in debug builds and in single-threaded mode to avoid high memory consumption
[827ff02] [YSQL] Basic framework for cache consistency with multi-head DDLs #679
[299c718] Add pre- for leader election logging
[bfd2019] #895: Rate-limit the warning of unsupported YCQL consistency level.
[ff38980] Break the long test TestPgRegressPgTypes into two separate tests
[492c67c] Bump up master memory limit threshold for ASAN and TSAN
[[abc6a2e](https://github.com/YugaByte/yugabyte-db/commit/abc6a2ead184c764a7732cfb2d...

Assets 4

19 Sep 23:35

ravimurthy

v1.1.0

0eb062a

v1.1.0 (Sep 17, 2018)