Experiments
Here we present some experimental results performed on machines with 2.13 GHz Dual-Core Intel(R) Xeon(R) processor, 2 MB cache, and 4 GB memory. We defined three types of transactions: (i) ReadTxn: where all operations only read, (ii) WriteTxn: where all operations only write, and (iii) ComplexTxn: consists of half read and half write operations. Each transaction operates on n rows, where n is a uniform random number between 0 and 20. Based on these types of transactions, we define four workloads: (i) Read-only: includes ReadTxn transactions, (ii) Write-only: comprises WriteTxn transactions,(iii) Complex-only: consists of ComplexTxn transactions,and (iv) Mixed: consists of half ReadTxn and half ComplexTxn transactions.
The major overhead of CrSO is the replication of the commit information onto the client nodes. To assess scalability with number of client nodes, we exponentially increase the number of clients from 1 to 1024 and plot the average latency vs. the average throughput in the figure above. The clients simulate transactions by sleeping for 50 ms before sending the commit request. The read-only transactions do not cause to the status oracle the cost of checking for conflicts as well as the cost of persisting data into the WAL. Besides, the more rows modified by transactions, the higher is the cos t of checking for conflicts at the status oracle. To evaluate the CrSO performance under high load, we, therefore, use a write-only workload where rows are randomly selected out of 20M rows. Moreover, to lower the granularity of the measured latency, w e disable the WAL timer and manually set a threshold for triggering a write into the WAL. When exponentially increasing the number of client nodes, the latency only linearly increases up to 9 ms mostly due to the status oracle overhead in maintaining the connections.
To assess scalability of CrSO with number of transactions, we repeat the same experiment with the difference that each client allows for 100 outstanding transactions with the execution time of zero, which mean s that the clients keep the pipe on the status oracle full. Moreover, we consider the heaviest workload, which is write-only. With more reads in a workload, the performance of CrSO will be higher. As the above figure depicts, by increasing the load on the status oracle, the throughput increases up to 46K TPS with average latency of 9.9 ms. After this point, with increasing the load the latency inc reases (mostly due to the buffering delay at the status oracle) with only marginal improvement in throughput (51K TPS). Rep-disabled in the figure represents the results of the same experiment with the difference that the client replication is disabled. The difference indicates that the replication only marginally decreases the throughput that CrSO could deliver.
Here, we evaluate the integrated prototypes to measure the overhead of transactional support on the data store. HBase is initially loaded with a table of size 100 GB comprising 100M rows. Since the allocated memory to each HBase process is 3 GB, this table size ensures that the data does not fit into the memory of data servers. A random read, therefore, causes an IO from either a local or remote hard disk. This setup makes random read operations much more expensive than random writes. This is not necessarily the case for all setups. For example, in a system that read operations are mostly sequential rather than random, most of read requests will be serviced from memory and hence are less expensive than writes. The same appliesto in-memory databases where the reads are entirely serviced from memory. In such systems, since the write operations are the main bottleneck, the ReBase overhead of writing the commit timestamp into the modified rows makes a more profound negative impact on the overall performance. Since the main purpose of this experiment is evaluating CrBase, which does not impose this overhead for write operations, we leave experimenting with such setups for future work. Figure above depicts the performance of the systems when increasing the number of clients from 1 to 5, 10, 20, 40, 80, 160, 320, and 640. We repeat each experiment five times and report on the average and varia nce in the figure. The HBase integration with ReTSO and CrSO is denoted ReBase and CrBase, respectively. As expected, the scalability of three systems under read-only workload is fairly the same, with the difference that ReBase and CrBase have a slightly higher latency due to contacting the status oracle for sta rt timestamp and commit requests. The good scalability reflects the fact that the overhead of read transactions on data servers is negligible for both ReBase and CrBase.
The next figure depicts the results of the same experiment with write-only workload. CrBase exhibits the same scalability as HBase since both perform the same number of writes per transaction. ReBase, on the other hand, has much lower scalability since it requires an extra write into HBase servers per each write operation. ReBase saturates with 2036 TPS, which is about half the traffic with which HBase and CrBase saturate, i.e., 4247 TPS. This was expected because each write operations in HBase and CrBase corresponds to two write operations in ReBase, one for writing the data and one for updating the commit timestamp.
Since the overhead of CrSO on HBase is negligible under both read and write workloads, we expect the same pattern under the complex-only and mixed workloads as well. This is verified in next figures, where we see the same scalability in CrBase as in HBase. ReBase, on the other hand, has a high overhead under the write workload and yet negligible overhead under the read workload. Its performance under a mixed workload, then, highly depends on proportion of read an d write operations. Due to the very high latency of the read operations in our setting (38.8 ms) compared to write latency (1.13 ms), the low performance of read operations dominates, and the overhead of write operations in ReBase, hence, does not make a tangible impact on the overall performance.
Here we break down the latency of different operations involved in a transaction: (i) start timestamp request, (ii) read from HBase, (iii) write to HBase, and (iv) commit request.
The average latency of start timestamp request is 0.17 ms.
Each read and write into HBase takes 38.8 ms and 1.13 ms on average, respectively. The writes are in general less expensive since they usually include only writing into memory and appending into a WAL. Random reads, on the other hand, might incur the cost of loading an entire block from HDFS, and therefore have higher delays.
The commit latency is measured from the moment that the commit request is sent to the status oracle until the moment its response is received. The average commit latency is 4.72 ms, which is roughly equal to timeout of sending the recent modifications to BookKeeper (5 ms). If we immediately write to BookKeeper after each commit, the average commit latency is 1.53 ms, which is 1.36 ms more than the latency of requesting a timestamp from the status oracle. The reason is mostly due to persistence of commit data via BookKeeper in multiple remote storages. Although the assigned start timestamps must also be persisted, the timestamp oracle could reserve thousands of timestamps per each write into WAL, and therefore on average servicing timestamps do not incur a persistence cost.