Skip to content

trantaiBD/driver-java

 
 

Repository files navigation

ByteHouse JDBC Driver

Introduction

ByteHouse provides a JDBC type 4 driver that supports core JDBC functionality. The JDBC driver must be installed in a 64-bit environment and requires Java 1.8. The driver can be used with most client tools/applications that support JDBC for connecting to ByteHouse datasource.

Table of Contents

Requirements

  • Java 1.8 or higher

Installation

Adding Driver as a Gradle Dependency

    repositories {
        // This is public bytedance repository for downloading artifacts
        maven {
            url "https://artifact.bytedance.com/repository/releases"
        }
    }
    
    dependencies {
        implementation 'com.bytedance.bytehouse:driver-java:1.1.24'
    }

Adding Driver as a Maven Dependency

        // This is public bytedance repository for downloading artifacts
        <repository>
            <id>bytedance</id>
            <name>ByteDance Public Repository</name>
            <url>https://artifact.bytedance.com/repository/releases</url>
        </repository>
        
        <dependency>
            <groupId>com.bytedance.bytehouse</groupId>
            <artifactId>driver-java</artifactId>
            <version>1.1.24</version>
        </dependency>

Direct Download

Usage

Creating ByteHouse Account

You need to create ByteHouse account in order to use JDBC Driver. You can simply create a free account with the process mentioned in our official website documentation: https://docs.bytehouse.cloud/en/docs/quick-start

You can also create ByteHouse account through Volcano Engine by ByteDance: https://www.volcengine.com/product/bytehouse-cloud

JDBC Connection URL Format

URL Format for password authentication

For password authentication, Region, Account ID, User ID & Password parameters are required.

    String connectionURL = String.format("jdbc:bytehouse:///?region=%s&account=%s&user=%s&password=%s", REGION, ACCOUNT, USER, PASSWORD);

URL Format for AK/SK authentication

For AK/SK authentication, Region, Access Key & Secret Key parameters are required.

    String connectionURL = String.format("jdbc:bytehouse:///?region=%s&access_key=%s&secret_key=%s&is_volcano=true", REGION, ACCESS_KEY, SECRET_KEY);

URL Format for IP & Port

You can directly use Host Name / IP & Port addresses instead of region.

    String connectionURL = String.format("jdbc:bytehouse://%s:%s/?account=%s&user=%s&password=%s", HOST, PORT, ACCOUNT, USER, PASSWORD);
    String connectionURL = String.format("jdbc:bytehouse://%s:%s/?access_key=%s&secret_key=%s&is_volcano=true", HOST, PORT, ACCESS_KEY, SECRET_KEY);

Adding additional parameters for URL

You can create Properties object to supply additional parameters alongside with JDBC connection URL. References of additional parameters can be found here.

    Properties properties = new Properies();
    properties.setProperty("secure", "false");

JDBC API

Implements Class
java.sql.Driver com.bytedance.bytehouse.jdbc.ByteHouseDriver
javax.sql.DataSource com.bytedance.bytehouse.jdbc.ByteHouseDataSource

Connecting using ByteHouseDriver

    Connection connection = new ByteHouseDriver().connect(connectionURL, properties);

Connecting using ByteHouseDataSource

    Connection connection = new ByteHouseDataSource(connectionURL, properties).getConnection();

Setting Virtual WareHouse

A virtual warehouse is a cluster of computing resources in Bytehouse that we can scale out on demand. A warehouse provides the required resources, such as CPU, memory, and temporary storage, to perform the database operations.

User can set the virtual warehouse using ByteHouseStatement object / Properties / Connection URL. After setting the virtual warehouse, user can obtain the connection object and can subsequently use the same connection object to perform database operations with the same virtual warehouse.

If users wants to change virtual warehouse, they have to create new connection object with the above mentioned procedure. In case, there is no virtual warehouse stated, the server may use the default virtual warehouse, if there is, or can throw exception message.

Statement Object

    try (Statement stmt = connection.createStatement()) {
        stmt.execute("SET WAREHOUSE vw_name");
    } catch (SQLException ex) {
        ex.printStackTrace();
    }

Properties

    properties.setProperty("virtual_warehouse", "virtual_warehouse_name");

Connection URL

    connectionURL = String.format("jdbc:bytehouse://%s&virtual_warehouse=%s", CONNECTION_URL, VIRTUAL_WAREHOUSE_NAME);

Setting Role

When using ByteHouse, you need to select an "Active Role", and all your behaviour will be restricted by the permissions assigned to this Active Role.

User can set the active role using ByteHouseStatement object / Properties / Connection URL. After setting the active role, user can obtain the connection object from ByteHouseStatement object and can subsequently use the same connection object to perform database operations with the same active role.

Statement Object

    try (Statement stmt = connection.createStatement()) {
        stmt.execute("SET ROLE role_name");
    } catch (SQLException ex) {
        ex.printStackTrace();
    }

Properties

    properties.setProperty("active_role", "active_role_name");

Connection URL

    connectionURL = String.format("jdbc:bytehouse://%s&active_role=%s", CONNECTION_URL, ACTIVE_ROLE_NAME);

Performing SQL Queries

DDL Query

    try (Statement stmt = connection.createStatement()) {
        String createDatabaseSql = "CREATE DATABASE IF NOT EXISTS inventory";
        stmt.execute(createDatabaseSql);
        
        String createTableSql = "CREATE TABLE IF NOT EXISTS inventory.orders (" +
                        " OrderID String, " +
                        " OrderName String, " +
                        " OrderPriority Int8 " +
                        " ) " +
                        " engine = CnchMergeTree()" +
                        " partition by OrderID" +
                        " order by OrderID";
        stmt.execute(createTableSql);                 
    } catch (SQLException ex) {
        ex.printStackTrace();
    }

DML Query

Row Insert
    try (Statement stmt = connection.createStatement()) {
        String insertSql = "INSERT INTO inventory.orders VALUES('54895','Apple',12)";
        stmt.executeUpdate(insertSql);                 
    } catch (SQLException ex) {
        ex.printStackTrace();
    }
Batch Insert
    String batchInsertSql = "INSERT INTO inventory.orders VALUES(?,'Apple',?)";
    try (PreparedStatement pstmt = connection.prepareStatement(batchInsertSql)) {
        int insertBatchSize = 10;
        for (int i = 0; i < insertBatchSize; i++) {
            pstmt.setString(1, "ID" + i);
            pstmt.setInt(2, i);
            pstmt.addBatch();
        }
        pstmt.executeBatch();              
    } catch (SQLException ex) {
        ex.printStackTrace();
    }

DQL Query

    try (Statement stmt = connection.createStatement()) {
        String selectSql = "SELECT * FROM inventory.orders";
        ResultSet rs = stmt.executeQuery(selectSql);              
    } catch (SQLException ex) {
        ex.printStackTrace();
    }

Supported DataTypes

ByteHouse Definition

Definition of datatypes in ByteHouse datasource can be found here: https://docs.bytehouse.cloud/en/docs/data-types

Driver Definition

  • Class - Class represents the top Java class name for each of the datatypes that are available in ByteHouse.
  • Driver Data Type - Represents the Java class for each of the ByteHouse data types. You can insert & select different datatypes using these classes.
  • JDBC Data Type - Represents the internal JDBC driver class. Driver data type is converted to JDBC data type before sending them to server.
  • Types - Representation of java.sql.Types for each of the ByteHouse datatypes.

No

Class (ByteHouse Type)

Driver Data Type

JDBC Data Type

Types

Precision

Scale

1

DataTypeUInt8

Short

Short

Types.TINYINT

3

0

2

DataTypeUInt16

Integer

Integer

Types.SMALLINT

5

0

3

DataTypeUInt32

Long

Long

Types.INTEGER

10

0

4

DataTypeUInt64

BigInteger

BigInteger

Types.BIGINT

19

0

5

DataTypeInt8

Byte

Byte

Types.TINYINT

4

0

6

DataTypeInt16

Short

Short

Types.SMALLINT

6

0

7

DataTypeInt32

Integer

Integer

Types.INTEGER

11

0

8

DataTypeInt64

Long

Long

Types.BIGINT

20

0

9

DataTypeFloat32

Float

Float

Types.FLOAT

8

8

10

DataTypeFloat64

Double

Double

Types.DOUBLE

17

17

11

DataTypeDecimal

BigDecimal

BigDecimal

Types.DECIMAL

User defined

User defined

12

DataTypeString

CharSequence

String

Types.VARCHAR

0

0

13

DataTypeFixedString

CharSequence

String

Types.VARCHAR

User defined

0

14

DataTypeIPv4

Long

Long

Types.INTEGER

0

15

15

DataTypeIPv6

Inet6Address

String

Types.VARCHAR

0

0

16

DataTypeUUID

UUID

String

Types.VARCHAR

36

0

17

DataTypeDate

LocalDate

Date

Types.DATE

10

0

18

DataTypeDateTime

ZonedDateTime

Timestamp

Types.TIMESTAMP

0

10

19

DataTypeEnum8

String

String

Types.VARCHAR

0

0

20

DataTypeEnum16

String

String

Types.VARCHAR

0

0

21

DataTypeNullable

Nested Type

Nested Type

Nested Type

0

0

22

DataTypeArray

ByteHouseArray

Array

Types.ARRAY

0

0

23

DataTypeMap

Map

Object

Types.OTHER

0

0

24

DataTypeTuple

ByteHouseStruct

Struct

Types.STRUCT

0

0

Usage of DataTypes

DataTypeUInt8

    // pstmt is the object for PreparedStatement class
    
    Short valueShort = 1;
    short valueShortPrimitive = 1;
    
    pstmt.setShort(1, valueShort);
    pstmt.addBatch();
    
    pstmt.setShort(1, valueShortPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueShort);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueShortPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeUInt16

    // pstmt is the object for PreparedStatement class
    
    Integer valueInteger = 35000;
    int valueIntegerPrimitive = 35000;
    
    pstmt.setInt(1, valueInteger);
    pstmt.addBatch();
    
    pstmt.setInt(1, valueIntegerPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueInteger);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueIntegerPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeUInt32

    // pstmt is the object for PreparedStatement class
    
    Long valueLong = Long.valueOf(256256256256L);
    long valueLongPrimitive = 256256256256L;
    
    pstmt.setLong(1, valueLong);
    pstmt.addBatch();
    
    pstmt.setLong(1, valueLongPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLong);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLongPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeUInt64

    // pstmt is the object for PreparedStatement class
    
    BigInteger bigIntegerValue = BigInteger.valueOf(256256256256L);
    
    pstmt.setObject(1, bigIntegerValue);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeInt8

    // pstmt is the object for PreparedStatement class
    
    Byte valueByte = 1;
    byte valueBytePrimitive = 1;
    
    pstmt.setByte(1, valueByte);
    pstmt.addBatch();
    
    pstmt.setByte(1, valueBytePrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueByte);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueBytePrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeInt16

    // pstmt is the object for PreparedStatement class
    
    Short valueShort = 1;
    short valueShortPrimitive = 1;
    
    pstmt.setShort(1, valueShort);
    pstmt.addBatch();
    
    pstmt.setShort(1, valueShortPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueShort);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueShortPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeInt32

    // pstmt is the object for PreparedStatement class
    
    Integer valueInteger = 35000;
    int valueIntegerPrimitive = 35000;
    
    pstmt.setInt(1, valueInteger);
    pstmt.addBatch();
    
    pstmt.setInt(1, valueIntegerPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueInteger);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueIntegerPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeInt64

    // pstmt is the object for PreparedStatement class
    
    Long valueLong = Long.valueOf(256256256256L);
    long valueLongPrimitive = 256256256256L;
    
    pstmt.setLong(1, valueLong);
    pstmt.addBatch();
    
    pstmt.setLong(1, valueLongPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLong);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLongPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeFloat32

    // pstmt is the object for PreparedStatement class
    
    Float valueFloat = Float.valueOf("123.45879");
    float valueFloatPrimitive = 123.45897f;
    
    pstmt.setFloat(1, valueFloat);
    pstmt.addBatch();
    
    pstmt.setFloat(1, valueFloatPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueFloat);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueFloatPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeFloat64

    // pstmt is the object for PreparedStatement class
    
    Double valueDouble = Double.valueOf("123.45879");
    double valueDoublePrimitive = 123.45897;
    
    pstmt.setDouble(1, valueDouble);
    pstmt.addBatch();
    
    pstmt.setDouble(1, valueDoublePrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueDouble);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueDoublePrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeDecimal

    // pstmt is the object for PreparedStatement class
    
    BigDecimal valueBigDecimal = BigDecimal.valueOf(123.456);
    
    pstmt.setBigDecimal(1, valueBigDecimal);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueBigDecimal);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeString

    // pstmt is the object for PreparedStatement class
    
    String valueString = "tobeInserted";
    CharSequence valueCharSequence = "charSequence";
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.setString(1, valueCharSequence.toString());
    pstmt.addBatch();
    
    pstmt.setObject(1, valueString);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueCharSequence);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeFixedString

    // pstmt is the object for PreparedStatement class
    
    String valueString = "abcdefghij";
    CharSequence valueCharSequence = "abcdefghij";
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.setString(1, valueCharSequence.toString());
    pstmt.addBatch();
    
    pstmt.setObject(1, valueString);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueCharSequence);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeIPv4

    // pstmt is the object for PreparedStatement class
    
    Long valueLong = Long.valueOf(256256256256L);
    long valueLongPrimitive = 256256256256L;
    
    pstmt.setLong(1, valueLong);
    pstmt.addBatch();
    
    pstmt.setLong(1, valueLongPrimitive);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLong);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLongPrimitive);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeIPv6

    // pstmt is the object for PreparedStatement class
    
    String valueString = "2001:44c8:129:2632:33:0:252:2";
    Inet6Address valueInet6Address = (Inet6Address) Inet6Address.getByName("2001:44c8:129:2632:33:0:252:2");
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.setString(1, valueInet6Address.getHostAddress());
    pstmt.addBatch();
    
    pstmt.setObject(1, valueString);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueInet6Address);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeUUID

    // pstmt is the object for PreparedStatement class
    
    String valueString = "cd175988-5fd2-11ec-bf63-0242ac130002";
    UUID valueUUID = UUID.fromString("cd175988-5fd2-11ec-bf63-0242ac130002");
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.setString(1, valueUUID.toString());
    pstmt.addBatch();
    
    pstmt.setObject(1, valueString);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueUUID);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeDate

    // pstmt is the object for PreparedStatement class
    
    Date valueDate = new Date(25369874L);
    LocalDate valueLocalDate = LocalDate.of(2012, 9, 9);
    
    pstmt.setDate(1, valueDate);
    pstmt.addBatch();
    
    pstmt.setDate(1, Date.valueOf(valueLocalDate));
    pstmt.addBatch();
    
    pstmt.setObject(1, valueDate);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueLocalDate);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeDateTime

    // pstmt is the object for PreparedStatement class
    
    ZonedDateTime valueZonedDateTime = ZonedDateTime.of(2012, 1, 1, 0, 0, 0, 0, ZoneId.of("Asia/Singapore"));
    Timestamp valueTimestamp = Timestamp.valueOf("2012-01-01 00:00:00");
    
    pstmt.setTimestamp(1, Timestamp.from(valueZonedDateTime.toInstant()));
    pstmt.addBatch();
    
    pstmt.setTimestamp(1, valueTimestamp);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueZonedDateTime);
    pstmt.addBatch();
    
    pstmt.setObject(1, valueTimestamp);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeEnum8

    // pstmt is the object for PreparedStatement class
    
    String valueString = "a";
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeEnum16

    // pstmt is the object for PreparedStatement class
    
    String valueString = "a";
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeNullable

    // pstmt is the object for PreparedStatement class
    
    String valueString = null;
    
    pstmt.setString(1, valueString);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeArray

ByteHouse Array

ByteHouseArray implements java.sql.array and is the object that you can use to ingest array types to ByteHouse. Construction of ByteHouseArray takes two parameters: DataType & Object[] (Object array containing the elements)

    ByteHouseArray byteHouseArrayIntegers = new ByteHouseArray(new DataTypeInt32(), integersArray);
    ByteHouseArray byteHouseArrayInts = new ByteHouseArray(new DataTypeInt32(), intsArray);
    ByteHouseArray byteHouseArrayFloat32 = new ByteHouseArray(new DataTypeFloat32(), integersArray);
    // pstmt is the object for PreparedStatement class
    
    Integer[] integers = new Integer[]{1, 2, 3};
    int[] ints =  new int[]{1, 2, 3};
    
    ByteHouseArray byteHouseArrayIntegers = new ByteHouseArray(new DataTypeInt32(), integers);
    ByteHouseArray byteHouseArrayInts = new ByteHouseArray(new DataTypeInt32(), ints);
    
    pstmt.setArray(1, byteHouseArrayIntegers);
    pstmt.addBatch();
    
    pstmt.setArray(1, byteHouseArrayInts);
    pstmt.addBatch();
    
    pstmt.setObject(1, byteHouseArrayIntegers);
    pstmt.addBatch();
    
    pstmt.setObject(1, byteHouseArrayInts);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeMap

    // pstmt is the object for PreparedStatement class
    
    Map<Integer, Integer> values = new HashMap<>();
    values.put(1, 1);
    values.put(2, 2);
    values.put(3, 3);
    
    pstmt.setObject(1, values);
    pstmt.addBatch();
    
    pstmt.executeBatch();

DataTypeTuple

ByteHouseStruct

ByteHouseArray implements java.sql.struct and is the object that you can use to ingest struct types to ByteHouse. Construction of ByteHouseStruct takes two parameters: DataType & Object[] (Object array containing the elements)

// pstmt is the object for PreparedStatement class

ByteHouseStruct byteHouseStruct = new ByteHouseStruct("Tuple", new Object[]{"test_string", 1});

pstmt.setObject(1, byteHouseStruct);
pstmt.addBatch();

pstmt.executeBatch();

Integration with BI Tools

Tableau Integration

  1. Download the Connector file (.taco)
  2. Move the .taco file here:
    • Windows: C:\Users[Windows User]\Documents\My Tableau Repository\Connectors
    • macOS: /Users/[user]/Documents/My Tableau Repository/Connectors
  3. Start Tableau and under Connect, select the [ByteHouse JDBC Connector] connector. (Note: You’ll be prompted if the driver is not yet installed.
  4. Driver Installation:
    • Go to the driver download page https://artifact.bytedance.com/repository/releases/com/bytedance/bytehouse/driver-java/
    • Click on the directory for the version that you need. The most recent version is not always at the end of the list.
    • Download the driver-java-#.#.#-all.jar file. Minimum required driver version: 1.1.0
    • Move jar file into the following directory:
      • Windows: C:\Program Files\Tableau\Drivers
      • macOS: /Users/[user]/Library/Tableau/Drivers
  5. Relaunch Tableau and connect using the [ByteHouse JDBC Connector] connector.

DataGrip Integration

DBeaver Integration

Parameters Reference

Authentication Parameters

ACCOUNT

String type

denotes the bytehouse account you're connecting to

USER

String type

denotes the bytehouse user that you're connecting to

PASSWORD

String type

denotes the password for this account & user

REGION

String type

denotes the region that you're connecting to

ACCESS_KEY

String type

denotes the access key for your volcano engine account

SECRET_KEY

String type

denotes the secret key for your volcano engine

IS_VOLCANO

Boolean type

set to true if you are connecting to volcano cloud using access_key & secret_key

Connection Parameters

Param name

Default value

Type

Description

secure

denotes whether the connection would use secure tcp/tls or not

queryTimeout

denotes query timeout value in seconds

connectTimeout

denotes connection timeout value in seconds

tcpKeepAlive

tcp connection properties

tcpNoDelay

tcp connection properties

enableCompression

denotes whether driver would use LZ4 compression or not

charset

denotes the character set used to encode or decode strings

max_block_size

denotes the internal buffer size for the number of rows before sending it to the server

Query / Server Side Parameters

Please refer to the ByteHouse documentation for available query / server side params.

Troubleshooting

Issue Reporting

If you have found a bug or if you have a feature request, please report them at this repository issues section. Alternatively, you can directly create an issue with our support platform here: https://bytehouse.cloud/support

License

This project is distributed under the terms of the Apache License (Version 2.0).

About

JDBC Driver for ByteHouse

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 100.0%