Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery: does not support sync column default value #96

Open
wd0517 opened this issue Mar 1, 2024 · 0 comments
Open

BigQuery: does not support sync column default value #96

wd0517 opened this issue Mar 1, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@wd0517
Copy link
Contributor

wd0517 commented Mar 1, 2024

Currently tidb2dw read table schema from https://github.com/pingcap-inc/tidb2dw/blob/main/pkg/tidbsql/ddl.go#L115

SELECT COLUMN_NAME, COLUMN_DEFAULT, IS_NULLABLE, DATA_TYPE, 
CHARACTER_MAXIMUM_LENGTH, NUMERIC_PRECISION, NUMERIC_SCALE, DATETIME_PRECISION, COLUMN_TYPE
FROM information_schema.columns
WHERE table_schema = "%s" AND table_name = "%s"

It can not recognize default string and default function as below:

mysql> create table t1 (a int primary key, b varchar(40) default 'uuid()');
mysql> create table t2 (a int primary key, b varchar(40) default uuid());
mysql> SELECT COLUMN_NAME, COLUMN_DEFAULT, IS_NULLABLE, DATA_TYPE,
    -> CHARACTER_MAXIMUM_LENGTH, NUMERIC_PRECISION, NUMERIC_SCALE, DATETIME_PRECISION, COLUMN_TYPE
    -> FROM information_schema.columns
    -> WHERE table_schema = "test" AND table_name in ('t1', 't2');
+-------------+----------------+-------------+-----------+--------------------------+-------------------+---------------+--------------------+-------------+
| COLUMN_NAME | COLUMN_DEFAULT | IS_NULLABLE | DATA_TYPE | CHARACTER_MAXIMUM_LENGTH | NUMERIC_PRECISION | NUMERIC_SCALE | DATETIME_PRECISION | COLUMN_TYPE |
+-------------+----------------+-------------+-----------+--------------------------+-------------------+---------------+--------------------+-------------+
| a           | NULL           | NO          | int       |                     NULL |                11 |             0 |               NULL | int(11)     |
| b           | uuid()         | YES         | varchar   |                       40 |              NULL |          NULL |               NULL | varchar(40) |
| a           | NULL           | NO          | int       |                     NULL |                11 |             0 |               NULL | int(11)     |
| b           | uuid()         | YES         | varchar   |                       40 |              NULL |          NULL |               NULL | varchar(40) |
+-------------+----------------+-------------+-----------+--------------------------+-------------------+---------------+--------------------+-------------+
4 rows in set (0.01 sec)

tidb2dw as a data changes replication tools from TiDB to data warehouse, user should not manual insert new data to warehouse. Therefore, it is safe to skip the default function, but can be better to handle normal default string/int value.

Now tidb2dw will not sync default value/function to bigquery, and will report an error if add new column with default value.

@zhangjinpeng87 zhangjinpeng87 added the bug Something isn't working label Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants