Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support inserting and selecting VECTORs as binary data #59225

Open
mjonss opened this issue Jan 28, 2025 · 2 comments
Open

Support inserting and selecting VECTORs as binary data #59225

mjonss opened this issue Jan 28, 2025 · 2 comments
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@mjonss
Copy link
Contributor

mjonss commented Jan 28, 2025

Feature Request

Is your feature request related to a problem? Please describe:

Since VECTOR datatype is implicitly parsed as text as well as converted to text implicitly, there are no way to insert and select VECTOR columns as binary data, causing unnecessary conversion and more network traffic.

Describe the feature you'd like:

Comparing to MySQL it is possible creating a program that inserts an array of 32-bit floats instead of STRING_TO_VECTOR() making it more efficient to insert vectors.

Comparing to MySQL and MariaDB the default format when selecting a VECTOR column is binary, so by default it will return the data as 32-bit float array if VECTOR_TO_STRING() or VEC_ToText() is not used. I cannot find any way to get that from TiDB (it also don't support HEX(vector_column)).

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

We should also aim to be compatible with MySQL (and possibly MariaDB) also for VECTOR support.

@mjonss mjonss added the type/feature-request Categorizes issue or PR as related to a new feature. label Jan 28, 2025
@mjonss
Copy link
Contributor Author

mjonss commented Jan 28, 2025

MySQL and MariaDB also supports insert Hex values, like INSERT INTO vt (v) VALUES (x'CDCC8C3FCDCC0C4033335340')

@mjonss
Copy link
Contributor Author

mjonss commented Jan 28, 2025

I created a small repository for testing the compatibility between MySQL/MariaDB/TiDB (as well as MySQL with MYVECTOR) https://github.com/mjonss/mysql-vector-compare.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant