-
-
Notifications
You must be signed in to change notification settings - Fork 508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Attempting to Import Nested JSON Objects to Dolt can Cause Dolt to Crash #5143
Comments
I changes the fields containing JSON from TEXT/LONGTEXT to JSON in an updated dolt-schema.sql file, specifically:
|
I debugged a bit and this has something to do with the expected format of a json file. I think the scanner is expecting every line to be a standalone json object, rather than the file itself being a json array. We still should not panic if the file is not formatted the way we expect. The following example works for me locally test.json: {"a": 1}
{"b": 2} test.sql: create table t (x varchar(10), y int); dolt table import -c -s test.sql t test.json
Rows Processed: 0, Additions: 0, Modifications: 0, Had No Effect: 0
Import completed successfully. I forked a PR workaround that imports the table with the CSV interface davidshq/oreillyas#1. |
Interesting. I tried removing the enclosing array and running it through that way, but no luck. I'm guessing it is catching on the nested fields that have arrays with objects inside them. Sounds like the scanner in general doesn't support nested objects? Thanks for the fork, I was able to run it successfully. |
The scanner splits on line, and tries force casting the line to |
Thanks again for taking the time to report this @davidshq. We've fixed the panic and added a better error message; we also made some documentation improvements to better document the require input JSON file structure. Thanks for helping us find this problem! Let us know if there's anything else we can help out with. |
Attempting to run an import to dolt using the following command:
dolt table import -c -s dolt-schema.sql books oreilly.json
Error:
SQL:
See
oreilly.txt
for JSONoreilly.txt
For repo with code that pulls from O'Reilly API see: https://github.com/davidshq/oreillyas
@max-hoffman noted in Discord:
The text was updated successfully, but these errors were encountered: