-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DataFusion should support casting strings such as "4e7" to decimal #10315
DataFusion should support casting strings such as "4e7" to decimal #10315
Comments
I suspect this issue is actually in arrow_cast - the error message seems to come from https://github.com/apache/arrow-rs/blob/ada986c7ec8f8fe4f94235c8aaeba4995392ee72/arrow-cast/src/cast.rs#L2753 |
I believe this will be supported in the next arrow release |
I'm working on this issue and need help here - #[test]
fn test_cast_with_options_utf8_to_decimal() {
let array = StringArray::from(vec!["4e7"]);
let result = cast_with_options(
&array,
&DataType::Decimal128(10,2),
&CastOptions {
safe: false,
format_options: FormatOptions::default(),
},
).unwrap();
let output_array = result.as_any().downcast_ref::<Decimal128Array>();
assert_eq!(output_array.unwrap().value(0) , 40000000);
}
returns 4000000000 so it expands till the precision value. should we replace parse_string_to_decimal_native with parse_decimal and fix this bug there? |
At first glance, For example, let value_str = value_str.trim();
let parts: Vec<&str> = value_str.split('.').collect(); It does seem to make sense to do what you suggest and use |
Is your feature request related to a problem or challenge?
DataFusion supports casting the string
4e7
to float but not to decimal. This is inconsistent with Postgres (and Apache Spark).Postgres
Apache Spark
DataFusion
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: