Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:use parse_decimal for generic_string_to_decimal conversion. #7191

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

himadripal
Copy link
Contributor

@himadripal himadripal commented Feb 25, 2025

Which issue does this PR close?

Few important consideration -

  • Existing string to decimal conversion uses parse_string_to_decimal_native
  • parse_string_to_decimal_native does not have support for e-notation
  • parse_string_to_decimal_native does rounding at scale, not truncate
  • parse_decimal an existing method has e-notation support and use elsewhere
  • add rounding support in parse_decimal
  • moved string to decimal conversion to use parse_decimal to get support for e-notation.

This PR is a 3rd one to break up #6905 , this one add rounding logic to parse_decimal to match the behavior in existing parse_string_to_decimal_native.

Closes #.

Rationale for this change

At present, string to decimal conversion does not support e-notation, in arrow, parse_string_to_decimal_native is called to get generic string to decimal. parse_decimal on the other hand is used from generic parse method and it has e-notation support. This PR is adding rounding and scale 0 handling to match the behavior or parse_string_to_decimal_native method. Then we can replace parse_string_to_decimal_native call with parse_decimal. This way, we will get e-notation support too.

What changes are included in this PR?

Are there any user-facing changes?

add precision and scale in the error message
@github-actions github-actions bot added the arrow Changes to the arrow crate label Feb 25, 2025
@himadripal
Copy link
Contributor Author

#7179 - adds the rounding logic and with that merged, tests will pass. All failed tests are due to parse_decimal truncates instead of round, existing parse_string_to_decimal_native defaults to rounding.

I can make changes to test now and change again when #7179 is merged..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DataFusion should support casting strings such as "4e7" to decimal
1 participant