Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add rounding logic and scale zero fix fro parse_decimal to match parse_string_to_decimal_native behavior #7179

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

himadripal
Copy link
Contributor

@himadripal himadripal commented Feb 23, 2025

Which issue does this PR close?

Few important consideration -

  • Existing string to decimal conversion uses parse_string_to_decimal_native
  • parse_string_to_decimal_native does not have support for e-notation
  • parse_string_to_decimal_native does rounding at scale, not truncate
  • parse_decimal an existing method has e-notation support and use elsewhere
  • Fix: Support for e notation using existing parse_decimal in string to decimal conversion #6905 added rounding support in parse_decimal
  • moved string to decimal conversion to use parse_decimal to get support for e-notation.

This PR is a 2nd one to break up #6905 , this one add rounding logic to parse_decimal to match the behavior in existing parse_string_to_decimal_native.

Closes #.

Rationale for this change

At present, string to decimal conversion does not support e-notation, in arrow, parse_string_to_decimal_native is called to get generic string to decimal. parse_decimal on the other hand is used from generic parse method and it has e-notation support. This PR is adding rounding and scale 0 handling to match the behavior or parse_string_to_decimal_native method. Then we can replace parse_string_to_decimal_native call with parse_decimal. This way, we will get e-notation support too.

What changes are included in this PR?

Are there any user-facing changes?

@himadripal himadripal force-pushed the fix_parse_decimal_for_rounding_scale_zero branch from 7e598c9 to bef2992 Compare February 27, 2025 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-change Changes to the arrow API arrow Changes to the arrow crate next-major-release the PR has API changes and it waiting on the next major version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DataFusion should support casting strings such as "4e7" to decimal
2 participants