Skip to content

round decimal to float once - #123

Merged
alii merged 1 commit into
masterfrom
decimal-to-float
Sep 24, 2026
Merged

alii merged 1 commit into
masterfrom
decimal-to-float

Conversation

@alii

@alii alii commented Sep 23, 2026

Copy link
Copy Markdown
Member

decimal.to_float rounds once, in the VM, instead of turning each half into a Float first.

It used to be float.from_int(units) / float.from_int(10^scale). Both halves clamp at the largest Float, so once either passed it the answer was wrong: 10^400 at 399 places read as 1.0 instead of 10.0. Every operand of a big enough decimal hit it, and mul reaches those scales on its own.

Now a private float__from_decimal(units, scale) builds <units>e-<scale> and parses it, so Rust's own decimal-to-binary conversion rounds to the nearest Float, subnormals included, and Value::float keeps the no-NaN rule: too large is the largest Float, too small is 0.0.

Checked in the test: 10^400 at 399 places is 10.0; a negative one keeps its sign; two decimals past the largest Float both give the largest Float; 10^-400 is 0.0; and new(1, 1) is the same Float as the literal 0.1.

All seven CI gates pass locally. hawk ran for the host target, and nothing here is gated on a cfg.

@alii
alii merged commit 40f74f7 into master Sep 24, 2026
4 checks passed
@alii
alii deleted the decimal-to-float branch September 24, 2026 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant