This repository has been archived by the owner on Jan 12, 2024. It is now read-only.
Questions about the DoubleAsFixedPoint function running out of memory #669
Labels
Kind-Bug
Something isn't working
Describe the bug
There was something wrong when I tried to print the result of
DoubleAsFixedPoint.
After debugging, I found two errors in its source code.integerBits
orfractionalBits
is negative, it undergoes no validation, and logically, it will execute successfully if the sum ofintegerBits
andfractionalBits
is greater than0
. However, negative numbers are meaningless here. For example, the official documentation's description of integerBits states: "Assumed number of integer bits (including the sign bit)."QuantumLibraries/Numerics/src/FixedPoint/Convert.qs
Line 35 in 025a812
rescaledConstant
is too big to calculate.To Reproduce
Expected behavior
Add fact statements to the
FixedPointAsBoolArray
function to promptintegerBits
andfractionalBits
should be positive.The overflow problem with the
Floor
is traced to theTruncate
function.Actual behavior
System.OverflowException: “Arithmetic operation resulted in an overflow.”
System information
Additional context
We can submit a PR to fix this bug QuantumLibraries.
The text was updated successfully, but these errors were encountered: