Basic Arithmetic goes wrong
(546 * 1552.244) + 175917.984 = 10234432.07 wrong round((546 * 1552.244) + 175917.984) = 1023443 correct
#2 Updated by Philip Fearon over 8 years ago
- Status changed from New to Resolved
- Found in version set to Beta 0.2
This has now been resolved. The issue affected the conversion of Double to string values for numbers between 1.0E6 and 1.0E7 and was due to a disparity between GWT number handling in developer and production modes in this particular case.
#5 Updated by Manfred Staudinger over 8 years ago
- File svg-t5-011.svg svg-t5-011.svg added
- File svg-t5-011.xhtml svg-t5-011.xhtml added
- File svg-t5-011.xml svg-t5-011.xml added
- File svg-t5-browser-v2.xsl svg-t5-browser-v2.xsl added
- File svg-t5-browser-v2-test.xsl svg-t5-browser-v2-test.xsl added
Original test case was maybe too simple. If I click on no. 27 the error still shows up.
#7 Updated by Philip Fearon over 8 years ago
Almost there, but the transform won't run because we're missing the file: svg-t5-tab.xml (from the parent directory) - could you send this also please? I've also had to create a dummy 'html-hide.png' file - but hopefully this doesn't adversely affect the UI for the test.
#9 Updated by Philip Fearon over 8 years ago
Thanks, the XSLT runs fine now - this is the output (from Saxon-CE 1.0 in Chrome) I get when clicking 27:
INFO: wrong? 1.493757383E6 correct: 1493757
INFO: wrong: 1.023443208E6 correct: 1023443
The part you've marked wrong? is actually the expected result according to the XSLT 2.0 specification (the initial problem in beta 0.2 with the wrong value was with the order of magnitude being out by 1). In this case, the only difference between the 2 numbers is due to the effect of rounding. There is a difference in formatting but this is because the XSLT 2.0 formatting rules for xs:double requires the use of the exponent E for numbers greater than 1 million - whereas for xs:integer (which you get after rounding) the rule doesn't apply. If you want to format an xs:double another way you can use the format-number() function.
Is that Ok, are you seeing the same result as this - or am I missing something else?
Please register to edit this issue