Bug #5622
closed
SaxonCS 11.4.0 - Division function generating 50 decimal places
Applies to branch:
11, trunk
Fix Committed on Branch:
11, trunk
Fixed in Maintenance Release:
Description
I am using C# API and I was comparing some transform result between Saxon9h and SaxonCS 11.4.0 where I found difference in decimal places. In new SaxonCS number of digits in scale are 50 where as in previous version Saxon9h was only 21 digits, refer below results.
Transform Result from Saxon9h :
<Quote_ProcessedTime Val="194.45083333333333333333"/>
Transform Result from SaxonCS :
<Quote_ProcessedTime Val="194.450833333333333333333333333333333333333333333333"/>
Can you please suggest has this been increase purposely or this needs to be fixed in next release? I have attached xslt for your reference.
Files
Thanks for reporting it. SaxonCS is using a completely different software library to support decimal arithmetic; we didn't go out of our way to change the default precision for decimal division, but it's not surprising that it's changed, and the XPath specification doesn't require a specific number of digits. You can round the value to any precision you like using the round
(or round-half-to-even
) function, and there's also a Saxon extension function saxon:decimal-divide that gives a user-specified precision.
We'll think about changing the code to do what the documentation for saxon:decimal-divide says we do:
(The default for decimal division in Saxon using the div operator is to return max(18, s1, s2) decimal places in the result, where s1 is the scale of the first operand and s2 the scale of the second.)
Alternatively, we'll update the documentation. (In fact we need to do that anyway, because the Singulink library that we use in SaxonCS doesn't have the concept of the "scale" of a decimal value.)
- Category set to Internals
- Status changed from New to Resolved
- Assignee set to Michael Kay
- Priority changed from Low to Normal
- Applies to branch trunk added
- Fix Committed on Branch trunk added
I have changed the implementation of decimal division to use 18 digits of precision so we get the same results as SaxonJ.
- Fixed in Maintenance Release 12.0 added
- Fixed in Maintenance Release deleted (
11.4)
Bug issue fix applied in the Saxon 12.0 Major Release.
Leaving this bug marked as Resolved until fix applied on the Saxon 11 branch.
- Fixed in Maintenance Release 11.5 added
Bug applied in the Saxon 11.5 Maintenance release.
- Status changed from Resolved to Closed
Please register to edit this issue
Also available in: Atom
PDF