Project

Profile

Help

Bug #5622

open

SaxonCS 11.4.0 - Division function generating 50 decimal places

Added by Vijay Bhushan 22 days ago. Updated 22 days ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
2022-07-29
Due date:
% Done:

0%

Estimated time:
Legacy ID:
Applies to branch:
11
Fix Committed on Branch:
11
Fixed in Maintenance Release:
Platforms:
.NET

Description

I am using C# API and I was comparing some transform result between Saxon9h and SaxonCS 11.4.0 where I found difference in decimal places. In new SaxonCS number of digits in scale are 50 where as in previous version Saxon9h was only 21 digits, refer below results.

Transform Result from Saxon9h : <Quote_ProcessedTime Val="194.45083333333333333333"/>

Transform Result from SaxonCS : <Quote_ProcessedTime Val="194.450833333333333333333333333333333333333333333333"/>

Can you please suggest has this been increase purposely or this needs to be fixed in next release? I have attached xslt for your reference.


Files

DivisionTest.xslt (542 Bytes) DivisionTest.xslt Vijay Bhushan, 2022-07-29 07:41
Actions #1

Updated by Michael Kay 22 days ago

Thanks for reporting it. SaxonCS is using a completely different software library to support decimal arithmetic; we didn't go out of our way to change the default precision for decimal division, but it's not surprising that it's changed, and the XPath specification doesn't require a specific number of digits. You can round the value to any precision you like using the round (or round-half-to-even) function, and there's also a Saxon extension function saxon:decimal-divide that gives a user-specified precision.

We'll think about changing the code to do what the documentation for saxon:decimal-divide says we do:

(The default for decimal division in Saxon using the div operator is to return max(18, s1, s2) decimal places in the result, where s1 is the scale of the first operand and s2 the scale of the second.)

Alternatively, we'll update the documentation. (In fact we need to do that anyway, because the Singulink library that we use in SaxonCS doesn't have the concept of the "scale" of a decimal value.)

Please register to edit this issue

Also available in: Atom PDF