Bug #6210
closedFormat of numbers generated by fn:json-to-xml
Added by fouad MOUTASSIM about 1 year ago. Updated 6 months ago.
0%
Description
Hello, We are using Saxonica 10.4 and we have a problem when using the function ** fn:json-to-xml.** ** We have numbers in the data. The sisue is we have scientific notation in the output xml (1.219496024E6 for example) Is there a way to keep the original number format without the scientific notation while using the fn:json-to-xml?
I saw a similar case with the function xml-to-json : https://saxonica.plan.io/issues/4860
Thank you in advance.
Files
payload.xml (6.04 KB) payload.xml | fouad MOUTASSIM, 2023-09-29 15:48 | ||
m-tbw-ws00x-filter-business-keys.xslt (49.6 KB) m-tbw-ws00x-filter-business-keys.xslt | fouad MOUTASSIM, 2023-09-29 15:48 | ||
Saxon_case.png (69.3 KB) Saxon_case.png | fouad MOUTASSIM, 2023-09-29 15:58 |
Updated by Michael Kay about 1 year ago
- Status changed from New to In Progress
The specification of json-to-xml() says that when the JSON contains a number, the lexical representation of the number is retained in the XML. This appears to be working correctly in Saxon 10, 11, and 12 - the XML will only contain a number in exponential notation if it was already in exponential notation in the JSON.
Could you clarify exactly what you are doing and what output it produces?
Updated by Michael Kay about 1 year ago
If I do this from the command line:
java net.sf.saxon.Query -qs:"json-to-xml('1234567890')" -t
the output is
<?xml version="1.0" encoding="UTF-8"?>
<number xmlns="http://www.w3.org/2005/xpath-functions">1234567890</number>
Updated by fouad MOUTASSIM about 1 year ago
- File m-tbw-ws00x-filter-business-keys.xslt m-tbw-ws00x-filter-business-keys.xslt added
- File payload.xml payload.xml added
Hi Sorry for the confusion.
Today after further investigation we got the root cause issue.
we are using "map{'number-formatter': function($x){$x}}" as parameter in the xml-to-json() function to avoid having the scientific notation in the output as you advised us in 2020 and it works well.
but last month we opened a case (Support #6190) for slash escaping and you adiviced us to add "=> parse-json() => serialize(map{ 'method' : 'json', 'use-character-maps' : map { '/' : '/' } })" afer the xml-to-json() function to ignore the slash escape. it works well but it seems that it canceled the first fix regarding the scientific notation.
please find below the used functions and the results:
fn:xml-to-json(fwkmapping:document/document:content/document:xml/*, map{'number-formatter': function($x){$x}})
"ShipViaId":"BESSON/LTL" "EstimatedVolume":1219496.024
fn:xml-to-json(fwkmapping:document/document:content/document:xml/*, map{'number-formatter': function($x){$x}}) => parse-json() => serialize(map{ 'method' : 'json', 'use-character-maps' : map { '/' : '/' } })
"ShipViaId":"BESSON/LTL" "EstimatedVolume":1.219496024E6
Please find attached the payload and the xslt map.
Thank you in advance.
Best Regards. Fouad
Updated by fouad MOUTASSIM about 1 year ago
- File Saxon_case.png Saxon_case.png added
please find a screenshot with actual results and desired result.
Updated by fouad MOUTASSIM about 1 year ago
Hello,
Could you please provide us an update regarding this issue.
Thank you in advance.
Best regards.
Updated by Norm Tovey-Walsh about 1 year ago
Indeed, running the output of xml-to-json
through parse-json
does "undo" the number formatting fix.
The JSON requirement to escape forward slashes is a frequent source of awkwardness.
I'm not immediately seeing a workaround that will accomplish both fixes, but I'll make sure we discuss it and see what we can come up with.
Updated by fouad MOUTASSIM about 1 year ago
Thank you for your response.
we will wait for updates from your side.
Updated by fouad MOUTASSIM about 1 year ago
Hello,
Any news regarding the issue ?
Best regards.
Updated by Vladimir Nesterovsky about 1 year ago
I would not rely for json serializer to escape or unescape values. Formally, it is the same data. E.g. we often deal with Hebrew and some json serializers decide to escape Hebrew text, some - not. It is pointless to appeal here.
Basically same story is with xmls, which in addition may reorder attributes.
Updated by fouad MOUTASSIM about 1 year ago
Thanks for the explanation. What do you suggest to solve the problem and get the desired result.
Updated by Vladimir Nesterovsky about 1 year ago
I would suggest undoing the part
-
serialize(map{ 'method' : 'json', 'use-character-maps' : map { '/' : '/' } })
that you intended to use to prevent escaping of'/'
; - treat result string as json, which would mean that
"\/"
and"/"
are the same values.
Updated by Michael Kay about 1 year ago
I'm afraid I can't offer any workaround with the current product that allows you both to control the number formatting, and to avoid escaping of "/".
In the 4.0 specs we have added an option escape-solidus="no"
to the JSON serialization method (so you no longer need the workaround with character maps), and at https://github.com/qt4cg/qtspecs/issues/530 I have proposed we should also add this option to the xml-to-json() function.
I'm also going to propose that we add an option for number formatting to JSON serialization.
Updated by Michael Kay about 1 year ago
- Tracker changed from Bug to Feature
Reclasifying this as a feature request. There is no bug -- Saxon is behaving correctly according to the specification.
Updated by Michael Kay about 1 year ago
After attempting to specify a new number-format serialization property, I've come to the conclusion that it gets terribly complicated, and probably won't meet the requirements, because you actually want different formats for different numbers in the document.
The JSON serialization method already says that the processor can emit any format that's legal in JSON. So I think the simplest thing to do is to change the Saxon implementation to use exponential notation only in extreme circumstances, for example if the value to be output is an xs:double
less than 1e-20 or greater than 1e20.
While looking into this, it also occurred to me that there is another workaround you could use: before serializing the output, transform it so that all the xs:double
values are converted to xs:decimal
. Saxon will not use exponential notation for xs:decimal
values.
Updated by Michael Kay about 1 year ago
- Tracker changed from Feature to Bug
In investigating further I have found a case that IS a bug: the JSON serialization method does not correctly handle integer values outside the range of a 64-bit long. Added test case serialize-json-136
Updated by Michael Kay about 1 year ago
- Category set to JSON
- Status changed from In Progress to Resolved
- Priority changed from High to Normal
- Applies to branch 11, 12, trunk added
- Fix Committed on Branch 12, trunk added
- Fix Committed on Branch deleted (
10) - Platforms .NET added
Patch for integers greater than a long applied to 11.x, 12.x, and main branches.
On 12.x and main branches, changed the formatting of floats and doubles so exponential notation is only used outside the range 1e-18 to 1e18.
Updated by Michael Kay about 1 year ago
- Status changed from Resolved to In Progress
Reopening.
The new rules are generating unreasonable output, such as
"price":8.049999999999979394260662957094609737396240234375
Updated by Michael Kay about 1 year ago
- Status changed from In Progress to Resolved
Previous patch amended.
Updated by O'Neil Delpratt about 1 year ago
- Fixed in Maintenance Release 12.4 added
Bug fix applied in the Saxon 12.4 Maintenance release. Leaving it marked as 'Resolved' until fix applied on Saxon 11.
Please register to edit this issue