Saxon

It seems I saved the files at https://github.com/Saxonica/Prague2016

Good luck with the project and please share the result. The main problem with this kind of exercise is that the results are likely to be very dependent on the precise workload you decide to analyse. For example, my experiments with the XX compiler showed that (for that workload) performance is very sensitive to the number of in-scope namespaces; it would be very easy to miss that this is one of the variables determining the outcome, because of course there are many others.

Than you!

I've cloned that repository and created a folder xml-vs-json.

I started my tests from preparation of xml and json data both for courses and prices.
So, there are:

• create-courses.xsl
• create-prices.xsl

that generate datasets for 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000 records. See input.

In addition I've added create-input.bat to run those stylesheets.

convert-courses-via-maps.xsl is left unchanged, while convert-courses-via-xml.xsl is changed to deal purely with xml.
Same is for convert-prices-via-maps.xsl vs convert-prices-via-xml.xsl.

I have added courses-map.bat, courses-xml.bat, prices-map.bat, prices-xml.bat. Those batch files run transformations for all different inputs.

I've run all transformations using saxon-he-11.3 for all inputs, and have found:

• convert-prices-via-maps.xsl runs into stack overflow for big inputs;
• initial impression is that json is still slower than xml.

I'm going to review and optimize stylesheets to avoid stack overflow and to try to make them faster, and then I'm going to compose final timing report.

I have committed some fixes and optimizations, and excel with comparison charts. Here I attach it as well.

Interesting.

I'm not very suprised that the courses inversion takes longer with JSON, given that is has to make a flattened copy of the data in its initial pass. I wonder if there's a way of avoiding that?

It's a little surprising that the XML version increases so little between 50K and 100K records. I wonder how much of the cost is parsing? I think if you supply the document as the value of a global parameter then it will only be parsed once and excluded from the measurement. But that's probably harder to achieve with the JSON version. Another way to get the parsing cost is to parse the document twice, but caching makes it a bit tricky.

In the convert-prices stylesheet, what proportion of the records are being updated? This one really should be quite efficient for both JSON and XML and that appears to be the case. A significant difference is that in the XML case, it doesn't build the result document in memory (the results are piped directly into the serializer as they are constructed), whereas in the JSON case, we're going to build the result structure in memory and then pass it to the serializer.

An observation: we have two internal implementations of maps, the Dictionary implementation which is optimised for read-only maps with string-valued keys, and the HashTrie implementation which allows any kind of key and supports update. Maps created using json-doc() or parse-json() are always Dictionary maps, which means each map that gets updated is going to be copied to a HashTrie map first. Probably not a big deal, but it means we're probably not choosing the optimum implementation for the workload. I don't think there's currently any switch to configure this.

In the convert-prices stylesheet, what proportion of the records are being updated?

I defined 11 tags with only one that is updated. Generated documents use up to half of tags. fn:random-number-generator() is used to produce tag permutations.

If I'm not wrong this means it's close to 10% of cases to update.

It's a little surprising that the XML version increases so little between 50K and 100K records. I wonder how much of the cost is parsing?

I checked this and have found my typos when I refactored convert-courses-via-xml.xsl. Plimary one was that it used: <xsl:output method="text"/> as in original stylesheet.

When I changed it to method="xml" things have changed.

Attached is updated timing.

So, remarkably close results!

Yes. Also this typo clearly shows that with increase of size of data IO dominates over implementation technique.

You might also be interested in

https://dev.saxonica.com/blog/mike/2022/05/xml-vs-json.html

Thanks.
I've can to similar conclusions that you point in your article.
In particular that format (xml or json) is not the argument deciding about performance.
Difference won't be dramatic.

In contrast IO should be scrutinized a lot.

E.g. we got big speed up once we implemented Saxon resolvers that know to read and write directly into ZIP.
See zipfilesystemprovider.html

There are certainly plenty of workloads where parsing and serialization costs are an order of magnitude higher than the transformation cost. We tend to neglect this because there's not much we can do to improve them.

Please watch out for my paper at Balisage 2022 which revisits this area.