Project

Profile

Help

Bug #5494

closed

Properties of attribute nodes have internal names which change from one release to another

Added by Jai B almost 2 years ago. Updated 11 months ago.

Status:
Closed
Priority:
Normal
Category:
API
Sprint/Milestone:
Start date:
2022-05-18
Due date:
% Done:

100%

Estimated time:
Applies to JS Branch:
Fix Committed on JS Branch:
Fixed in JS Release:
SEF Generated with:
Platforms:
Company:
Jai B
Contact person:
Jai B
Additional contact persons:
-

Description

Hi,

This very well may not be a bug but perhaps user error, but it appeared when upgrading to 2.4.0.

Have XSLT files that are used for templates for a website generating XHTML5 output.

We have some tags in them like, <ff:component src="" />, we have our namespace 'ff' set in the templates and also set with XPathOptions in order to make Saxon not complain.

After compiling the stylesheet and getting the result, we do SaxonJS.XPath.evaluate for the component elements and add them to an array of objects with the components name and the element itself.

On Saxon 2.3.0, when calling .next() on a returned element from evaluate, the node in the object returned was referenced as .kb, since 2.4.0 it is now referenced as .eb.

In the first place, I'm not sure why the names would be 'eb' or 'kb' versus something like 'childNode'.

I am passing .principalResult from SaxonJS.transform() on a sef file that was originally compiled from the XSLT sources; we cache the sef and only recompile if the XSLT has changed, while the input XML obviously would change more often.

		this.XPathOptions = {
			resultForm:'iterator',
			namespaceContext:{
				'xsl':'http://www.w3.org/1999/XSL/Transform',
				'ff':'http://www.shaped.ca/freeform'
			}
		};

...

		let ffComponentNodes = this.saxon.XPath.evaluate('//ff:component/@name',doc,this.XPathOptions);

		for (let x=ffComponentNodes.next();x!=null;x=ffComponentNodes.next()) {
			logger.log(util.inspect(x))
			ffComponents.push({
				name:x.value,
				node:x.eb // wtf? x.kb for saxon 2.3.0?? ughwtf
			});
		}

		for (let component of ffComponents) {
...
			component.node.parentNode.replaceChild(componentChunk.firstChild,component.node);
		}

I'm not sure if I'm doing something wrong, if this is an actual issue/bug but it only popped up when using 2.4.0 and jumping back to 2.3.0 resolved it; also, changing from using x.kb to x.eb also seemed to fix it.

Are these variables just the result of some sort of transpilation?

Also, I can't seem to find any information regarding this and I've searched pretty heavily. I know that JSON is an option for input in a way, although most of the references I can seem to find about this seem to imply that people are using an XSLT stylesheet to dictate how to translate JSON to XML or vice-versa.

What I want to do is a little different; I want to provide the input data for an XSLT template that would normally be XML, as JSON instead. My XSLT stylesheets are basically webpage templates with the XML input being the data to fill them out. Now that my system is all on node, I feel like I'm doing a needless step converting my data from JSON->XML before translating it, I'd like to be able to just use JSON as input but I haven't had any luck by setting sourceType: 'json' in the .transform({options})..? If this is possible is there somewhere I can find some better docs on it?

I feel like there's potentially more functionality available in SasonJS too that's not as well documented; I would love to see the documentation improved perhaps.

Previously my web platform used PHP and XSLT templates for just about everything; it seems time to perhaps use Node instead and SaxonJS seems to be my best bet for templates and so far it's mostly working great, although there's a few oddities I haven't figured out, I will likely post in the forum now that I'm signed up.


Related issues

Related to SaxonJS - Bug #5535: Improve documentation for exposed node propertiesNew2022-05-24

Actions
Actions #1

Updated by Martin Honnen almost 2 years ago

It is certainly possible to process JSON as the input of XSLT 3 but you have to understand the XPath 3.1 data model for the JSON, i.e. JSON objects are XDM 3.1 maps and JSON arrays are XDM 3.1 arrays. An example to process JSON would be e.g. JSON is

[
    { "cat" : "a", "values" : [1, 2] },
    { "cat" : "b", "values" : [5, 6, 7] },
    { "cat" : "a", "values" : [3, 4] },
    { "cat" : "b", "values" : [8, 9] },
    { "cat" : "c", "values" : [10, 11] }
]

XSLT 3 is e.g.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
        xmlns:xs="http://www.w3.org/2001/XMLSchema"
        exclude-result-prefixes="#all"
        version="3.0">

  <xsl:output method="json" indent="yes"/>

  <xsl:template match="." >
     <xsl:map>
       <xsl:for-each-group select="?*" group-by="?cat">
         <xsl:map-entry key="current-grouping-key()" select="array{ current-group()?values?* }"/>
       </xsl:for-each-group>
     </xsl:map>
  </xsl:template>

</xsl:stylesheet>

Use of xslt3 command line:

xslt3 -json:input.json -xsl:json-grouping.xsl
{
   "a": [
      1,
      2,
      3,
      4
   ],
   "b": [
      5,
      6,
      7,
      8,
      9
   ],
   "c": [
      10,
      11
   ]
}
Actions #2

Updated by Martin Honnen almost 2 years ago

As for your attempt to process attribute nodes selected with XPath with e.g. saxon.XPath.evaluate('//ff:component/@name', I think you have run into a quirk with both Saxon-JS 2.3 and 2.4 as well that you don't get an iterator over attribute nodes but an iterator over an internal representation whose properties unfortunately changed (randomly) through the compiler. It might be easier to select e.g. //ff:component, I think then you get an iterator over DOM element nodes and as you use Javascript anyway furtherone, can work with those nodes and the DOM API.

Actions #3

Updated by Michael Kay almost 2 years ago

The names .kb and .eb are allocated by the Closure compiler when we compile the SaxonJS source code - they are liable to change every time we build the product. The fact that you're using these short names essentially means you're accessing an internal method/interface that we didn't realise users should want or need. We need to look more closely at what you're doing here and whether we need to make something internal into a public/stable API.

Actions #4

Updated by Jai B almost 2 years ago

  • Company set to Jai B
  • Contact person set to Jai B

I'm basically just trying to replace an element, ff:component with the
output of a partial template.

Basically I load a main template file, default.xslt, which uses xsl imports
to load other needed templates and does what it needs to build the page.

When that gets spit out after transformation, I then take the ff:component
tags and replace them with the output of the component which is xhtml5 as
well (in this case also transformed from an xslt file but can be any output)

I have no preference to whether the tag is replaced after the initial
transformation or before but it seemed easier to do it after.

As for using the internal methods, that just seemed the only way to get it
done and I wasn't having much luck with the docs for js version, though I'm
not understanding why with your suggestion selecting it without referencing
the attribute would be any different, but I think I had tried without the
attribute in the XPath and had no luck getting it to work.

If you would like, I can try and post a clearer example of what I'm
doing/trying to do.

As for using JSON, your example looks reasonable, I was thinking I would
have to reference it slightly differently than xml input but that may still
be better than converting it. I haven't found anything yet that really does
json to xml the way I kind of expect although previously when I was doing
this with PHP I did have to write my own logic to convert objects to xml (I
swear everything wants to convert arrays to elements whos tagname is the
index of the array, which would be fine if your index wasn't an integer and
having a <0></0> tag doesn't fly with xml lol)

On Wed., May 18, 2022, 5:52 a.m. Saxonica Developer Community, <
> wrote:

Actions #5

Updated by Michael Kay almost 2 years ago

What property of the attribute node are you trying to retrieve here? It's not easy for us to work out which property has been renamed as ".eb".

And are you in Node.js or in the browser?

In the case of elements, Saxon exposes the nodes in its DOM implementation in the result of an XPath evaluation, and these will have public properties as defined in the DOM specification. When the XPath expression returns attribute nodes, we're returning instances of our own internal AttributeNode class, and it looks to me as if we're not exposing the properties of this object as public names. We should fix that.

In the meantime, there are a couple of possible workarounds. You could write your XPath expression so it returns elements rather than attributes (as suggested by Martin), and then use the DOM methods on the element node (Element.attributes) to get the attribute information. Or you could use the XPath expression name() applied with the attribute node as the context item to get the attribute's name.

Can I suggest you raise a new question about your JSON processing requirements? It seems an unrelated topic and it would be less confusing if they were treated separately. There's certainly plenty of ways Saxon can help you with this.

Actions #6

Updated by Michael Kay almost 2 years ago

  • Subject changed from New Issue with 2.4.0 to Properties of attribute nodes have internal names which change from one release to another
  • Status changed from New to In Progress
Actions #8

Updated by Norm Tovey-Walsh almost 2 years ago

  • Assignee set to Norm Tovey-Walsh

The property you're after is called 'parent' in our API where DOM Attr nodes would call it 'ownerElement'. Exposing the property named parent with a consistent name is straightforward. I think we're unlikely to try to change its name. While 'ownerElement' would be more consistent with the DOM, 'parent' is more consistent with XDM.

Actions #9

Updated by Norm Tovey-Walsh almost 2 years ago

  • Status changed from In Progress to Resolved

The parent property is now exposed as an extern so it will not be renamed.

Actions #10

Updated by Jai B almost 2 years ago

As I said, wasn't sure if was using wrong or actual issue (partly due to
lacking docs specifically for js version; but I get you're working on it
and I appreciate having it at all)

I will try and implement your suggestions next time I have a chance to go
into xsltHandler.js, sounds like I should be able to access what I need
w/out needing to access the compiled internals.

On Mon, May 23, 2022 at 3:49 AM Saxonica Developer Community <
> wrote:

Actions #11

Updated by Norm Tovey-Walsh almost 2 years ago

  • Related to Bug #5535: Improve documentation for exposed node properties added
Actions #12

Updated by Norm Tovey-Walsh over 1 year ago

  • Sprint/Milestone set to SaxonJS 2.5
Actions #13

Updated by Norm Tovey-Walsh over 1 year ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in JS Release set to SaxonJS 2.5

Fixed in SaxonJS 2.5.

Actions #14

Updated by Jai B over 1 year ago

Jai B wrote:

Hi,

This very well may not be a bug but perhaps user error, but it appeared when upgrading to 2.4.0.

Have XSLT files that are used for templates for a website generating XHTML5 output.

We have some tags in them like, <ff:component src="" />, we have our namespace 'ff' set in the templates and also set with XPathOptions in order to make Saxon not complain.

After compiling the stylesheet and getting the result, we do SaxonJS.XPath.evaluate for the component elements and add them to an array of objects with the components name and the element itself.

On Saxon 2.3.0, when calling .next() on a returned element from evaluate, the node in the object returned was referenced as .kb, since 2.4.0 it is now referenced as .eb.

In the first place, I'm not sure why the names would be 'eb' or 'kb' versus something like 'childNode'.

I am passing .principalResult from SaxonJS.transform() on a sef file that was originally compiled from the XSLT sources; we cache the sef and only recompile if the XSLT has changed, while the input XML obviously would change more often.

		this.XPathOptions = {
			resultForm:'iterator',
			namespaceContext:{
				'xsl':'http://www.w3.org/1999/XSL/Transform',
				'ff':'http://www.shaped.ca/freeform'
			}
		};

...

		let ffComponentNodes = this.saxon.XPath.evaluate('//ff:component/@name',doc,this.XPathOptions);

		for (let x=ffComponentNodes.next();x!=null;x=ffComponentNodes.next()) {
			logger.log(util.inspect(x))
			ffComponents.push({
				name:x.value,
				node:x.eb // wtf? x.kb for saxon 2.3.0?? ughwtf
			});
		}

		for (let component of ffComponents) {
...
			component.node.parentNode.replaceChild(componentChunk.firstChild,component.node);
		}

I'm not sure if I'm doing something wrong, if this is an actual issue/bug but it only popped up when using 2.4.0 and jumping back to 2.3.0 resolved it; also, changing from using x.kb to x.eb also seemed to fix it.

Are these variables just the result of some sort of transpilation?

Also, I can't seem to find any information regarding this and I've searched pretty heavily. I know that JSON is an option for input in a way, although most of the references I can seem to find about this seem to imply that people are using an XSLT stylesheet to dictate how to translate JSON to XML or vice-versa.

What I want to do is a little different; I want to provide the input data for an XSLT template that would normally be XML, as JSON instead. My XSLT stylesheets are basically webpage templates with the XML input being the data to fill them out. Now that my system is all on node, I feel like I'm doing a needless step converting my data from JSON->XML before translating it, I'd like to be able to just use JSON as input but I haven't had any luck by setting sourceType: 'json' in the .transform({options})..? If this is possible is there somewhere I can find some better docs on it?

I feel like there's potentially more functionality available in SasonJS too that's not as well documented; I would love to see the documentation improved perhaps.

Previously my web platform used PHP and XSLT templates for just about everything; it seems time to perhaps use Node instead and SaxonJS seems to be my best bet for templates and so far it's mostly working great, although there's a few oddities I haven't figured out, I will likely post in the forum now that I'm signed up.

Norm Tovey-Walsh wrote in #note-13:

Fixed in SaxonJS 2.5.

hm.. fixed in SaxonJS 2.4 maybe..?

I did an npm upgrade the other day, again almost forgetting about this issue, remembering, going to update the property name, checking my log and seeing that the property was now named parent instead of random letters, awesome!!

So regardless, wanted to come and say thanks; whether my usage unusual or otherwise, I have stuff that relies on this and it's one less thing that will arise anytime npm upgrade or new branch etc.

cheers!

/closed from OP's perspective as well :-)

Actions #15

Updated by Jai B 11 months ago

Martin Honnen wrote in #note-1:

It is certainly possible to process JSON as the input of XSLT 3 but you have to understand the XPath 3.1 data model for the JSON, i.e. JSON objects are XDM 3.1 maps and JSON arrays are XDM 3.1 arrays. An example to process JSON would be e.g. JSON is

[
    { "cat" : "a", "values" : [1, 2] },
    { "cat" : "b", "values" : [5, 6, 7] },
    { "cat" : "a", "values" : [3, 4] },
    { "cat" : "b", "values" : [8, 9] },
    { "cat" : "c", "values" : [10, 11] }
]

XSLT 3 is e.g.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
        xmlns:xs="http://www.w3.org/2001/XMLSchema"
        exclude-result-prefixes="#all"
        version="3.0">

  <xsl:output method="json" indent="yes"/>

  <xsl:template match="." >
     <xsl:map>
       <xsl:for-each-group select="?*" group-by="?cat">
         <xsl:map-entry key="current-grouping-key()" select="array{ current-group()?values?* }"/>
       </xsl:for-each-group>
     </xsl:map>
  </xsl:template>

</xsl:stylesheet>

Use of xslt3 command line:

xslt3 -json:input.json -xsl:json-grouping.xsl
{
   "a": [
      1,
      2,
      3,
      4
   ],
   "b": [
      5,
      6,
      7,
      8,
      9
   ],
   "c": [
      10,
      11
   ]
}

Hopefully I can still get a response here; I know this is OT from the original bug/issue (again, thanks for fixing!!) but since my question was sort of answered here...

I just looked at your example here again while digging into the docs as I'm updating some things..

Perhaps I should've clarified but while transforming JSON -> JSON is nice, my actual desire is to transform JSON (or even just JS objects) to xHTML5..

My XSL templates are for webpages / web output. Data comes from whatever source, database, etc and then gets applied with an XSL template to produce the final xHTML5 output.

This works fine but I have to convert any data into XML first before translation, which as I previously mentioned there's not many good options for this for Node let alone any that were built with this in mind and ages ago when my stuff was based off PHP I even built my own converter to convert PHP objects to appropriate XML for XSLT translation.

Considering the following XSLT:

<?xml version="1.0" encoding="utf-8" ?>
<xsl:stylesheet
	xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
	version="1.0">

<xsl:output
	method="xhtml"
	omit-xml-declaration="true"
	indent="yes"
	encoding="utf-8"
/>

<xsl:template 
	name="default"
	match="/">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
	<title><xsl:value-of select="/page/title" /></title>
</head>

<body>
</body>
</html>
</xsl:template>
</xsl:stylesheet>

Then I have a JS/JSON object like this:

{
  "page" : "title"
}

which I would then convert to XML like this:

<?xml version="1.0" encoding="utf-8" ?>
<page>
  <title>pageTitle</title>
</page>

to transform into something like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
	<title>pageTitle</title>
</head>

<body>
</body>
</html>

I want to skip the step of converting that JS/JSON object to XML first, thus using JS objects or JSON directly as input/data for XSLT to transform to XML (xHTML5) output.

I haven't been able to glean whether this is actually even possible; at first I assumed it was and that was why JSON was allowed as an input type but after re-reading your example I'm wondering if that's only for transforming JSON->JSON itself and not what I'm looking for?

If it is indeed possible then I at least am failing on knowing how / finding how to access the parameters properly; I suppose if it is possible that your example does give a bit of insight but I'm guessing using standard XPath in stuff like value-of="" isn't the way or at least quite the way (although I could see at least some XPath syntax being translatable enough); I see you've used ? in the for-each-group select and otherwise; is there somewhere the syntax is documented for this type of usage then?

Cheers!

Actions #16

Updated by Michael Kay 11 months ago

This doesn't seem related to the original issue. I'm reluctant to re-open the issue as that implies a bug has re-appeared, but if I leave it as closed, then this won't appear on anyone's action list. Could you please raise this as a new issue so that we can manage it properly?

Actions #17

Updated by Martin Honnen 11 months ago

The syntax is all "documented", of course, in the XPath 3.1 specification, see e.g. https://www.w3.org/TR/xpath-31/#id-maps and https://www.w3.org/TR/xpath-31/#id-arrays and https://www.w3.org/TR/xpath-31/#id-lookup.

As with any XSLT, the type of input (e.g. JSON) doesn't restrict you to the same type of output so it is perfectly fine and possible to construct HTML/XHTML output when processing JSON input.

Example fiddle using the lookup operator ? and "text value "templates.

Actions #18

Updated by Jai B 11 months ago

Edit: Did some testing today and made some progress.. At first I was getting some errors but with this JS object converted to JSON w/JSON.stringify()

{ test : "testValue" }

and this XSLT:

<xsl:value-of select="?test" />

I'm still not sure of the proper syntax to use for selecting and such; I only tried the ? as a guess because of your past provided example; I'm guessing it has something to do with XDF but if I can get some direction on docs for that I think I'll be good to go!

Actions #19

Updated by Jai B 11 months ago

Michael Kay wrote in #note-16:

This doesn't seem related to the original issue. I'm reluctant to re-open the issue as that implies a bug has re-appeared, but if I leave it as closed, then this won't appear on anyone's action list. Could you please raise this as a new issue so that we can manage it properly?

Thanks for the quick reply; was hoping not to bother anyone and definitely no need to re-open the issue (or any at all, this is a user needs to RTFM thing not an issue thing; I just don't know which FM to read :-) ) so definitely keep it closed (the original issue was/is solved and much appreciated)! As I said it's basically off-topic and unrelated but I had hoped I could get an answer here simply because of your previously provided example to my question regarding that.

The links you provided seem to be a start in the right direction; I haven't updated any of my knowledge past XSLT/XPath 1.0 as it suited my needs fine; if you have any other resources I'd be glad to look, reviewing the spec itself isn't always the easiest approach especially when not implementing the spec itself.

Otherwise the doc here: https://www.saxonica.com/saxon-js/documentation2/index.html#!xdm/conversions and the ones you provided should be a good start; thanks.

Actions #20

Updated by Jai B 11 months ago

edit: missed the fiddle when I first replied too; didn't know there was a saxon fiddle! super! this will be a big help.

Unless you have any other doc suggestions, I think I'm good to go on this; thanks again for your help - super happy to not have to convert to XML first anymore, even if my XPath will have to be slightly different which is not really a big deal as I was planning on updating and maybe taking advantage of some other features, perhaps using Saxon on the client side as well and some ixslt (although to be honest the 500kb package size is a bit heavy for my tastes but I was going to poke around with it anyway and see if it'll work for me)

thanks again!

Actions #21

Updated by Martin Honnen 11 months ago

XSLT 3.0 and XPath 3.1 and XQuery 3.1 share the same data model with XDM maps and arrays as representations for JSON objects and arrays and share the same operator and the same functions on maps and arrays. There is no book dedicated to XSLT 3.0 and XPath 3.1 but there are at least two books on XQuery 3.1 which have a chapter or two dedicated to processing JSON:

You might also want to check https://github.com/joewiz/learn-xquery with the entries for "Using XPath 3.1 (and its friends) to work with JSON" and "New in XQuery 3.1: Maps and arrays"

Actions #22

Updated by Jai B 11 months ago

Martin Honnen wrote in #note-21:

XSLT 3.0 and XPath 3.1 and XQuery 3.1 share the same data model with XDM maps and arrays as representations for JSON objects and arrays and share the same operator and the same functions on maps and arrays. There is no book dedicated to XSLT 3.0 and XPath 3.1 but there are at least two books on XQuery 3.1 which have a chapter or two dedicated to processing JSON:

You might also want to check https://github.com/joewiz/learn-xquery with the entries for "Using XPath 3.1 (and its friends) to work with JSON" and "New in XQuery 3.1: Maps and arrays"

Awesome. Thanks again for the quick responses, the fiddle and you/your team/whomever was responsible fixing the original bug too. Apologies again too for bumping this issue for semi-off topic questions, I only did because of the previous provided example but was happy to open a thread on the forum otherwise since this wasn't any actual issue/bug but I did get the answers I was looking for very quickly and it's been a productive morning, can't thank you enough.

The fiddle will definitely help a lot getting up to speed! I've already figured out probably 90% of what I wanted/need to drop the XML conversion step and am now just digging further to see what else I've missed since XSLT/XPath/XQuery 3 stuff arrived.

An 'auto-update' option that performs the transform after a small delay (like other fiddly things) would make it perfect.

Cheers! This thread can RIP now; if I have anything further I'll post on the forums/create a new issue as appropriate. Have a great weekend!!

Please register to edit this issue

Also available in: Atom PDF Tracking page