Project

Profile

Help

Bug #3545

Inconsistencies using map functions on JS objects

Added by Debbie Lockett 10 months ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Sprint/Milestone:
-
Start date:
2017-11-23
Due date:
% Done:

100%

Applies to JS Branch:
1.0, Trunk
Fix Committed on JS Branch:
1.0, Trunk
Fixed in JS Release:
SEF Generated with:
Company:
-
Contact person:
-
Additional contact persons:
-

Description

Within XPath, JavaScript objects are treated as XDM maps, allowing properties to be accessed using the XPath map functions. However it seems there are a number of problems with the way this is implemented.

Firstly, map:contains($obj, 'prop') returns true when 'prop' is NOT a property on the JS Object obj. It appears we are currently testing obj[prop] !== null, but this is wrong since obj[prop] is undefined (not null) if prop is not a property of obj!

Secondly, we are inconsistent about whether we include only direct properties of an object, or also the inherited properties (from the object's prototype chain; e.g. 'toString' is a property on all JS Objects):

  • currently map:get($obj, 'prop') returns any property on the JS object obj, direct or inherited, since internally we use obj[prop]

  • while map:keys($obj) returns only the direct properties of obj, since internally we use obj.hasOwnProperty

History

#1 Updated by Debbie Lockett 10 months ago

  • Status changed from New to In Progress

JS unit tests global/global08 and global/global09 added.

As well as the ownership of JS object properties (direct or inherited from prototype chain), we also need to consider their enumerability (properties are enumerable or nonenumerable, which determines whether they show up in for...in loops) - and be consistent. We have decided that the XDM map representation of a JS object will contain only its direct enumerable properties as keys.

Fix committed on 1.0 and trunk branches: methods 'containsKey' and 'get' on JSValue (wrapper for JS object) updated to be consistent with the 'keys' method (i.e. only direct enumerable properties are keys in the XDM map representation).

The documentation also needs to be updated to clarify this information (see https://www.saxonica.com/saxon-js/documentation/index.html#!ixsl-extension/conversions and https://www.saxonica.com/saxon-js/documentation/index.html#!ixsl-extension/functions/get).

#2 Updated by Debbie Lockett 10 months ago

Back to the drawing board.

As stated above, the current 'fix' means that 'the XDM map representation of a JS object will contain only its direct enumerable properties as keys.' But this means that you can no longer reliably use the map lookup operator as an alternative to ixsl:get() on JS objects, and this is something we want.

e.g. ixsl:event()?clientX, used when processing an onclick event so ixsl:event() is a MouseEvent, as in JS unit test ixsl2/setAttr02. This used to work, but with the new implementation it does not, because clientX is not a direct property of MouseEvent.

#3 Updated by Debbie Lockett 10 months ago

So really in the XDM map representation of a JS object (JSValue) we want to return to the original implementation for map:get()

i.e. map:get($obj, 'prop') returns any property on the JS object obj, direct or inherited, by internally using obj[prop]

and map:contains() should be fixed to correspond with the above

i.e. map:contains($obj, 'prop') returns true if and only if typeof obj[prop] !== "undefined"

The problematic map function is map:keys(). Actually, I'm thinking that this function should not be available for JSValue. To correspond with the above, map:keys() should return all properties (inherited and direct, enumerable and non-enumerable) of an object. But this would be awkward to implement since there is no easy way to do this - see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Enumerability_and_ownership_of_properties

#4 Updated by Debbie Lockett 10 months ago

Perhaps JSValue.keys could return only the direct enumerable properties (as originally), but then we have the problem that map:keys() and map:get() are inconsistent - i.e. $obj?prop can return a result even when 'prop' is not in map:keys($obj), as in the case that 'prop' is an inherited property. This doesn't seem sensible, it's not what you'd expect from an XDM map.

So far so good, but if keys is not available on JSValue, then forAllPairs and conforms will not work either.

Well when we are working with the XDM map representation of a JS object which originated as JavaScript (i.e. built in JS objects, or objects constructed using script elements in the HTML page), then I think this is probably fair enough (I don't think we'd expect to do things with the XDM map representation, that we couldn't do for the original JS object).

However, consider the case that we store an XDM map as a global JavaScript variable.

e.g. say we have a script element with:

var obj = null;
function setObj(map) {
   obj = map;
}
function getObj() {
   return obj;
}

And in our XSLT:

<xsl:variable name="map" select="map{'k1':'v1', 'k2':'v2'}"/>

<xsl:sequence select="js:setObj($map)"/>

I think it is reasonable to want to do the following:

<xsl:variable name="retrievedMap" select="js:getObj()" as="map(*)?"/>
<xsl:sequence select="map:keys($retrievedMap)"/>

But without JSValue.conforms implemented, we cannot do type tests (which definitely is a problem). And without JSValue.keys we can't use map:keys().

#5 Updated by Debbie Lockett 10 months ago

Apparently the problem is our conversion from XDM to JS and vice versa. Currently in convertToJS, an XDM map (which is represented internally as a HashTrie) is converted to a JS object by:

if (item instanceof HashTrie) {
   var object = {};
   item.forAllPairs(function (x) {
      object[x.k.toString()] = convertToJS(x.v);
   });
   return object;
}

But we should not be doing this! We should retain the HashTrie, but wrap it with a new wrapper (e.g. XDMValue).

Then in convertFromJS we should unwrap the XDMValue to return the original HashTrie, rather than producing a JSValue which wraps the object (which results in the problems with available map functions).

#6 Updated by Debbie Lockett 10 months ago

Note that these changes necessitate a rewrite of Browser.makeHttpRequest, and the code where this function is used (in the implementation of ixsl:schedule-action in Expr.js). We currently use the convertToJS and convertFromJS functions to convert the XDM map representation of the HTTP request to a pure JS object, and convert the generated response JS object to the returned XDM map representation. Such conversions will no longer be possible. So we should probably just deal with HashTries directly, rather than using JS literal objects to represent the request and response.

#7 Updated by Debbie Lockett 10 months ago

Further changes committed on 1.0 and trunk branches:

  • Only containsKey and get methods are available on JSValue, other methods produce errors.

  • Updated convertToJS and convertFromJS as suggested above (to wrap HashTries with XDMValue wrapper, and unwrap respectively)

  • Preliminary changes for Browser.makeHttpRequest so that it at least works, but should really rewrite to use HashTries directly

Note that supplying stylesheet parameters of type map(*) is now broken. Need to rethink what should actually be allowed, and update convertParamFromJS in Expr.js accordingly.

#8 Updated by Debbie Lockett 9 months ago

Fixed supplying stylesheet parameters of type map(*) again now. A new parameter 'objectToMap' is supplied to the convertFromJS function, which indicates whether or not JavaScript literal objects should be converted to XDM maps. In general, JS objects will just be wrapped in a JSValue; however when objectToMap is true, the object is actually converted to a HashTrie, with the key-value pairs converted accordingly. For now, objectToMap is only true when calling convertFromJS from convertParamFromJS (i.e. for conversion of stylesheet parameters).

Changes committed on 1.0 and trunk branches.

Still to do: update documentation https://www.saxonica.com/saxon-js/documentation/index.html#!ixsl-extension/conversions and https://www.saxonica.com/saxon-js/documentation/index.html#!api/transform/parameters

#9 Updated by Debbie Lockett 9 months ago

Like the 'objectToMap' parameter for the convertFromJS function, which forces conversion from a JS object to an XDM map (HashTrie); the 'mapConvert' parameter has been added for the convertToJS function, to force conversion from an XDM map (HashTrie) to a JS object.

Browser.makeHttpRequest has been updated to use these. Changes committed on 1.0 and trunk branches.

Various JS unit tests updated (global and paramsMap tests).

Still to do: documentation updates.

#10 Updated by Debbie Lockett 8 months ago

Still want to think about what happens when an XDM map is supplied in the array of arguments within an ixsl:call(). Currently this will be wrapped as an XDMValue, but it seems reasonable that really the JavaScript function called would expect a JS object.

#11 Updated by Debbie Lockett 6 months ago

Note 8 explains code changes to allow conversion for stylesheet parameters, with declared type @map()@. However, I don't think I've got this quite right. Currently, the 'objectToMap' parameter is set to true unconditionally in convertParamFromJS - and this means that JS objects not of a more specific recognised type (i.e. JS objects other than null, undefined, String, Number, Boolean, Date, Node, Array, or Function) are converted to HashTries. But I think it should actually only be true when the parameter has a declared type of something other than @item()@. Then within stylesheet parameters with declared type other than @item()*@, JS objects (not of a more specific recognised type) get converted to HashTries. But for stylesheet parameters with no declared type, such objects do not get converted to HashTries, and instead get wrapped as JSValues.

Ex 1. Supplying the JavaScript object {'x':1, 'y':2} for a stylesheet parameter with declared type @map(*)@, the object is converted to the XDM map @map{x":1, "y

Ex 2. Nested objects within stylesheet parameters with other specific declared types are also converted to XDM maps. e.g. supplying [['a string', {'x':1, 'y':2}]] for a stylesheet parameter with declared type @array(*)@, the object is converted to an XDM array containing an XDM map, @array{a string", map{"x":1, "y

Ex 3. Supplying the JavaScript object {'x':1, 'y':2} for a stylesheet parameter with no declared type, the object is converted to a JSValue object, (which behaves a bit like an XDM map, but isn't a true XDM map).

#12 Updated by Debbie Lockett 5 months ago

We should use the same conversion rules for template and function parameters, as for stylesheet parameters. In particular, JS objects are only converted to HashTries if the parameter has a declared type other than @item()*@.

Note that bug #3535 points out other type checking improvements required for function parameters. SEFs do not (currently) have the same generated type check functions for function parameters as for stylesheet parameters. But we can at least use the @as attribute to see if the declared type is something other than @item()*@, to determine how JS objects are converted.

#13 Updated by Debbie Lockett 5 months ago

Previous note suggested using the same conversion rules for template and function parameters, as for stylesheet parameters. i.e. JS objects are only converted to HashTries if the parameter has a declared type other than @item()*@.

However, this proved to be awkward. We have different compile time information, and the way the parameters are stored and used internally is different. Values for stylesheet parameters can only be supplied externally, and so are held in the unconverted JS form, and only converted when they are used. But this is not true for template parameters and function parameters: the values may be supplied externally for transformation invocation, in which case conversion is required; but more generally they are internal. (One thing which became awkward is the fact that we allow any object with a toString() method to be supplied for a stylesheet param with declared type xs:string; this was not straight forward to transfer across for template and function params.)

Instead, for template parameters and function parameters, how about we just use the standard JS to XDM conversion rules (whether or not there is a declared type), with "other" JS objects always converted to maps (i.e. with 'objectToMap' set to true).

Various JS unit tests added and amended.

Documentation updated (probably with more work to be done).

Code changes committed on Saxon-JS 1.x and 2.0 branches.

#14 Updated by Debbie Lockett 5 months ago

Re Note 10:

Still want to think about what happens when an XDM map is supplied in the array of arguments within an ixsl:call(). Currently this will be wrapped as an XDMValue, but it seems reasonable that really the JavaScript function called would expect a JS object.

We have stuck with XDM maps always being converted to wrapped XDMValue objects when they are passed to JS functions. It may not be obvious that the converted JS object will not just be a literal object (this should be made clear in the documentation), but since there is not a direct correspondence between XDM maps and JS objects, I think it's reasonable. If a user actually wants to work with JS objects, then it will be best to create and amend these in the JavaScript space, by calling JS global functions from the XSLT. The alternative is to work with XDM maps but provide your own conversion to JS (using a JS global function). But working with XDM maps on the XSLT side and using our internal conversion will no longer work.

#15 Updated by Debbie Lockett 5 months ago

  • Status changed from In Progress to Resolved
  • Fix Committed on JS Branch 1.0, Trunk added

Saxon-JS 1.1. documentation updated with the new rules for conversions (http://www.saxonica.com/saxon-js/documentation/index.html#!ixsl-extension/conversions and https://www.saxonica.com/test_website/saxon-js/documentation110/index.html#!api/transform/parameters). It has become a little complicated, but at least the rules should now be fully documented, with no ambiguities.

#16 Updated by Debbie Lockett 5 months ago

  • Status changed from Resolved to Closed
  • % Done changed from 0 to 100
  • Fixed in JS Release set to Saxon-JS 1.1.0

Bug fix applied in the Saxon-JS 1.1.0 maintenance release.

Please register to edit this issue

Also available in: Atom PDF Tracking page