Notes on the Dojo XML Parser, by Dylan Schiemann

Introduction

Our goals for the parser are:

• provide a single, uniform data structure from all of the various supported markup encodings
• instantiate and provide properties (data and configuration directives) to components and widgets
• a second pass of the parser must provide deeper linking structures such as property sets shared among items, and event handling linkage

First Pass Parser

Overview

The Dojo Parser has two major pieces. The first piece is a generic XML Parser. Basically this will take any XML node or fragment, and parse it into a JavaScript hierarchical object structure. The idea is that the fastest way to parse XML with JavaScript is to convert the XML into an object tree as quickly as possible. Therefore, little or no analysis of the structure is done in this pass. For example, the following dojoml fragment:

	<dojo:button dataProvider="#buttonDataProvider">
		<dojo:image xlink:href="test.png">
	</dojo:button>

is parsed to:

	{"dojo:button"[0]:{
		dataProvider[0]: { value: "#buttonDataProvider"},
		"dojo:image"[0]: {
			xlink:href[0]: { value : "test.png"},
			tagName:"dojo:image",
			nodeRef:[object DojoImageElement]
		},
		tagName:"dojo:button",
		nodeRef:[object DojoButtonElement]
	}}  

Note that tagName and nodeRef are special values stored for convenience. Tags are normalized to try to eliminate inconsistencies across different environments.

Attributes (e.g. 'dataProvider') and child elements (e.g. 'dojo:image') are stored together in the array structure. In fact, any attribute may be provided as a child element, and any child element can be provided as an attribute (if it can be expressed in a single node). This structure should also facilitate round-tripping (i.e. regenerating markup from parser output) should someone define the same property as both an attribute and a child element.

A testcase is available for testing the first pass of the parser.

This portion of the parser also works with HTML and SVG documents (with dojo attributes for widget placeholders), and supports external property sets.

Using It

The package name for the first pass parser is dojo.xml.Parse . To use it, simply do the following:

	var testObjects = new dojo.xml.Parse();
	testItems = testObjects.parseElement(document.getElementById("test"));

We create a new instance of the Parser, and then provide a root node to parse. This pass parses attributes and elements into a JavaScript object structure. Because we are parsing arbitrary XML, there is no guarantee that anything is unique. Therefore, each member of the created structure is an array of objects (to store, for example, each instance of multiple attributes called 'dojoProvider'). This added an order of magnitude of complexity, but we (I?) feel it was a good design decision.

Second Pass Parser

Overview

The second pass parser is where the more interesting stuff happens. This pass takes the structure provided from the generic XML parser, and creates components/widgets (note that as a group, we're somewhat split on the use of the word component vs. widgets, so we're somewhat inconsistent in our naming). The second pass parser also normalizes properties and property sets into a more consumable structure. It is also possible to provide this step of the parser with a JavaScript object structure that mimics the first pass, allowing the initialization of components from script in addition to the normal markup way of doing things.

A testcase is available for testing the second pass of the parser.

Using It

The package name for the second pass parser is dojo.widget.Parse. Simply pass the return value from the first pass to the createComponents method.

	dojo.widget.getParser().createComponents(testItems);

So how does the parser know which tags correspond to widgets? There's simply an Array, dojo.widget.tags. I tend to append each item to this within my code for the widget. For example, in the code for our Button widget, we define a button as:

	dojo.widget.tags["dojo:button"] = function(fragment, widgetParser){
		dojo.widget.buildWidgetFromParseTree("dojo:button", fragment, widgetParser);
	}

Support was added in late April, 2005, to wrap this with a simpler API:

	dojo.widget.tags.addParseTreeHandler("dojo:button");

Note that the tagName is all lowercase for normalization issues with differing host environments.

For each widget that the parse finds, a call is made to the relevant Widget constructor code. The Widget constructor code then makes calls back to the parser to get properties and property sets. The current model for parsing property sets is to somewhat mimic the cascading mechanism found in CSS, without some of the complexity. This means that duplicate properties override each other, and that properties are taken in the order of global, then property sets defined for all widgets of a type, then for the property set and inline properties for the specific widget instance. Support for properties is currently limited to the most common cases, and does not include all of the cases detailed in requirements document

More Ways to Use It

Script Generated Widgets

As mentioned above, it is possible to create widgets directly from JavaScript using only the second pass parser. This is done by passing the expected JavaScript structure from the generic XML parser to the second pass parser. To simplify the process, there is a convenience method, createComponentFromScript, that will take a simple flat value:property JavaScript property structure and create the structure expected. A usage example involves defining the following file (note the () wrapping the object):

({
	label: 		"this is a test button!",
	onClick:	"alert(this.domNode.innerHTML);",
	onFoo:		"dojo.hostenv.println('onFoo');"
})

Then, using dojo.io.bind, you can load this file, have it eval-ed (hence the () wrapping the object above), and send it directly to the second pass parser.

	dojo.io.bind({
		url: "buttonProperties.js",
		load: function(type, js, evt){ 
			if(type=="load"){
				dojo.widget.fromScript("Button", js, document.body, "last");
			}
		},
		mimetype: "text/javascript"
	});

Data Providers

Work is in progress to define and implement our data provider mechanism (Data Provider IRC chat). Data providers serve several purposes for Dojo:

• abstract data independent of a widget instance (so, for example, data can be loaded before a widget is constructed, and retained after a widget is removed)
• provide APIs to tie the widget state, its properties, and/or its data to some dynamic data source (In mda's words, an example would be that "the width of this div should be equal to the temperature in alaska as obtained from some weather service"

Currently we have a very primitive data provider mechanism, where you can specify a dataProvider attribute in the widget declaration language of a JavaScript file on a remote server. This basically just grabs the contents of this file, evals it, and makes it available to your widget. A testcase is available.

An abstract data provider structure for a widget is very simple to implement. We do not currently have a finished API for this. In the interim, you can do something like this:

	dojo.provide("dojo.widget.custom.AbstractDataProvider");

	dojo.widget.custom.AbstractDataProvider = function(widgetRef, dataSource) {
		this.widgetRef = widgetRef;
		// add relevant code here to convert properties from the data source
		// and also add code that allows getting/setting of data
	}
}

And then in your widget code, you set a reference to the instance of the AbstractDataProvider, and get/set data inside the AbstractDataProvider. There are probably much better ways to go about this, but it works for me now until we get further along with this part of the toolkit.

Additional Information

• Requirements Documents

Revisions

• 1.0, Initial Article, April 27, 2005
• 1.1, Add updates to reflect getParser() and fromScript() convenience methods, July 6, 2005
• 1.2, Add revised, more concise fromScript() syntax, July 27, 2005. RAR