Observables, uiElement Objects, and Variable Tracking

After last time’s observables primer, we’re ready to jump back into some related features of the UI Component system and uiElement based objects.

There’s no getting past it — this is an advanced tutorial. We’ll do our best to point you in the right direction, but you’ll need to have a grounding in Magento’s Advanced Javascriptfeatures as well as the UI Component system. If this doesn’t make sense it’s not because you’re not intelligent, it’s just because it’s new, confusing, and the systems we’re discussing aren’t fully baked.

We’ll run all our code examples in the javascript console (View -> Developer -> Javascript Console in Google Chrome) of a loaded Magento 2 page. Our examples will also load modules via the requirejs function — normally we would load these modules as part of a define or require dependency list. Finally, the specifics here were tested against Magento 2.1.1, but the concepts should apply across all Magento 2 versions.

Using Observables as Object Properties

We’re going to start with a common pattern Magento uses with uiElement derived objects. First, let’s create a uiElement derived object.

//get the constructor function
UiElement = requirejs('uiElement');

//create the object
ourViewModel = new UiElement;

Our object created, give the following a try

UiElement = requirejs('uiElement');    
ourViewModel = new UiElement;

//get the knockout library object
ko = requirejs('ko');

//create a "observable object" and assign it to the `title` 
//property of our object
ourViewModel.title = ko.observable('Default Title');

//view the value of the observable object
console.log(ourViewModel.title());    

//set a new value for the observable object
ourViewModel.title('A new Title');

//view the new value of the observable object
console.log(ourViewModel.title());        

This is pretty standard observable stuff. If a Knockout template accesses the title value via a data-binding, any later call to our observable’s “setter” will trigger a UI re-render.

Magento’s uiElement derived objects (including the important uiComponent anduiCollection constructor functions) make heavy use of ko.observables as data properties. This makes sense, as these objects are the view models Magento’s customscope binding uses to access data from template. Magento is so fond of observables, that there’s some extra observables features baked into uiElement derived objects.

Subscriber Wrapper

Give the following a try

ourViewModel.on('title', function(value){
    console.log("Someone just set the title to: " + value);    
});

ourViewModel.title("A Third Title")

You should see the output

Someone just set the title to: A Third Title

displayed on your screen. The on method is a special event handling method that everyuiElement has (and comes from the Magento_Ui/js/lib/core/events module). As in “on this event (the first argument), call this function (the second argument)”.

If you pass (as the first argument, title above) the name of an object property, and that property contains an observable, the uiElement class will automatically setup a Knockout.js subscriber for that observable. If you’re not familiar with what a subscriber is, you may want to read our Knockout Observables for Javascript Programmers article.

Tracking any Property

The uiElement derived objects also have the ability to automatically setup any property for tracking. In order to take advantage of this feature, we’ll need to create a new constructor function from uiElement (i.e. create a new UI Component class withuiElement as its ancestor). Give the following a try

UiElement = requirejs('uiElement');

//define a new constructor function based on uiElement
OurClass = UiElement.extend({
    defaults:{
        tracks:{
            title: true,
            foo: true
        }
    }
});    

//create a new object
ourViewModel = new OurClass;

//set a default value
ourViewModel.title = 'a default value';

//setup a callback
ourViewModel.on('title', function(value){
    console.log("Another callback: " + value);
});

//set the value (notice the normal javascript assignment)
//and you should see the "Another callback" output
ourViewModel.title = 'a new value';

If you run the above program, you should see your callback function (fromourViewModel.on) called.

While this is similar to using an observable, there are a few differences. The first difference is the tracks parameter in the defaults object (if you’re not familiar with defaults, you should review the UiComponent series, in particular The uiClass Data Features article)

OurClass = UiElement.extend({
    defaults:{
        tracks:{
            title: true,
            foo: true
        }
    }
});    

The tracks object is a set of key/value pairs. The key is the object property name you want to track, the value is a boolean true. This must be an actual boolean, not just a truth-y value. When you configure a tracks default, you’re telling Magento

Hey, if someone uses this constructor function to create an object, make the following properties (title and foo above) trackable

The other difference? If you look at how we’re setting a value

ourViewModel.title = 'a new value';

you’ll see we’re using a regular assignment instead of calling an observable function.

The tracks default is a neat, and powerful, feature. Once the novelty wears off, you may be left wondering how this even works.

The knockout-es5 Library

To talk about the tracks feature, we’ll need to talk about Steve Sanderson’s knockout-es5 plugin. The knockout-es5 plugin adds support for ECMAScript 5 properties. Magento includes the knockout-es5 plug as part of its front end enviornment.

This plugin adds a ko.track function that lets you track properties on an object withoutsetting up observables for each property. Or, to be more accurate, without manually setup observers. When you call track, the plugin will automatically create observables for the object’s properties, and swap these observables in as ES5 setters and getters. The theory is this offers a cleaner syntax than the sometimes awkward observable objects.

Behind the scenes, the uiElement‘s tracks feature uses knockout-es5 to implement its tracking. If you take a look at our object in a debugger, we’ll see that title has both a setter, and a getter, and that these are an observable function/object.

Screenshot of setters/getters from javascript console

Update: This article previously contained a warning against directly using the trackfunction that powers the tracks default.

track: function (properties) {
    this.observe(true, properties);

    return this;
},

This warning was off-base. The problem I was seeing with track was not a bug in theuiElement object, but my being forgetful about observables only firing when a valuechanges. i.e.

object.foo = 'Hello';
object.track('foo');
object.on('foo', function(value){
    console.log("Fired!");
});

//won't fire, because same value
object.foo = "Hello";

//will fire
object.foo = "Hello Again"; 

So, rather than a cautionary tale about systems level APIs being public, instead this should serve as a warning about the difficulties in trusting an undocumented system.

Tracked Variables are Observables. Mostly.

Because they’re hidden behind setters and getters, and not part of the official Knockout.js API (i.e. they come from a plugin and custom Magento code), these tracked variables can sometimes produce strange results. For example, they’ll fail a Knockout isObservable test

> ko.isObservable(ourViewModel.title);
false

But you can still use Knockout’s getObservable method to fetch the underlying observable object

observableFunctionObject = ko.getObservable(ourViewModel, 'title')
console.log(observableFunctionObject());

Also, in case it’s not obvious, if you data-bind (via Knockout.js) a DOM node to atracked variable, Knockout.js will treat the DOM node as an observable and re-render your UI whenever the variable is updated.

Wrap Up

That’s it for our whirlwind tour of the uiElement observable event/subscriber on method and related variable tracking features. These two competing approaches (properties as observables vs. knockout-es5 tracked variables) are yet another sign that point to Magento’s javascript frameworks being rushed out the door in a not fully baked way.

We don’t have any clear recommendations on “the right” way to develop your own javascript based UIs in Magento 2 — but regardless of your own approach, you’ll need to be aware that Magento’s core code uses both, and plan your debugging accordingly.

Magento 2: UI Component Retrospective

Today we’re going to wrap up our UI Component series with some odds, ends, and thoughts on how we got here. Let’s kick things off with some blatant self promotion.

Commerce Bug 3.2

Pulse Storm has just released Commerce Bug 3.2. This is relevant to us because 3.2 brings with it UI Component debugging features — check out the screencast for more information

It’s features like these that make Commerce Bug 3 the world’s best Magento developer toolbar. When Magento 2 was first released a number of free competitor toolbars popped up, but none of those have kept the pace with subsequent Magento 2 releases, and none of them provide debugging features for Magento 2’s newer systems. Even a powerhouse like Zend’s Z-ray can’t keep up, with no stand-alone support for PHP 7 and zero support for modern Magento front end features. We know you probably didn’t chose Magento as your platform, but once you have a copy of Commerce Bug installed you’ll be cutting through Magento’s red tape like a hot knife through butter.

If marketing hyperbole isn’t your thing, remember that revenue from Commerce Bug, along with Pulse Storm’s other products, is what allows me to spend time digging into Magento 2’s features as deeply as I do. If I’ve ever bailed you out of a jam please consider a purchase, or if you don’t think any of Pulse Storm’s products are for you please consider donating to my patreon.

Chrome Knockout Debugger

Flipping PT Barnum mode off, another tool that’s useful for debugging UI Components isGoogle Chrome’s Knockout.js context debugger.

This tool will let you debug a specific node’s Knockout.js context — i.e. you’ll be able to see the data the view has access to. While this tool doesn’t have access to Magento’s deep view models tree, it’s often useful to quickly peek at data without adding a <pre data-bind="text: ko.toJSON($data, null, 2)"></pre> to your nodes.

UI Component Critique

Even with these tools, the UI Component system can be difficult to get your head around, or even identify as a single system. You have

  1. A new PHP XHTML template layer
  2. A domain specific language (DSL) for creating JSON in a particular format
  3. A RequireJS application for instantiating and registering objects from that particular format
  4. An extension to Knockout.js to use those models as multiple view models on a single page
  5. An extension to Knockout.js to allow remote template loading
  6. An extension to Knockout.js to replace the Knockout.js “tag-less” syntax with XHTML based tags and attributes

While Magento’s backend UI grids and forms make full use of this tenuously threaded together UI Component system, even the Magento core front end development team have abandoned/ignored large parts of this system. For example, the “mini-cart” application appears on most Magento frontend pages. Here’s part of its UI Component tree as viewed via Commerce Bug 3

image of UI Component debugger

While the mini-cart is a Magento_Ui/js/core/app application,

<a class="action showcart" href="http://magento-2-1-1.dev/checkout/cart/"
   data-bind="scope: 'minicart_content'">
    <span class="text">My Cart</span>
    <span class="counter qty empty"
          data-bind="css: { empty: !!getCartParam('summary_count') == false }, blockLoader: isLoading">
        <span class="counter-number"><!-- ko text: getCartParam('summary_count') --><!-- /ko --></span>
        <span class="counter-label">
        <!-- ko if: getCartParam('summary_count') -->
            <!-- ko text: getCartParam('summary_count') --><!-- /ko -->
            <!-- ko i18n: 'items' --><!-- /ko -->
        <!-- /ko -->
        </span>
    </span>
</a>
<!-- ... -->
<script type="text/x-magento-init">
{
    "[data-block='minicart']": {
        "Magento_Ui/js/core/app": {
            "components": {
                "minicart_content": {
                    "children": {
                        "subtotal.container": {
                            "children": {
                                "subtotal": {
                                    "children": {
                                        "subtotal.totals": {
                                            "config": {
                                                "display_cart_subtotal_incl_tax": 0,
                                                "display_cart_subtotal_excl_tax": 1,
                                                "template": "Magento_Tax\/checkout\/minicart\/subtotal\/totals"
                                            },
                                            "children": {
                                                "subtotal.totals.msrp": {
                                                    "component": "Magento_Msrp\/js\/view\/checkout\/minicart\/subtotal\/totals",
                                                    "config": {
                                                        "displayArea": "minicart-subtotal-hidden",
                                                        "template": "Magento_Msrp\/checkout\/minicart\/subtotal\/totals"
                                                    }
                                                }
                                            },
                                            "component": "Magento_Tax\/js\/view\/checkout\/minicart\/subtotal\/totals"
                                        }
                                    },
                                    "component": "uiComponent",
                                    "config": {
                                        "template": "Magento_Checkout\/minicart\/subtotal"
                                    }
                                }
                            },
                            "component": "uiComponent",
                            "config": {
                                "displayArea": "subtotalContainer"
                            }
                        },
                        "item.renderer": {
                            "component": "uiComponent",
                            "config": {
                                "displayArea": "defaultRenderer",
                                "template": "Magento_Checkout\/minicart\/item\/default"
                            },
                            "children": {
                                "item.image": {
                                    "component": "Magento_Catalog\/js\/view\/image",
                                    "config": {
                                        "template": "Magento_Catalog\/product\/image",
                                        "displayArea": "itemImage"
                                    }
                                },
                                "checkout.cart.item.price.sidebar": {
                                    "component": "uiComponent",
                                    "config": {
                                        "template": "Magento_Checkout\/minicart\/item\/price",
                                        "displayArea": "priceSidebar"
                                    }
                                }
                            }
                        },
                        "extra_info": {
                            "component": "uiComponent",
                            "config": {
                                "displayArea": "extraInfo"
                            }
                        },
                        "promotion": {
                            "component": "uiComponent",
                            "config": {
                                "displayArea": "promotion"
                            }
                        }
                    },
                    "config": {
                        "itemRenderer": {
                            "default": "defaultRenderer",
                            "simple": "defaultRenderer",
                            "virtual": "defaultRenderer"
                        },
                        "template": "Magento_Checkout\/minicart\/content"
                    },
                    "component": "Magento_Checkout\/js\/view\/minicart"
                }
            },
            "types": []
        }
    },
    "*": {
        "Magento_Ui/js/block-loader": "http://magento-2-1-1.dev/pub/static/frontend/Magento/luma/en_US/images/loader-1.gif"
    }
}
</script>

it does not use the XML based UI Component DSL to generate its JSON, nor does it use the XHTML based PHP template system to output its initial nodes. It also targets a specificCSS selector in its x-magento-init script

"[data-block='minicart']":  {
    "Magento_Ui/js/core/app": {/*...*/}
}

and invokes a completely separate program via the more-familiar-to-us * syntax

"*": {
        "Magento_Ui/js/block-loader": "http://magento-2-1-1.dev/pub/static/frontend/Magento/luma/en_US/images/loader-1.gif"
    }

On the backend the mini-cart code is rendered by a good old fashioned Magento block, created via a layout handle XML file.

#File: vendor/magento/module-checkout/view/frontend/layout/default.xml
<referenceContainer name="header-wrapper">
    <block class="Magento\Checkout\Block\Cart\Sidebar" name="minicart" as="minicart" after="logo" template="cart/minicart.phtml">
        <arguments>
            <argument name="jsLayout" xsi:type="array">
                <item name="types" xsi:type="array"/>
                <item name="components" xsi:type="array">
                    <item name="minicart_content" xsi:type="array">
                        <item name="component" xsi:type="string">Magento_Checkout/js/view/minicart</item>
                        <item name="config" xsi:type="array">
                            <item name="template" xsi:type="string">Magento_Checkout/minicart/content</item>
                        </item>
                        <item name="children" xsi:type="array">
                            <item name="subtotal.container" xsi:type="array">
                                <item name="component" xsi:type="string">uiComponent</item>
                                <item name="config" xsi:type="array">
                                    <item name="displayArea" xsi:type="string">subtotalContainer</item>
                                </item>
                                <item name="children" xsi:type="array">
                                    <item name="subtotal" xsi:type="array">
                                        <item name="component" xsi:type="string">uiComponent</item>
                                        <item name="config" xsi:type="array">
                                            <item name="template" xsi:type="string">Magento_Checkout/minicart/subtotal</item>
                                        </item>
                                    </item>
                                </item>
                            </item>
                            <item name="extra_info" xsi:type="array">
                                <item name="component" xsi:type="string">uiComponent</item>
                                <item name="config" xsi:type="array">
                                    <item name="displayArea" xsi:type="string">extraInfo</item>
                                </item>
                            </item>
                            <item name="promotion" xsi:type="array">
                                <item name="component" xsi:type="string">uiComponent</item>
                                <item name="config" xsi:type="array">
                                    <item name="displayArea" xsi:type="string">promotion</item>
                                </item>
                            </item>
                        </item>
                    </item>
                </item>
            </argument>
        </arguments>
        <container name="minicart.addons" label="Mini-cart promotion block"/>
    </block>
</referenceContainer>

The entire x-magento-init JSON block is configured via <argument> nodes, and then rendered in the phtml template via the block’s getJsLayout method.

<script type="text/x-magento-init">
{
    "[data-block='minicart']": {
        "Magento_Ui/js/core/app": <?php /* @escapeNotVerified */ echo $block->getJsLayout();?>
    },
    "*": {
        "Magento_Ui/js/block-loader": "<?php /* @escapeNotVerified */ echo $block->getViewFileUrl('images/loader-1.gif'); ?>"
    }
}
</script>

Other frontend UI Components (checkout, messages, etc.) are similarly divergent. So — given all this: Are these true UI Components? They certainly use some of the UI Component system’s RequireJS/Knockout.js code, but with a completely separate method for rendering that code on the server side.

Did these front end XML-less UI Components come first and then Magento’s core developers created a DSL to help manage the complexity of those nodes?

Or did the DSL come first, and the above is an example of front end developers taking only what they needed to get their jobs done?

We say all this not in condemnation of Magento’s new front end systems — this sort of thing happens all the time in software and is part of the process of building out brand new systems. It does leave a working Magento developer with some puzzling choices on how to proceed with their own code, and some real head scratching as to why a system that’s clearly in a rough beta phase is being marketed as production ready.

Javascript Critique

Part of the issue here may be a simultaneously ambitious but impossibly vague goal around Magento 2 embracing our brave new javascript world. In this, Magento isn’t alone. Thanks to the ubiquity of javascript in all major web browsers, the tremendous efforts put forth by companies like Google, Apple, and Microsoft, etc. in optimizing the performance of the Javascript runtime, and the community of open source developers that have coalesced behind all this, it’s hard to be involved in a software project that doesn’t use javascript in some way, shape, or form. It makes sense that, in 2016, one of Magento 2’s goals should be embracing this new world.

The problem, as is all too often the case, is you can’t just glop technology on top of a project an expect it to work like magic fairy dust. Javascript is popular for a number of different, overlapping, but often incompatible reasons.

For example, javascript is popular with agency-style businesses because a javascript based user interface is often powered by a “server-less” HTTP based API. The nature and structure of agencies mean they’re often pretty bad at maintaining server side, business logic oriented software, and much better at building out user/customer driven interfaces built on top of APIs that have already solved a particular business problem.

Implicit in the above scenario, javascript/API based user interfaces are often popular with traditional software companies because they allow engineers to focus on the problem of scaling out their high demand systems and adjusting to changes in business logic withoutgetting bogged down in the world of UI/UX. All an engineering team needs to do is expose and support a fully featured HTTP based API and let folks build whatever sort of UI/UX they’d like. Both jobs are important, but separating them out like this can reduce a lot of friction between teams with different skill sets.

Finally, javascript is also popular with smaller startups because of the efficiencies gained by having your server side code and your front end code written in the same language.

Magento 2, unfortunately, doesn’t address any of the above use cases in a compelling or complete way. The UI Component/x-magento-init system we’ve explored allows you to embed javascript application inside a HTML page, but much of Magento 2 still relies on PHP’s tried and true server rendered HTML, using javascript to enhance page functionality.

Modernizing a classic PHP based software system isn’t an easy task, but it is possible and it does have a precedent. Consider WordPress/Automattic, in some ways the grandfather of all these PHP based software systems. Automattic have never used their influence with the WordPress community to force the PHP based WordPress into adopting modern javascript patterns wholesale. While doing so may have turned WordPress into something interesting (like the Ghost blogging platform), leaving behind so much working code and so many community members wasn’t worth it.

Instead, Automattic created Calypso, a ground up re-imagining of WordPress.com as a javascript based application. The only way this directly impacted the existing WordPress user base was the WordPress PHP application (and many WordPress plugins) got a new, modern, RESTFul API system.

Magento’s approach, on the other hand, feels a bit like a plate of mush — there’s good ideas in there, but when combined the systems feel rickety, confusing, or both. Word out of Magento Inc. is the 2.2 release will bring myriad improvements to “developer experience” (i.e. stuff might work), but in the meantime working Magento developers will need to make do with the incomplete systems available to them.

Practical Solutions

So, what’s a Magento developer to do? My advice is to tread lightly. If you’re building a Magento extension, copying the UI Component DSL for grids and simple forms seems like a safe bet, but otherwise I’d stay away from UI Component XML. Beyond the burden of boilerplate and fighting Magento’s XSDs, the XML based DSL is likely to change in random and unpredictable ways. It seems better to stick to tried and true server rendered HTML enhanced with a single level x-magento-init/Magento_Ui/js/core/appprogram.

The same applies to system integrators, but if you’re the one building and maintaining a specific Magento instance I’d say it’s safe to also use RequireJS’s selective “monkey patching” abilities to modify the behavior of core javascript objects. Keep an eye on Magento Quickies for an upcoming article on how to do this with Magento’s specialrequirejs-config.js files. While an extension developer can do this, too many extension developers monkey patching the same module means someone’s functionality ends up losing. If you’re in a position to take responsibility for your changes, this RequireJS monkey patching is often more stable that the wholesale file replacement Magento system integrators fall back on.

In general, while you’ll need to understand Magento’s new javascript systems to work with and analyze the platform, be careful when embracing them for your own functionality. Hopefully, as Magento 1’s end of life looms closer and closer, Magento’s engineering teams will be able to get these new and incomplete systems under control. Until then, the old ways often remain the best ways. Remember that, at the end of the day, our job isn’t to solve technology problems, it’s to solve problems using technology. The less time we spend on the former, the more we’ll have for the later.

Magento 2: UI Component Data Sources

Today we’ll discuss how Magento 2’s UI Component system gets configuration and data source data into a javascript and Knockout.js context. i.e. How is something configured via PHP, but used in the browser.

This article will focus on the Customer Listing grid. While we’ve experimented with creating XSD valid UI Components, it turns out our reliance on the root <container/>node makes Magento ignore the inner dataSource nodes when rendering the x-magento-init scripts. We’re going to focus on the Customer Listing grid, and hope that Magento 2.2 loosens its grip a little w/r/t to ui_component XML.

Also, if it’s not obvious from the above jargon, this article borrows heavily from concepts introduced in earlier series articles. While not absolutely required reading, if you get stuck you may want to start from the beginning. The specifics here are Magento 2.1.x, but the concepts should apply across versions.

The Magento_Ui/js/core/app Application

The Magento_Ui/js/core/app application is the program that handles instantiating Magento’s view models, and registering them in the uiRegistry. This meansMagento_Ui/js/core/app is also the place where configuration data gets moved from the XML/x-magento-init script, and into our view models.

//File: vendor/magento//module-ui/view/base/web/js/core/app.js
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
define([
    './renderer/types',
    './renderer/layout',
    '../lib/knockout/bootstrap'
], function (types, layout) {
    'use strict';

    return function (data, merge) {
        types.set(data.types);
        layout(data.components, undefined, true, merge);
    };
});

This program is deceptively simple. There’s two function calls — the first, (a call to the setmethod on the Magento_Ui/js/core/renderer/types module object) is a simple registry/dictionary that’s used to serialize data from the types attribute of the rendered x-magento-init JSON.

//embeded in the HTML page
{
    "*": {
        "Magento_Ui/js/core/app": {
            "types": {/* handled by Magento_Ui/js/core/renderer/types */},
            "components": { /*...*/}
        }
    }
}

This is not a straight serialization of the data. You can peek at an individual entry in the registry with code like this

//try this in the javascript console on the customer grid listing page
typesReg = requirejs('Magento_Ui/js/core/renderer/types');
console.log(
    typesReg.get('text');
);

console.log(
    typesReg.get('customer_listing');
);

However, unlike the uiRegistry, there’s no way to view everything that’s stored in this types registry.

The majority of the work in Magento_Ui/js/core/app happens in the function returned by the Magento_Ui/js/core/renderer/layout module.Magento_Ui/js/core/renderer/layout is a — complicated — module. This module appears to be responsible for instantiating Knockout.js view model objects and registering them in the uiRegistry. It does this by using data from the components attributes of the rendered x-magento-init script.

//embeded in the HTML page    
{
    "*": {
        "Magento_Ui/js/core/app": {
            "types": { /*...*/},
            "components": { /* handled by Magento_Ui/js/core/renderer/layout */}
        }
    }
}

The javascript patterns used in Magento_Ui/js/core/renderer/layout are a little unorthodox (when compared to other modules).

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
define([
    'underscore',
    'jquery',
    'mageUtils',
    'uiRegistry',
    './types'
], function (_, $, utils, registry, types) {
    'use strict';

    var templates = registry.create(),
        layout = {},
        cachedConfig = {};

    function getNodeName(parent, node, name) {/*...*/}

    function getNodeType(parent, node) {/*...*/}

    function getDataScope(parent, node) {/*...*/}

    function loadDeps(node) {/*...*/}

    function loadSource(node) {/*...*/}

    function initComponent(node, Constr) {/*...*/}

    function run(nodes, parent, cached, merge) {/*...*/}

    _.extend(layout, /*... methods and properties ...*/);

    _.extend(layout, /*... methods and properties ...*/);

    _.extend(layout, /*... methods and properties ...*/);

    return run;
});      

The first thing you’ll notice about this module is, it returns a function (run). Most Magento 2 RequireJS modules return an object literal, an object created with .extend, or an object created with a javascript constructor function/new. The run function serves as a main entry point for the program, which (mostly) operates on the locally defined layout object.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
function run(nodes, parent, cached, merge) {
    if (_.isBoolean(merge) && merge) {
        layout.merge(nodes);

        return false;
    }

    if (cached) {
        cachedConfig[_.keys(nodes)[0]] = JSON.parse(JSON.stringify(nodes));
    }

    _.each(nodes || [], layout.iterator.bind(layout, parent));
}

The layout object is the second weird thing about this file. You’ll see Magento uses the underscore JS library to extend a raw object literal with methods.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
var /*...*/ layout = {}, /*...*/;
/*...*/
_.extend(layout, /*... methods and properties ...*/);

_.extend(layout, /*... methods and properties ...*/);

_.extend(layout, /*... methods and properties ...*/);    

There’s nothing odd about this in and of itself — but doing this in three separate statements seems a little off, and suggests a series of developers working on this module, each afraid to touch each other’s code.

Magento kremlinology aside, the run method kicks off processing with the following

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
_.each(nodes || [], layout.iterator.bind(layout, parent));

Magento uses underscore.js’s each method to run through each of the nodes from x-magento-init‘s components attribute, and call that node’s iterator method (usingjavascript’s bind method to call iterator). This module liberally uses bind, as well asapply, to call methods.

Running through the full execution chain of this module is beyond the scope of this article, and would probably require an entire new series. Instead we’re going to highlight a few important sections.

View Model Instantiation

The first part of layout.js that’s worth highlighting is the instantiation of our view models and their registration in the uiRegistry. All this happens here.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
function initComponent(node, Constr) {
    var component = new Constr(_.omit(node, 'children'));
    registry.set(node.name, component);
}

Of course, you’re probably wondering what the node and Constr parameters are, and where they come from.

Regarding “what they are”, the node variable is an object that contains values from thecomponents property of Magento’s x-magento-init array, and the Constr variable is a view model constructor object created via a view model constructor factory.

Regarding “where they come from”, that brings us to the other parts of layout.js worth highlighting. The data in the node object starts with the aforementioned x-magenti-initobject, but it really comes to life in the build method.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
build: function (parent, node, name) {
    var defaults    = parent && parent.childDefaults || {},
        children    = node.children,
        type        = getNodeType(parent, node),
        dataScope   = getDataScope(parent, node),
        nodeName;

    node.children = false;

    node = utils.extend({
    }, types.get(type), defaults, node);

    nodeName = getNodeName(parent, node, name);

    _.extend(node, node.config || {}, {
        index: node.name || name,
        name: nodeName,
        dataScope: dataScope,
        parentName: utils.getPart(nodeName, -2),
        parentScope: utils.getPart(dataScope, -2)
    });

    node.children = children;

    delete node.type;
    delete node.config;

    if (children) {
        node.initChildCount = _.size(children);
    }

    if (node.isTemplate) {
        node.isTemplate = false;

        templates.set(node.name, node);

        return false;
    }

    if (node.componentDisabled === true) {
        return false;
    }

    return node;
},

The component data from x-magento-init is a series of nested objects, and each parent/child object will make its way through this method. There’s a lot of default value setting in this method. This is also where Magento uses the values from theMagento_Ui/js/core/renderer/types registry to add values to node.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
node = utils.extend({
}, types.get(type), defaults, node);

You’ll also notice layout.js looks at the node’s parent node to set default values. Magento’s core engineers manipulate this node object throughout layout.js, but buildseems to be where the majority of the work happens.

The third area worth highlighting is the loadSource function. This is where Magento loads the view model constructor factory (i.e. the RequireJS model configured in component) to get a view model constructor object. The constr below is the Constr we pass in toinitComponent.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js

function loadSource(node) {
    var loaded = $.Deferred(),
        source = node.component;

    require([source], function (constr) {
        loaded.resolve(node, constr);
    });

    return loaded.promise();
}

The loaded = $.Deferred(), loaded.resolve(node, constr), andloaded.promise() objects are part of the jQuery promises API. They don’t have anything to do with how Magento uses the view model constructor factory — they’re here as a consequence of the way Magento calls the loadSource function.

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js

loadDeps(node)
    .then(loadSource)
    .done(initComponent);

Unfortunately, jQuery’s promises library and Magento’s use of the same is beyond the scope of what we can cover today.

Speaking broadly, the use of promises, the nested/recursive nature of parsing thecomponents data, and some use of promise-ish like features in the registry (in methods likewaitParent and waitTemplate) make reading through this module a bit more involved than your average code safari. That said, if you remember the ultimate job here is to get a view model constructor loaded, and a view model instantiated and registered, the above three methods should be enough to help you when it comes time to debug.

Practical Debugging Tips

Next, we’ll run through a few tips for those times when you need to trace down a piece of data from the backend to the view model/Knockout.js level.

When Magento encounters a bit of UI Component layout XML like this

<!-- File: vendor/magento/module-customer/view/adminhtml/ui_component/customer_listing.xml -->

<bookmark name="bookmarks">
    <argument name="data" xsi:type="array">
        <item name="config" xsi:type="array">
            <item name="storageConfig" xsi:type="array">
                <item name="namespace" xsi:type="string">customer_listing</item>
            </item>
        </item>
    </argument>
</bookmark>

Magento merges in the data from definition.xml

<!-- vendor/magento//module-ui/view/base/ui_component/etc/definition.xml -->
<bookmark class="Magento\Ui\Component\Bookmark">
    <argument name="data" xsi:type="array">
        <item name="config" xsi:type="array">
            <item name="component" xsi:type="string">Magento_Ui/js/grid/controls/bookmarks/bookmarks</item>
            <item name="displayArea" xsi:type="string">dataGridActions</item>
            <item name="storageConfig" xsi:type="array">
                <item name="saveUrl" xsi:type="url" path="mui/bookmark/save"/>
                <item name="deleteUrl" xsi:type="url" path="mui/bookmark/delete"/>
            </item>
        </item>
    </argument>
</bookmark>

and serializes the merged node out to the x-magento-init JSON/javascript. This data looks something like this

"bookmark": {
    "extends": "customer_listing",
    "current": {/* ... */},
    "activeIndex": "default",
    "views": {/* ... */},
    "component": "Magento_Ui\/js\/grid\/controls\/bookmarks\/bookmarks",
    "displayArea": "dataGridActions",
    "storageConfig": {/* ... */}
},

You’ll notice the storageConfig, component, and displayArea properties from the XML configuration are represented in the JSON. You’ll also notice some other properties (views,current, activeIndex) are not represented in the XML. Remember that the UI Component’s PHP class (in this case, Magento\Ui\Component\Bookmark) can render anyJSON here. You may need to look at your rendered x-magento-init source rather than the XML files when you’re looking for a value.

When layout.js gets a hold of this node, it will load component (in this case,Magento_Ui/js/grid/controls/bookmarks/bookmarks) via RequireJS. We’ve been calling components view model constructor factories to better describe their role in the entire process. This component module returns our view model constructor. This view model constructor factory’s defaults looks something like this

//File: vendor/magento/module-ui/view/base/web/js/grid/controls/bookmarks/bookmarks.js

return Collection.extend({
    defaults: {
        template: 'ui/grid/controls/bookmarks/bookmarks',
        viewTmpl: 'ui/grid/controls/bookmarks/view',
        newViewLabel: $t('New View'),
        defaultIndex: 'default',
        activeIndex: 'default',
        viewsArray: [],
        storageConfig: {
            provider: '${ $.storageConfig.name }',
            name: '${ $.name }_storage',
            component: 'Magento_Ui/js/grid/controls/bookmarks/storage'
        },

You’ll remember from the UI Class data features article that the defaults array of a view model constructor controls default data values in an instantiated view model object. Lacking other context, the values set above will carry through to the instantiated view model object.

However, you’ll want to keep view model instantiation in mind as well. i.e. we don’t lack other context!

#File: vendor/magento/module-ui/view/base/web/js/core/renderer/layout.js
function initComponent(node, Constr) {
    var component = new Constr(_.omit(node, 'children'));
    registry.set(node.name, component);
}

When Magento instantiates the view model, Magento passes in an array of data to the constructor (node above — after using the omit method of the underscore.js library to remove the children property). This means values set via x-magento-init (which comes from the ui_component XML object’s config data property) will override the view model constructor’s defaults.

Data Source Data

If you’re not too dizzy, we have one last thing to talk about. The values we discussed above are a view model’s configuration values. There is, however, another source of data for backend UI Components. The data from the dataSource nodes.

<!-- File: vendor/magento/module-customer/view/adminhtml/ui_component/customer_listing.xml -->
<dataSource name="customer_listing_data_source">
    <argument name="dataProvider" xsi:type="configurableObject">
        <argument name="class" xsi:type="string">Magento\Customer\Ui\Component\DataProvider</argument>
        <argument name="name" xsi:type="string">customer_listing_data_source</argument>
        <argument name="primaryFieldName" xsi:type="string">entity_id</argument>
        <argument name="requestFieldName" xsi:type="string">id</argument>
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="component" xsi:type="string">Magento_Ui/js/grid/provider</item>
                <item name="update_url" xsi:type="url" path="mui/index/render"/>
            </item>
        </argument>
    </argument>
</dataSource>

You’ll remember from our Simplest UI Component that the dataSource node (when used inside a listing node) with the above configuration is responsible for the extra data node in the UI Component’s x-magento-init data

"customer_listing_data_source": {
    /* ... */
    "config": {
        "data": {
            "items": [/*... JSON representation of Grid Data ...*/],
            "totalRecords": 1
        },
        "component": "Magento_Ui\/js\/grid\/provider",
        "update_url": "http:\/\/magento-2-1-1.dev\/admin\/mui\/index\/render\/key\/a9a9aa9e29b8727c7eaa781ffc7e2619e0cf12b2cf36f7458d93ae6e2412e50b\/",
        "params": {
            "namespace": "customer_listing"
        }
    }
    /* ... */        
}

Despite this extra super power, a dataSource node is just another UI Component that will end up in the uiRegistry. Its view model constructor factory isMagento_Ui/js/grid/provider. The Magento_Ui/js/grid/provider view model factory returns a view model constructor with the following defaults

//File: vendor/magento//module-ui/view/base/web/js/grid/provider.js
return Element.extend({
    defaults: {
        firstLoad: true,
        storageConfig: {
            component: 'Magento_Ui/js/grid/data-storage',
            provider: '${ $.storageConfig.name }',
            name: '${ $.name }_storage',
            updateUrl: '${ $.update_url }'
        },
        listens: {
            params: 'onParamsChange',
            requestConfig: 'updateRequestConfig'
        }
    },
    //...

Accessing data source data directly view the uiRegistry is pretty straight forward. If you’re on a grid listing page, give the following a try in your javascript console.

requirejs(['uiRegistry'], function(reg){
    dataSource = reg.get('customer_listing.customer_listing_data_source')
    console.log(dataSource.data.items);    
});

Assuming you have customers registered in your system, you should see output something like the following

0:Object
    _rowIndex:0
    actions:Object
    billing_city:"Calder"
    billing_company:null
    billing_country_id:Array[1]
    billing_fax:null
    billing_firstname:"Veronica"
    billing_full:"6146 Honey Bluff Parkway Calder Michigan 49628-7978"
    billing_lastname:"Costello"
    billing_postcode:"49628-7978"
    billing_region:"Michigan"
    billing_street:"6146 Honey Bluff Parkway"
    billing_telephone:"(555) 229-3326"
    billing_vat_id:null
    confirmation:"Confirmation Not Required"
    created_at:"2016-09-01 10:01:38"
    created_in:"Default Store View"
    dob:"1973-12-15 00:00:00"
    email:"roni_cost@example.com"
    entity_id:"1"
    gender:Array[1]
    group_id:Array[1]
    id_field_name:"entity_id"
    lock_expires:"Unlocked"
    name:"Veronica Costello"
    //...    

If you need to access data source data via javascript, this is the most direct way to do it. However, this presents a problem for Magento’s “one view model, one Knockout.js template” front end model. If you look at a Knockout.js view

<!-- File: vendor/magento/module-ui/view/base/web/templates/grid/listing.html -->
<div class="admin__data-grid-wrap" data-role="grid-wrapper">
    <table class="data-grid" data-role="grid">
       <thead>
            <tr each="data: getVisible(), as: '$col'" render="getHeader()"/>
        </thead>
        <tbody>
            <tr class="data-row" repeat="foreach: rows, item: '$row'" css="'_odd-row': $index % 2">
                <td outerfasteach="data: getVisible(), as: '$col'"
                    css="getFieldClass($row())" click="getFieldHandler($row())" template="getBody()"/>
            </tr>
            <tr ifnot="hasData()" class="data-grid-tr-no-data">
                <td attr="colspan: countVisible()" translate="'We couldn\'t find any records.'"/>
            </tr>
        </tbody>
    </table>
</div>

You’ll see the view model gets at the data via a rows property, foreached over here

<!-- File: vendor/magento/module-ui/view/base/web/templates/grid/listing.html -->    
<tr class="data-row" repeat="foreach: rows, item: '$row'" css="'_odd-row': $index % 2">

To understand where this rows property comes from, you’ll need to look at the Knockout.js template’s view model constructor.

//File: vendor/magento/module-ui/view/base/web/js/grid/listing.js    
return Collection.extend({
    defaults: {
        //...
        imports: {
            rows: '${ $.provider }:data.items'
        },
        //...
    },     

The Magento_Ui/js/grid/listing view model uses the imports feature to load a rowsproperty. We learned about imports in the UiClass Data Features article. The template literal (or template literal-ish thing, also covered in our ES6 Template Literals article) above expands to customer_listing.customer_listing_data_source, which means the values for rows will be imported by javascript that looks something like this

requirejs('uiRegistry') .
    get('customer_lising.customer_listing_data_source') .
    data .
    items

While indirect and hard to follow, this mechanism does allow a Knockout.js template to have easier access to this central store of data. Only time will tell if this pattern makes it easier, or harder, for Magento developers trying to create and maintain their expert system user interface elements.

Wrap Up

That brings us to the end of our article, and the near-end of our UI Components series. Next time we’ll reflect on where the UI Components system is at, how Magento’s core developers are coping with it, and some parting tips for surviving forays in to Magento’s front end systems.

Magento 2: uiClass Data Features

Our last article covered Magento’s implementation of ES6 template literals. Near its end, we started to bump up against some features (the defaults array) of Magento’s UI Component focused classical-style object system. Before we can get to UI Component data sources, we’ll need to slog through a few more object system features.

We’ll be running all of today’s code via a browser’s (Google Chrome’s) javascript debugging console/REPL. Also, the specifics here are Magento 2.1.1, but the concept should apply across Magento versions.

The Story so Far

To start with, let’s review what we’ve learned about Magento 2’s front end systems. If any of the following sounds unfamiliar, you may want to review our Advanced Javascriptseries, as well as the previous articles in this UI Component series.

  • Magento uses Knockout.js to render front end interfaces.
  • By default, Knockout.js uses the entire HTML page as a view, and a javascript object for the view model.
  • To use Knockout.js, programmers create a javascript constructor function. Knockout.js uses this constructor function to instantiate a view model.
  • Magento added a custom Knockout.js binding named scope. Scope allows different areas of the page to use different view models.
  • These different view models are instantiated by the Magento_Ui/js/core/appRequireJS module. This module is embedded in the page via an x-magento-initscript, and this module uses a large data structure rendered via the backend’s UI Component classes and XML. This data structure configures a number of RequireJS modules called “components”.
  • These components are “view model constructor factories”. TheMagento_Ui/js/core/app application uses these RequireJS components to create view model constructors, and then uses the instantiated view model constructor function to instantiate view models. Finally, the Magento_Ui/js/core/appapplication registers each instantiated view model object, (by name), with theuiRegistry RequireJS module. The view model’s registered names come from the large data structure rendered via the backend’s UI Component classes and XML.
  • The view model constructor objects are based on a Magento built javascript object system. This object system uses RequireJS modules as classes, and usesunderscore.js for inheritance and method/property sharing. There’s also some Magento secret sauce in there. The base class is the uiClass module. TheuiElement class/module extends from the uiClass class/module. Most of Magento’s Knockout.js view model constructors are uiElement objects.
  • Above, we mentioned the uiRegistry, uiClass, and uiElement RequireJS modules. These are RequireJS “map aliases” that point to the real modules atMagento_Ui/js/lib/registry/registry, Magento_Ui/js/lib/core/class, and Magento_Ui/js/lib/core/element/element respectively.

Today we’re going to look at some features of the uiElement based objects.

Javascript History

For some of you (or at least, for myself) the first thing you’ll find intimidating about Magento’s object system is the use of javascript constructor functions.

In many ways, the dawn of the modern javascript era started with Douglas Crockford’s JavaScript: The Good Parts. This slim volume laid javascript’s lispy heart bare, and helped the world see the language as more than a janky scripting language. Or, depending on your point of view, it dressed up a janky scripting language as a tool for software engineers.

Either way, it’s the volume a lot of people look to when they’re trying to learn how to program javascript.

One of Crockford’s peccadillos was an aversion to the new keyword in javascript. He preferred developers create new objects directly with the object literal syntax

var foo = {};

rather than using constructor functions.

var FooConstructor = function(){
    this.message = "Hello World";
};

var foo = new FooConstructor;    

His reasons were myriad (you should really read The Good Parts – it’s a good book), but the primary one was the ambiguity of javascript constructors. A javascript constructor function is just a regular function. It becomes a constructor when used with the newkeyword – however, it’s still possible to call a constructor function without this keyword. This usually results in javascript just chugging along and your program doing something you likely did not intend. Further complicating things – when you use a constructor function with the new keyword, the magic variable “this” is bound to the object the constructor function is creating. i.e. in our above examples, the variable foo will have amessage property with the string "Hello World" inside. If invoked as a regular function,this is (per standard javascript) bound to the function itself.

This ambiguity creates a class of bugs that are hard to track down, which is why Crockford recommended eschewing javascript constructors and sticking to object literals, factories, and his module pattern. Much of the early “modern javascript” movement followed his lead.

There are, however, programers who think “don’t do that” is bad advice, and started experimenting with javascript constructor functions. There are also programmers who never heard of Douglas Crockford. As time went on, more frameworks started eschewing his advice, and javascript constructor functions remained a thing.

The Knockout.js framework’s view models are based on end-user-programers creating view model constructor functions, and Magento’s object system follows that lead.

Creating uiElement Objects

History lesson out of the way, we’re going to start by instantiating a Magento uiElementobject. Browse to a Magento 2 page, open your browser’s debugging console/REPL, and type the following

var Element = requirejs('uiElement');

Here we’re using the RequireJS shorthand to load a module directly into the current namespace. Normally you would use this inside a RequireJS program or module definition

define(['uiElement'], function(Element){
    //... use Element here ...
});    

What we’ve done is load the uiElement module. The Element variable is our javascript constructor function (i.e. view model constructor). To instantiate a javascript object from this constructor, we’d do the following.

var viewModel = new Element;
console.log(viewModel);

UiClass {_super: undefined, ignoreTmpls: Object, _requesetd: Object, containers: Array[0], exports: Object…}

You may wonder why this object looks like a UiClass object to javascript. That’s beyond the scope of this article, but if you’re curious you can start debugging in uiElement‘s source file.

#File: vendor/magento//module-ui/view/base/web/js/lib/core/element/element.js

At this point, we now have a uiElement object that’s ready to use as a simple view model. This object also has a number of default properties and methods that Magento’s Knockout.js scope binding parameter, as well as the uiRegistry, expect to find.

Defaults Feature and Data Properties

We’ve already discussed the defaults array in previous articles, but it’s worth reviewing.

With code like the following (give it a try!)

var Element = requirejs('uiElement');
var viewModelConstructor = Element.extend({
    defaults:{
        'message':'Hello World'}
    }
);

viewModel = new viewModelConstructor;
console.log(viewModel.message);
Hello World

we can ensure our instantiated view models have default values. When we use code like this

var viewModelConstructor = Element.extend({...});

It’s sort of like saying the following in PHP.

class viewModelConstructor extends uiElement
{
}

We’re sub-classing the uiElement object/class. We say sort of because, despite some dressing up with classical inheritance, we’re still writing javascript code and using javascript’s object system. Additionally, the extend method comes from the underscore.jslibrary, which is most definitely not a classical inspired object-system. Again, it’s a little beyond the scope of this article, but Magento 2’s javascript object system is its own weird thing. Today we’re going to try and stay concentrated on using this object system instead of getting bogged down in its implementation details.

In addition to setting defaults values in your constructor, you can also set properties at instantiation time. Consider the following program.

var Element = requirejs('uiElement');
var viewModelConstructor = Element.extend({
    defaults:{
        'message':'Hello World'}
    }
);

viewModel = new viewModelConstructor({
    'message':'Goodbye World'
});
console.log(viewModel.message);

Here we’ve set a default message to Hello World. However, at instantiation time we’ve passed in a new object literal as a parameter. Magento will use this object to set data properties on the newly instantiated object — overriding any defaults set via the constructor function.

We’ve reviewed defaults because the last two features we’ll explore today are similar, in that they related to setting property values on Magento’s uiElement view model objects at instantiation time.

A uiRegistry Review

Before we can talk about the imports and exports feature of the defaults array, let’s review the Magento 2 uiRegistry. Navigate to the Customer -> All Customers page in Magento 2’s backend, and then enter the following code in your javascript console.

reg = requirejs('uiRegistry');
var viewModel = reg.get('customer_listing.customer_listing');
console.log(viewModel);

The uiRegistry object is where Magento registers its instantiated view model objects. Above we have fetched the view model registered with the namecustomer_listing.customer_listing. The following code

reg = requirejs('uiRegistry');    
var viewModel = reg.get('customer_listing.customer_listing_data_source');
console.log(viewModel.data.items);

fetches the data source view model, which contains the row data Magento’s grid component needs to render.

The instantiation and registration of these view models happens in theMagento_Ui/js/core/app module.

While this instantiation is beyond the scope of today’s article, behind the scenes Magento’s running code that looks something like this.

requirejs(['Magento_Ui/js/form/components/html', 'uiRegistry'], function(viewModelConstructorFormComponentHtml, registry){
    var viewModel = new viewModelConstructorFormComponentHtml;     
    registry.set('the_name_of_the_view_model', viewModel);
});

That is, Magento uses the configured component (or “view model constructor factory” —Magento_Ui/js/form/components/html above) to fetch a view model constructor, uses that view model constructor to instantiate an object, and then registers that object in the registry. The actual code is, of course, much more complicated, and involves many of the rendered data properties from the ui_component XML. That said, at its heart, view model registration is as simple as the above two line RequireJS program.

The uiRegistry ties in heavily to the features of the uiElement objects we’re exploring today.

Default Imports

The imports feature of Magento 2’s object system allows you to, at the time of instantiation, link a property on your instantiated object with a property in a registereduiRegistry object. Remember our code from above?

var viewModel = reg.get('customer_listing.customer_listing_datasource');
console.log(viewModel.data.items);

With the import feature, we can ensure our objects are linked to thecustomer_listing.customer_listing_datasource object’s data.items row, giving our object access to the grid component’s data. Let’s give it a try.

var Element = requirejs('uiElement');
var viewModelConstructor = Element.extend({
    defaults:{
        'imports':{
            ourLinkedRows:'customer_listing.customer_listing_data_source:data.items'
        }
    }
});         

viewModel = new viewModelConstructor;
console.log(viewModel.ourLinkedRows);

Run the above program, and you’ll see our instantiated object now contains anourLinkedRows property, and that property contains the data source’s data object.

The imports property of the defaults object is an object of key/value pairs.

'imports':{
    ourLinkedRows:'customer_listing.customer_listing_data_source:data.items'
}

The key (ourLinkedRows above) is the property of your instantiated object that you want to populate with data. The value (customer_listing.customer_listing_data_source:data.items above) is a special string that Magento will use to pull data out of the registry. The string is colon (:) separated. The left side of the string (customer_listing.customer_listing_data_source) is the registry key, and the right side of the string (data.items) is the data property.

Using imports, you can give your viewModel easy access to any data currently in theuiRegistry, which means (in turn) your Knockout view can have access to any data from the entire UI Component tree.

Default Exports

The exports feature works similarly, but in reverse. Consider the following program

reg = requirejs('uiRegistry');
var Element = requirejs('uiElement');
var viewModelConstructor = Element.extend({
    defaults:{
        'message':'Hello World',
        'exports':{
            message:'customer_listing.customer_listing_data_source:theMessagePropertyFromExport'
        }
    }
});         

viewModel = new viewModelConstructor({
    'message':'Goodbye World'
});

viewModelObject = reg.get('customer_listing.customer_listing_data_source');
console.log(viewModelObject.theMessagePropertyFromExport);

Here, the exports object

'exports':{
    message:'customer_listing.customer_listing_data_source:theMessagePropertyFromExport'
}

has a key of message, and value ofcustomer_listing.customer_listing_data_source:theMessagePropertyFromExport. With this exports configuration, when our object is instantiated, the uiElement system will look at the message property of the instantiated object, and link it with thetheMessagePropertyFromExport property of the uiRegistry object registered to the keycustomer_listing.customer_listing_data_source.

In other words, the exports feature allows you modify objects that already exist in theuiRegistry.

Wrap Up

The imports and exports features are interesting ones. On the server side, Magento’s core engineering teams seem — obsessed? — with using Gang of Four style design patterns to thinly slice any possible dependency an object may have. However, on the client side, Magento’s core team have built an entire object system that has a huge glaring dependency (the uiRegistry) baked right into their classes, and imports and exports seem like dependency factories. Right or wrong, it’s easy to see why real certified Engineers roll their eyes whenever programmers call themselves Software Engineers.

Regardless, these are the patterns we have to work with in Magento 2, and you’d be wise to learn their ins and outs. With imports and exports covered, and our toes dipped into Magento’s UI Component data source, we’re finally ready to dive deep on how Magento 2 gets server side data into its javascript programs.

Magento 2: ES6 Template Literals

Today we need to take a small detour into ES6 template literals. Template literals are a newish javascript feature that Magento 2’s UI Component system ends up leaning on for some key pieces of functionality. They’re an important concept to understand by themselves, but it’s even more important to understand the extra abstractions Magento have built up on top of template literals, as well as how Magento’s uiClass object system has these template literals baked into its bones.

Also, before we get started, a big thanks to “bassplayer7” over on the Magento StackExchange. This answer provided a key piece of information that helped me finish up this article.

ES6 Template Literals

For many of you, your first question is likely – what the heck is ES6?!

The ES is short for ECMAScript. The ECMA is short for (Ecma_International)[https://en.wikipedia.org/wiki/Ecma_International], the body that standardized the various javascript/jscript/actionscript programming languages into one unified standard.

ECMAScript 6 is the latest version of this standard. Browser makers (Apple, Google, Firefox) are implementing features from ES6 over time (similar to how HTML5 rolled out).

One of those features is something called template literals. Template literals provide javascript with a simple, built-in template language. We’re going to use the javascript console in Google Chrome to run through some template literal code samples, but you should be able to use these in any javascript enviornment that supports them.

In their simplest form, template literals are almost indistinguishable from strings. Consider the Hello World text below.

> results = `Hello World`
> console.log(results);

Hello World

Notice we’ve placed the text “Hello World” between backticks (the ` character). By placing the text between backticks, we’re telling javascript this string is a template literal.

So far, there’s not much difference between a string and a template literal. Template literals are (from a userland perspective) immediately rendered as strings. Consider the following.

> var results = `Hello World`
> var type    = typeof results;
> console.log(type)

string

To javascript, the results variable looks like a string. Given what we’ve seen so far, template literals seem useless.

Of course, we wouldn’t be telling you about them if they were useless! Template literals support template variables/placeholders. Consider the following

> var salutation = "Goodbye"
> var results = `${salutation} World`
> console.log(results);

Above, ${salutation} is a template literal variable (or placeholder). These variables read from javascript’s current scope. So, javascript will render the template variable${salutation} as the word Goodbye. Javascript will do this because we assigned the string Goodbye to the global variable salutation.

While covering template literal’s functionality in full is beyond the scope of this article, The Mozilla developer network has more information on template literals, including advanced features like “tag” functions.

Browser Support and Magento 2

Like any new javascript feature, a client side developer needs to be careful before adopting template literals. Template literals are not supported across all browsers currently in use, and trying to use them in an old browser will likely result in javascript errors that crash your application.

Magento 2’s developers worked around this limitation by creating a RequireJS module for rendering template literals. This module’s identifier is mage/utils/template, and you can use it like this.

//requires an enviornment bootstrapped with Magento 2
//javascript.  i.e. open you debugger on a Magento page

> requirejs(['mage/utils/template'], function(templateRenderer){
    window.salutation      = 'Brave New';
    var templateLiteral = '${salutation} World';    
    var results         = templateRenderer.template(templateLiteral);
    console.log(results);
}); 

Brave New World

Behind the scenes, the mage/utils/template module will check for template literal support. If this support is there, the module uses the native browser implementation. If support’s not there, the module will use a (slower) pure userland javascript implementation. I believe the kids call this sort of abstraction a polyfill.

While this module is useful, there are a few caveats. First, you’ll notice we needed to assign the salutation variable to the global namespace (“window” in browser javascript).

window.salutation      = 'Brave New';
var templateLiteral = '${salutation} World';  

Native template literals will read from the current functional scope, but Magento 2’s module (or any userland code) doesn’t have automatic access to that scope. Therefore, the template literals can only read from global scope. The following code

> requirejs(['mage/utils/template'], function(templateRenderer){
        var salutation      = 'Brave New';
        var templateLiteral = '${salutation} World';    
        var results         = templateRenderer.template(templateLiteral);
        console.log(results);
    }); 

results in the following javascript error (unless, of course, you have a global variable named salutation defined)

VM1627:1 Uncaught ReferenceError: salutation is not defined

The second mage/utils/template caveat, and likely a direct result of the above scope problem, is Magento 2’s template literals have extra abilities that go above and beyond those defined in the standard.

Binding View Variables to a Template Literal

Magento 2’s template literals allow you to bind a specific object to a specific template literal, and then reference those variables with a special syntax. The syntax for binding the object looks like this.

> requirejs(['mage/utils/template'], function(templateRenderer){
        var viewVars        = {
            'salutation':'What a Crazy'
        };   

        var templateLiteral = '${salutation} World';
        var results         = templateRenderer.template(templateLiteral, viewVars);
        console.log(results);
    }); 

That is, the template method has a second parameter that accepts an object.

templateRenderer.template(templateLiteral, viewVars);

However, the above program still results in an error. That’s because of the aforementioned special syntax. You need to reference this object with a second $ symbol inside a ${}variable/placeholder. In other words, this

var templateLiteral = '${salutation} World';

needs to be this

var templateLiteral = '${$.placeholder} World';

It’s a slightly awkward syntax, but easy enough to mentally parse once you understand that$. is just a place holder for your passed in object. If you try running this corrected program, you should see the correct What a Crazy World output.

> requirejs(['mage/utils/template'], function(templateRenderer){
        var viewVars        = {
            'salutation':'What a Crazy'
        };   

        var templateLiteral = '${$.salutation} World';
        var results         = templateRenderer.template(templateLiteral, viewVars);
        console.log(results);
    }); 

What a Crazy World

Connection to UI Components

You may be wondering why we’re covering template literals in a series focused on Magento 2’s UI Components. It turns out template literals are baked into the view model constructor object system we’ve discussed in previous articles.

As we know, the Knockout.js view model constructor objects Magento uses with its UI Components are based on the uiElement/Magento_Ui/js/lib/core/element/elementRequireJS module. We’re going to take a step away from Knockout and try using this object system via pure javascript code to better understand it. Here’s a simple example that creates a new view model constructor, and uses that constructor to create a view model.

> requirejs(['uiElement'], function(Element){
    viewModelConstructor = Element.extend({});
    viewModel = new viewModelConstructor;
    console.log(viewModel);
});

UiClass {_super: undefined, ignoreTmpls: Object, ...}

One of the features of this object system (implemented in theuiClass/Magento_Ui/js/lib/core/class module that uiElement extends from) is the ability to have your instantiated object include “default” values. If you provide a defaultsobject for your view model constructor

viewModelConstructor = Element.extend({
    'defaults':{
        'ourDefaultValue':'Look at our value!'
    }
});

then any object instantiated from this constructor will have a default value for itsourDefaultValue property.

> requirejs(['uiElement'], function(Element){
    viewModelConstructor = Element.extend({
        'defaults':{
            'ourDefaultValue':'Look at our value!'
        }
    });
    viewModel = new viewModelConstructor;
    console.log(viewModel.ourDefaultValue);
});

Look at our value!

It turns out Magento’s object system will scan any default string value for a template literal, and automatically render those template literals. Consider the following program.

> requirejs(['uiElement'], function(Element){
    window.salutation = 'Hello';
    viewModelConstructor = Element.extend({
        'defaults':{
            'message':'${salutation} World. ',
            'salutation':'Goodbye'
        }
    });
    viewModel = new viewModelConstructor;
    console.log(viewModel.message);
});

Hello World. 

Here we’ve provided the view model constructor with a default message

'defaults':{
    'message':'${salutation} World. '
}

and Magento 2’s javascript objects automatically expand that literal in the instantiatedviewModel object, using the globally set salutation variable.

Hello World.

These defaults template literals can also take advantage of Magento’s “bound object” feature. The $. object accessor will read from any other defaults variable.

requirejs(['uiElement'], function(Element){
    viewModelConstructor = Element.extend({
        'defaults':{
            'message':'${$.salutation} World. ',
            'salutation':'Goodbye'
        }
    });
    viewModel = new viewModelConstructor({
        'salutation':'This is still a crazy'
    });
    console.log(viewModel.message);
});

As well as an object passed to the view model constructor.

requirejs(['uiElement'], function(Element){
    viewModelConstructor = Element.extend({
        'defaults':{
            'message':'${$.salutation} World. ',
            'salutation':'Goodbye'
        }
    });
    viewModel = new viewModelConstructor({
        'salutation':'This is still a crazy'
    });
    console.log(viewModel.message);
});

As you can see from our final program, the values passed to the constructor have precedence over those passed in as defaults. i.e. The This is still a crazy salutation wins out over the Goodbye salutation.

Wrap Up

Today’s tutorial is a good example of the challenges facing a developer who wants to adopt Magento 2 as a platform. Not only are there bleeding edge javascript concepts to pickup, but you also need to understand how Magento has extended these concepts in non-standard ways (binding view variables), and also understand how Magento has composed these objects into their systems (the uiClass object system). Without the concepts above, a developer who encounters code like this

return Element.extend({
    defaults: {
        clientConfig: {
            urls: {
                save: '${ $.submit_url }',
                beforeSave: '${ $.validate_url }'
            }
        }
    },

will be left scratching their head.

Fortunately, we now have Magento’s ES6-like template literals in our tool-belt. We’re one step closer to being able to explore how Magento gets backend data into a view model constructor’s defaults array, as well as how Magento handles the UI Component generated data sources.

Magento 2: Simplest XSD Valid UI Component

In our previous two articles we ran through creating a new UI Component from scratch. While we were successful, we needed to add a class <preference/> (i.e. a rewrite) that disabled Magento 2’s XSD validation. While this was useful as a learning exercise, it’s not that helpful in a real world.

It’s probably OK for a developer managing a single Magento system to use a class<preference/>, but those sorts of developers probably aren’t creating UI Components. For an extension developer, disabling XSD validation with a class preference seriously compromises the stability of the systems an extension will be deployed to.

This problem with the XSD files sheds light on an untrue thing the software industry likes to tell itself over and over. Namely, that it’s possible, purely through use of advanced design patterns, to crete a system that will be both flexible and stable for developers who didn’t create the system.

It’s clear that Magento 2 set out to create a flexible system for developers — plugins, dependency injection, RequireJS map features, etc. However, the XSD schema files (a tool meant to reign in the complexity of the XML files) ended up limiting the flexibility of the UI Component system.

Whatever role systems design, and gang-of-four style design patterns play, time and time again we rediscover that the only way to build flexible and stable systems for your end-user-programmers is to listen to them and adjust the system over time toward real world usage patterns. Even if (or especially if) this means abandoning whatever high minded ideals your project’s trying to bring to the table.

All that, however, is a topic for another time (and possibly for all-time). Today, carrying with us the lessons learned so far, we’re going to revisit our simple UI Component. This time though we’re going to do it with XSD validation on, and come up with a pattern we can reuse in real world projects.

The specifics here were tested against Magento 2.1.1, but the concepts should apply across all Magento 2 versions.

Setup

The first thing we’ll want to do is disable our Pulsestorm_SimpleUiComponent module. We’re doing this to reenable the XSD validation for XML files. Run the following command, and you should be all set. If you never created aPulsestorm_SimpleUiComponent module, this step is not necessary.

$ php bin/magento module:disable Pulsestorm_SimpleUiComponent
The following modules have been disabled:
- Pulsestorm_SimpleUiComponent

Cache cleared successfully.
Generated classes cleared successfully. Please run the 'setup:di:compile' command to generate classes.
Info: Some modules might require static view files to be cleared. To do this, run 'module:disable' with the --clear-static-content option to clear them.

Once we’ve done that, we’ll want to create a new admin module for our UI Component. As usual, we’ll use pestle for this.

pestle.phar generate_module Pulsestorm SimpleValidUiComponent 0.0.1

pestle.phar generate_acl Pulsestorm_SimpleValidUiComponent Pulsestorm_SimpleValidUiComponent::top,Pulsestorm_SimpleValidUiComponent::menu_1

pestle.phar generate_menu Pulsestorm_SimpleValidUiComponent Magento_Backend::system_other_settings Pulsestorm_SimpleValidUiComponent::a_menu_item Pulsestorm_SimpleValidUiComponent::menu_1 "Hello Simple Valid Ui Component" pulsestorm_simplevaliduicomponent/index/index 1

pestle.phar generate_route Pulsestorm_SimpleValidUiComponent adminhtml pulsestorm_simplevaliduicomponent

pestle.phar generate_view Pulsestorm_SimpleValidUiComponent adminhtml pulsestorm_simplevaliduicomponent_index_index Main content.phtml 1column

php bin/magento module:enable Pulsestorm_SimpleValidUiComponent

php bin/magento setup:upgrade

With the above in place, clear your cache, and you should be able to navigate to a System -> Other Settings -> Hello Simple Valid Ui Component menu in your Magento backend.

With our boilerplate generated, lets get started!

A New UI Components

As we did earlier in this series, we’ll add a new <uiComponent/> to our layout handle XML file.

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/layout/pulsestorm_simplevaliduicomponent_index_index.xml
<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <referenceBlock name="content">
        <block template="content.phtml" class="Pulsestorm\SimpleValidUiComponent\Block\Adminhtml\Main" name="pulsestorm_simplevaliduicomponent_block_main" />

        <uiComponent name="pulsestorm_simple_valid"/>

    </referenceBlock>
</page>

and then we’ll create an XML file for this named UI Component.

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/pulsestorm_simple_valid.xml
<pulsestorm_simple_valid xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
</pulsestorm_simple_valid>

If we clear our cache and reload the page, we’ll end up with an error something like this.

Exception #0 (Magento\Framework\Exception\LocalizedException): Element
'pulsestorm_simple_valid': No matching global declaration 
available for thevalidation root.

As previously mentioned, this error happens because there’s nopulsestorm_simple_valid in Magento’s vendor/magento/module-ui/view/base/ui_component/etc/definition.xml file, and we can’t add one because Magento’s XSD schema validation file for UI Component files doesn’t allow root nodes named <pulsestorm_simple_valid/>.

Unfortunately, there’s no way we can work around this. The schema is the schema. Even if we removed the xsi:noNamespaceSchemaLocation from our XML files, Magento’s merging our nodes into XML trees that use this XSD file. As of Magento 2.1, there’s no way for third party developers to distribute UI Components without some half-baked module that disables or modifies the XSD validation routines. This is disappointing.

However, we can still take advantage of the <uiCompnent/> tag if we change our root node to the following.

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/pulsestorm_simple_valid.xml
<container xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
</container>

The <container/> node is a valid root level node. If we reload with the above in place, we’ll end up with the following error instead.

Fatal error: Method Magento\Ui\TemplateEngine\Xhtml\Result::__toString()
must not throw an exception, caught Error: Call to a member function
getConfigData() on null in 
/path/to/magento/vendor/magento/module-ui/
Component/Wrapper/UiComponent.php on line 0

In other words, our schema validation errors are gone, and Magento’s just complaining about a missing dataSource node. We can fix that with a new dataSource node

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/pulsestorm_simple_valid.xml
<container xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <dataSource name="pulsestorm_simple_valid_data_source">                        
        <argument name="dataProvider" xsi:type="configurableObject">
            <!-- the PHP class that implements a data provider -->
            <argument name="class" xsi:type="string">Pulsestorm\SimpleValidUiComponent\Model\DataProvider</argument>    

            <!-- redundant with the `dataSource` name -->
            <argument name="name" xsi:type="string">pulsestorm_simple_valid_data_source</argument>

            <!-- required: means ui components are meant to work with models -->
            <argument name="primaryFieldName" xsi:type="string">entity_id</argument>

            <!-- required: means ui components are meant to work with URL passing -->
            <argument name="requestFieldName" xsi:type="string">id</argument>
        </argument>        
    </dataSource>
</container>

and a new data provider class

#File: app/code/Pulsestorm/SimpleValidUiComponent/Model/DataProvider.php  
<?php
namespace Pulsestorm\SimpleValidUiComponent\Model;
class DataProvider extends \Magento\Ui\DataProvider\AbstractDataProvider
{
}    

Reload your page with the above in place, and you should see your page rendered again, sans errors.

What Did We Render?

If we take a look at our raw page source (i.e. not from a DOM Inspector, but the pre-javascript “View Source” source), we’ll see we’ve rendered the following

<div>
    <div data-bind="scope: 'pulsestorm_simple_valid.areas'" class="entry-edit form-inline">
        <!-- ko template: getTemplate() --><!-- /ko -->
    </div>
    <script type="text/x-magento-init">
    {
        "*": {
            "Magento_Ui/js/core/app": {
                "types": {
                    "dataSource": [],
                    "container": {
                        "extends": "pulsestorm_simple_valid"
                    },
                    "html_content": {
                        "component": "Magento_Ui\/js\/form\/components\/html",
                        "extends": "pulsestorm_simple_valid"
                    }
                },
                "components": {
                    "pulsestorm_simple_valid": {
                        "children": {
                            "pulsestorm_simple_valid": {
                                "type": "pulsestorm_simple_valid",
                                "name": "pulsestorm_simple_valid",
                                "config": {
                                    "component": "uiComponent"
                                }
                            },
                            "pulsestorm_simple_valid_data_source": {
                                "type": "dataSource",
                                "name": "pulsestorm_simple_valid_data_source",
                                "dataScope": "pulsestorm_simple_valid",
                                "config": {
                                    "params": {
                                        "namespace": "pulsestorm_simple_valid"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }    
    </script>
</div>

If we take a look at the <container/> node’s definition.xml configuration.

<!-- File: vendor/magento//module-ui/view/base/ui_component/etc/definition.xml -->

<!-- ... -->
<container class="Magento\Ui\Component\Container">
    <argument name="data" xsi:type="array">
        <item name="config" xsi:type="array">
            <item name="component" xsi:type="string">uiComponent</item>
        </item>
        <item name="template" xsi:type="string">templates/container/default</item>
    </argument>
</container>
<!-- ... -->

We see its XHTML template is templates/container/default, and its component is the RequireJS map alias uiComponent (Magento_Ui/js/lib/core/collection).

The reason we chose the <container/> component is two fold. First, it’s one of the few generic components we can use at the root of our ui_compnent file without tripping a Magento XSD validation error. Second though, the uiComponent javascript component is exactly what we want. You’ll remember from last time a uiComponent‘s Knockout.js template (ui/collection) will run through all its child elements and render theirtemplates – similar to a layout update XML <container/> node, or a Magento 1core/text_list block.

The templates/container/default XHTML template, however, does not suit our needs.

#File: vendor/magento/module-ui/view/base/ui_component/templates/container/default.xhtml
<div xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../../Ui/etc/ui_template.xsd">
    <div data-bind="scope: '{{getName()}}.areas'" class="entry-edit form-inline">
        <!-- ko template: getTemplate() --><!-- /ko -->
    </div>
</div>

It’s not 100% clear what this template is for. Magento’s core ui_component files use<container/> as a sub-node, which means the XHTML template is never rendered. It’s likely this is some legacy cruft left over from earlier days when container was used by Magento as a root level node. Or maybe it’s something forward looking. Whatever the reason, this is probably why we can use <container/> as a root level node in the first place. It’s hard to say if this “feature” will stick around, but for now it’s the best we have.

Changing the Template

So if this XHTML template is no good for us, are we stuck? Of course not — we can just configure a new template in our pulsestorm_simple_valid.xml file

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/pulsestorm_simple_valid.xml
<container xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <argument name="data" xsi:type="array">
        <item name="template" xsi:type="string">templates/pulsestorm_simple_valid/default</item>
    </argument>    
    <!-- ... -->

</container>

Remember, the definition.xml file is where the default sub-nodes for a particular node are set, but each individual file in the ui_component folder can override these values.

We’ll also want to create the xhtml file for ourtemplates/pulsestorm_simple_valid/default template.

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/templates/pulsestorm_simple_valid/default.xhtml
<div    
    data-bind="scope: '{{getName()}}.{{getName()}}'"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="../../../../../../Ui/etc/ui_template.xsd">

    <!-- ko template: getTemplate() --><!-- /ko -->
</div>

The code above is based on the default XHTML for a listing grid (vendor/magento/module-ui/view/base/ui_component/templates/listing/default.xhtml). While it performs the same Knockout.js scope kickoff as the template from our previous articles, there’s a few more things going on here that are worth mentioning.

First, let’s start with what we know. We have our standard Knockout tag-less template binding.

<!-- ko template: getTemplate() --><!-- /ko -->

We also know that Knockout’s view model for this section is set by the outer scopebinding. However, if we look at that scope binding.

<div data-bind="scope: '{{getName()}}.{{getName()}}'" ...>

We see the first unfamiliar bit. Instead of a hard coded scope, we have {{getName()}}.{{getName()}}. The text inside the {{...}} brackets are part of the XHTML template language, and will call through to the underlying UI Component object’s getName()method. This name will be the name we used in the layout handle XML file —<uiComponent name="pulsestorm_simple_valid"/>. Meaning the above will render as

<div data-bind="scope: 'pulsestorm_simple_valid.pulsestorm_simple_valid'" ...>

We talked more about these template tags a few articles ago.

The next confusing bit is this

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="../../../../../../Ui/etc/ui_template.xsd"

We have an XML namespace declaration in the root level <div/>, as well as a schema validation file (xsi:noNamespaceSchemaLocation). Remember — these are XHTML, not HTML templates. They behave like XML files. This also means you’re only allowedone top level node in an XHTML template.

While these attributes aren’t strictly necessary, they’re used in the Magento core XHTML templates so it’s best to use them here for consistency.

If you’re curious why these attributes aren’t rendered in the final HTML, Magento removes them before rendering here

#File: vendor/magento/module-ui/TemplateEngine/Xhtml/Result.php
    public function __toString()
    {
        //...
        foreach ($templateRootElement->attributes as $name => $attribute) {
            if ('noNamespaceSchemaLocation' === $name) {
                $this->getDocumentElement()->removeAttributeNode($attribute);
                break;
            }
        }
        $templateRootElement->removeAttributeNS('http://www.w3.org/2001/XMLSchema-instance', 'xsi');
        //...
    }

Finally, it’s worth taking a peek at the ui_template.xsd file

#File: vendor/magento/module-ui/etc/ui_template.xsd
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
    <xs:element name="form">
        <xs:complexType >
            <xs:sequence>
                <xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />
            </xs:sequence>
        </xs:complexType>
    </xs:element>
    <xs:element name="div" >
        <xs:complexType >
            <xs:sequence>
                <xs:any minOccurs="0" maxOccurs="unbounded" processContents="lax" />
            </xs:sequence>
            <xs:anyAttribute processContents="lax" />
        </xs:complexType>
    </xs:element>
</xs:schema>

Covering the entire xs:schema language is a task beyond the scope of this article, but the above says that our xhtml files must have a root node of div, or form. Also, leaving thexsi:noNamespaceSchemaLocation out of our file won’t skip validation, as these .xhtmlfiles are merged into an XML tree that includes this schema.

Adding to the Collection

Alright, if we clear our cache and reload the page with the above in place, we won’t see anything changed on the page. However, using the Sources tab of Chrome’s debugger, we can see Magento has included collection.js (fromMagento_Ui/js/lib/core/collection, which is the RequireJS library uiCollectionpoints at).

We can also (via XHR debugging) see that Magento has downloaded thecollection.html Knockout.js template.

We can also pop into the javascript console, and we’ll see a registered view model constructor named pulsestorm_simple_valid.pulsestorm_simple_valid.

> reg = requirejs('uiRegistry');
> reg.get(function(item){
    console.log(item.name);
    console.log(item);
});

pulsestorm_simple_valid.pulsestorm_simple_valid
UiClass {_super: undefined, ignoreTmpls: Object, _requesetd: Object, containers: Array[0], exports: Object…}

If you’re unsure of the significance here, you probably want to review our last article. Our next step is adding some child components to render.

We’re going to use an htmlContent component/node. Add the following to yourpulsestorm_simple_valid.xml file

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/pulsestorm_simple_valid.xml
<container>
    <!-- ... -->
    <htmlContent name="our_first_content">
        <argument name="block" xsi:type="object">Pulsestorm\SimpleValidUiComponent\Block\Example</argument>
    </htmlContent>           
    <!-- ... -->
</container>

The htmlContent node allows you to render the contents of a Magento block object into our x-magento-init script, and then have those contents rendered onto the page via Knockout.js. The above example will render the block namedPulsestorm\SimpleValidUiComponent\Block\Example. If we look at thedefinition.xml file for <htmlContent/>

<htmlContent class="Magento\Ui\Component\HtmlContent">
    <argument name="data" xsi:type="array">
        <item name="config" xsi:type="array">
            <item name="component" xsi:type="string">Magento_Ui/js/form/components/html</item>
        </item>
    </argument>
</htmlContent>

We see this rendering happens via the RequireJSMagento_Ui/js/form/components/html module. Or, said more completely, theMagento_Ui/js/form/components/html module returns a Knockout.js view model constructor with a Knockout.js “Magento remote” template of ui/content/content.

//File: vendor/magento/module-ui/view/base/web/js/form/components/html.js
//...
return Component.extend({
    defaults: {
        content:        '',
        showSpinner:    false,
        loading:        false,
        visible:        true,
        template:       'ui/content/content',
        additionalClasses: {}
    },  
//...

We’ll leave the specifics of this rendering as an advanced exercise for the user.

If we clear our cache and reload with the above in place, we’ll get the following error.

Exception #0 (ReflectionException): Class Pulsestorm\SimpleValidUiComponent\Block\Example does not exist

Whoops! We forgot to create ourPulsestorm\SimpleValidUiComponent\Block\Example class. Lets do that now.

#File: app/code/Pulsestorm/SimpleValidUiComponent/Block/Example.php     
<?php
namespace Pulsestorm\SimpleValidUiComponent\Block;

use Magento\Framework\View\Element\BlockInterface;

class Example extends \Magento\Framework\View\Element\AbstractBlock
{
    public function toHtml()
    {
        return '<h1>Hello PHP Block Rendered in JS</h1>';
    }
}    

These are standard block classes rendered via the current area’s layout, and need to extend the Magento\Framework\View\Element\AbstractBlock class. Normally these are phtmltemplate blocks, but we’re using a block with a hard coded toHtml method for simplicity’s sake.

If we reload with the above in place, we should see our block rendered.

Hijacking htmlContent

While the htmlContent nodes are interesting, if only for their amusing “render some server side code that renders some front-end code that renders some more server side code” pattern, we’re not interested in them today for their core functionality. We chosehtmlContent nodes because

  1. They’re “XSD allowed” as children of <container/> elements
  2. Their base functionality is relatively simple/uncomplicated
  3. They’re generic, and not likely to imply a specific piece of functionality (vs., say,<listingToolbar/>)

This makes them ideal blocks to hijack. By hijack, we mean we’re going to take advantage of the UI Component’s XML merging to make our htmlContent blocks

  1. Use a different component class
  2. Use a different RequireJS view model constructor factory
  3. Have that RequireJS view model constructor factory point to a new Knockout.js template

Regarding Use a different component class, all we need to do is add a new class attribute to our htmlContent XML node

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/ui_component/pulsestorm_simple_valid.xml
<htmlContent class="Pulsestorm\SimpleValidUiComponent\Component\Simple" name="our_first_content">
</htmlContent>   

We’ve also removed the block argument, as this was required by theMagento\Ui\Component\HtmlContent class ourPulsestorm\SimpleValidUiComponent\Component\Simple replaces. We’ll also (of course) want to create our Pulsestorm\SimpleValidUiComponent\Component\Simpleclass (confused by this? Checkout our previous article for the server side functionality of UI Components).

#File: app/code/Pulsestorm/SimpleValidUiComponent/Component/Simple.php
<?php
namespace Pulsestorm\SimpleValidUiComponent\Component;
class Simple extends \Magento\Ui\Component\AbstractComponent
{
    const NAME = 'html_content_pulsestorm_simple';    
    public function getComponentName()    
    {
        return self::getName();
    }
}

Regarding Use a different RequireJS view model constructor factory, all we need to do is add a new @name="data"/@name="config"/@name="component" argument

<htmlContent class="Pulsestorm\SimpleValidUiComponent\Component\Simple"  name="our_first_content">
    <argument name="data" xsi:type="array">
        <item name="config" xsi:type="array">
            <item name="component" xsi:type="string">Pulsestorm_SimpleValidUiComponent/js/pulsestorm_simple_component</item>
        </item>
    </argument>
</htmlContent>

and then, (conveniently covering our third Have that RequireJS view model constructor factory point to a new Knockout.js template point), have that RequireJS module return a view model constructor with a new Knockout.js remote template.

#File: app/code/Pulsestorm/SimpleValidUiComponent/view/adminhtml/web/js/pulsestorm_simple_component.js
define(['uiElement','ko'], function(Element, ko){
    viewModelConstructor = Element.extend({
        defaults: {
            template: 'Pulsestorm_SimpleValidUiComponent/pulsestorm_simple_template'
        }
    });

    return viewModelConstructor;
});

and then create thePulsestorm_SimpleValidUiComponent/pulsestorm_simple_template template.

<!-- File: app/code/Pulsestorm/SimpleValidUiComponent//view/adminhtml/web/template/pulsestorm_simple_template.html -->
<h1>Our Remote Knockout Template!</h1>

With the above in place, clear your cache and reload the page

Congratulations! You’ve just successfully created a UI Component, fully under our programmatic control, without violating Magento’s XSD schema validations.

Wrap Up

Whether or not this is a good idea or not remains to be seen. While we’ve taken every step possible to keep our htmlContent node under our control (custom PHP component class, custom RequireJS component, custom Knockout.js template), it’s still theoretically possible that a future change by the core Magento engineering team might break what we’ve done here. Right now, a fundamental problem with UI Components is all the programmatic and political evidence points to them being for the core team only, and only time will tell if third party developers are meant to, or will be able to, incorporate them stably into their extensions.

Magento 2: Simplest UI Knockout Component

Last time we created (with the help of some <preference/> hackery) the simplest possible Magento 2 UI Component. If you made it all the way through, I bet you were a little disappointed that we left out the javascript. Today we’ll try to sooth that disappointment.

There’s two big reasons we didn’t discuss javascript last time.

The first was complexity — between the XML, the fact UI Component files are a newdomain specific language (DSL), the need to side step XSD validation, and the strangeness of an XHTML template system, adding the new javascript systems on top of that felt like too much.

The second reason was — it’s not 100% clear where Magento’s new RequireJS and Knockout.js systems start, and the UI Component systems begin.

As always, make sure your system is running in developer mode, and keep in mind the specifics here refer to Magento 2.1.1, but the concepts should apply across versions. You really should work your way through the previous article before continuing, but if you’re the reckless type we’ve put up our starting module on GitHub (you want the first-pass-unstable module).

An App for Knockout.js View Models

Articles two through six in our Magento 2 for PHP MVC developers series covered the basics of serving and using frontend files in Magento 2. In our short Advanced Javascriptseries, we covered Magento’s x-magento-init scripts, and the ground-up basics of Magento’s Knockout.js implementation. Back in July we also covered some of the strange tags you’ll find in Magento’s Knockout.js templates. You can probably get something out of this article without having read those previous articles, but they’ll help tremendously if you lose your footing here.

When we finished up our last article, we had managed to create aPulsestorm_SimpleUiComponent module with a simple UI Component configuration. The rendered UI Component included an x-magento-init section that looked something like this.

<script type="text/x-magento-init">
    {
        "*": {
            "Magento_Ui/js/core/app": {
                "types": {
                    "dataSource": [],
                    "html_content": {
                        "extends": "pulsestorm_simple",
                        "component": "Magento_Ui\/js\/form\/components\/html"
                    },
                    "pulsestorm_simple": {
                        "extends": "pulsestorm_simple"
                    }
                },
                "components": {
                    "pulsestorm_simple": {
                        "children": {
                            "pulsestorm_simple": {
                                "type": "pulsestorm_simple",
                                "name": "pulsestorm_simple",
                                "children": {
                                    "example_content": {
                                        "type": "html_content",
                                        "name": "example_content",
                                        "config": {
                                            "component": "Pulsestorm_SimpleUiComponent\/js\/pulsestorm_simple_component_child",
                                            "content": null
                                        }
                                    }
                                },
                                "config": {
                                    "component": "uiComponent"
                                }
                            },
                            "hello_world_data_source": {
                                "type": "dataSource",
                                "name": "hello_world_data_source",
                                "dataScope": "pulsestorm_simple",
                                "config": {
                                    "data": {
                                        "foo": "bar"
                                    },
                                    "params": {
                                        "namespace": "pulsestorm_simple"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }    
</script>

This x-magento-init script will pass the JSON object into the RequireJS program located in the Magento_Ui/js/core/app module. If we take a look at that module’s source file (info on deriving the file name is over here).

//File: vendor/magento//module-ui/view/base/web/js/core/app.js
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
define([
    './renderer/types',
    './renderer/layout',
    '../lib/knockout/bootstrap'
], function (types, layout) {
    'use strict';

    return function (data, merge) {
        types.set(data.types);
        layout(data.components, undefined, true, merge);
    };
});

This seemingly simple program is actually one of the most important javascript files in Magento’s UI Component system. This code (or more specifically, the code in theMagento_Ui/js/core/renderer/types and Magento_Ui/js/core/renderer/layoutmodules) is responsible for creating and registering any and all Knockout.js view model constructor objects.

A view model is the javascript object that Knockout.js’s inline javascript uses to fetch data and/or perform complex actions. If you’ve worked your way through the official Knockout.js tutorials and our Advanced Javascript Tutorial series you should have a solid understanding of view model basics.

Less familiar though will be the registration of these view model constructor objects. We’re not going to cover this registration in full — just know that afterMagento_Ui/js/core/app runs Magento will have added a number of view model constructor objects to a global registry.

The quickest way to understand this is to take a look at the registry on a core grid page. Navigate to the product grid listing at Products -> Catalog and open up your browser’s javascript debugger (View -> Developer -> Javascript Console in Google Chrome).

The uiRegistry

Magento registers each Knockout.js view model constructor into the object returned by theuiRegistry RequireJS module. I wasn’t deeply involved in the javascript world when theAMD specification came to life, so I’m not sure if this storing of global state in a module is considered a good practice or not, but it’s what Magento does so it’s best to accept it and move on.

The uiRegistry is a key in a RequireJS map.

// File: vendor/magento/module-ui/view/base/requirejs-config.js
var config = {
    paths: {
        'ui/template': 'Magento_Ui/templates'
    },
    map: {
        '*': {
            uiElement:      'Magento_Ui/js/lib/core/element/element',
            uiCollection:   'Magento_Ui/js/lib/core/collection',
            uiComponent:    'Magento_Ui/js/lib/core/collection',
            uiClass:        'Magento_Ui/js/lib/core/class',
            uiEvents:       'Magento_Ui/js/lib/core/events',
            uiRegistry:     'Magento_Ui/js/lib/registry/registry',
            uiLayout:       'Magento_Ui/js/core/renderer/layout',
            buttonAdapter:  'Magento_Ui/js/form/button-adapter'
        }
    }
};

This key points to the actual module — Magento_Ui/js/lib/registry/registry, defined in vendor/magento/module-ui/view/base/web/js/lib/registry/registry.js. The registry object functions similarly to a dictionary or hash map — you can use the registry set method to set a value, and use the registry’s get method to fetch a value. Let’s give that a try. First, load theuiRegistry module/object in your debugger.

> reg = requirejs('uiRegistry');
Registry {}

You won’t be able to see the items in the debugger. Magento’s core team programmed the registry in such a way that the data properties are private — the get method is how you’ll want to fetch a registered value. Give the following a try

> reg.get('product_listing.product_listing');
UiClass {_super: undefined, ignoreTmpls: Object, _requesetd: Object, 
    containers: Array[0], exports: Object…}

Here we’ve fetched the Knockout.js view model registered with the nameproduct_listing.product_listing.

Where the uiRegistry differs from your average dictionary or hash map is its get method supports a query syntax for fetching items. You can find a brief description of this query language in the vendor/magento/module-ui/view/base/web/js/lib/registry/registry.js definition file. We’ll skip to the chase though and let you know there’s support for a callback method that will fetch everyobject in the registry. Give the following a try

> reg.get(function(item){
    console.log(item.name);
    console.log(item);
});
//long list of view model constructor and names snipped

This callback query lets us work around the private data member problem, and peek at all the registered view models.

Configuring a View Model Constructor

The product listing grid contains a wealth of view models, but let’s return to our simpler model. Navigate back to our page at System -> Other Settings -> Hello Simple UI Component. If we try the javascript debugging method here

> reg = requirejs('uiRegistry');
reg.get(function(item){
    console.log(item.name);
    console.log(item);
});    
undefined      

We’ll get no results. The UI Component system does not automatically create view models. We need to configure our UI Component with a RequireJS module, and then program that module to return a view model constructor.

To start, we’ll need to add the following configuration node to our definition.xml file.

#File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml
<components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
    <pulsestorm_simple class="Pulsestorm\SimpleUiComponent\Component\Simple">
        <argument name="data" xsi:type="array">
            <!-- ... -->
            <item name="config" xsi:type="array">
                <item name="component" xsi:type="string">Pulsestorm_SimpleUiComponent/js/pulsestorm_simple_component</item>
            </item>            
        </argument>        
    </pulsestorm_simple>
</components> 

Here we’ve added an item node named config and given it a sub-node named component. The value of this node,Pulsestorm_SimpleUiComponent/js/pulsestorm_simple_component, is the name of our RequireJS module. If you clear your cache, reload with the above in place, and view the rendered source of the UI Component, you’ll see the following in that blob of JSON.

"components": {
    "pulsestorm_simple": {
        "children": {
            "pulsestorm_simple": {
                "type": "pulsestorm_simple",
                "name": "pulsestorm_simple",
                "config": {
                    "component": "Pulsestorm_SimpleUiComponent\/js\/pulsestorm_simple_component"
                }
            },
//...

However, you’ll also see the following error in your javascript console.

error message screen shot

The Magento_Ui/js/core/app tried to load ourPulsestorm_SimpleUiComponent\/js\/pulsestorm_simple_component, but failed. Let’s fix that. Add the following file, clear the cache, and reload the page

//File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/web/js/pulsestorm_simple_component.js
define([], function(){
    console.log("Called");
});         

With the above in place, you’ll see a new error in the console

Called    
Uncaught TypeError: Constr is not a constructor    

This is progress. We know Magento loaded our RequireJS module — the Called text makes this clear. However, our module failed to return a view model constructor. Let’s fix that — make your javascript file match the following

//File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/web/js/pulsestorm_simple_component.js    
define(['uiElement'], function(Element){
    viewModelConstructor = Element.extend({
        defaults: {
            template: 'Pulsestorm_SimpleUiComponent/pulsestorm_simple_template'
        }
    });

    return viewModelConstructor;
});  

Here we’re importing the uiElement RequireJS module, using that module’s extendmethod to create a new object with some data, and then we return that new object. This object is our view model constructor.

The uiElement module (a RequireJS map key toMagento_Ui/js/lib/core/element/element) is part of Magento’s custom class based javascript object system, built for the UI Component system. It’s beyond the scope of this article to cover this object system in full, but it’s based on underscore JS, and this quickie is a good place to start if you’re the curious type.

The template property of the defaults object above defines the Magento 2 Knockout.js remote template our view model should use.

Hooking up the View Model

With the above in place, if we clear our Magento cache, reload the page, and enter the following in the javascript debugger

reg = requirejs('uiRegistry');
//hold your questions on pulsestorm_simple.pulsestorm_simple
//we'll get there in a second
viewModelConstructor = reg.get('pulsestorm_simple.pulsestorm_simple')

we’ll see a single returned view model.

UiClass {_super: undefined, ignoreTmpls: Object, _requesetd: Object, containers: Array[0], exports: Object…}

Our next step is linking this view model constructor with a DOM node in our HTML page. This is where Magento’s special Knockout.js scope binding comes into play. Edit your UI Component’s XHTML template so it matches the following.

<!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/templates/different.xhtml -->
<?xml version="1.0" encoding="UTF-8"?>
<div>
    <h1>Hello Brave New World</h1>
    <div data-bind="scope: 'pulsestorm_simple.pulsestorm_simple'" class="entry-edit form-inline">
        <!-- ko template: getTemplate() --><!-- /ko -->
    </div>    
</div>       

Here we’ve done two things. First, we’ve added the following attribute: data-bind="scope: 'pulsestorm_simple.pulsestorm_simple'" This attribute invokes Magento’s Knockout.js scope binding. The scope binding takes a single argument (pulsestorm_simple.pulsestorm_simple above). Magento will use this argument to lookup a view model in the uiRegistry, and make this view model the current Knockout.js view model for every inner node. The scope data binding allows you to have different Knockout.js view models used on different parts of the page.

The second thing we’ve done is include the following “tag-less” Knockout.js binding: <!-- ko template: getTemplate() --><!-- /ko -->. This will render the current view model’s template. The getTemplate method is one of the methods we get “for free” by basing our view model on the uiElement class above.

With the above in place, if we clear our cache and reload the page, we’ll see the following error.

Unable to resolve the source file for ‘adminhtml/Magento/backend/enUS/PulsestormSimpleUiComponent/template/pulsestormsimpletemplate.html’ #0 /path/to/magento/vendor/magento/framework/App/StaticResource.php(97): Magento\Framework\View\Asset\File->getSourceFile() #1 /path/to/magento/vendor/magento/framework/App/Bootstrap.php(258): Magento\Framework\App\StaticResource->launch() #2 /path/to/magento/pub/static.php(13): Magento\Framework\App\Bootstrap->run(Object(Magento\Framework\App\StaticResource)) #3 {main}

Whoops! Back when we configured a template

//File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/web/js/pulsestorm_simple_component.js    
defaults: {
    template: 'Pulsestorm_SimpleUiComponent/pulsestorm_simple_template'
}

we forgot to create the template file. Let’s do that now. If you create a file for thePulsestorm_SimpleUiComponent/pulsestorm_simple_template template URN

<!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/web/template/pulsestorm_simple_template.html -->
<h1>Rendered with Knockout.js</h1>

and then clear your cache and reload the page, you should see a rendered template.

Congratulations, you just created your first Knockout.js based Magento UI Component.

Using Knockout

Of course, all of this seems like a lot of work to render a static HTML template. If you want to really take advantage of Knockout.js, you’ll need to import Knockout into your RequireJS module.

For example, to get knockout-observable data into the following data-bind="text: message" binding

<!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/web/template/pulsestorm_simple_template.html -->
<h1>Rendered with Knockout.js</h1>
<strong data-bind="text: message"></strong>

You’ll create your view model constructor like this

//File: vendor/magento//module-ui/view/base/web/js/core/app.js
define(['uiElement','ko'], function(Element, ko){
    viewModelConstructor = Element.extend({
        defaults: {
            template: 'Pulsestorm_SimpleUiComponent/pulsestorm_simple_template'
        },
        message: ko.observable("Hello Knockout.js!")    
    });

    return viewModelConstructor;
});

Above, we’ve imported the ko module into our module. This ko module is a replacementfor the global ko normally available in Knockout.js. We’ve also added a message property to our view model constructor, and set this property to a ko.observable object. This isnuts and bolts Knockout.js coding. If you reload the page, you should see the Hello Knockout.js text rendered in the strong tag.

Since the message is a Knockout.js observable, we can change it with the following (try it out via the javascript debugger)

> reg = requirejs('uiRegistry');
> reg.get('pulsestorm_simple.pulsestorm_simple').message("Change Me");

The above code snippet uses the uiRegistry to fetch our view model (with the namepulsestorm_simple.pulsestorm_simple), and then call the observable messageproperty.

Modern Javascript and the Browser Debugger

One of the challenges in working with Magento 2’s (and a lot of other modern) javascript is keeping track of what is and isn’t loaded. It’s no longer as simple as viewing your page source and looking for a <script/> tag.

For Google Chrome’s debugger, if you’re looking for your RequireJS module(s), theSource tab is what you want

If you’re looking for your Knockout.js remote template, Network -> XHR is your friend

You’ll want to pay particular attention to the actual text these debugging tools report. Between Magento’s own cache and some aggressive headers set by Magento’s custom front-end file serving application, the files you’re working with on disk may not be the files your browser sees. In addition to clearing our your Magento cache, a full browser cache refresh is another good sanity check to use during development.

Why the Double Name

One thing that may be bothering you is the “double naming” of the view model constructor.

product_listing.product_listing
pulsestorm_simple.pulsestorm_simple

This name comes from the invoked name in our layout handle XML file/ui_component/*.xml filename

<!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->        
<uiComponent name="pulsestorm_simple"/>    

However — based on the examples we’ve provided so far, it’s not 100% clear why we need to use the name twice, and in a fashion that implies a hierarchy of some kind. This is where the final features of the UI Component system come into play, and the features that will let us fully understand the listing and form components that ship with Magento.

First, let’s go back to our definition.xml file and change the configured component.

#File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml
<components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
    <pulsestorm_simple class="Pulsestorm\SimpleUiComponent\Component\Simple">
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <!-- <item name="component" xsi:type="string">Pulsestorm_SimpleUiComponent/js/pulsestorm_simple_component</item> -->
                <item name="component" xsi:type="string">uiComponent</item>
            </item>            
        </argument>        

    </pulsestorm_simple>
</components>  

Here we’ve replaced ourPulsestorm_SimpleUiComponent/js/pulsestorm_simple_component component/view model constructor with a uiComponent. This is another RequireJS map key that points to the Magento_Ui/js/lib/core/collection module.

If we clear our cache and reload with the above in place, we’ll see our template is no longer rendered. This makes sense — different view model, different template. If we take a look at the view model’s template URN in the javascript debugger.

> reg = requirejs('uiRegistry');
> viewModelConstructor = reg.get('pulsestorm_simple.pulsestorm_simple')
> viewModelConstructor.getTemplate()
ui/collection

We’ll see the template’s URN is a ui/collection. This corresponds to the following file.

<!-- File: vendor/magento//module-ui/view/base/web/templates/collection.html -->
<!--
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<each args="data: elems, as: 'element'">
    <render if="hasTemplate()"/>
</each>

If you’re new to Magento 2’s frontend code, the tags probably confuse you. These tags are a Magento 2 extension to the Knockout.js rendering engine — Magento expands these into Knockout.js tag-less bindings. We wrote about this a bit over on Magento Quickies. In plain Knockout.js, the above looks like the following

<!-- ko foreach: {data: elems, as: 'element'} -->
    <!-- ko if: hasTemplate() --><!-- ko template: getTemplate() --><!-- /ko --><!-- /ko -->
<!-- /ko -->

This Knockout.js template foreachs over an elems property of our view model, and if the object inside elems has a template, it renders that template. If we look at elems

> reg = requirejs('uiRegistry');
> viewModelConstructor = reg.get('pulsestorm_simple.pulsestorm_simple')
> viewModelConstructor.elems()
[]

We see its an empty array. So how can we populate this array? Via UI Component configuration!

Let’s add the following to our UI Component.

<!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->
<pulsestorm_simple xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <!--  ... -->
    <htmlContent name="first_ever_child">
        <argument name="block" xsi:type="object">Magento\Framework\View\Element\Text</argument>
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="component" xsi:type="string">Pulsestorm_SimpleUiComponent/js/pulsestorm_simple_component</item>
            </item>            
        </argument>         
    </htmlContent>
</pulsestorm_simple>

Here we’ve added an <htmlContent/> sub-node to our pulsestorm_simple.xml file. This is a stock UI Component node Magento provides in definition.xml. The specific UI node isn’t important — what’s important is we’ve configured this node with ourPulsestorm_SimpleUiComponent/js/pulsestorm_simple_component component, and it’s a sub-node of pulsestorm_simple.

Clear your cache, and reload the page. You should see the rendered template fromPulsestorm_SimpleUiComponent/js/pulsestorm_simple_component again!

screenshot

More interesting to us is the uiRegistry.

> reg = requirejs('uiRegistry');
> reg.get(function(item){
    console.log(item.name);
})
undefined
pulsestorm_simple.pulsestorm_simple
pulsestorm_simple.pulsestorm_simple.first_ever_child
undefined

Here we see a hierarchy of components defined. If we go back to our UI Component XML and add another node.

<!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->    
<pulsestorm_simple xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <!--  ... -->
    <htmlContent name="first_ever_child">
        <argument name="block" xsi:type="object">Magento\Framework\View\Element\Text</argument>
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="component" xsi:type="string">Pulsestorm_SimpleUiComponent/js/pulsestorm_simple_component</item>
            </item>            
        </argument>         
    </htmlContent>

    <htmlContent name="second_ever_child">
        <argument name="block" xsi:type="object">Magento\Framework\View\Element\Text</argument>
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="component" xsi:type="string">Pulsestorm_SimpleUiComponent/js/pulsestorm_simple_component</item>
            </item>            
        </argument>         
    </htmlContent>        
</pulsestorm_simple>

and clear cache/reload, we’ll see the template rendered twice

and another node added to the hierarchy.

> reg = requirejs('uiRegistry');
> reg.get(function(item){
    console.log(item.name);
})
pulsestorm_simple.pulsestorm_simple
pulsestorm_simple.pulsestorm_simple.first_ever_child
pulsestorm_simple.pulsestorm_simple.second_ever_child

While there are many ways you could use the UI Component system to compose your Magento frontend code, in the end this is how it’s primarily used in Magento 2. TheuiComponent/Magento_Ui/js/lib/core/collection module collects and renders a series of Knockout.js view models.

The root level UI Component node is responsible for rendering an XHTML template, but if the configuration for this code includes a uiComponent component attribute, and the XHTML template invokes this component via a scope binding, the sub-nodes of the UI Component tree become named view models in the uiRegistry. Somewhat confusingly, the root node is also registered as a view model constructor, which is where thepulsestorm_simple.pulsestorm_simple comes from.

Wrap Up

Phew! It was quite a journey, but you should now have a better understanding of one of Magento 2’s more mysterious new systems. That said, there’s still plenty to explore in the UI Component system. In our next few articles, we’re going to dive even deeper, and discuss how UI Components access the data created by the <dataProvider/> node, and revisit our “simplest” UI Component to see if there’s a way to use the system without resorting to a class <preference/>.

Magento 2: Simplest UI Component

  1. Today we’re going to YOLO deep dive into Magento 2’s UI Components and attempt to create one from scratch. At this stage in Magento 2’s lifecycle this isn’t something third party developers can do without taking extra ordinary “not production safe” measures, but sometimes the only way to really understand a system is to inhabit it from the ground up.

    Like most of these tutorials, you’ll want to make sure you’re running Magento 2 indeveloper mode (as opposed to production or default mode). Also, in case any of the following gets too crazy, we’ve put a completed module up on GitHub. Also also, these specifics here have been tested and developed against Magento 2.1.1 — but the concepts should apply across versions.

    Baseline Admin Module with Pestle

    To start, we’re going to use pestle to create a boilerplate module with a backend menu item. If you’ve not sure what the below commands are doing, you may want to work your way through the Magento 2 for PHP MVC Developers developers series.

    pestle.phar generate_module Pulsestorm SimpleUiComponent 0.0.1
    
    pestle.phar generate_acl Pulsestorm_SimpleUiComponent Pulsestorm_SimpleUiComponent::top,Pulsestorm_SimpleUiComponent::menu_1
    
    pestle.phar generate_menu Pulsestorm_SimpleUiComponent Magento_Backend::system_other_settings Pulsestorm_SimpleUiComponent::a_menu_item Pulsestorm_SimpleUiComponent::menu_1 "Hello Simple Ui Component" pulsestorm_simpleuicomponent/index/index 1
    
    pestle.phar generate_route Pulsestorm_SimpleUiComponent adminhtml pulsestorm_simpleuicomponent
    
    pestle.phar generate_view Pulsestorm_SimpleUiComponent adminhtml pulsestorm_simpleuicomponent_index_index Main content.phtml 1column
    
    php bin/magento module:enable Pulsestorm_SimpleUiComponent
    
    php bin/magento setup:upgrade
    

    After running the above commands, you should be able to navigate to Magento’s backend and click on the System -> Other Settings -> Hello Simple Ui Component menu to bring up our new backend section.

    Configuring a UI Component

    If you click on the System -> Other Settings -> Hello Simple Ui Component link, you should see the stock pestle auto-generated page.

    The first thing we’ll want to do is add a <uiComponent> configuration to our layout handle XML file.

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/layout/pulsestorm_simpleuicomponent_index_index.xml -->
    <?xml version="1.0"?>
    <page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
        <referenceBlock name="content">
            <block template="content.phtml" class="Pulsestorm\SimpleUiComponent\Block\Adminhtml\Main" name="pulsestorm_simpleuicomponent_block_main" />
    
            <!-- START: our new ui component -->
            <uiComponent name="pulsestorm_simple"/>
            <!-- END:   our new ui component -->
        </referenceBlock>
    </page>
    

    In the code above, we’re telling Magento we want to add a pulsestorm_simple UI Component to the content block on our page. With the above in place, if we clear the Magento cache and reload the page, we’ll see the following error

    1 exception(s):
    Exception #0 (Magento\Framework\Exception\LocalizedException): Object 
    DOMDocument should be created.
    
    Exception #0 (Magento\Framework\Exception\LocalizedException): Object 
    DOMDocument should be created.
    #0 /path/to/magento/
    vendor/magento/framework/View/Element/UiComponent/Config/Reader.php(95): 
    Magento\Framework\View\Element\UiComponent\Config\DomMerger->getDom()
    

    The reason we’re seeing this error is we’ve configured a UI Component namedpulsestorm_simple, but Magento couldn’t find a configuration file for it. The errorObject DOMDocument should be created comes from PHP code trying to read an XML object that wasn’t created.

    Every named UI Component needs a defined ui_component/[...].xml configuration file. So, let’s get that configuration file in place so PHP stops complaining about theDOMDocument. Create the following file.

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->
    <pulsestorm_simple xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    </pulsestorm_simple>
    

    A UI Component’s configured name attribute will match its XML filename —pulsestorm_simple and pulsestorm_simple.xml in our examples above. All UI Component files are found in a ui_component sub-folder of a specific area’s view folder. Although there’s nothing stopping you from using a UI Component in the frontend area, it’s not 100% clear if this will work as you’d expect, as Magento’s core team have mostly (only?) released UI Components configured on backend layouts.

    If we clear the Magento cache and reload with the above in place, we’ll be rewarded with anew error message.

    1 exception(s):
    Exception #0 (Magento\Framework\Exception\LocalizedException): Element
    'pulsestorm_simple': No matching global declaration available for the
    validation root.
    Line: 1
    

    Here Magento’s objecting to our top level node name — pulsestorm_simple. As you’ll recall from the first article in our UI Component series, a UI Component file is a domain specific language (DSL) that controls the instantiation of nested PHP object files. Each node in a ui_component file is linked up with a configuration node in in the following file.

    vendor/magento/module-ui/view/base/ui_component/etc/definition.xml
    

    So, the first problem is our pulsestorm_simple node does not exists in definition.xml, and Magento’s UI Component DSL wouldn’t know which PHP class to instantiate when it encountered this node. Now, thanks to Magento’s merged configuration file loading, we can add-to/change the final merged definition.xml by adding the following file (note: thismust be in the base folder for this to work)

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml -->
    <components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
        <pulsestorm_simple class="Pulsestorm\SimpleUiComponent\Component\Simple"/>
    </components>
    

    With the above configuration, we’re telling Magento

    Whenever you encounter a pulsestorm_simple UI Component node, you should instantiate a Pulsestorm\SimpleUiComponent\Component\Simple object.

    You’ll want to be careful naming your nodes here — this file will be merged with Magento’s core definition.xml file, and if you use a name that’s already in use, you may change core system behavior. Including your vendor namespace (pulsestorm_ above) is a good best practice here.

    If we clear our cache with the latest file in place, we’ll manage to get a changed error message. (Clearly Sisyphus’ rock was made from XML).

    1 exception(s):    
    Exception #0 (Magento\Framework\Exception\LocalizedException): Element 
    'pulsestorm_simple': This element is not expected. Expected is one of 
    ( range, tab, dataSource, paging, massaction, listing, form, fieldset, 
    field, filters ).
    

    Our problem here? Magento is merging our definition.xml file into an XML document with schema validation. Specifically, Magento demands that the final definition.xmlmatch the structure rules setup in

    vendor/magento/module-ui/etc/ui_definition.xsd
    

    Unfortunately, there’s no supported way of adding to these rules in Magento 2. If you know where to look (Magento\Framework\Config\Dom::validateDomDocument) and aren’t above using the object manager’s preference system to inject some custom behavior, it’s possible to have Magento ignore these XSD validation rules. Unfortunately, there’s no way to do this that wouldn’t conflict with another extension trying to do the same thing, so it’s not really an option if you’re trying to redistribute code. Magento’s more stable plugin system isn’t an option, because the validateDomDocument, while public, is a staticmethod, and Magento’s plugin system doesn’t work with static methods.

    At this point, we’re out of luck if we want to create a new, top level ui_component node. This is the first sign that the UI Component system is either reserved for Magento’s core developers, was released before it was feature complete, or both.

    Skipping Schema Validation

    Of course, when we said “out of luck”, we meant “out of luck, unless we want to attack the problem with a possibly unstable class preference” (i.e. YOLO).

    Class preferences are the system where Magento developers (core or third party) define concrete classes that the object manager can use to instantiate interfaces. i.e. They link a default class to a PHP interface, and then when a programmer asks the object manager to instantiate that interface, Magento returns an object of the linked class.

    Class preferences can also be used to replace concrete class definitions, providing functionality that’s very similar to class rewrites in Magento 1 (with all the same downsides as the rewrite system).

    To keep this tutorial going, we’re going to gin up a class preference that will skip XSD validation for XML files. This is not something you’ll want to do in a production system or distributable extension. We’re only doing it now because there’s no other way to proceed.

    Create the following di.xml file

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/etc/di.xml -->
    <?xml version="1.0"?>
    <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">                                                                -->
        <preference for="Magento\Framework\App\Arguments\ValidationState" type="Pulsestorm\SimpleUiComponent\Model\ValidationState"/>
    
    </config>
    

    and add the following class file to your module

    #File: app/code/Pulsestorm/SimpleUiComponent/Model/ValidationState.php
    <?php
    namespace Pulsestorm\SimpleUiComponent\Model;
    class ValidationState extends \Magento\Framework\App\Arguments\ValidationState
    {
        public function isValidationRequired()
        {        
            return false;
        }
    }
    

    The specifics of why/how this works are left as an exercise for the reader, but this Magento quickie should give you a head start on debugging.

    If you clear your cache with the above in place, you should now see a different error.

    1 exception(s):
    Exception #0 (ReflectionException): Class
    Pulsestorm\SimpleUiComponent\Component\Simple does not exist
    

    XSD validation left behind, we’re ready to continue our exploration.

    UI Component Rendering Class

    Before we went down that schema validation hole, we’d just added the following configuration to definition.xml

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml -->
    <components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
        <pulsestorm_simple class="Pulsestorm\SimpleUiComponent\Component\Simple"/>
    </components>
    

    The etc/definition.xml configuration sets the default attributes and nodes that will be used whenever Magento encounters a particular parent node in aui_component/[somefile].xml file. In the above example, we’ve configuredPulsestorm\SimpleUiComponent\Component\Simple as pulsestorm_simple‘s default class. This means when we use pulsestorm_simple here

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->
    <pulsestorm_simple xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    </pulsestorm_simple>
    

    Magento will attempt to instantiate aPulsestorm\SimpleUiComponent\Component\Simple object, and use that object to render our UI Component. So, our error

    1 exception(s):
    Exception #0 (ReflectionException): Class Pulsestorm\SimpleUiComponent\Component\Simple does not exist
    

    Is Magento complaining it can’t find thePulsestorm\SimpleUiComponent\Component\Simple class it needs to instantiate. Let’s fix that! Create the following class file.

    #File: app/code/Pulsestorm/SimpleUiComponent/Component/Simple.php
    <?php
    namespace Pulsestorm\SimpleUiComponent\Component;
    class Simple extends \Magento\Ui\Component\AbstractComponent
    {
        const NAME = 'pulsestorm_simple';
        public function getComponentName()
        {
            return static::NAME;
        }
    }     
    

    A Magento 2 UI Component class file should extend the base abstractMagento\Ui\Component\AbstractComponent class, and will need to define agetComponentName method. It’s not clear if a component name needs to be the same as the UI Component node name or ui_componont/[filename].xml (pulsestorm_simpleabove), but it’s best to follow the guidelines set by Magento’s core code here. For similar reasons, we’ve also given our component class a NAME constant.

    With the above in place, let’s clear the Magento cache and reload our page to get the next error!

    1 exception(s):
    Exception #0 (Magento\Framework\Exception\LocalizedException): Object
    DOMDocument should be created.
    
    Exception #0 (Magento\Framework\Exception\LocalizedException): Object
    DOMDocument should be created.
    

    Once again Magento is complaining about a missing XML file. Here’s where we let you in on one of the biggest surprises of the UI Component system — a UI Component object, one that extends Magento\Ui\Component\AbstractComponent, is a system for renderingXHTML templates. XHTML — as in the series of specifications that attempted to replace the non-well-formed standard of HTML4 with an HTML that had XML’s draconian parsing rules.

    We’ve told Magento we want it to render aPulsestorm\SimpleUiComponent\Component\Simple object, we have not told Magento which template a Pulsestorm\SimpleUiComponent\Component\Simple object should use. Let’s change that! Add the following to our definition.xml file

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml -->
    <components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
        <pulsestorm_simple class="Pulsestorm\SimpleUiComponent\Component\Simple">
            <argument name="data" xsi:type="array">
                <item name="template" xsi:type="string">templates/our-template</item>
            </argument>        
        </pulsestorm_simple>
    </components>
    

    The above tells Magento we want to render the templates/our-template XHTML template. Let’s add that template to our system.

    #File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/templates/our-template.xhtml
    <?xml version="1.0" encoding="UTF-8"?>
    <div>
        <h1>Hello World</h1>
    </div>    
    

    The UI Component system will look for these templates in a module’sview/[area]/ui_component folder. The value from the definition.xml file is transformed into a template path by appending a .xhtml to the file name. Notice that, although these files look like HTML, they have an XML prolog. There are XHTML files, and will need to be well formed XML.

    With the above in place, clear your cache and reload the page. You’ll see yet another error, but we promise you we’re almost there.

    ( ! ) Fatal error: Method Magento\Ui\TemplateEngine\Xhtml\Result::__toString() must not throw an
     exception, caught Error: Call to a member function getConfigData() on null
     in /path/to/magento/
     vendor/magento/module-ui/Component/Wrapper/UiComponent.php on line 0
    

    Here Magento ran into an error while trying to render the XHTML template, and we’ve stumbled into another aspect of the UI Component system. In addition to being a system for rendering XHTML templates, UI Components are also a system that match up a data provider class with a specific XHTML template. The idea is a UI Component is meant to render server side data, and the data provider is the formal method for getting that information to the component.

    This means our final (we swear) step for a bare bones UI Component object is configuring a data provider class. This happens in the pulsestorm_simple.xml file since each theoretical component instance renders a specific UI Component. Add the followingdataSource node to our pulsestorm_simple.xml file.

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->
    <pulsestorm_simple xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    
        <dataSource name="pulsestorm_simple_data_source">                        
            <argument name="dataProvider" xsi:type="configurableObject">
                <!-- the PHP class that implements a data provider -->
                <argument name="class" xsi:type="string">Pulsestorm\SimpleUiComponent\Model\DataProvider</argument>    
    
                <!-- redundant with the `dataSource` name -->
                <argument name="name" xsi:type="string">pulsestorm_simple_data_source</argument>
    
                <!-- required: means ui components are meant to work with models -->
                <argument name="primaryFieldName" xsi:type="string">entity_id</argument>
    
                <!-- required: means ui components are meant to work with URL passing -->
                <argument name="requestFieldName" xsi:type="string">id</argument>
            </argument>        
    
        </dataSource>
    
    </pulsestorm_simple>
    

    There’s some redundant boilerplate naming conventions to be aware of in the<dataSource/> tree. First, the name attribute

    <dataSource name="pulsestorm_simple_data_source">...</dataSource>                       
    

    Is a combination of the UI Component name (pulsestorm_simple), prepended to the string _data_source. Similarly, the following argument node

    <argument name="name" xsi:type="string">pulsestorm_simple_data_source</argument>      
    

    is required, even though it’s just a redundant naming of the node.

    The following two nodes are also required

    <argument name="primaryFieldName" xsi:type="string">entity_id</argument>
    <argument name="requestFieldName" xsi:type="string">id</argument>
    

    and hint at functionality that’s not present in our bare bones component, but PHP will raise an error if they’re not there.

    Finally, the class argument

    <argument name="class" xsi:type="string">Pulsestorm\SimpleUiComponent\Model\DataProvider</argument>    
    

    tells our component which PHP data provider class to instantiate. We’d better create this class! Make sure the following class is a part of your module.

    #File: app/code/Pulsestorm/SimpleUiComponent/Model/DataProvider.php
    <?php
    namespace Pulsestorm\SimpleUiComponent\Model;
    class DataProvider extends \Magento\Ui\DataProvider\AbstractDataProvider
    {
    }
    

    DataProvider classes need to extend the baseMagento\Ui\DataProvider\AbstractDataProvider class — although this class has zeroabstract methods for us to define.

    Alright. With the above in place, lets cross our fingers, clear the Magento cache, and reload the page.

    Eureka! We’ve rendered an XHTML template!

    What’s Happening Behind the Scenes

    Before we get into some of the things we can do with this rendered XHTML template, let’s take a second to talk about what’s going on behind the scenes. When Magento’s layout rendering code encounters a UI Component tag, a bunch of code runs that’s equivalent to the following pseudo code

    $data = functionThatLoadsArgumentNodesFromXmlFiles();
    $ui_component = new Pulsestorm\SimpleUiComponent\Component\Simple(
        //...
        $data,    
    );    
    echo $ui_component->render();
    

    The entire process of configuring a ui_component is to pick the class that’s instantiated, and to set data properties on that class. In our example the instantiation calls looks like this

    $data = functionThatLoadsArgumentNodesFromXmlFiles();
    $ui_component = new Pulsestorm\SimpleUiComponent\Component\Simple(
        //...
        [
            'template'=>'templates/our-template'
        ],    
    );    
    echo $ui_component->render();    
    

    A good DSL usually lets you forget about implementation details like this — but if you’ve never encountered a DSL this sort of thing can seem strange and foreign. Whenever you’re stuck with a bit of UI Component configuration, try to remember you’re preparing values for Magento to convert into PHP code. These are not simple data attributes.

    Raw Template Source

    Let’s come back to our rendered template.

    That’s how it looks in the browser — but what’s the actual rendered source look like. If we take a look (using the browser’s View Source menu and not the rendered DOM of a browser debugger), we’ll see the following (formatted for easier viewing below)

    <div>
        <h1>
            Hello World
        </h1>
        <script type="text/x-magento-init">
            {
                "*": {
                    "Magento_Ui/js/core/app": {
                        "types": {
                            "dataSource": [],
                            "pulsestorm_simple": {
                                "extends": "pulsestorm_simple"
                            },
                            "html_content": {
                                "component": "Magento_Ui\/js\/form\/components\/html",
                                "extends": "pulsestorm_simple"
                            }
                        },
                        "components": {
                            "pulsestorm_simple": {
                                "children": {
                                    "pulsestorm_simple": {
                                        "type": "pulsestorm_simple",
                                        "name": "pulsestorm_simple"
                                    },
                                    "pulsestorm_simple_data_source": {
                                        "type": "dataSource",
                                        "name": "pulsestorm_simple_data_source",
                                        "dataScope": "pulsestorm_simple",
                                        "config": {
                                            "params": {
                                                "namespace": "pulsestorm_simple"
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }        
        </script>
    </div>
    

    Not only has Magento rendered our <div> and <h1> tags from the XHTML template — they’ve also rendered a text/x-magento-init script. This is the final aspect of the UI Component system we’ll cover today. Not only does the UI Component render an XHTML template, not only does it bind that template to a data provider object: The UI Component system also renders a JSON object, and uses that JSON object to initialize an instance of the Magento_Ui/js/core/app RequireJS app/module via an x-magento-init script.

    Now that we know the scope of a UI Component, let’s take a look at some features of this template/rendering engine.

    XHTML Template Tags

    Similar to phtml templates — you can “call through” to the underlying UI Component class in an XHTML template. There’s a special {{...}} template directive syntax you can use. For example, is we add the getEvenMoreData method to our component class

    #File: app/code/Pulsestorm/SimpleUiComponent/Component/Simple.php
    <?php
    namespace Pulsestorm\SimpleUiComponent\Component;
    class Simple extends \Magento\Ui\Component\AbstractComponent
    {
        const NAME = 'pulsestorm_simple';
        public function getComponentName()
        {
            return static::NAME;
        }
    
        //added this method
        public function getEvenMoreData()
        {
            return 'Even More Data!';
        }
    }    
    

    We can use the following {{...}} calls in our xhtml template.

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/templates/our-template.xhtml -->
    <?xml version="1.0" encoding="UTF-8"?>
    <div>
        <h1>Hello World</h1>
    
        <p>
            {{getComponentName()}}
        </p>
    
        <p>
            {{getEvenMoreData()}}
        </p>
    </div>
    

    with the above in place, clear your cache and reload the page. You should see the data pass through from the class methods/properties to the template.

    In addition to calling methods on the object, we should be able to fetch data properties by using a data configuration like this

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml -->
    <argument name="data" xsi:type="array">
        <item name="template" xsi:type="string">templates/our-template</item>
    
        <!-- NEW NODE HERE -->
        <item name="message" xsi:type="string">Hello World</item>                          
    </argument>        
    

    And then referencing the data property by name in a xhtml {{template}} variable.

    <?xml version="1.0" encoding="UTF-8"?>
    <div>
        <!-- ... -->
        {{message}}
    </div>
    

    However, there’s a bug in the XHTML rendering that will prevent this from working unlessyour data variable is in a tag’s attribute

    <?xml version="1.0" encoding="UTF-8"?>
    <div class="{{message}}">
        <!-- ... -->
        {{message}}
    </div>
    

    Super annoying, and another sign that the UI Component system isn’t fully baked.

    Understanding UI Component Inheritance

    When you place a top level node in definition.xml, you’re creating a reusable UI Component tag. This lets a UI Component programmer use your tag in a UI Component XML file loaded in through a <uiComponent/> layout tag.

    The definition.xml file also lets you set defaults for your UI Component, but an end user programmer can override them.

    For example, we set a default template with the following

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/base/ui_component/etc/definition.xml -->
    <components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
        <pulsestorm_simple class="Pulsestorm\SimpleUiComponent\Component\Simple">
            <argument name="data" xsi:type="array">
                <item name="template" xsi:type="string">templates/our-template</item>
            </argument>        
        </pulsestorm_simple>
    </components>
    

    If a theoretical UI Component programmer wanted a pulsestorm_simple UI Component, but wanted to change the template, all they’d need to do is create the same structure in their XML file. For example, if we wanted a new template

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/templates/different.xhtml -->
    <?xml version="1.0" encoding="UTF-8"?>
    <div>
        <h1>Hello Brave New World</h1>
    </div>    
    

    for pulsestorm_simple, all we need to do is add this to our XML file.

    <!-- File: app/code/Pulsestorm/SimpleUiComponent/view/adminhtml/ui_component/pulsestorm_simple.xml -->
    <pulsestorm_simple xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">        
        <argument name="data" xsi:type="array">
            <item name="template" xsi:type="string">templates/different</item>
        </argument>
        <!-- ... -->         
    </pulsestorm_simple>    
    

    While this feature isn’t often used for templates — it is used for other UI Component configuration parameters, and is fundamental to using the system. While you won’t be adding information to definition.xml in the real world, you will be referencingdefinition.xml when you need to, say, debug a grid listing’s rendering parameters.

    Adding Data

    The last thing we’ll want to talk about today is a UI Component’s data. A UI Component’s data would be something like the rows of information for a grid listing, or the default values of a form. Behind the scenes, the UI Component system can render this backend data for you in the frontend as a javascript array/object.

    All we need to do is define a getDataSourceData method on our component class.

    #File: app/code/Pulsestorm/SimpleUiComponent/Component/Simple.php    
    <?php
    namespace Pulsestorm\SimpleUiComponent\Component;
    class Simple extends \Magento\Ui\Component\AbstractComponent
    {
        <!-- ... -->
        public function getDataSourceData()
        {
            return ['data' => ['foo'=>'bar']];
        }
    }
    

    If you clear your cache and reload the page with the above in place, you’ll see our littlefoo=>bar data structure rendered as JSON.

    "pulsestorm_simple_data_source": {
        //...
        "config": {
            "data": {
                "foo": "bar"
            }
        //...
        }
    }
    

    Depending on how fried your brain is, this may be a little confusing, or a lot confusing, to you. If the data comes from the getDataSourceData method on our component class — why’d we need to configure a Pulsestorm\SimpleUiComponent\Model\DataProviderclass?

    Unfortunately, I don’t have a great answer for you there. Based on core code, it looks like the “correct” usage pattern is to have your component class fetch the data provider and call its getData method.

    #File: app/code/Pulsestorm/SimpleUiComponent/Component/Simple.php 
    public function getDataSourceData()
    {
        return ['data' => $this->getContext()->getDataProvider()->getData()];
    }
    

    Then the data provider’s getData method is the one that returns the actual data.

    #File: app/code/Pulsestorm/SimpleUiComponent/Model/DataProvider.php
    <?php
    namespace Pulsestorm\SimpleUiComponent\Model;
    class DataProvider extends \Magento\Ui\DataProvider\AbstractDataProvider
    {
        public function getData()
        {
            return [['foo'=>'baz']];
        }
    }    
    

    While the UI Component system, at first glance, seems like a fully object oriented domain specific language for building user interface components, and may indeed have started out as that, looking deeper it seems like a system that was nowhere near complete as Magento 2’s launch date closed in and is filled with the sort of edge cases, bugs, and weird omissions that a poorly managed or directionless dev cycle brings to mind.

    Next Steps

    That, in a nutshell, is the PHP portion of the UI Component system. At the end of the day, all this complexity boils down to rendering an xhtml template and tying it to a data source.

    In our next UI Component article, we’re going to dive a little bit deeper, and look at the ways Magento’s javascript systems (both RequireJS, and knockout.js) interact with this system. It’s here that the bulk of the work rendering Magento’s grid listings and backend forms happens, and understanding these systems will be vital for customizing Magento’s backend UI.

Magento 2: Introducing UI Components

Today we’re going to talk about a new feature in Magento 2 — UI Components. This may end up being a stand alone article, or it may be the start of a longer in depth series. I’m still figuring out the best way to cover this large, complex topic.

UI Components are an ambitious new approach to building user interface elements in Magento 2, and much of the new admin console is built on this functionality. Today we’ll take a high level tour of what the UI Component System’s goals are, cover implementation details at as high a level as possible, and then wrap up by using pestle to generate a grid/listing UI Component.

The Positive Spin

The easiest way to understand the goals of UI Components is to talk about Magento 1’s backend user interface generating code. Here’s an example of some Magento 1 layout update XML code

<!-- #File: app/design/adminhtml/default/default/layout/catalog.xml -->
<adminhtml_catalog_product_new>
    <update handle="editor"/>
    <reference name="content">
        <block type="adminhtml/catalog_product_edit" name="product_edit"></block>
    </reference>
    <reference name="left">
        <block type="adminhtml/catalog_product_edit_tabs" name="product_tabs"></block>
    </reference>
    <reference name="js">
        <block type="adminhtml/catalog_product_edit_js" template="catalog/product/js.phtml" name="catalog_product_js"></block>
        <block type="core/template" template="catalog/wysiwyg/js.phtml"/>
    </reference>
</adminhtml_catalog_product_new>

That’s 4 separate layout update XML nodes to add a single product editing form. If you consider the layout update XML behind the reusable <update handle="editor"/> node

<!-- #File: app/design/adminhtml/default/default/layout/main.xml -->
<editor>
    <reference name="head">
        <action method="setCanLoadExtJs"><flag>1</flag></action>
        <action method="addJs"><script>mage/adminhtml/variables.js</script></action>
        <action method="addJs"><script>mage/adminhtml/wysiwyg/widget.js</script></action>
        <action method="addJs"><script>lib/flex.js</script></action>
        <action method="addJs"><script>lib/FABridge.js</script></action>
        <action method="addJs"><script>mage/adminhtml/flexuploader.js</script></action>
        <action method="addJs"><script>mage/adminhtml/browser.js</script></action>
        <action method="addJs"><script>prototype/window.js</script></action>
        <action method="addItem"><type>js_css</type><name>prototype/windows/themes/default.css</name></action>
        <action method="addCss"><name>lib/prototype/windows/themes/magento.css</name></action>
    </reference>
</editor>

You see that adding the product editing form to a page is even more complex.

The intention behind UI Components is to do away with and/or hide this complexity. Magento 2 introduces a new <uiComponent/> tag for its layout handle XML files (Magento 2 handle XML files are analogous to Magento 1’s layout update XML files). In Magento 2, you can add a product editing form to your page with the following configuration

<uiComponent name="product_form"/>

By introducing the concept of a <uiComponent/>, Magento 2 makes it easier for developers to reuse different components in different locations. While It was possible to drop different Magento 1 UI forms and grids in different areas, you needed to know which blocks and javascript files made up a particular component. Magento 1’s approach made it easy to accidentally setup a grid listing or a form so the component almost worked.

Magento 2’s UI Components set out to solve this problem, and simplify everyone’s layout handle XML files considerably.

The Actual Spin

While everything we just said it true enough, the reality of the UI Component system is a little murkier than that rosy picture. That’s because the UI Component system has a number of other goals, and those goals introduce a significant amount of complexity.

As near as I can tell, the UI Component System

  • Simplifies Layout Handle XML files
  • Moves admin user interface elements from HTML+Javascript to a “pure javascript” custom widget system
  • Is a system for building more complex UI components out of smaller components
  • Pre-renders data for UI components as JSON, binding closely to Magento backend data objects
  • Uses ajax to update component data
  • Introduce a new DSL for creating all of the above

The UI Component system is an ambitious one, and like a lot of things in Magento 2, it’s not quite fully baked. While you might want to stay away from a system that’s not quite done, most core grids and many forms use the UI Component system to render their interface elements, and others use a mix of a traditional block rendering and javascript files. If you want to build a fully featured module, you’ll need to work with the UI Component system.

The rest of this article represents my best understanding of UI Components at this time, (the Magento 2.1 era). Many of the specifics are likely to change in the future, but hopefully the core concepts will stay the same. There’s no great standard course of action for developers who want (or need) to develop backend UI interface elements — as per usual its best to look at what the core team is doing with their own components, mimic that, and keep a close eye on your module/extension code whenever there’s a Magento version update.

Unless you’re interested in complex implementation details, you may want to skip to the end where we use pestle to create a UI Component.

Pure Javascript

If you navigate to Content -> Block in Magento 2’s backend, you’ll see a grid listing of all the CMS Blocks in your Magento system. If you’re not familiar with Blocks, they’re a way to create usable chunks of HTML for your store. Block information is stored in Magento’s backend using CRUD Models.

The listing you see is a UI Component, configured with the following layout handle XML

<!-- File: vendor/magento/module-cms/view/adminhtml/layout/cms_block_index.xml -->
<!-- ... -->
<referenceContainer name="content">
    <uiComponent name="cms_block_listing"/>
</referenceContainer>
<!-- ... -->

If you’re new to Magento’s layout XML, a plain english reading of the above is

Get a reference to the already created container named content, and add thecms_block_listing UI Component to it

If you view the raw source of the HTML page, the <uiComponent/> tag is responsible for rendering the following HTML

<div class="admin__data-grid-outer-wrap" data-bind="scope: 'cms_block_listing.cms_block_listing'">
    <div data-role="spinner" data-component="cms_block_listing.cms_block_listing.cms_block_columns" class="admin__data-grid-loading-mask">
        <div class="spinner">
            <span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span>
        </div>
    </div>
    <!-- ko template: getTemplate() --><!-- /ko -->        
    <script type="text/x-magento-init">
        {"*": {"Magento_Ui/js/core/app": {...very large js object...}}}
    </script>
</div>

If you’ve worked through the Magento 2 Advanced Javascript series, particularly theJavascript Init Scripts tutorial, you know the x-magento-init script tag will invoke theMagento_Ui/js/core/app RequireJS module as a program, passing in the very large js object as a parameter.

Without getting too deep into the implementation details (some of which you can read about in these Stack Exchange answers), this javascript code ends up creating a series of javascript constructor objects that Magento will use as KnockoutJS view models.

The actual rendering of the interface element in the browser is handled by KnockoutJS. The outer div of the skeleton HTML uses Magento’s custom KnockoutJS scope binding to bind a view model that was created by the text/x-magento-init javascript.

<div ... data-bind="scope: 'cms_block_listing.cms_block_listing'">
</div>

and then rendering of the UI Component happens via the KnockoutJS “tag-less” template binding

<!-- ko template: getTemplate() --><!-- /ko -->

The call to getTemplate actually kicks off a number of KnockoutJS nested template renderings — starting with a file named collection.html. You can find all these templates by looking for .html files in your browser’s XHR debugging window. If you’re not familiar with Magento’s extension of KnockoutJS templates to XHR, or any of the other KnockoutJS code, try reading through the KnockoutJS Integration article that’s part of the Magento 2: Advanced Javascript series. Also, keep in mind that Magento’s core team has enhanced KnockoutJS templates with some custom tags and attributes that can be a little disorienting.

In summary, Magento 1 rendered a listing in HTML, and then used javascript to provide the enhanced user interface functionality. Magento 2, while it still uses some skeleton HTML, has shifted most of the rendering of these interface elements to RequireJS modules and KnockoutJS templates.

Sub Components

If you take a closer look at the x-magento-init JSON object, you’ll see there’s a number of nested child javascript objects.

{
    "*": {
        "Magento_Ui/js/core/app": {
            "types": /*...*/
            "components": {
                "cms_block_listing": {
                    "children": {
                        "cms_block_listing": {
                            /*...*/
                            "children": {
                                "listing_top": {
                                    "type": "container",
                                    "name": "listing_top",
                                    "children": {
                                        "bookmarks": {/*...*/},
                                        "columns_controls": {/*...*/},
                                        "fulltext": {/*...*/},
                                        "listing_filters": {/*...*/},
                                        "listing_massaction": {/*...*/},
                                        "listing_paging": {/*...*/}
                                    },

Older developers will be bemused to note the return of nodes named children — a practice we thought was left behind in Magento 1. These child element are each, themselves, fully featured UI Components. The cms_block_listing component is made up of components named listing_top, bookmarks, etc.

As we mentioned earlier, that initial getTemplate call ends up rendering many sub-components. The first KnockoutJS template, collection.html, is so named because its acollection of many different UI Components. Covering this rendering process in full is, unfortunately, not something we have time for today.

One thing we can cover today is how a PHP developer controls what’s rendered in that javascript tree. If we jump back to our <uiComponent/> tag

<!-- #File: vendor/magento/module-cms/view/adminhtml/layout/cms_block_index.xml -->
<uiComponent name="cms_block_listing"/>

Magento uses the uiComponent‘s name to look for a new XML file namedcms_block_listing.xml.

#File: vendor/magento//module-cms/view/adminhtml/ui_component/cms_block_listing.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<listing xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <argument name="data" xsi:type="array">
        <item name="js_config" xsi:type="array">
            <item name="provider" xsi:type="string">cms_block_listing.cms_block_listing_data_source</item>
            <item name="deps" xsi:type="string">cms_block_listing.cms_block_listing_data_source</item>
        </item>
        <item name="spinner" xsi:type="string">cms_block_columns</item>
        <item name="buttons" xsi:type="array">
            <item name="add" xsi:type="array">
                <item name="name" xsi:type="string">add</item>
                <item name="label" xsi:type="string" translate="true">Add New Block</item>
                <item name="class" xsi:type="string">primary</item>
                <item name="url" xsi:type="string">*/*/new</item>
            </item>
        </item>
    </argument>
    <!-- ... we'll get to this in a second ... -->
</listing>

These UI Component XML files are a new domain specific language (DSL). The above instructions tell Magento to

  1. Look up a PHP class name and default arguments for the root level listingnode
  2. Instantiate that class, using the argument node as constructor arguments.

Magento will look up the PHP class name and default arguments in the following file

#File: vendor/magento/module-ui/view/base/ui_component/etc/definition.xml
<components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
    <!-- ... -->
    <listing sorting="true" class="Magento\Ui\Component\Listing">
        <argument name="data" xsi:type="array">
            <item name="template" xsi:type="string">templates/listing/default</item>
            <item name="save_parameters_in_session" xsi:type="string">1</item>
            <item name="client_root" xsi:type="string">mui/index/render</item>
            <item name="config" xsi:type="array">
                <item name="component" xsi:type="string">uiComponent</item>
            </item>
        </argument>
    </listing>    
    <!-- ... -->
</components>

So, when Magento renders <uiComponent name="cms_block_listing"/> as JSON, it starts by running code that (oversimplified) looks like this

$uiComponent = new Magento\Ui\Component\Listing(
    $context, $components, [
        'template'=>'templates/listing/default',
        'save_parameters_in_session'=>'1',
        'client_root'=>'mui/index/render',
        'config'=>[
            'component'=>'uiComponent'
        ],
        'js_config'=>[
            'provider'=>'',
            'deps'=>''
        ],
        'spinner'=>'cms_block_columns',
        'buttons'=>[
            'add'=>[
                'name'=>'add',
                'label'=>'Add New Block',
                'class'=>'primary',
                'url'=>'*/*/new'
            ]
        ],
    ]
)

The data for the arguments above comes from merging the <argument/> nodes together. Each of these parameters has a different effect — but the one we’re interested in is thetemplates/listing/default parameter. This specifies the XHTML template to render for this UI Component. The templates/listing/default string corresponds to the following template.

#File: vendor/magento//module-ui/view/base/ui_component/templates/listing/default.xhtml
<div
    class="admin__data-grid-outer-wrap"
    data-bind="scope: '{{getName()}}.{{getName()}}'"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="../../../../../../Ui/etc/ui_template.xsd">
    <div data-role="spinner" data-component="{{getName()}}.{{getName()}}.{{spinner}}" class="admin__data-grid-loading-mask">
        <div class="spinner">
            <span/><span/><span/><span/><span/><span/><span/><span/>
        </div>
    </div>
    <!-- ko template: getTemplate() --><!-- /ko -->
</div>    

This XHTML template is rendered by a completely different rendering engine than a standard Magento phtml template.

Magento replaces the {{...}} text by calling the PHP method on the UI Component object (getName()), or directly accessing a data property of the same object ({{spinner}}).

The more astute among you may have noticed there’s no x-magento-init listed in the template. Rendering the x-magento-init portion of the UI Component is still handled by the XHTML rendering engine — specifically in the appendLayoutConfiguration method called here

#File: vendor/magento/module-ui/TemplateEngine/Xhtml/Result.php
public function __toString()
{
    try {
        //...
        $this->appendLayoutConfiguration();
        $result = $this->compiler->postprocessing($this->template->__toString());
    } catch (\Exception $e) {
        $this->logger->critical($e->getMessage());
        $result = $e->getMessage();
    }
    return $result;
}
//...
public function appendLayoutConfiguration()
{
    $layoutConfiguration = $this->wrapContent(
        json_encode(
            $this->structure->generate($this->component)
        )
    );
    $this->template->append($layoutConfiguration);
}
//...
protected function wrapContent($content)
{
    return '<script type="text/x-magento-init"><![CDATA['
    . '{"*": {"Magento_Ui/js/core/app": ' . str_replace(['<![CDATA[', ']]>'], '', $content) . '}}'
    . ']]></script>';
}    

Magento will render the structure of the UI Component object as the JSON string, and then append that string to the template.

What is the structure of a UI Component you ask? Remember the we’ll get to the rest in a second hand waving we did here?

#File: vendor/magento//module-cms/view/adminhtml/ui_component/cms_block_listing.xml
<?xml version="1.0" encoding="UTF-8"?>
<listing xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <argument name="data" xsi:type="array">
        <item name="js_config" xsi:type="array">
            <item name="provider" xsi:type="string">cms_block_listing.cms_block_listing_data_source</item>
            <item name="deps" xsi:type="string">cms_block_listing.cms_block_listing_data_source</item>
        </item>
        <item name="spinner" xsi:type="string">cms_block_columns</item>
        <item name="buttons" xsi:type="array">
            <item name="add" xsi:type="array">
                <item name="name" xsi:type="string">add</item>
                <item name="label" xsi:type="string" translate="true">Add New Block</item>
                <item name="class" xsi:type="string">primary</item>
                <item name="url" xsi:type="string">*/*/new</item>
            </item>
        </item>
    </argument>
    <!-- ... we'll get to this in a second ... -->
</listing>

If we look at the actual contents of those nodes

#File: vendor/magento//module-cms/view/adminhtml/ui_component/cms_block_listing.xml

<listingToolbar name="listing_top">
    <argument name="data" xsi:type="array">
        <!-- ... -->
    </argument>    
</listingToolbar>
<columns name="cms_block_columns">
    <argument name="data" xsi:type="array">
        <!-- ... -->
    </argument>    
</columns>

we see more configured UI Components. Any sub-node of a UI Component that’s notnamed argument is considered a child node of the parent object. i.e. When Magento renders the listing component, it also looks up classes and arguments forlistingToolbar, columns, etc. in definitions.xml

#File: vendor/magento/module-ui/view/base/ui_component/etc/definition.xml
<components xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_definition.xsd">
    <listingToolbar class="Magento\Ui\Component\Container"><!--...--></listingToolbar>
    <columns class="Magento\Ui\Component\Listing\Columns"><!--...--></columns>
</components>

and the pseudo code we used earlier actually looks more like this

$uiComponent = new Magento\Ui\Component\Listing(...);

$listingToolbar = new Magento\Ui\Component\Container(...);
$columns        = new Magento\Ui\Component\Listing\Columns(...);

$uiComponent->addComponent($listingToolbar);     
$uiComponent->addComponent($columns);

As a (potentially overwhelming) side note, these child components are the ones configured with the RequireJS module names

#File: vendor/magento//module-cms/view/adminhtml/ui_component/cms_block_listing.xml
<columns class="Magento\Ui\Component\Listing\Columns">
    <argument name="data" xsi:type="array">
        <item name="config" xsi:type="array">
            <item name="component" xsi:type="string">Magento_Ui/js/grid/listing</item>
            <!-- ... -->
        </item>
    </argument>
</columns>

These are the RequireJS modules that Magento turns into KnockoutJS view models. If you lookup the source to these KnockoutJS view models — you’ll usually find the KnockoutJS template configured on the view model constructor.

#File: vendor/magento//module-ui/view/base/web/js/grid/listing.js
define([
    'ko',
    'underscore',
    'Magento_Ui/js/lib/spinner',
    'uiLayout',
    'uiCollection'
], function (ko, _, loader, layout, Collection) {
    'use strict';

    return Collection.extend({
        defaults: {
            template: 'ui/grid/listing',
        }
        //...
    });
});

Data Source Nodes

Finally, there’s one more special UI Component sub-node — the <dataSource/> node.

#File: vendor/magento//module-cms/view/adminhtml/ui_component/cms_block_listing.xml
<listing xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
    <argument name="data" xsi:type="array">
        <item name="js_config" xsi:type="array">
            <item name="provider" xsi:type="string">cms_block_listing.cms_block_listing_data_source</item>
            <item name="deps" xsi:type="string">cms_block_listing.cms_block_listing_data_source</item>
        </item>
        <item name="spinner" xsi:type="string">cms_block_columns</item>
        <item name="buttons" xsi:type="array">
            <item name="add" xsi:type="array">
                <item name="name" xsi:type="string">add</item>
                <item name="label" xsi:type="string" translate="true">Add New Block</item>
                <item name="class" xsi:type="string">primary</item>
                <item name="url" xsi:type="string">*/*/new</item>
            </item>
        </item>
    </argument>
    <!-- ... -->
    <dataSource name="cms_block_listing_data_source">
        <!-- ... -->
    </dataSource>
</listing>

The nodes named dataSource are still UI Components, but they get special treatment. When Magento renders the JSON for the UI component, dataSource nodes get pulled out of the children structure, and Magento renders them right along side the main, top level component (using the name of the component appended with the string _data_source as an object key).

{
    "*": {
        "Magento_Ui/js/core/app": {
            "types": {/*...*/},
            "components": {
                "cms_block_listing": {
                    "children": {
                        "cms_block_listing": {/*...*/},
                        "cms_block_listing_data_source": {
                            "type": "dataSource",
                            "name": "cms_block_listing_data_source",
                            "dataScope": "cms_block_listing",
                            "config": {
                                "data": {
                                    "items": [],
                                    "totalRecords": 0
                                },
                                "component": "Magento_Ui\/js\/grid\/provider",
                                "update_url": "http:\/\/magento-2-1-0.dev\/admin\/mui\/index\/render\/key\/e628fdf18db9219474935e85ab3f25b445287503a00a230704b4168c566f8059\/",
                                "storageConfig": {
                                    "indexField": "block_id"
                                },
                                "params": {
                                    "namespace": "cms_block_listing"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

The dataSource component is where Magento will look for the actual data that populates your UI Component (i.e. the rendered collection data for a model)

Summary of the UI Component Rendering DSL

OK — that was a bananas-pants amount of information. I just finished writing it and I’m not sure even I followed all of it, so don’t worry if your head is spinning. Here’s a very high level summary.

  1. UI Components render an x-magento-init script that populates a global registry of KnockoutJS view models
  2. UI Components also render skeleton HTML that uses KnockoutJS and the customscope binding to rendering the DOM nodes that make up a component
  3. The ui_component XML files are a domain specific language for instantiating a nested hierarchy of UI Component objects, which Magento will ultimately use to render the JSON for the x-magento-init script
  4. A ui_component‘s XML node name is used to lookup PHP classes to instantiate
  5. Magento uses any sub-<agument/> nodes as constructor arguments for that class
  6. Magento uses any sub-node named <dataSource/> to render the actual data used in a UI component (i.e. grid listing information)
  7. Any other sub-node will be used to render a child UI Component — those child UI Components follow the same rules as their parent
  8. The top level UI node configures an XHTML template, which Magento renders via PHP
  9. UI Component nodes configure the RequireJS module(s) that Magento uses as KnockoutJS view model constructors

As you can see, while the uiComponent tags greatly simplifies Magento 2’s layout handle XML files, they also hide a much more complex UI rendering system that includes both front end and backend Magento systems code, and requires developers to understand Magento’s customizations to RequireJS and KnockoutJS as well.

Creating a Grid Listing with Pestle

As you can see from the above (whether you read it or not), the UI Component system rivals Magento 1’s layout update XML system in both its complexity and the lack of clear guidance on usage. In other words, it’s exactly the sort of place a code generation tool like pestle can make things better for working Magento 2 developers. The most recent versions of pestle include a magento2:generate:ui:grid command for creating UI listings, with more commands to come soon.

We’re going to run through using pestle to create a UI grid. We’re going to assume you’ve worked your way through the CRUD Models for Database Access tutorial and have a working Pulsestorm_ToDoCrud module. We’ll also assume you’ve been able to create an admin endpoint with a layout handle XML file, and have a backend page you can navigate to.

In order to create a grid listing, invoke pestle’s magento2:generate:ui:grid command with the following arguments

$ pestle.phar magento2:generate:ui:grid
Which Module? (Pulsestorm_Gridexample)] Pulsestorm_ToDoCrud
Create a unique ID for your Listing/Grid! (pulsestorm_gridexample_log)] pulsestorm_todo_listing
What Resource Collection Model should your listing use? (Magento\Cms\Model\ResourceModel\Page\Collection)] Pulsestorm\ToDoCrud\Model\ResourceModel\TodoItem\Collection
What's the ID field for you model? (pulsestorm_gridexample_log_id)] pulsestorm_todocrud_todoitem_id

The Which Module? argument tells pestle the Magento module you want to create your grid listing in. This is, generally speaking, the same module as the collection file, but there’s nothing enforcing this convention. We’ve specified the Pulsestorm_ToDoCrud from the previous tutorials

The Create a unique ID for your Listing/Grid! argument is the name we want for our UI Component. This will be the name="" attribute we use in the <uiComponent/> tag, as well as the base file name on disk for the UI Component XML file.

The What Resource Collection Model should your listing use? argument is the class name of the collection model to use. We want our grid listing to display Pulsestorm_ToDoCrudmodels, so we use thePulsestorm\ToDoCrud\Model\ResourceModel\TodoItem\Collection collection.

The What’s the ID field for you model? argument is the primary key database column for a model’s database table.

After running the above command, add the following <uiComponent/> to your admin module’s layout handle XML file.

<!-- File: app/code/Pulsestorm/ToDoCrud/view/adminhtml/layout/pulsestorm_admin_todocrud_index_index.xml -->
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <body>
        <referenceBlock name="content">                
            <uiComponent name="pulsestorm_todo_listing"/> 
        </referenceBlock>
    </body>
</page>

With the above in place, clear your cache, and you should have a simple UI Grid that lists the model ID for each individual Pulsestorm\ToDoCrud\Model\ToDoItem model. If you want to add a column for the model’s title attribute, just add the following <column/>node to the generated UI Component XML file

<!-- File: app/code/Pulsestorm/ToDoCrud/view/adminhtml/ui_component/pulsestorm_todo_listing.xml -->
<!-- ... -->
<columns>

    <!-- ... --->

    <column name="title">
        <argument name="data" xsi:type="array">
            <item name="config" xsi:type="array">
                <item name="filter" xsi:type="string">text</item>
                <item name="label" xsi:type="string" translate="true">Item Title</item>
                <item name="sortOrder" xsi:type="number">20</item>
            </item>
        </argument>
    </column>

    <!-- ... ---> 

</columns>
<!-- ... -->    

In addition to generating the UI Component pulsestorm_todo_listing.xml file, pestle also generates a “data provider” class, and a “page action” class.

The data provider class wraps the collection resource model

#File: app/code/Pulsestorm/ToDoCrud/Ui/Component/Listing/DataProviders/Pulsestorm/Todo/Listing.php
<?php
namespace Pulsestorm\ToDoCrud\Ui\Component\Listing\DataProviders\Pulsestorm\Todo;

class Listing extends \Magento\Ui\DataProvider\AbstractDataProvider
{    
    public function __construct(
        $name,
        $primaryFieldName,
        $requestFieldName,
        \Pulsestorm\ToDoCrud\Model\ResourceModel\TodoItem\CollectionFactory $collectionFactory,
        array $meta = [],
        array $data = []
    ) {
        parent::__construct($name, $primaryFieldName, $requestFieldName, $meta, $data);
        $this->collection = $collectionFactory->create();
    }
}

and the page action class is responsible for rendering the edit link in the final column.

#File: app/code/Pulsestorm/ToDoCrud/Ui/Component/Listing/Column/Pulsestormtodolisting/PageActions.php
<?php
namespace Pulsestorm\ToDoCrud\Ui\Component\Listing\Column\Pulsestormtodolisting;

class PageActions extends \Magento\Ui\Component\Listing\Columns\Column
{
    public function prepareDataSource(array $dataSource)
    {
        if (isset($dataSource["data"]["items"])) {
            foreach ($dataSource["data"]["items"] as & $item) {
                $name = $this->getData("name");
                $id = "X";
                if(isset($item["pulsestorm_todocrud_todoitem_id"]))
                {
                    $id = $item["pulsestorm_todocrud_todoitem_id"];
                }
                $item[$name]["view"] = [
                    "href"=>$this->getContext()->getUrl(
                        "adminhtml/pulsestorm_todo_listing/viewlog",["id"=>$id]),
                    "label"=>__("Edit")
                ];
            }
        }

        return $dataSource;
    }    

}

While not yet fully featured, the magento2:generate:ui:grid command will get you started with a base grid listing configuration. From there, you should be able to examine Magento’s core grid classes and replicate any functionality you see in a core module.

Magento 2: Composer and Components

This entry is part 3 of 3 in the series Magento 2 and Composer. Earlier posts include Magento 2: Composer, Marketplace, and Satis, and Magento 2: Composer Plugins. This is the most recent post in the series.

One question I keep getting from new Magento 2 developers, (and we’re all new Magento 2 developers) is, “How should I organize my project files?”. Since Magento has heavily restructured code organization around Composer, it’s not always clear how files should be organized during development, organized for distribution, and how (if at all) to move between these two modes.

While this article won’t definitively answer those questions, we will dive into some of the formalization behind Magento 2 components, as well as how Magento 2’s Composer integration works. With this information in hand, you should be able to come up with a project structure that works for you and your team.

Magento 2 Components

Magento 1 had an informal, inconsistent idea of components. The best way to think about Magento components is

A group of source files, in various formats, with a single purpose in the system

If that’s a little vague, see our previous comments about informal and inconsistent. Speaking more concretely, the four component types in Magento are

  • Modules
  • Themes
  • Language Packs
  • Code Libraries

In Magento 1, modules were pretty well defined and self contained

A folder of files with an etc/config.xml, with developers making Magento aware of the module via an app/etc/modules/Package_Namespace.xml file

Themes were a little less defined and a little less self contained

A collection of files under app/design/[area]/[package]/[name], with developers making Magento aware of the theme via a setting in core_config_data. Unless it’s the admin theme in which case you need a module. Also, modules are responsible for adding the layout XML files to themes. Also, while we’re here, modules aren’t all that self contained either because if they want to use phtml templates then the templates need to be in a theme folder

Things start to get really vague with language packs

A collection of key/value csv files located inapp/locale/[language]/Packagename_Modulename.csv. Also we’re just going to drop email templates in here because reasons

And Magento 1 barely had the concept of a generic code library.

Um, yeah, maybe just drop them in lib and add that path to PHP’s autoloader? And look in the code pool folders too? And maybe just add a top level js folder for javascript libraries? Unless they go in skin?

Oh right! Skins! Magento 1 also had a (now dropped) concept of skins. Skins were best defined as

Any CSS or javascript file that doesn’t belong in a theme.

Where CSS or javascript file that doesn’t belong in a theme was defined as

Any CSS or javascript file that doesn’t belong in a skin

While, in practice, norms developed over time and development wasn’t as chaotic as I’m describing, Magento 1’s lack of formalization around components did make the system harder to work with, particularly if you were trying to redistribute Magento 1 code for reuse.

Magento 2 formalizes the idea of components, and this formalization means the core Magento system, and other external systems (i.e. Composer) can deal with these components in a sane and reasonable way.

Magento 2 Components

In Magento 2, a component is

A group of files, under a top level directory (with sub-folders allowed), with aregistration.php file defining the type of component.

That’s it. There’s nothing about how the components work, interact with the system, or interact with other components. Those things aren’t the concern of the component system.

As of this writing, there are four component types in Magento 2

  • Modules
  • Themes
  • Libraries
  • Language Packs

Let’s take a look at this in action. Consider the Magento_AdminNotification module. This is a collection of files, under a top level directory (AdminNotification), with aregistration.php file. If we take a look at registration.php

#File: app/code/Magento/AdminNotification/registration.php
<?php
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Magento_AdminNotification',
    __DIR__
);

We can see this file registers a component via the static\Magento\Framework\Component\ComponentRegistrar::register method. This component is a module (\Magento\Framework\Component\ComponentRegistrar::MODULE), its identifier isMagento_AdminNotification, and you can find its files in the __DIR__ folder, (i.e. the same directory registration.php is in via PHP’s magic __DIR__ constant).

Next, consider the Luma theme. Again, a collection of files, under a top level directory (luma), with a registration.php file.

#File: app/design/frontend/Magento/luma/registration.php
<?php
/**
 * Copyright © 2016 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::THEME,
    'frontend/Magento/luma',
    __DIR__
);

Here the component type is a theme (\Magento\Framework\Component\ComponentRegistrar::THEME), and its name isfrontend/Magento/luma.

Even though the Magento GitHub project has these files in familiar locations, (app/code,app/design, etc.), thanks to Magento 2’s new component system, these directories can be located anywhere, so long as as the module, theme, library, or language pack correctly defines its registration.php file.

How Magento Loads Components

At this point, the systems minded among you are probably wondering how Magento 2 loads and identifies components. It’s one thing to say so long as the module, theme, library, or language pack correctly defines its registration.php file, but the system still needs to load these files, and that means there are rules.

In order to get Magento to recognize your module, theme, code library, or language pack (i.e. your component), you need Magento to read your component’s registration.phpfile. There are two ways to get Magento to read your registration.php file

  1. Place your component in one of several predefined folders
  2. Distribute your module via Composer, and use Composer’s autoloader features

Of the two methods, the second is the preferred and recommended way of distributing Magento 2 modules. For development, the first offers a convenient way to get started on a component, or checkout/clone a version control repository to a specific location. The first also offers a non-composer way for extension developers to distribute their components.

Predefined Folders

At the time of this writing, Magento 2 will scan the following folders/files for components (the patterns below are for the glob function).

app/code/*/*/cli_commands.php
app/code/*/*/registration.php
app/design/*/*/*/registration.php
app/i18n/*/*/registration.php
lib/internal/*/*/registration.php
lib/internal/*/*/*/registration.php

This is what allows you to place modules in app/code/Packagename/Modulename, or themes in app/design/[area]/[package]/[name], etc. Magento will explicitly look forregistration.php files to load at these locations. Also of interest are theapp/code/*/*/cli_commands.php files — this appears to be a way for a module to register command line classes without using di.xml.

It’s not clear if these folders were added as a stop-gap measure while Magento 2 gets everyone moved over to Composer distribution, or if they’ll stick around for the long term. If you’re curious, Magento does this registration check in the following file

#File: app/etc/NonComposerComponentRegistration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

$pathList[] = dirname(__DIR__) . '/code/*/*/cli_commands.php';
$pathList[] = dirname(__DIR__) . '/code/*/*/registration.php';
$pathList[] = dirname(__DIR__) . '/design/*/*/*/registration.php';
$pathList[] = dirname(__DIR__) . '/i18n/*/*/registration.php';
$pathList[] = dirname(dirname(__DIR__)) . '/lib/internal/*/*/registration.php';
$pathList[] = dirname(dirname(__DIR__)) . '/lib/internal/*/*/*/registration.php';
foreach ($pathList as $path) {
    // Sorting is disabled intentionally for performance improvement
    $files = glob($path, GLOB_NOSORT);
    if ($files === false) {
        throw new \RuntimeException('glob() returned error while searching in \'' . $path . '\'');
    }
    foreach ($files as $file) {
        include $file;
    }
}    

If you’re researching how a future version of Magento 2 handles scanning for components, this would be a good place to start.

Composer Distribution

The other way to have Magento notice your component is to distribute your component via Composer. We’re going to assume you have a basic familiarity with Composer, but for the purposes of this article, all you really need to know is

Composer allows you to ask for a package of PHP files, and have that package downloaded to the vendor/ folder

If you’re interested in learning more about Composer, the previous articles in this series, my Laravel, Composer, and the State of Autoloading, and the Composer manual are a good place to start.

So, assuming you have your Magento component in GitHub (or a different source repository Composer can point at), and your component has a registration.php file, the only question left is How do we get Magento to look at our registration.php file.

Rather than have Magento scan all of vendor/ for registration.php files, (an approach that could quickly get “O^N out of hand” as the number packages grows), Magento uses Composer’s file autoloader feature to load each individual component’sregistration.php file.

If that didn’t make sense, an example should clear things up. Assuming you’ve installed Magento via the Composer meta-package, or installed it via the archive available via magento.com (which is based on the meta-package), take a look at the catalog module’scomposer.json file.

#File: vendor/magento/module-catalog/composer.json
{
    "name": "magento/module-catalog",
    //...
    "autoload": {
        "files": [
            "registration.php"
        ],
        "psr-4": {
            "Magento\\Catalog\\": ""
        }
    }
    //...
}

The autoload section is where you configure the PHP class autoloader for a Composer package. This is covered in great detail in my Laravel, Composer, and the State of Autoloading series. The section we’re interested in today is here

#File: vendor/magento/module-catalog/composer.json
"files": [
    "registration.php"
],

The files autoloader section was originally intended as a stop gap measure for older PHP packages that had not moved to a PSR-0 (and later, PSR-4) autoloader system. Composer’s autoloader (not during install or update, but when your application is running) will automatically include any files listed in files (with the specific package as the base directory), and package developers can do whatever they need to do to setup their pre-PSR autoloaders.

Over the years, many frameworks have taken the simplicity and flexibility of the filesautoloader and turned it to different purposes. Magento 2 is no exception. The above autoload configuration ensures Composer will always load the file at

#File: vendor/magento/module-catalog/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Magento_Catalog',
    __DIR__
);    

This, as we’ve already learned, will register the component. The same holds true for third party components (i.e. yours!) — make sure you’ve created a registration.php file with the correct registration code for your component type, and then include an identical filesautoloader.

Here’s an example of each component type from Magento’s core.

Module

#File: vendor/magento/module-weee/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'Magento_Weee',
    __DIR__
);

Theme

#File: vendor/magento/theme-frontend-luma/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::THEME,
    'frontend/Magento/luma',
    __DIR__
);

Library

#File: vendor/magento/framework/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::LIBRARY,
    'magento/framework',
    __DIR__
);

Language Pack

#File: vendor/magento/language-de_de/registration.php
<?php
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::LANGUAGE,
    'magento_de_de',
    __DIR__
);

Invalid Assumptions

There’s one last important thing to take away from this, even if you’re not responsible for packaging your company’s Magento work. It’s no longer safe to make assumptions aboutwhere a folder is located in located in the Magento hierarchy. If you’re trying to find a specific file in Magento, it’s more important than ever to learn your way around theMagento\Framework\Module\Dir helper class.

Daily Work

So, now that we have a better understanding of what a component is, and how Magento loads components into the system, that still leaves us with our original question. Where should the code for our in progress Magento projects go? How should we store our projects in source control?

Unfortunately — there’s no clear answer, and a lot will depend on the sort of project you’re working on. Are you an extension developer? A theme developer? A system integrator/store builder or someone integrating with a Magento system? Do you want your working source repository to be the same repository Composer reads from? What tooling is your team is familiar with? While there certainly are approaches that are “better” for each scenario, from a programmer’s point of view Magento 2’s still too new to know for sure.

For what its worth, I’ve been creating symlinks to my source repositories so thatNonComposerComponentRegistration.php finds my components, using a build process to create the final Composer accessible repository, and temporarily patching over any issues Magento has with symlinks.

Part of being a Magento 2 developer will be figuring this out for your own team, even if you’re just a team of one.