Magento 2: Admin MVC/MVVM Endpoints

Now that we know how to create Access Control List Rules, and how to generate Admin Menu Items with Magento 2’s CSFR protection, we’re ready to create an MVC/MVVM URL endpoint (i.e. an HTML page) in Magento 2’s admin.

While you’ll find useful information here without it, we’re assuming a basic familiarity with creating routes on Magento’s front end. If there’s a concept below that confuses you, work through the first article in this series before linking to your question on the Magento Stack Exchange in the comments.

Recreate Previous Progress with Pestle

Before we start, we’ll need a module with a few predefined ACL rules, and an admin Menu Item. We’re going to use pestle to quickly get a module up and running. See the previous articles in this series if you’re not sure what the following commands do.

After running the following five commands

$ pestle.phar generate_module Pulsestorm HelloAdminBackend 0.0.1

$ pestle.phar generate_acl Pulsestorm_HelloAdminBackend Pulsestorm_HelloAdminBackend::top,Pulsestorm_HelloAdminBackend::menu_1

$ pestle.phar generate_menu Pulsestorm_HelloAdminBackend Magento_Backend::system_other_settings Pulsestorm_HelloAdminBackend::a_menu_item Pulsestorm_HelloAdminBackend::menu_1 "Hello Admin Backend Pestle" pulsestorm_hello_admin_backend/index/index 1

$ php bin/magento module:enable Pulsestorm_HelloAdminBackend

$ php bin/magento setup:upgrade

you’ll have a Pulsestorm_HelloAdminBackend module with an ACL hierarchy defined, and a single Menu Item under System -> Other Settings -> Hello Admin Backend Pestle. With that, we’re ready to get started.

Magento 2 Admin URL Structure

If you look at the URL of your generated Menu Item hyperlink (Right click, Copy Link As)

http://magento.example.com/admin/pulsestorm_hello_admin_backend/index/index/key/...

you’ll see Magento 2’s backend URLs have a four segment URL structure. All Magento 2 admin urls start with the segment /admin/. This sets the area code for the backend. The next three segments

pulsestorm_hello_admin_backend/index/index

are your module front name (pulsestorm_hello_admin_backend), the controller name (index) and the action name (index). For now, just make a note of these, as we’ll be referencing them below.

Create Files

We’re going to start by using pestle to create a routes.xml file and controller file for our URL. Run the following command

$ pestle.phar generate_route  
Which Module? (Pulsestorm_HelloWorld)] Pulsestorm_HelloAdminBackend
Which Area (frontend, adminhtml)? (frontend)] adminhtml
Frontname/Route ID? (pulsestorm_helloworld)] pulsestorm_hello_admin_backend

and pestle will add two files to your system

app/code/Pulsestorm/HelloAdminBackend/etc/adminhtml/routes.xml
app/code/Pulsestorm/HelloAdminBackend/Controller/Adminhtml/Index/Index.php

Clear your cache and generated class files, and then click on the Hello Admin Backend Pestle menu. If you’re logged in as the Magento super user, you should be brought to a new, blank admin page!

We’ll get to why this doesn’t work with a non super-user in a moment.

The generate_route command accepts three arguments. The first (Pulsestorm_HelloAdminBackend) is the name of the module you want to add yourroutes.xml to. The second, (adminhtml) is the Magento area to use for your route. The area normally defaults to frontend, but here we’ve told Magento we want to create a route and controller file for the adminhtml area. The final argument, (pulsestorm_hello_admin_backend), is the frontname to use for our URLs. This serves the same purpose as a front end front name, but with a few key differences we’ll get to in a moment.

First though, let’s take a look at the generated routes.xml file.

Magento 2 adminhtml Routes

The first thing to make a note of is the final path of routes.xml

#File: app/code/Pulsestorm/HelloAdminBackend/etc/adminhtml/routes.xml

You’ll notice Magento wants this file in the adminhtml sub-folder of etc. That’s because we’re setting up a route for the backend admin area. If we open this file

<!-- File: app/code/Pulsestorm/HelloAdminBackend/etc/adminhtml/routes.xml -->
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:App/etc/routes.xsd">
    <router id="admin">
        <route id="pulsestorm_hello_admin_backend" frontName="pulsestorm_hello_admin_backend">
            <module name="Pulsestorm_HelloAdminBackend"/>
        </route>
    </router>
</config>

we’ll see a familiar structure. A routes.xml file is a collection of <router/> nodes. Under the <router/> nodes you’ll find individual <route/> nodes. Each of these <route/>nodes tells Magento that a particular URL front name is claimed by a particular module. Above, we’ve told Magento the module Pulsestorm_HelloAdminBackend claims the front name pulsestorm_hello_admin_backend.

Compared to the front end, there are two major differences to be aware of in Magento backend routing. First, the top level <router/> node does not use the standard id.

<router id="admin">

Instead, all Magento 2 admin backend <router/> tags use an id of admin.

The second thing to be aware of is, as previously mentioned, a backend admin URL’s front name (pulsestorm_hello_admin_backend) is actually the second segment of a URL.

http://magento.example.com/admin/pulsestorm_hello_admin_backend/index/index/key/...

All Magento’s backend URLs have /admin as the first segment of a URL. The front name is the second segment. This has some implications for legacy backend routes and Magento’s internal URL identifiers, which we’ll cover in a later article. For now though, let’s take a look at the generated controller file.

Magento 2 Backend Controllers

The first thing you’ll notice about Magento 2 admin controllers is their file names

app/code/Pulsestorm/HelloAdminBackend/Controller/Adminhtml/Index/Index.php

Like all Magento 2 PHP class files, a controller file’s path and name are based on the full class name of the controller class. Magento front end controller’s use a [Package]\[Module]\Controller\[Controller Segment]\[Action Segment] naming convention. Magento 2 backend controller files use a similar naming convention — the only difference is an additional Adminhtml segment.

[Package]\[Module]\Controller\Adminhtml\[Controller Segment]\[Action Segment]

In our case, our controller class name isPulsestorm\HelloAdminBackend\Controller\Adminhtml\Index\Index. When you create a route with pestle, pestle automatically assumes you want an[frontname]/index/index URL, and creates the controller class name with a corresponding Index\Index.

If you wanted a different URL (say pulsestorm_hello_admin_backend\foo\baz), then you’d manually create aPulsestorm\HelloAdminBackend\Controller\Adminhtml\Foo\Bar class.

There’s more differences inside a backend controller’s class definition. Let’s take a look

#File: app/code/Pulsestorm/HelloAdminBackend/Controller/Adminhtml/Index/Index.php
<?php
namespace Pulsestorm\HelloAdminBackend\Controller\Adminhtml\Index;
class Index extends \Magento\Backend\App\Action
{
    protected $resultPageFactory;
    public function __construct(
        \Magento\Backend\App\Action\Context $context,
        \Magento\Framework\View\Result\PageFactory $resultPageFactory)
    {
        $this->resultPageFactory = $resultPageFactory;        
        return parent::__construct($context);
    }

    public function execute()
    {
        return $this->resultPageFactory->create();  
    }    
    protected function _isAllowed()
    {
        return $this->_authorization->isAllowed('ACL RULE HERE');
    }            

}

There’s a few things to make note of here. First, all Magento backend controllers need to inherit from the Magento\Backend\App\Action class. Not doing so may introduce instability into the system, and open up unintended privacy and security leaks.

The next thing to make note of is the __construct method. While you don’t need a__construct method in your controller, if you do use one you’ll need to include abackend context object

\Magento\Backend\App\Action\Context $context

and make sure you pass that object as the first argument when calling the parent constructor.

return parent::__construct($context);

Context objects are beyond the scope of this article, but in short they’re a cheat the Magento core team uses to inject multiple objects via automatic constructor dependency injection without having multiple constructor arguments. If you didn’t understand that, don’t worry. Just make sure if you use a __construct method, that you follow the above advice.

The last thing to make note of w/r/t to Magento backend admin controllers is the_isAllowed method.

#File: app/code/Pulsestorm/HelloAdminBackend/Controller/Adminhtml/Index/Index.php

protected function _isAllowed()
{
    return $this->_authorization->isAllowed('ACL RULE HERE');
}       

The _isAllowed method is required, and is where you, as the module developer, define which ACL rule a user needs to access this URL endpoint. If _isAllowed returns true, then the user is allowed. If not, they’ll get an access denied screen. By convention, this method uses the object in the _authorization property to check a specific ACL rule. The sole exception to this is any account with super user rights. For super users, the results of_isAllowed are ignored.

The pestle generate_route command does not insert a rule for you automatically — that’s why you see the text ACL RULE HERE above. This is why our route only worked for Magento admin super users. If you edit this file to include the ACL rule we generated withgenerate_acl

protected function _isAllowed()
{
    return $this->_authorization->isAllowed('Pulsestorm_HelloAdminBackend::menu_1');
}       

then any user that’s assigned the Pulsestorm_HelloAdminBackend::menu_1 permission (via System -> Permissions) will be able to access this controller endpoint. By convention, this rule should be the same as the rule for accessing the Menu Item, but it’s up to each individual module developer to decide if they want to enforce this convention.

Create View Files

Now that we’ve created an admin endpoint, you’re probably wondering how to add content to it. Like the front end, Magento’s backend is controlled by Magento’s domain specific language for layouts. Below we’re going to use pestle to add a layout handle XML file, as well as a block and template.

$ pestle.phar generate_view
Which Module? (Pulsestorm_HelloGenerate)] Pulsestorm_HelloAdminBackend
Which Area? (frontend)] adminhtml
Which Handle? (pulsestorm_helloadminbackend_index_index)] pulsestorm_hello_admin_backend_index_index
Block Name? (Main)] Main
Template File? (content.phtml)] content.phtml
Layout (ignored for adminhtml) ? (1column)] 

After running pestle with the above arguments and clearing your cache, reload your page and you should see pestle’s default block content.

The main pestle difference, (compared to the front end), is the Which Area? argument — you’ll need to specify adminhtml.

This command creates three files — a layout handle XML file, a block view file, and aphtml template for the block. First, let’s take a look at the layout handle XML file.

<!-- File: app/code/Pulsestorm/HelloAdminBackend/view/adminhtml/layout/pulsestorm_hello_admin_backend_index_index.xml -->
<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <referenceBlock name="content">
        <block  template="content.phtml" 
                class="Pulsestorm\HelloAdminBackend\Block\Adminhtml\Main" 
                name="pulsestorm_helloadminbackend_block_main" />
    </referenceBlock>
</page>

You’ll see the format is nearly the same as a layout handle XML file for the front end. The main difference is pestle creates this file in the view/adminhtml folder instead ofview/frontend folder. The view/adminhtml folder is where Magento looks for layout handle XML files for backend admin console requests. Keen observers will also notice pestle does not add a layout="..." attribute to the top level page tag. While the backend does support this attribute for setting different page level layouts, most Magento backend layout handle files omit it.

This layout handle XML file adds a block whose class isPulsestorm\HelloAdminBackend\Block\Adminhtml\Main, and whose template iscontent.html. If we look at the class generated for that block

#File: app/code/Pulsestorm/HelloAdminBackend/Block/Adminhtml/Main.php 
<?php
namespace Pulsestorm\HelloAdminBackend\Block\Adminhtml;
class Main extends \Magento\Backend\Block\Template
{
    function _prepareLayout(){}
}

we’ll see two main differences from a front end block. First, by convention, and admin block’s class name includes Adminhtml. While this isn’t strictly necessary, most of Magento uses this convention, and it never hurts to follow along with what the platform owner is doing. Second, and more importantly, admin blocks need to inherit from the baseMagento\Backend\Block\Template class. This class gives admin block objects a different set of injected dependencies (a form key helper, an authorization helper, etc), and a different set of helper methods. While covering these differences is beyond the scope of this article, if you’re curious, take a look at the backend block’s class definition file.

#File: vendor/magento/module-backend/Block/Template.php

Finally, if we take a look at the generated phtml template file.

#File: app/code/Pulsestorm/HelloAdminBackend/view/adminhtml/templates/content.phtml
<h1>This is my template, there are many like it, but this one is mine.</h1>

We see a standard Magento 2 phtml template file. The only difference from a front end template is, again, that we create this file in the adminhtml folder.

At this point, all standard Magento 2 block and template programming techniques apply, and you can build out your admin console page however you want.

Setting the Current Menu and Page Title

There’s two last things we need to cover before moving on. You may have noticed that while navigating the admin, Magento will highlight the left side Menu Item of the page you’re currently on (Dashboard below)

This isn’t the case for our Menu Item. The System menu does not have a highlight. You also may have noticed that our page has a generic Magento Admin title.

There’s two additional steps you’ll need to take in your controller action to match the behavior of the core system. For the menu highlight, you’ll need to use the setActiveMenumethod of the page object

#File: app/code/Pulsestorm/HelloAdminBackend/Controller/Adminhtml/Index/Index.php    

public function __construct(
    \Magento\Backend\App\Action\Context $context,
    \Magento\Framework\View\Result\PageFactory $resultPageFactory)
{
    $this->resultPageFactory = $resultPageFactory;        
    return parent::__construct($context);
}
//...    
public function execute()
{
    $page = $this->resultPageFactory->create();  
    $page->setActiveMenu('Pulsestorm_HelloAdminBackend::a_menu_item');
    return $page;
}    

The setActiveMenu method accepts a single string parameter — the name of the Menu Item you want to set as active. When you add this code, Magento (at the time of this writing) will do two things. The Menu Item’s top level parent (System) will be highlighted and the page’s default title will be set to second level Menu Item’s title (Other Settings).

If you’d like to set a different title, you can do so my directly manipulating the page object’s configuration.

#File: app/code/Pulsestorm/HelloAdminBackend/Controller/Adminhtml/Index/Index.php    
public function execute()
{
    $page = $this->resultPageFactory->create();  
    $page->setActiveMenu('Pulsestorm_HelloAdminBackend::a_menu_item');
    $page->getConfig()->getTitle()->prepend(__('Our Custom Title'));
    return $page;
} 

You’ll notice we’ve wrapped our string in the global “translate this symbol” function (__). This ensures our title is available to be translated into other languages. With the above in place, you’ll see your page title reflected in the browser.

If you’re thinking the syntax and behavior of these features is a little odd, I’m right there with you. The following

  • The setActiveMenu method is only available if we inject an object (vs. being something available directly on the backend controller
  • That setActiveMenu sets a page title, but not the actual page title of a third level menu
  • That the prepend method seems to set a title, but the available set method does not

all point to the backend controller features being unfinished. You’ll also see core controller actions that look more like this

#File: vendor/magento/module-user/Controller/Adminhtml/Locks/Index.php
public function execute()
{
    $this->_view->loadLayout();
    $this->_setActiveMenu('Magento_User::system_acl_locks');
    $this->_view->getPage()->getConfig()->getTitle()->prepend(__('Locked Users'));
    $this->_view->renderLayout();
}

This pattern (loadLayout, renderLayout) may look familiar to Magento 1 developers. Although this technique is present in the core, the _ leading variables are a hint (although not a guarantee) that the developers don’t intend to keep this technique around. Completing/finalizing these features and deciding on one path forward was likely dropped in favor of more visible functionality for the initial release of Magento 2.0. Such is the life of a software developer under the regime of architects and investors.

Wrap Up

With the above techniques you should be able to start building out your admin features. If you’re still having trouble, this Magento Quickies tutorial has some controller action debugging tips that apply to the front and backends.

All that said, we’ve only touched the surface of what’s happening in Magento’s backend. Next time, we’ll cover some advanced backend routing and layout topics, and prepare ourselves for a deeper dive into Magento’s backend UI functionality.

Magento 2: Advanced Routing

Today we’re going to cover a few of the advanced features of Magento’s routing system, and discuss the history of some seemingly acute sharp edges. While all the techniques available in this article may not be the best way to achieve your goals, as a working Magento developer you’ll need to be aware they’re possible, and that other developers (including Magento’s core engineering team) may have used them.

This article assumes you have solid Magento 2 module experience, and it may gloss over details that are critical for a beginner. If that’s you, use the comments to ask your question or point to your Stack Exchange question.

Magento URL Action Strings

As a PHP programmer, when you need to create a URL in Magento, you use the getUrlmethod of a block or URL model object.

Mage::getUrl('foo/baz/bar');
$urlModel->getUrl('foo/baz/bar');
$this->getUrl('foo/baz/bar');

This is true in Magento 1 and Magento 2. In Magento 2 you don’t have access to the globalMage::getUrl method, so you’ll need to inject the specific URL model you want viaautomatic constructor dependency injection. URL models allow PHP programmers to treat URLs as a simple action string, and then have the core system code construct the final rendered URL. If you’ve only ever done front end work, this may seem like overkill, but it’s advantageous if the URL structure needs to change in the future or accommodate multiple contexts. Most web programming frameworks consider this a necessary feature.

We’re not going to dive too deeply into URL models today — what we are interested in is the action string these URL methods accept. At first glance, you may think thefoo/baz/baz structure maps directly to the frontName/controllerName/actionNamestructure of a Magento URL. You’d almost be right. The actual structure isrouteId/controllerName/actionName.

When you setup a routes.xml file for your module, you use an XML structure that looks like this

<!-- File: app/code/Package/Module/etc/[area]/routes.xml -->
<route id="routerId" frontName="urlFrontName">
    <module name="Package_Module" />
</route>

That is, you setup a <route/> node with an id and frontName attributes. The id identifies the route node uniquely in the Magento system, and the frontName defines the first segment of your URL.

When Magento generates a URL from an actions string like foo/baz/bar, it uses the first segment to lookup a <route/> node in the merged XML tree, and then use that route node’s frontName as the first URL segment.

This isn’t an obvious thing, and you may be developing on Magento for years without realizing it. That’s because most modules use route IDs and front names that are identical.

For example, you can see this in the catalog module

<!-- File: app/code/Magento/Catalog/etc/frontend/routes.xml -->
<route id="catalog" frontName="catalog">
    <module name="Magento_Catalog" />
</route>

both the id and frontName attributes are catalog. This convention also existed in Magento 1.

<!-- File: app/code/core/Mage/Catalog/etc/catalog.xml -->
<frontend>
    <routers>
        <catalog>
            <use>standard</use>
            <args>
                <module>Mage_Catalog</module>
                <frontName>catalog</frontName>
            </args>
        </catalog>
    </routers>
    <!-- ... -->
</frontend>

The above is Magento 1’s catalog router configuration. A single <catalog/> node under<routers/> configures a frontName named catalog. In Magento 2, that node under<routers/> has been turned in <module/>, with the id attribute replacing the Magento 1 node name. (i.e. <catalog/> becomes <module id="catalog"/>.

This convention makes it easy to look at a URL action string and get a rough idea of what the final URL will look like. However, there’s one huge exception: Magento’s admin URLs.

Magento Admin URLs

In Magento 1, the main admin route was configured with the following (in the Magento core)

<!-- File: app/code/core/Mage/Adminhtml/etc/config.xml -->
<admin>
    <routers>
        <adminhtml>
            <use>admin</use>
            <args>
                <module>Mage_Adminhtml</module>
                <frontName>admin</frontName>
            </args>
        </adminhtml>
    </routers>
</admin>

That is, the single node in <routers/> named <adminhtml/> sets up a frontName namedadmin. This carries over into Magento 2.

#File: vendor/magento/module-backend/etc/adminhtml/routes.xml 
<route id="adminhtml" frontName="admin">
    <module name="Magento_Backend" />
</route>

This means, in Magento 1, an action string like adminhtml/foo/bar will translate into a full URL that looks something like http://example.magento.com/admin/foo/bar. That is, the first URL segment, and front name, will be admin instead of adminhtml.

This is an odd exception to Magento’s general convention of matching front name and router id, and is likely less a deliberate design decision than it is Magento 1 launching with a half finished concept of areas (adminhtml, frontend) implemented.

Regardless, you can still find evidence of this non-decision in Magento 2’s codebase.

Magento 2 Admin URLs

In Magento 1, the adminhtml router ID is part of what ensured admin URLs began with the string /admin, and that the Magento area be set to adminhtml.

However, in Magento 2, every route setup in etc/adminhtml/routes.xml files will automatically be prepended with the string admin. This is what lets us use our ownfrontName in these adminhtml/routes.xml files.

This ends up having a curious side effect on legacy admin URLs that still use theadminhtml router ID. Consider the URL rewrite module

<!-- File: vendor/magento/module-url-rewrite/etc/adminhtml/menu.xml -->
<add id="Magento_UrlRewrite::urlrewrite" 
     title="URL Rewrites" module="Magento_UrlRewrite"
     sortOrder="20" 
     parent="Magento_Backend::marketing_seo"
     action="adminhtml/url_rewrite/index"
     resource="Magento_UrlRewrite::urlrewrite"
     />

The above menu.xml file uses an action string of adminhtml/url_rewrite/index. Magento ends up generating a URL like this

http://magento.example.com/admin/admin/url_rewrite/index

That’s a URL that begins with /admin/admin. That’s two admin strings. The first comes from the /admin URL segment that Magento prepends to every admin URL. The second comes from Magento using the adminhtml route ID to lookup a frontName attribute.

These appear to be legacy URLs automatically converted into menu items, although when Iasked Magento’s architects about this back in January they were initially confused by the question, and then stated both admin URL formats (adminhtml route IDs/front names and custom route IDs/front names) were valid, but that custom route ID and front names were preferred.

Given the lack of clear initial rules, and the lack of the core team’s adherence to these later decreed rules, working Magento developers will want to be familiar with both URL formats, and be ready to debug them as needed.

URL Route Sharing

If you have not dug too deeply into Magento’s core code, you may be wondering

Wait — I thought each Magento module claimed a specific front name — how can multiple modules claim the admin front name via the adminhtml router?

This brings us to another feature from Magento 1 that made the jump to Magento 2: Route sharing.

Magento’s “1 module, 1 front name” policy traces its roots back to the Zend Framework’s early routing/MVC system. Magento 1, while a framework all its own, based a lot of its work on core Zend Framework classes, and this “1 module, 1 front name” feature came along for the ride. Module developers quickly noticed how limiting this was, and Magento introduced the ability for multiple modules to claim a particular front name.

If that didn’t make sense

  1. When Magento 1 was created, you could only create Controller files for a URL starting with /foo/... in a single module
  2. Magento introduced the ability for multiple modules to have controller files for URLs starting with /foo/ in Magento 1.3

The ability persists in Magento 2, and syntax for it has been greatly simplified. For example, if you wanted to go back to the first module in this series (Pulsestorm_HelloWorldMVVM) and add a front end URL endpoint at catalog/foo/bar, all you’d need to do is add the following configuration

<!-- File: app/code/Pulsestorm/HelloWorldMVVM/etc/frontend/routes.xml-->
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:App/etc/routes.xsd">
    <router id="standard">
        <route id="hello_mvvm" frontName="hello_mvvm">
            <module name="Pulsestorm_HelloWorldMVVM"/>
        </route>

        <!-- START: new configuration -->        
        <route id="catalog">
            <module name="Pulsestorm_HelloWorldMVVM" after="Magento_Catalog"/>
        </route>
        <!-- END:   new configuration -->                
    </router>
</config>

and the following controller file.

#File: app/code/Pulsestorm/HelloWorldMVVM/Controller/Foo/Bar.php
<?php    namespace Pulsestorm\HelloWorldMVVM\Controller\Foo;
class Bar extends \Magento\Framework\App\Action\Action
{    
    public function execute()
    {
        var_dump("Proof of life");
    }
}

with the above in place, you now have a second module with controller files for thecatalog front name, and Magento will use your controller when you load thecatalog/foo/bar URI.

Unlike Magento 1, the configuration for this is almost identical to setting up routing for a single module

<!-- File: app/code/Pulsestorm/HelloWorldMVVM/etc/frontend/routes.xml-->
<route id="catalog">
    <module name="Pulsestorm_HelloWorldMVVM" after="Magento_Catalog"/>
</route>

The two main differences are

  1. You do not configure a frontName attribute in the <route/> tag
  2. You need (or are strongly advised) to use a before or after tag to control the order Magento will check for matches in

When you use the above configuration, you’re telling Magento

Hey, you know that <route/> tag with an id of catalog? I want you to merge in an extra<module/> node.

When Magento encounters multiple <module/> nodes in its global configuration, it will check each module for a controller match until it finds one. Using the after tag above ensures Magento checks our module after the Magento_Catalog module. Without this, it would be possible for us to accidentally create a controller file that replaced the core controllers in vendor/magento/module-catalog/Controller.

It’s important to note that you’re looking for the <route/> id attribute, and not thefrontName attribute here. For example, when a module wants to add to the admin front name, a Magento core developer will

  1. Identify the module that first added the frontName="admin"
  2. Identify that <module/>‘s <route/> id
  3. Use that ID in their own modules
  4. Ensure a proper before or after tag is in place

So, step one — the module that initially added the admin front name is Magento_Backend

#File: vendor/magento/module-backend/etc/adminhtml/routes.xml 
<route id="adminhtml" frontName="admin">
    <module name="Magento_Backend" />
</route>

This <route/> node’s id is adminhtml. So, the other Magento modules that use the adminfront name configure themselves with id="adminhtml".

<!-- File: vendor/magento/module-variable/etc/adminhtml/routes.xml -->
<route id="adminhtml">
    <module name="Magento_Variable" before="Magento_Backend" />
</route>

<!-- File: vendor/magento/module-widget/etc/adminhtml/routes.xml -->
<route id="adminhtml">
    <module name="Magento_Widget" before="Magento_Backend" />
</route>    

Default Action String Segments

Before we move on from advanced routing, there’s a few last things to mention. You may occasionally see an action string that’s missing its second or third segment.

<!-- File: vendor/magento/module-tax/etc/adminhtml/menu.xml -->
<add 
    id="Magento_Tax::sales_tax_rules" 
    title="Tax Rules" 
    module="Magento_Tax" 
    sortOrder="10" 
    parent="Magento_Tax::sales_tax" 
    action="tax/rule" 
    resource="Magento_Tax::manage_tax"/>

When Magento encounters an action string with a missing segment, it will substitute the string index. In other words, the action string of tax/rule used above is functionally equivalent to an action string of tax/rule/index.

Also, in PHP code, you may occasionally see the * character in a URL action string.

$this->getUrl('*/*/*')

These asterisks will be translated as the current front name, controller name, or action name. In other words, they create context dependent URLs, and are useful in base UI classes meant to be used in multiple modules.

Wrap Up

Like a lot of the “in the trenches” decisions made by Magento 2’s non-architects (i.e. the programmers actually implementing the features), it’s not 100% clear why theseadmin/admin URLs stuck around. As third party developers, it’s probably best for us to create our own URL front names, and only rely on the admin front name if there’s some feature that makes it absolutely necessary.

Magento 2: Admin Menu Items

Last time we discussed how Magento serves and generates front end (javascript, css) files to end users from its own modules. Like most of Magento’s feature, if it’s done in a core module, third party developers can do it in their own modules. This time we’ll be creating a module of our own, and using it to add front end javascript/css files to the system.

The specifics in this article refer to the official Magento 2.0 released in the fall of 2015. While the specifics may change in future versions, the concepts should apply to all versions of Magento 2.

A Quick Note on File Permissions

If you take a close look at how Magento handles unix file permissions in its code generation systems, one thing is clear. The core team are not fans of Apache’s mod_phpmodule, and probably run their systems using some sort of PHP-FPM/FastCGI implementation.

As a result, if you’re running PHP with the Apache mod_php module (the most common out of the box way of running PHP) you may end up running into problems with Magento created files. Specifically, files created via command line mode that PHP can’t read/write when running in web server mode, or vice versa.

“The right” thing to do here would be to ensure the apache user and your own shell user are in the unix same group. That’s a non-trivial thing to do though. If you’re not up for it, and you understand the security implications, another way of dealing with it is just chmod 777ing your development files in pub/static and var/ folders. Here’s two quick findcommands that will do this for you.

$ find /path/to/magento2/pub/static -exec chmod 777 '{}' +
$ find /path/to/magento2/var/ -exec chmod 777 '{}' + 

This isn’t my favorite approach, but until Magento 2 gets its permission situation in order it’s the only simple way of dealing with your development environment.

Creating the Module

Concerns about file permissions aside — step one is creating a new module. We’re going to run through the steps, cookbook style, below. If you’re curious on a more in depth look at creating modules in Magento, try our Introduction to Magento 2 — No More MVC article.

We’re going to name our module Pulsestorm_FrontendTutorial1. To create this module, we’ll need to create two files. First, create the module.xml file.

#File: app/code/Pulsestorm/FrontendTutorial1/etc/module.xml
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../lib/internal/Magento/Framework/Module/etc/module.xsd">
    <module name="Pulsestorm_FrontendTutorial1" setup_version="0.0.1" />
</config>

Then, create the register.php file

#File: app/code/Pulsestorm/FrontendTutorial1/registration.php
<?php
    \Magento\Framework\Component\ComponentRegistrar::register(
        \Magento\Framework\Component\ComponentRegistrar::MODULE,
        'Pulsestorm_FrontendTutorial1',
        __DIR__
    );?>

With these two files in place, use Magento’s CLI program to enable your module.

$ php bin/magento module:enable Pulsestorm_FrontendTutorial1
The following modules have been enabled:
- Pulsestorm_FrontendTutorial1

To make sure that the enabled modules are properly registered, run 'setup:upgrade'.
Cache cleared successfully.
Generated classes cleared successfully. Please re-run Magento compile command
Info: Some modules might require static view files to be cleared. Use the optional --clear-static-content option to     clear them.

And then run the CLI’s setup:upgrade command.

$ php bin/magento setup:upgrade
Cache cleared successfully
File system cleanup:
/Users/alanstorm/Sites/magento-2-with-keys/magento2/var/generation/Composer
/Users/alanstorm/Sites/magento-2-with-keys/magento2/var/generation/Magento
/Users/alanstorm/Sites/magento-2-with-keys/magento2/var/generation/Symfony
Updating modules:
Schema creation/updates:
Module 'Magento_Store':

//...

Module 'Pulsestorm_FrontendTutorial1':

Please re-run Magento compile command

After doing the above, your (functionless) module will be installed into the Magento system.

System Assumptions

If you finished the article from last week, you know that Magento’s front end file serving behaves differently depending on

  1. The root folder you’ve chosen for your website
  2. The “mode” Magento is running in

We’re going to start this article assuming

  1. That you’ve setup your web site’s root folder as the pub/ folder
  2. That you’re running in developer mode

If you’re unsure of how to do this, or curious why we need to have this disclaimer, be sure to read Magento 2: Serving Front End Files

Adding Front End Files

Our goal for this article is to have URLs similar to the following

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

return javascript files from our module. Let’s start with hello.js. First, add the following file to our Magento module.

#File: app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello.js
alert("Hello World");

With the above in place, load the following URL in your browser or via a command line program like curl

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

You should see the contents of your file! Congratulations, you’ve just added your first front end file to a Magento 2 system

What Just Happened — Module Files

Magento 2 allows a module developer (some might say forces them) to include javascript and CSS files under their main module folder. The old Magento top level skin folder is gone, and app/design is reserved exclusively for Magento theme files.

The top level module view folder (app/code/Pulsestorm/FrontendTutorial1/view) above, is where a module developer places all files related to Magento 2’s user interface. This includes the front end files we’re interested in today, as well as Magento’s layout handle XML files and phtml template files.

The next folder, view, is the area folder. Areas area a way to split individual Magento 2 applications into different areas of functionality, based on the URL. i.e. The cart application is Magento’s frontend area, the backend admin console is the adminhtml area.

So what’s the base area? This is one of those places where Magento 2 has improved on Magento 1 — the base area is a special folder that will allow you to serve your files fromeither the frontend or adminhtml areas. We’ll talk more about this below.

Next up is the web folder. Files in web are ones that will be served via http or https. While they’re beyond the scope of this article, other folders at this level are email, layout,page_layout, templates, and ui_component.

Finally, we have our file, hello.js. Notice, unlike Magento 1, there’s no need for us to create a sub-folder for our assets. Because these files already live in a Magento module folder, Magento’s smart enough to know where they should go. That said, some Magento core modules still separate out these files with an additional folder/namespace. While you’re free to do this, there’s no need to do this.

What Just Happened — Asset URLs

Module files explained, next up let’s take a look at that URL.

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

We’re going to examine each URL segment and describe where it comes from. For the most part, you won’t need to remember all this — Magento will do most of the URL generating for you. However, understanding the URL asset path will be useful if you’re debugging a system that’s returning 404s, 500s, or some other error when browsers request front end asset files.

The first part of a Magento 2 front end asset URL is static. This points to the actual

pub/static

folder in your Magento install. If you’re using the root level index.php file instead ofpub/index.php, this should be

http://magento.example.com/pub/static/...

The next URL segment is the Magento area. In our example URL above, this is frontend. However, because we created a file in the base area folder, we can also fetch this file via the following URL

http://magento.example.com/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

i.e., using adminhtml instead of frontend. We’ll talk more about this in the area section below — for now all we need to know is this URL segment is the area.

The next URL segment is Magento — this is the vendor prefix for the theme name. This is similar to the design package in Magento 1. This is Magento because the theme we’re using is in the app/design/frontend/Magento folder.

After the vendor prefix is the theme’s actual name. In our case, we’re using the blanktheme in the URL. However, we could also use the luma theme that ships with Magento 2.0, or any theme installed in the system.

The penultimate folder, en_US, is the locale folder. Magento allows different front end asset files per locale — think of hard coded language strings in javascript, or locale specific images in CSS files. If we wanted a version of our file for french, we’re use fr_FR in the URL

http://magento.example.com/static/adminhtml/Magento/blank/fr_FR/Pulsestorm_FrontendTutorial1/hello.js

and create our french version of hello.js in

app/code/Pulsestorm/FrontendTutorial1/view/base/web/i18n/fr_FR/hello.js

Notice the i18n folder — this stands for internationalization, and is topic well beyond the scope of this, or any single, article.

Our final url segment is the name of our module — Pulsestorm_FrontendTutorial1. This is why Magento 2 doesn’t need you to self-organize your files in the view/[area]folder. By generating URLs with the module name in them, Magento 2 has enough information to find the file in your module folder. Change this segment of the URL, and Magento won’t find your file.

http://magento.example.com/static/science/Magento/blank/en_US/Some_OtherModuleWithAHelloJs/hello.js

With developer mode enabled, Magento will use each of these URL segments to find the correct file, read it into memory via PHP, and echo it back out to the end user. While this is appropriate for an individual developer’s machine, this sort of dynamic lookup would be unacceptable for a medium to high traffic production system.

That’s where static asset generation comes into play.

Generating Static Assets for Production

If you’ve read the previous article in this series, you know the first time we accessed each of the above URLs, Magento automatically generated a file for us in pub/static. In the first non-beta Magento 2.0 release, these files generated during development mode are symlinks to the actual file. You can see this for yourself with ls -lh

$ ls -lh pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js 
lrwxrwxrwx  1 _www  staff   112B Dec 27 11:19 pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/ hello.js -> /path/to/magento2/app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello.js

This is a common approach for many PHP frameworks. However, another common approach is to just copy the file over the first time its requested. There’s no clear right way here, and I wouldn’t be surprised to see the Magento core team flip-flop its approach in a future release.

Regardless of whether they’re symlinks or actual files, generating these on the fly is inappropriate for a production environment. In addition to putting additional burden on the server’s file system, it also introduces a potential attack vector for black hat hackers. That’s why if we remove the generated file(s)

$ rm pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
$ rm pub/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

and then switch our system into production mode

#If you're having trouble, did you check pub/.htaccess for mode setting?
#File: .htaccess
SetEnv MAGE_MODE production

Magento will return a blank screened, 404 for the file.

$ curl -i 'http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js'
HTTP/1.1 404 Not Found
Date: Sun, 27 Dec 2015 20:30:30 GMT
Server: Apache/2.4.10 (Unix) PHP/5.6.16
X-Powered-By: PHP/5.6.16
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=UTF-8

When you deploy your Magento application, you need to generate these files on your own. You can do this by running the following command

$ php bin/magento setup:static-content:deploy
Requested languages: en_US
=== frontend -> Magento/blank -> en_US ===
.....................................

...

---

New version of deployed files: 1451248980

After the above finishes running, you’ll have a statically generated file for every web file inevery Magento module. We can see this if we use the command line find program to look for generated hello.js file.

$ !find
find pub/ -name hello.js
pub//static/adminhtml/Magento/backend/en_US/Pulsestorm_FrontendTutorial1/hello.js
pub//static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
pub//static/frontend/Magento/luma/en_US/Pulsestorm_FrontendTutorial1/hello.js

Here we can see that Magento has created three hello.js files. One is for the backendtheme in the Magento adminhtml area. The other two are for the frontend area — one for the blank theme, another for the luma theme.

Before we move on to our promised discussion of areas — there’s a few things to take note of. First (at the time of this writing), the files generated for production are not symlinks, and are actual copies of the file

$ ls -lh pub/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js 
-rw-rw-rw-  1 alanstorm  staff    21B Dec 27 13:10 pub/static/frontend/Magento/blank/en_US/ Pulsestorm_FrontendTutorial1/hello.js

This means if you change something directly in a module folder on your production system, you won’t see the change reflected on the live site. Also, if you do directly edit the files in the pub/static sub-folders, your changes will be deleted the next time someone deploys to the production server.

Another thing to watch out for here: If the symlinks from development mode are still present in the pub/static sub-folders when you run setup:static-content:deploy, Magento will not remove them. On one hand — this shows whomever implementedsetup:static-content:deploy cared enough to make sure their command wasn’t destructive. On the other hand — if your deployment procedure isn’t super tight, this means you may end up with symlinks on your production website.

Understanding the Area Hierarchy

The first thing we had you do in this tutorial was create the following file

#File: app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello.js
alert("Hello World");

This added the hello.js file to both the adminhtml and frontend areas.

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
http://magento.example.com/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

It’s also possible, and far more common, to have a javascript or css file that’s only needed in one of the two areas. For example, try creating the following file

#File: app/code/Pulsestorm/FrontendTutorial1/view/frontend/web/hello-2.js
alert("Hello Frontend World");

Notice its location is almost exactly the same as our original file — the only difference is we’ve replaced the base folder with a folder named frontend. With the above in place, and developer mode reenabled, try loading the file’s corresponding URL

$ curl -i 'http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello-2.js'
HTTP/1.1 200 OK
Date: Mon, 28 Dec 2015 02:06:30 GMT
Server: Apache/2.4.10 (Unix) PHP/5.6.16
Last-Modified: Mon, 28 Dec 2015 02:04:32 GMT
ETag: "1e-527ebb9d38c00"
Accept-Ranges: bytes
Content-Length: 30
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Cache-Control: public
Content-Type: application/javascript

alert("Hello Frontend World");

You should see your file returned without a hitch. However, if you attempt to load the same file using an adminhtml area, you should see a 404 Not Found error.

$ curl -I 'http://magento.example.com/static/adminhtml/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello-2.js'
HTTP/1.1 404 Not Found

If there’s a file in both a specific area folder and the base area folder

app/code/Pulsestorm/FrontendTutorial1/view/base/web/hello-2.js
app/code/Pulsestorm/FrontendTutorial1/view/frontend/web/hello-2.js

the file in the specific area folder (frontend above) will win out.

Generally speaking, you should try to keep your front end asset files in the appropriate area folder. If you need a javascript file that adds features for the backend admin console, it’s best to keep that in the adminhtml folder. However, if there’s a library file your module needs in both locations, then using the base folder is appropriate.

Developer Mode Quirks

Finally, when you’re working with your system in developer mode, there are a few quirks to be aware of. Consider the URL for our original helloworld.js file

http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

If you change the URL to use a fake area like the one below

http://magento.example.com/static/fake-area-that-is-not-there/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

You’ll be surprised to find out Magento still returns the file found in the base folder

$ curl -I http://magento.example.com/static/fake-area-that-is-not-there/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
HTTP/1.1 200 OK

The same is true for the theme vendor, theme name, and locale portions of the URL (replaced with bar, foo, and baz below)

http://magento.example.com/static/fake-area-that-is-not-there/bar/foo/baz/Pulsestorm_FrontendTutorial1/hello.js

This behavior can go from curious anomaly to crazy making if you’re manually creating URLs (say, for a tutorial). Consider the following typos

http://magento.example.com/static/frotnend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
http://magento.example.com/static/frontend/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

http://magento.example.com/static/front-end/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js
http://magento.example.com/static/fronternd/Magento/blank/en_US/Pulsestorm_FrontendTutorial1/hello.js

Fortunately though, as hinted at throughout this article, you shouldn’t need to manually create paths to front-end static assets. Magento 2 contains a number of different systems for pulling in javascript and CSS files, and a number of PHP objets and methods for generating these paths programmatically.

Now that we have a better understanding of how Magento serves front end static assets, and also understand how we can add individual front end assets to our own modules, our next steps in exploring the various Magento systems that will let you use these assets. That’s where we’ll start next time.

Magento 2: Understanding Access Control List Rules

The Magento backend application, (sometimes called “The Admin” or adminhtml area), is where a system owner manages their Magento store. This is where users interact with web forms to add new products, change configurations, etc. Magento is a multiuser application — i.e. a business owner may have a backend account for herself, but also give each individual member of her staff an account to access the Magento backend. Furthermore, that business owner can turn off features for different accounts.

For example, the customer support staff may only have access to the customer and orders sections, while the sales staff may have access to both these sections and the marketing section. In the Magento backend, a system owner can accomplish this via the System -> Users and Systems -> Roles sections. These two sections implement an authenticationand authorization system.

For those of you too busy to read the wikipedia article, authentication is the act of ensuring a user is who they say they are. In simplified terms, this is the user entering an account name and password in a login screen. Systems with a higher level of general security or special PCI compliance requirements are often required to implement two factor authentication. A common two factor authentication process is a password, combined with an SMS/Text message sent to their phone.

Once a user proves who they are, the next step is an authorization system. Authorization systems implement rules that say what a user is allowed to do in a system. Magento’s authorization system allows a system owner to

  1. Create an unlimited number of logical Roles. Some example of roles might include Sales Staff, Support Staff, IT Staff, Contract Developers, Executive Team, etc.
  2. Assign a set of Access Control List (ACL) rules to each individual role

Each access control rule defines a specific permission granted to the user in the system. You can see a list of these rules by navigating to

System -> User Roles -> Add/Edit Role -> Role Resources

and selecting Custom from the drop down menu

Each individual rule controls access to a system feature. Tailoring a set of rules into a set of Roles that an individual business can use to run their online store is one of the many things a Magento system integrator or store owner will need to do.

The special Resource Access: all role is a super user role. These users are granted access toevery resource in the system.

ACL Rules for Developers

As a module developer, ACL rules present a few interesting challenges. First, there are several places that you, as a module developer, are expected to add ACL rule checks to your module. A few examples

  1. Every URL endpoint/controller in the admin application must implement an_isAllowed method that determines if a user can access the URL endpoint.
  2. Every Menu Item in the left hand navigation also has a specific ACL rule that controls whether or not the menu displays for the logged in user. This is often the same rule from _isAllowed)
  3. Every configuration field in System -> Configuration has a specific ACL rule that controls whether or not the menu displays

Despite being required fields, there are no hard and fast rules as to how a module developer should setup and structure their own rules. Also, a module developer will likely want additional rules that are specific to their module. This article can’t answer these hard questions for you, but we will show you how to check the current user against a specific ACL rule, look up ID values for existing rules, and how to create your own tree of ACL rules.

Looking up a Rule ID

In Magento 2, each specific ACL rule is assigned an arbitrary string ID. You can find these IDs in a module’s etc/acl.xml file. For example, Magento defines the Stores -> Settings -> Google API ACL rule here

<!-- File: vendor/magento/module-google-analytics/etc/acl.xml -->
<?xml version="1.0"?>
<!--
/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Acl/etc/acl.xsd">
    <acl>
        <resources>
            <resource id="Magento_Backend::admin">
                <resource id="Magento_Backend::stores">
                    <resource id="Magento_Backend::stores_settings">
                        <resource id="Magento_Config::config">
                            <resource id="Magento_GoogleAnalytics::google" title="Google API" />
                        </resource>
                    </resource>
                </resource>
            </resource>
        </resources>
    </acl>
</config>

You can find the specific rule by its title attribute. If you’re having trouble, it’s this one

<!-- File: vendor/magento/module-google-analytics/etc/acl.xml -->    
<resource id="Magento_GoogleAnalytics::google" title="Google API" />    

The ACL rule with the title “Google API” has an ACL ID ofMagento_GoogleAnalytics::google. This is the ID you’ll use to check if the logged in user has access to this resource.

ACL IDs are mostly arbitrary strings. By convention, a Magento 2 ACL rule ID name is

  1. The name of the module
  2. Followed by two colon characters ::
  3. Followed by a lower case string describing the rule’s purpose

Also, Magento’s new XSD/XML validation system

<!-- File: vendor/magento/framework/Acl/etc/acl.xsd -->
<xs:simpleType name="typeId">
    <xs:annotation>
        <xs:documentation>
            Item id attribute can has only [a-z0-9/_]. Minimal length 3 symbol. Case insensitive.
        </xs:documentation>
    </xs:annotation>

    <xs:restriction base="xs:string">
        <xs:pattern value="([A-Z]+[a-zA-Z0-9]{1,}){1,}_[A-Z]+[A-Z0-9a-z]{1,}::[A-Za-z_0-9]{1,}" />
    </xs:restriction>
</xs:simpleType>

forces the rule to conform to the following regular expression

([A-Z]+[a-zA-Z0-9]{1,}){1,}_[A-Z]+[A-Z0-9a-z]{1,}::[A-Za-z_0-9]{1,}

All in all, while it’s possible to deviate slightly from Magento’s convention, it’s best to stick to it.

If you’re curious about the placement of the Magento_GoogleAnalytics::google rule inside the node structure of acl.xml files (i.e. what are its parent nodes), an acl.xml file is an infinitely deep XML tree of <resource/> nodes, with each node defining a new level in the ACL hierarchy. If that didn’t make sense, don’t worry, we’ll be running through a few examples later that should clear things up.

Before we move on to creating our own ACL rules, Magento 1 developers will want to take note. Magento 1 ACL rules did not have an explicit ID. Instead, the system derived an ID from the names of the XML nodes that defined the rules (i.e. foo/baz/bar). This means, in Magento 2, you can no longer look at an ID and be sure where it lives in the hierarchy, you can only be sure of the module that created it.

When I’m hunting through a Magento 2 installation for a specific ACL rule, I use a number of different unix commands in a terminal to search through the acl.xml files. For example, to find the Google API rule above, I used the following

$ find vendor/magento/ -name 'acl.xml' -exec grep -i 'Google API' '{}' +
vendor/magento//module-google-analytics/etc/acl.xml:
<resource id="Magento_GoogleAnalytics::google" title="Google API" />

This searches every acl.xml in vendor/magento for the string Google API.

Creating Your Own ACL Rules

When you create Magento admin features, you’ll need to add ACL rules to your module. We’re going to show you how to do this using pestle, and then explain what each file pestle created is for. If you’re not familiar with it, pestle is a free and open source PHP command line system that features a number of useful Magento 2 code generation tools.

First, we’ll want to create a blank module named Pulsestorm_AclExample. You can create the base module files using pestle’s generate_module command.

$ pestle.phar generate_module Pulsestorm AclExample 0.0.1

and then enable the module in Magento by running the following two commands

$ php bin/magento module:enable Pulsestorm_AclExample
$ php bin/magento setup:upgrade    

If you’re interested in creating a module by hand, or curious what the above pestle command is actually doing, take a look at our Introduction to Magento 2 — No More MVCarticle.

Next, we’re going to use pestle’s generate_acl command.

# with interactive input

$ pestle.phar generate_acl
Which Module? (Pulsestorm_HelloWorld)] Pulsestorm_AclExample
Rule IDs? (Pulsestorm_AclExample::top,Pulsestorm_AclExample::config,)] 
Created /path/to/magento/app/code/Pulsestorm/AclExample/etc/acl.xml

#without interactive input
$ pestle.phar generate_acl Pulsestorm_AclExample Pulsestorm_AclExample::top,Pulsestorm_AclExample::config

The first argument (Which Module?) lets pestle know which module’s acl.xml file it should use or create. The second argument (Rule IDs), is a comma separated list of Rule IDs — each comma represents a <resource/> sub-node in acl.xml.

If that didn’t make sense, open up the generated acl.xml

<?xml version="1.0"?>
<!-- File: app/code/Pulsestorm/AclExample/etc/acl.xml -->    
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Acl/etc/acl.xsd">
    <acl>
        <resources>
            <resource id="Magento_Backend::admin">
                <resource id="Pulsestorm_AclExample::top" title="TITLE HERE FOR">
                    <resource id="Pulsestorm_AclExample::config" title="TITLE HERE FOR"/>
                </resource>
            </resource>
        </resources>
    </acl>
</config>

Here you can see pestle created a Pulsestorm_AclExample::top rule that’s a parent of the Pulsestorm_AclExample::config rule (i.e. the first two rules in our comma separated list). You’ll also notice pestle created both these rules under the Magento_Backend::adminresource — all Magento ACL rules go under this node — it’s the top level node.

Let’s edit this file to give our rules some less generic titles

<!-- File: app/code/Pulsestorm/AclExample/etc/acl.xml -->    

<resource id="Pulsestorm_AclExample::top" title="Pulse Storm ACL Example Module">
    <resource id="Pulsestorm_AclExample::config" title="The First Rule"/>
</resource>

And, while we’re at it, lets add a second rule!

<!-- File: app/code/Pulsestorm/AclExample/etc/acl.xml -->    

<resource id="Pulsestorm_AclExample::top" title="Pulse Storm ACL Example Module">
    <resource id="Pulsestorm_AclExample::config" title="The First Rule"/>
    <resource id="Pulsestorm_AclExample::more_rules" title="The Second Rule"/>        
</resource>

With the above in place, clear your cache, and take a look at the Roles menu at System -> User Roles -> Add/Edit Role -> Role Resources. You should see your rules added to the system

That’s all there is to it! Your rules can now be assigned to admin roles in the system, and those admin roles can be assigned to users.

Programmatically Checking ACL Rules

The following assumes some base familiarity with Magento’s object and automatic constructor dependency injection systems, although if you’re not 100% comfortable with these concepts, just power through for the code snippets you’ll need!

Magento provides an abstract type, Magento\Framework\AuthorizationInterface, which a client programmer (you!) can use to validate the currently logged in user against a specific access control rule. i.e., if you were playing fast and loose with Magento’s Don’t use the Object Manager guidelines, the following

$auth = $object_manger->get('Magento\Framework\AuthorizationInterface');
if($auth->isAllowed('Pulsestorm_AclExample::config'))
{
    //user is logged in here
}
else
{
    //user is not logged in here
}

would check if the currently logged in user was assigned ourPulsestorm_AclExample::config rule. If you’re not playing fast and loose with Magento’s Don’t use the Object Manager guidelines, you can inject the auth checking object with something like this

public function __construct(Magento\Framework\AuthorizationInterface $auth)
{
    $this->authorization = $auth;
}

If you’re in a controller that extends the \Magento\Backend\App\Action controller, you automatically have access to the authorization checking object via the _authorizationproperty.

namespace Pulsestorm\HelloAdmin\Controller\Adminhtml\Index;
class Index extends \Magento\Backend\App\Action
{
    protected function someControllerMethod()
    {
        return $this->_authorization->isAllowed('Pulsestorm_HelloAdmin::pulsestorm_helloadmin_index_index');
    }            

}

Regarding the controller method above — if you’re injecting additional arguments via the__construct method, don’t forget to include the admin context object (Magento\Backend\App\Action\Context). This context object is where the auth checking object is, itself, instantiated and injected.

class Index extends \Magento\Backend\App\Action
{
    protected $resultPageFactory;
    public function __construct(
        \Magento\Backend\App\Action\Context $context,
        \Magento\Framework\View\Result\PageFactory $resultPageFactory)
    {
        $this->resultPageFactory = $resultPageFactory;        
        return parent::__construct($context);
    }
    //...
}

Finally, for the curious, in a stock Magento install (circa spring 2016), theMagento\Framework\AuthorizationInterface object type resolves to aMagento\Framework\Authorization object. The class for this object is found here

#File: vendor/magento/framework/Authorization.php

If you’re having trouble with ACL rule debugging, this is where you’ll want to start.

Wrap Up

Access Control Rules are an important, but often overlooked part, of Magento extensions. While an extension’s functionality and ability to solve a business or technical problem is paramount, giving your extension users the ability to turn certain admin features on and off for certain users can often be the differences between a manager or store owner choosing your extension over a competitor’s.

However, even if you don’t want to slice your extension’s functionality into narrow bands, there are places in the Magento Admin where you’ll need to add ACL rules. Next time we’ll explore one of these sections, when we cover how to create Magento Admin MCV/MVVM controller endpoints.

Magento 2: Understanding Object Repositories

Last time we explored database access in Magento via creating simple Magento 2 CRUD objects, and we explored the various source files involved. While some of the window dressing has changed, Magento 1 developers probably felt right at home. Conceptually, Magento 1’s Model/ResourceModel/Collection ORM still exists in Magento 2. Today we’re going to discuss a new feature of Magento 2’s Model layer — repository objects.

The Repository Pattern

One of Magento 2’s goals was a complete overhaul of the API system. Magento needed to

  1. Provide a modern RESTful based API with oAuth authentication
  2. Keep a SOAP based API for corporate shops/merchants that speak SOAP.
  3. Maintain feature parity between the API regardless of REST/SOAP/FutureAPI
  4. All while refactoring Magento’s underlying business objects and the ORM layer
  5. Do all this with a team that had various levels of experience with Magento 1

One of the tools the core team used to rein in the resulting chaos was the repository pattern. Like many design patterns, the repository pattern is an old one that first came to prominence in java and C++ based systems. These systems needed to map data objects (or “business objects”) to virtual/machine memory. Like all the great 90s design patterns, the repository has been used, abused, and repurposed for different means in the ensuing decades.

To my mind, the best way to understand repository objects is

A repository object is responsible for reading and writing your object information to an object store (i.e. a database, the file system, physical memory, etc.), freeing the repository-user-programmer from worrying about computer problems, and letting them focus on using the data in their business objects and business logic

In other words, a repository becomes the source of truth for fetching and saving objects. With this source of truth in place, the API team was able to use these repositories in thewebapi.xml files while other teams worked on the refactoring effort.

Understanding Magento’s use of repositories is an important part of being a Magento 2 programmer, but at this point in Magento 2’s lifecycle, repositories are not ready to shoulder the full burden of Magento 2’s model layer.

This article will explore using Magento 2’s repositories to fetch and manipulate objects, but also serve as a critique of where the repository implementation introduces new unneeded complexity and confusion. Consider this your guide through the fire swamp.

Article Conventions

We’re going to work through some code examples using a Magento command line script. To simplify things, we’ll be fetching and instantiating objects directly via the object manager. At the end of this article, we’ll provide a class that shows how to use automatic constructor dependency injection with these objects. While direct use of the object manager is discouraged for product systems, it can be useful for teaching users about the underlying objects, and for exploring areas of the system you’re not familiar with.

If any of the above, or the following, sounds like ancient greek (and you’re not a time traveler!), you may want to review our object manager series and the articles so far in this series, in particular the first article where we discuss manually creating a Magento module. You should be able to follow along without fully understanding everything we’ve done so far, but there articles might help you fill the gaps in your understanding.

We’ll use pestle to create a module for our command line application.

$ pestle.phar generate_module Pulsestorm RepositoryTutorial 0.0.1

and then add a command class with pestle’s generate_command

$ pestle.phar generate_command Pulsestorm_RepositoryTutorial Examples    

We’ll enable our module with the Magento command line

$ php bin/magento module:enable Pulsestorm_RepositoryTutorial

and then install it with the setup:upgrade command

$ php bin/magento setup:upgrade

Finally, we can test that the command was added successfully by running

$ php bin/magento ps:examples
Hello World

and looking for the “Hello World” output.

Injecting the Object Manager

The example command we just created should be in the following file.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php

We’ll need to modify this command slightly to give us access to the object manager, as well as make a small adjustment related to Magento’s areas. Add the following property and constructor to your command file.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php
class Examples extends Command
{
    //...    
    protected $objectManager;
    public function __construct(
        \Magento\Framework\ObjectManagerInterface $objectManager,
        \Magento\Framework\App\State $appState,
        $name=null
    )
    {
        $this->objectManager = $objectManager;
        $appState->setAreaCode('frontend');
        parent::__construct($name);
    }
    //...
}

This constructor does two things. First, it uses automatic constructor dependency injection to insert an object manager instance we’ll use later. Second, it uses automatic constructor dependency injection to insert a Magento\Framework\App\State object to work around some issues related to Magento areas. The later is beyond the scope of this article, but we’ve talked about it a bit over on Magento Quickies.

For the remainder of this article, we’ll be running code samples from the execute method.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php

protected function execute(InputInterface $input, OutputInterface $output)
{
    $repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');
    $page = $repo->getById(2);        
    echo get_class($page),"\n";
}

With that out of the way, we’re ready to get started!

Getting Started with Repositories

To start with, a repository is just another object. For each “business object” (product, category, CMS page, etc), there’s a single corresponding repository object. We’ll start with CMS Page objects (cms/page objects for folks still thinking in terms of Magento 1,Magento\Cms\Model\Page objects for folks thinking in terms of Magento 2). If you’re not familiar with them, CMS page objects hold all the data (content, title, additional layout rules, etc.) related to a single page in Magento’s built-in CMS.

The repository object for CMS objects is a Magento\Cms\Model\PageRepository object. You can use this object to load a CMS Page, by its id, with the getById method.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php

$repo = $this->objectManager->get('Magento\Cms\Model\PageRepository');
$page = $repo->getById(2);

echo $page->getTitle(),"\n";    

If you have a CMS Page object in your system with an ID of 2, the above code will output its title.

When you’re looking for a business object’s corresponding repository object, there’s nothing formal in the system that ties a particular repository to a particular object. Generally speaking though, the pattern of appending Repository to the base business object class name holds

Magento\Cms\Model\Page
Magento\Cms\Model\PageRepository

If you take a look at the source of the PageRepository

#File: vendor/magento/module-cms/Model/PageRepository.php

namespace Magento\Cms\Model;
//...
class PageRepository implements PageRepositoryInterface
{    
    //...
}

you’ll see the repository implements a PageRepositoryInterface. If we take a look at this interface,

#File: vendor/magento/module-cms/Api/PageRepositoryInterface.php

interface PageRepositoryInterface
{

    public function save(\Magento\Cms\Api\Data\PageInterface $page);

    public function getById($pageId);

    public function getList(\Magento\Framework\Api\SearchCriteriaInterface $searchCriteria);

    public function delete(\Magento\Cms\Api\Data\PageInterface $page);

    public function deleteById($pageId);
}

We see there’s five methods the page repository must implement. (save, getById,getList, delete, and deleteById). We’ve already seen one of them — the getByIdmethod. This method fetches an object from the system by its assigned database ID. Thesave method will persist an object to the database, the delete and deleteById methods are for removing an object, and the getList method will fetch a number of objects from the database based on search criteria. The remainder of this article will explore each of these methods.

One last thing before we leave the interface behind — it’s important to note that eachrepository in Magento 2 implements its own interface.

#File: vendor/magento/module-catalog/Model/ProductRepository.php
class ProductRepository implements \Magento\Catalog\Api\ProductRepositoryInterface
class OrderRepository implements \Magento\Sales\Api\OrderRepositoryInterface    
etc.

While most (if not all?) business object repositories in Magento 2 share the save, getById,getList, delete, and deleteById methods, there’s nothing in the system forcing them to. If you’re having trouble with a particular repository type, it’s always a good idea to investigate the underlying interface, and its implementation in the concrete repository class.

Persisting to/from the Database

Coming back to our client code, so far we’ve loaded an object using the repository.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');
$page = $repo->getById(2);    
echo $page->getTitle(),"\n";   

Next up is saving objects. In the Magento 1 view of the world, this was as simple as callingsave on the object you wanted to persist to the database

//magento 1
$page->save();

However, this violates a core repository principle — the logic that saves an object’s data to the system should not be part of the business object. Instead, in Magento 2, you tell your repository to save an object. If you give the following code a try, you’ll find your page object saved with an appended title.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Cms\Model\PageRepository');
$page = $repo->getById(2);    
echo $page->getTitle(),"\n";  
$page->setTitle($page->getTitle() . ', Edited by code!');
$repo->save($page);

If you wanted to duplicate a page, you could load the page to duplicate, set its ID to NULL, and pass the duplicated object to the repository.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Cms\Model\PageRepository');
$page = $repo->getById(2);
$page->setId(null);
$page->setTitle('My Duplicated Page');
$repo->save($page);                
echo $page->getId(),"\n";

The delete methods work similarly — either pass the repository a full object to delete, or pass an ID

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo->delete($page);
$repo->deleteById($page_id);

The deleteById method is useful if you have a database ID from a previous operation and don’t want to load a new object just to delete it.

Getting a List of Objects

Nothing we’ve covered so far is too complicated. Instead of objects using their own methods to interact with the database, we’re using the repository’s.

The biggest change you’ll need to overcome with repositories is the syntax/APIs for fetching a list of objects. Magento 1 provided collection objects, and collection objects exposed a SQL-like API for fetching objects.

With repositories, collections have been replaced with a getList method. While using this method may seem simple on the surface, there’s hidden complexity at every step along the way.

If you try calling the getList method (notice we’ve switched to a ProductRepository)

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');        
$repo->getList();

you’ll get an error like the following

Argument 1 passed to Magento\Catalog\Model\ProductRepository\Interceptor::g etList() must implement interface Magento\Framework\Api\SearchCriteriaInter face, none given

Whenever you call getList, you need to pass in a “search criteria” object. We’ll discuss why in a moment, but for now just give the following a try.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');        
$search_criteria = $this->objectManager->create(
    'Magento\Framework\Api\SearchCriteriaInterface'
);
$result = $repo->getList($search_criteria);
$products = $result->getItems();
foreach($products as $product)
{
    echo $product->getSku(),"\n";
}

Running our command with the above code in the execute method should output a list of product SKUs. We’ve used the object manager to create a search criteria object and passed that search criteria object in to the getList method. After doing this, you can use the result object’s getItems method to grab a PHP array of the returned product objects, and thenforeach over that array to get the actual product objects.

Using Search Criteria

Of course, it’s rare that you want every object of a particular type in the system. More often than not, you want a subset of objects that match some search criteria. For example, you may want all SKUs that start with the text WSH11. This is where we actually use the search criteria object.

A search criteria object contains a number of grouped filter objects. These filter objects control what objects the repository will return.

If that didn’t make sense, a code sample might. The following code implements the “all SKUs that start with the text WSH11” use case we described earlier.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter = $this->objectManager->create('Magento\Framework\Api\Filter');
$filter->setData('field','sku');
$filter->setData('value','WSH11%');
$filter->setData('condition_type','like');

//add our filter(s) to a group
$filter_group = $this->objectManager->create('Magento\Framework\Api\Search\FilterGroup');
$filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager->create('Magento\Framework\Api\SearchCriteriaInterface');
$search_criteria->setFilterGroups([$filter_group]);

//query the repository for the object(s)
$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');                        
$result = $repo->getList($search_criteria);
$products = $result->getItems();
foreach($products as $product)
{
    echo $product->getSku(),"\n";
}

The above code

  1. Creates a LIKE filter for SKUs using the % wildcard (you can find a list of valid condition_types invendor/magento/framework/Api/CriteriaInterface.php)
  2. Creates a filter group object, and adds our single filter to that group
  3. Adds the filter group to our search criteria object
  4. Uses the search criteria object to fetch the product objects we want from the repository

If you run the above code, you should get a list of products whose SKUs begin with WSH11.

If all that arbitrary setData seems a little loosey goosey, you’re right. Magento providesBuilder objects to build your filter, filter group, and search criteria objects. The above is equivalent to the following, similar code.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11%')
->create();

//add our filter(s) to a group
$filter_group = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter)
->create();
// $filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->setFilterGroups([$filter_group])
->create();

When working with the product repository, filters within a group are added as OR filters. For example, the following search criteria would return two products — WHERE sku LIKE 'WSH11-28%Blue' OR sku = 'WSH11-28%Red' (assuming, of course, the sample data is installed)

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter_1 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Red')    
->create();

$filter_2 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Blue')    
->create();

//add our filter(s) to a group
$filter_group = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter_1)
->addFilter($filter_2)
->create();
// $filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->setFilterGroups([$filter_group])
->create();

Filter groups, on the other hand, are combined as AND filters (again, when working with product repositories). The following code would return no items, as WHERE sku LIKE 'WSH11-28%Blue' AND sku = 'WSH11-28%Red' is impossible.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter_1 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Red')    
->create();

$filter_2 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Blue')    
->create();

//add our filter(s) to a group
$filter_group_1 = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter_1)
->create();

$filter_group_2 = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter_2)
->create();        
// $filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->setFilterGroups([$filter_group_1, $filter_group_2])
->create();

Finally, when working with non-complicated queries, the search criteria builder has short cut methods for adding “single-filter” filter groups. For example, the following code will create a search criteria object, with a single filter group, and that single filter group will contain a sku LIKE 'WSH11-28%Blue' filter

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->addFilter('sku','WSH11-28%Blue', 'like')
//->addFilter('sku','WSH11-28%Blue', 'like') //additional addFilters will 
                                             //add another group
->create();        

This is convenient for creating a series of simple AND filters.

The Case Against Repositories

It makes sense that Magento 2 chose the repository pattern for its new API work. By starting with a single, simple, unified RepositoryInterface, Magento’s architects gave the system implementors a tool that could abstract away whatever ugliness was necessary on the persistence/database layer to unify Magento 2 into a new API. If Magento 2’s core team was a team of developers familiar with Magento 1, this might not have been necessary. However, since they were getting the a new team up to speed on an existing platform, it made sense to start with repositories on the top.

Repositories also make it easier to write tests — breaking filters out in to a well factored, multiple object hierarchy means there’s a path forward for getting full test coverage on the repository object queries.

All that said, (and while I’m using repository objects in my Magento 2 work), there’s a lot about Magento’s particular implementation of this platform I don’t like, and I’m not afraid to fall back on using Magento’s CRUD models and collections when the repository API fails me.

Too Many Ways to Do The Same Thing

First off, there’s no clear, right way to use the getList method of a repository object. Put more succinctly, there’s too many ways to create a search criteria object. This is on display in our code samples above — we started directly creating filter, group, and criteria objects, but then moved to using Magento’s builder objects. Nothing in the architecture points to a clean, single, API for creating search criteria objects. Even now, I’m not 100% certain my code with the builder objects is right.

Additionally, since the final implementation features multiple repository interfaces, each repository can potentially drift on how search criteria are applied, and may not even provide a getList method. This means a product repository may work differently than a category repository, and they both may work differently than a CMS Page repository.

With systems like Laravel’s Eloquent available, which starts with a unified collection interface for all objects, even modern PHP developers will be looking sideways at the criteria/filter group/filter system. Java developers should feel right at home.

Too Verbose

Consider — Magento 2

$search_criteria = $this->searchCriteriaBuilder
->addFilter('sku','WSH11-28%Blue', 'like')
->create();

$repo->getList($search_criteria);

$items = $repo->getItems();

foreach($items ...){}

vs. Magento 1

$items = Mage::getModel('catalog/product')
->getCollection()
->addFieldToFilter('sku',['like'=>'WSH11-28%Blue']);    

foreach($items ...){}

One expression vs. three expressions for the simplest case. Start adding more complex filters and this difference becomes even more striking. Magento 2’s repository implementation is biased towards making it easier for the core team to implement new SOAP and REST APIS. Magento 1’s collections, and ActiveRecord in general, are biased towards giving programmers easy, SQL like access to a set of objects.

Repositories are Built on top of CRUD models

There’s a few very smart folks in the Magento community who are gung-ho about repositories being the new way forward, and the only way to write new code. It’s hard to take these claims too seriously when Magento repositories, are themselves, based on Magento’s old CRUD models. For example, if you take a look at the product repository source file

#File: vendor/magento/module-catalog/Model/ProductRepository.php
public function __construct(
    //...
    \Magento\Catalog\Model\ResourceModel\Product\CollectionFactory $collectionFactory,
    //...
) {
    $this->collectionFactory = $collectionFactory;
}

public function getList(\Magento\Framework\Api\SearchCriteriaInterface $searchCriteria)
{
    /** @var \Magento\Catalog\Model\ResourceModel\Product\Collection $collection */
    $collection = $this->collectionFactory->create();
    //...
}

You’ll see that getList relies on a converted Magento 1 Collection resource model.

Given the Magento core developers needed to dip into Magento’s old CRUD system, it seems likely Magento system integrators and extension developers will need to as well, if only to debug code related to using Magento 2’s repositories.

On top of that, the objects Magento 2 repositories return still have their own load, save,delete, and getCollection methods. While this was smart from a practical migration point of view, it creates ambiguity as to where and when a Magento 2 programmer should use the new repository classes. We’ll likely see a lot of code where Magento developers are still relying on the old CRUD methods without even realizing the repositories are available. It also makes the whole “we’ve abstracted away persistence from business objects” logic a tough sell when the persistence logic still lives in the business objects.

Now that Magento 2 has shipped, and applications are shipping inside of large corporate enterprise organizations, it seems likely that Magento 2’s core developers will face the same “de-facto frozen API” as Magento 1’s core developers did, or risk breaking what their partner agencies and independent developers are doing with the platform. While there’s no guarantees of the CRUD models not changing, doing so would cause a cascade of problems that could seriously impact Magento’s credibility.

No Mandated Consistent Repository Interfaces

As previously mentioned, each Magento 2 repository object implements its own interface. It’s only convention and process that keeps the consistent getById, getList, save,delete, and deleteById methods available to all repositories. As time goes on, I wouldn’t be surprised to see this consistent API drift, and new future third party repositories notusing these methods.

Potential for Inconsistent Filter and Group Interactions

Finally, Magento’s implementation of the repository pattern leaves the system open for inconsistent filter and group behavior between repositories. Earlier we told you that inside a group, filters are applied as OR criteria, but that groups are combined with AND, with a warning that we were talking about the product repository only.

That’s because it’s the responsibility of of each repository’s getList method to implement filtering logic. Again, using the product repository as an example

#File: vendor/magento/module-catalog/Model/ProductRepository.php    

public function getList(\Magento\Framework\Api\SearchCriteriaInterface $searchCriteria)
{
    //...
    //Add filters from root filter group to the collection
    foreach ($searchCriteria->getFilterGroups() as $group) {
        $this->addFilterGroupToCollection($group, $collection);
    }
    //...
}     

protected function addFilterGroupToCollection(
    \Magento\Framework\Api\Search\FilterGroup $filterGroup,
    Collection $collection
) {
    $fields = [];
    $categoryFilter = [];
    foreach ($filterGroup->getFilters() as $filter) {
        $conditionType = $filter->getConditionType() ? $filter->getConditionType() : 'eq';

        if ($filter->getField() == 'category_id') {
            $categoryFilter[$conditionType][] = $filter->getValue();
            continue;
        }
        $fields[] = ['attribute' => $filter->getField(), $conditionType => $filter->getValue()];
    }

    if ($categoryFilter) {
        $collection->addCategoriesFilter($categoryFilter);
    }

    if ($fields) {
        $collection->addFieldToFilter($fields);
    }

}

The only reason product repositories have the AND/OR behavior we mentioned is an implementation detail of the specific repository. A different repository might implement a different bit of filtering logic. i.e. there’s nothing in the framework that forces filters and filter group to implement the same logic.

Helper Class with Dependency Injection

As promised, before we wrap up, here’s a single helper class with examples of how to use automatic constructor dependency injection to grab references to search criteria builders, filter group builders, and filter builders.

#File: app/code/Pulsestorm/RepositoryTutorial/Model/Helper.php
<?php
namespace Pulsestorm\RepositoryTutorial\Model;
/**
* Example of automatic constructor dependency injection
* for repository and filter objects
*/
class Helper
{
    protected $pageRepository;
    protected $productRepository;
    protected $filterBuilder;
    protected $filterGroupBuilder;
    protected $searchCriteriaBuilder;

    public function __construct(
        \Magento\Cms\Model\PageRepository $pageRepository, 
        \Magento\Catalog\Model\ProductRepository $productRepository,
        \Magento\Framework\Api\FilterBuilder $filterBuilder,
        \Magento\Framework\Api\Search\FilterGroupBuilder $filterGroupBuilder,
        \Magento\Framework\Api\SearchCriteriaBuilder $searchCriteriaBuilder    
    )
    {
        $this->pageRepository           = $pageRepository;
        $this->productRepository        = $productRepository;
        $this->filterBuilder            = $filterBuilder;
        $this->filterGroupBuilder       = $filterGroupBuilder;
        $this->searchCriteriaBuilder    = $searchCriteriaBuilder;
    }
}

Wrap Up

While there’s a lot to critique about Magento’s repository implementation, they do seem like the new way forward for Magento 2. It would be wise to become familiar with using them in your own extensions and integration work.

That said, the things you’ll need to do with Magento often go beyond what’s available via the new officially sanctioned API classes — don’t be afraid to dip back into your Magento 1 CRUD knowledge to get your work done. This will mean moving beyond Magento sanctioned @api methods. Getting solid integration and acceptance tests around your work will protect you from any surprises when and if things break in the future.

As Magento’s consultants, its partners, and its unaffiliated developers start deploying Magento across the world, a @defacto-api will start to develop, and the core team will be forced into a decision where they either support the old APIs, or let an untold number of systems break. Unless they’re recklessly irresponsible, conventional wisdom points to the later being the case.

Magento 2: CRUD Models for Database Access

Today we’re going to cover creating Magento 2 CRUD models. CRUD stands for Create, Read, Update, and Delete, and commonly refers to framework features used to read and write information to/from the underlying database without directly writing any SQL statements.

Magento 1’s ORM (object relationship mapper) was not abandoned in Magento 2. It’s still an Active Record inspired pattern that uses Model, Resource Model, and Resource Model collection classes. However, due to major changes in Magento’s underlying object system, you’ll need to learn a new set of boilerplate class creation to use Magento’s CRUD features in your own modules.

This article will use the pestle command line framework’s generate_crud_modelcommand to generate this new boiler plate, and then explain what each class in the boilerplate does. We’re going to create a model for a fictitious “To Do List” application.

Creating a Base Module

Before we get to the CRUD specific portion of this article, we’ll need to create a base Magento 2 module to work in.

You can create the base module files using the pestle command line framework’sgenerate_module command.

$ pestle.phar generate_module Pulsestorm ToDoCrud 0.0.1

Normally, we’d enable this module in Magento by running the following two commands

$ php bin/magento module:enable Pulsestorm_ToDoCrud
$ php bin/magento setup:upgrade    

However, you’ll want to hold off on running setup:upgrade for reasons we’ll mention below.

If you’re interested in creating a module by hand, or curious what the above pestle command is actually doing, take a look at our Introduction to Magento 2 — No More MVCarticle.

Next, we’re going to create a URL endpoint and view for our model. This is not directly related to the task at hand, but will provide us with a place to write and test PHP code that uses our model.

To create the URL endpoint, run the following pestle commands.

$ pestle.phar generate_route Pulsestorm_ToDoCrud frontend pulsestorm_todocrud

Then, create your view with the generate_view command

$ pestle.phar generate_view Pulsestorm_ToDoCrud frontend pulsestorm_todocrud_index_index Main content.phtml

Normally, you’d clear your Magento cache

php bin/magento cache:clean

and then access your endpoint at the following URL.

http://magento.example.com/pulsestorm_todocrud

However, since we’re holding off on running setup:upgrade, you’ll need to get through the Generating Crud Files section below before accessing your URL.

If you’re curious what the above commands are actually doing, be sure to read theIntroduction to Magento 2 — No More MVC article.

Generating Crud Files

As previously mentioned, we’re going to create a model for a “To Do” item in our imaginary productivity application. We’ll want this model to have two main fields — thetext of the to do item, and a date completed field.

To generate the base files needed for this module, run the followinggenerate_crud_model command

$ pestle.phar generate_crud_model Pulsestorm_ToDoCrud TodoItem
Creating: /path/to/magento2/app/code/Pulsestorm/ToDoCrud/Model/TodoItemInterface.php
Creating: /path/to/magento2/app/code/Pulsestorm/ToDoCrud/Model/ResourceModel/TodoItem/Collection.php
Creating: /path/to/magento2/app/code/Pulsestorm/ToDoCrud/Model/ResourceModel/TodoItem.php
Creating: /path/to/magento2/app/code/Pulsestorm/ToDoCrud/Model/TodoItem.php
Creating: /path/to/magento2/app/code/Pulsestorm/ToDoCrud/Setup/InstallSchema.php
Creating: /path/to/magento2/app/code/Pulsestorm/ToDoCrud/Setup/InstallData.php

That’s 6 files in total. You’ll notice there’s no configuration files created or edited. Since Magento 2 no longer uses string based class aliases, the base CRUD models need zeroconfiguration. All we need to do is create class files with the correct names.

Once we’ve created the class files, our next step is creating the database table for our model.

To add the database table, we’ll use the Install Schema feature of Magento’s ORM. Pestle has already created a boiler plate install schema class at the following location

#File: app/code/Pulsestorm/ToDoCrud/Setup/InstallSchema.php
public function install(\Magento\Framework\Setup\SchemaSetupInterface $setup, \Magento\Framework\Setup\ModuleContextInterface $context)
{
    //...

    //START table setup
    $table = $installer->getConnection()->newTable(
                $installer->getTable('pulsestorm_todocrud_todoitem')
        )->addColumn(
                'pulsestorm_todocrud_todoitem_id',
                \Magento\Framework\DB\Ddl\Table::TYPE_INTEGER,
                null,
                array (
      'identity' => true,'nullable' => false,'primary' => true,'unsigned' => true,
    ),
                'Entity ID'
            )->addColumn(
                'title',
                \Magento\Framework\DB\Ddl\Table::TYPE_TEXT,
                255,
                array (
      'nullable' => false,
    ),
                'Demo Title'
            )->addColumn(
                'creation_time',
                \Magento\Framework\DB\Ddl\Table::TYPE_TIMESTAMP,
                null,
                array (
    ),
                'Creation Time'
            )->addColumn(
                'update_time',
                \Magento\Framework\DB\Ddl\Table::TYPE_TIMESTAMP,
                null,
                array (
    ),
                'Modification Time'
            )->addColumn(
                'is_active',
                \Magento\Framework\DB\Ddl\Table::TYPE_SMALLINT,
                null,
                array (
      'nullable' => false,'default' => '1',
    ),
                'Is Active'
            );

    //...
    $installer->getConnection()->createTable($table);        
}

This code creates a PHP data structure that represents a MySQL table. Magento will automatically run this install schema class when we run bin/magento setup:upgrade. You may see this install schema class referred to as a migration, or by its name in Magento 1 — a setup resource class.

Pestle creates a stock database table migration for you. This stock migration will create a table named pulsestorm_todocrud_todoitem, with the following columns

pulsestorm_todocrud_todoitem_id
title
creation_time
update_time
is_active

The first column is the table’s primary key, and will serve as the model’s ID. Pestle bases its name on the name of the table.

The second column, title, is optional, and not used my Magento’s ORM.

The last three columns (creation_time, update_time, and is_active) are fields that Magento expects to find in a model. While not strictly required, having these fields in your models is always a good idea.

Before we run this migration, we’ll want to add our columns. Change the code block above so it looks like the following.

#File: app/code/Pulsestorm/ToDoCrud/Setup/InstallSchema.php    
$table = $installer->getConnection()->newTable(
            $installer->getTable('pulsestorm_todocrud_todoitem')
    )->addColumn(
            'pulsestorm_todocrud_todoitem_id',
            \Magento\Framework\DB\Ddl\Table::TYPE_INTEGER,
            null,
            array (
  'identity' => true,'nullable' => false,'primary' => true,'unsigned' => true,
),
            'Entity ID'
        )->addColumn(                
            'item_text',
            \Magento\Framework\DB\Ddl\Table::TYPE_TEXT,
            255,
            array (
  'nullable' => false,
),
            'Text of the to do item'

        )->addColumn(                
            'date_completed',
            \Magento\Framework\DB\Ddl\Table::TYPE_DATETIME,
            null,
            array (
  'nullable' => true,
),
            'Date the item was completed'
 )->addColumn(
            'creation_time',
            \Magento\Framework\DB\Ddl\Table::TYPE_TIMESTAMP,
            null,
            array (
),
            'Creation Time'
        )->addColumn(
            'update_time',
            \Magento\Framework\DB\Ddl\Table::TYPE_TIMESTAMP,
            null,
            array (
),
            'Modification Time'
        )->addColumn(
            'is_active',
            \Magento\Framework\DB\Ddl\Table::TYPE_SMALLINT,
            null,
            array (
  'nullable' => false,'default' => '1',
),
            'Is Active'
        );

We’ve removed the title column, and added an item_text column, and adate_completed column. Covering Magento’s data definition language classes in full is beyond the scope of this article, but you can find the base table class (and its TYPE_constants) here.

#File: vendor/magento//framework/DB/Ddl/Table.php

With the above in place, we’re ready to run our migrations.

Running a Magento 2 Migration

In Magento 1, the core system code automatically ran any needed migrations whenever an uncached HTTP(S) request was made. When it worked, this feature was super useful, as the simple act of adding a module to the system also automatically added its data tables. Unfortunately, when this didn’t work, it could leave the system in a half updated state that was hard to recover from.

In Magento 2, a system owner is required to run the setup:upgrade command when they want to run a migration. Let’s give that a try with the above code in our system. Run the following

php bin/magento setup:upgrade

and you should see the pulsestorm_todocrud_todoitem table created in your database.

mysql> show create table pulsestorm_todocrud_todoitem\G;
*************************** 1. row ***************************
       Table: pulsestorm_todocrud_todoitem
Create Table: CREATE TABLE `pulsestorm_todocrud_todoitem` (
  `pulsestorm_todocrud_todoitem_id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Entity ID',
  `item_text` varchar(255) NOT NULL COMMENT 'Text of the to do item',
  `date_completed` datetime DEFAULT NULL COMMENT 'Date the item was completed',
  `creation_time` timestamp NULL DEFAULT NULL COMMENT 'Creation Time',
  `update_time` timestamp NULL DEFAULT NULL COMMENT 'Modification Time',
  `is_active` smallint(6) NOT NULL DEFAULT '1' COMMENT 'Is Active',
  PRIMARY KEY (`pulsestorm_todocrud_todoitem_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='pulsestorm_todocrud_todoitem'
1 row in set (0.00 sec)    

If your migration did not run, it may be because you ran setup:upgrade before you added the install schema class. If this is the case, you’ll want to remove the information that lets Magento know the Pulsestorm_ToDoCrud module is installed in the system. You can find this information in the setup_module table.

mysql> select * from setup_module where module = 'Pulsestorm_ToDoCrud';
+---------------------+----------------+--------------+
| module              | schema_version | data_version |
+---------------------+----------------+--------------+
| Pulsestorm_ToDoCrud | 0.0.1          | 0.0.1        |
+---------------------+----------------+--------------+
1 row in set (0.00 sec)

If you delete this row

mysql> DELETE from setup_module where module = 'Pulsestorm_ToDoCrud';

and try running setup:upgrade again, Magento should call the install method and install your table.

The InstallSchema.php file is meant to hold code for creating the structure of your database tables. If you wanted to install actual data into the tables you’ve created, you’d use the InstallData.php file

app/code/Pulsestorm/ToDoCrud/Setup/InstallData.php

The InstallData.php file is beyond the scope of this article, but take a look at some core modules to get an idea for how you’d use it

$ find vendor/magento/ -name InstallData.php
vendor/magento//magento2-base/dev/tests/api-functional/_files/Magento/TestModuleIntegrationFromConfig/Setup/InstallData.php
vendor/magento//module-authorization/Setup/InstallData.php
vendor/magento//module-bundle/Setup/InstallData.php
//...
vendor/magento//module-widget-sample-data/Setup/InstallData.php
vendor/magento//module-wishlist-sample-data/Setup/InstallData.php    

A Base Magento 2 CRUD Model

Now that we’ve got a table definition installed, lets take a look at the other four files pestle created for us.

First is the base model file

#File: app/code/Pulsestorm/ToDoCrud/Model/TodoItem.php
<?php
namespace Pulsestorm\ToDoCrud\Model;
class TodoItem extends \Magento\Framework\Model\AbstractModel implements TodoItemInterface, \Magento\Framework\DataObject\IdentityInterface
{
    const CACHE_TAG = 'pulsestorm_todocrud_todoitem';

    protected function _construct()
    {
        $this->_init('Pulsestorm\ToDoCrud\Model\ResourceModel\TodoItem');
    }

    public function getIdentities()
    {
        return [self::CACHE_TAG . '_' . $this->getId()];
    }
}

This is our main model class file. Objects instantiated with this class are the bread and butter for Magento 2 CRUD programming. Like Magento 1, all Magento CRUD models extend the base abstract model class.

Magento\Framework\Model\AbstractModel

Unlike Magento 1, all CRUD models also implement an IdentityInterface. This interface forces model developers to define a getIdentities method

<?php
#File: vendor/magento/framework/DataObject/IdentityInterface.php
namespace Magento\Framework\DataObject;
interface IdentityInterface
{
    public function getIdentities();
}

You’ll also notice the model implements aPulsestorm/ToDoCrud/Model/TodoItemInterface class.

#File: app/code/Pulsestorm/ToDoCrud/Model/TodoItemInterface.php
<?php
namespace Pulsestorm\ToDoCrud\Model;
interface TodoItemInterface 
{

}    

While not strictly necessary, the model specific TodoItemInterface interface plays an important role when it comes time to exporting CRUD models to Magento’s new service contracts based API. While beyond the scope of this article, the model specific interfaces for Magento CRUD models will determine which class methods are available via the Magento API.

The last thing to make note of is the _construct method

#File: app/code/Pulsestorm/ToDoCrud/Model/TodoItem.php
protected function _construct()
{
    $this->_init('Pulsestorm\ToDoCrud\Model\ResourceModel\TodoItem');
}

A model’s _construct method is a leftover concept from Magento 1. It’s an alternative constructor. The implementation of this _construct method is beyond the scope of this article. All you need to know is that _construct will be called whenever a model is instantiated. Every CRUD model in Magento must use the _construct method to call the_init method. The _init method accepts a single string parameter — the name of this model’s resource model.

A Magento 2 Resource Model

In Magento 2, the model class defines the methods an end-user-programmer will use to interact with a model’s data. A resource model class contains the methods that will actually fetch the information from the database. Each CRUD model in Magento 2 has a corresponding resource model class.

#File: app/code/Pulsestorm/ToDoCrud/Model/ResourceModel/TodoItem.php
<?php
namespace Pulsestorm\ToDoCrud\Model\ResourceModel;
class TodoItem extends \Magento\Framework\Model\ResourceModel\Db\AbstractDb
{
    protected function _construct()
    {
        $this->_init('pulsestorm_todocrud_todoitem','pulsestorm_todocrud_todoitem_id');
    }
}

Every CRUD resource model class extends theMagento\Framework\Model\ResourceModel\Db\AbstractDb class. This base class contains the basic logic for fetching information from a single database table.

For a basic model like ours, the only thing a resource model must do is call the _initmethod from _construct. The _init method for a resource model accepts two arguments. The first is the name of the database table (pulsestorm_todocrud_todoitem), and the second is the ID column for the model (pulsestorm_todocrud_todoitem_id).

While it’s beyond the scope of this article, Magento 2’s active record implementation contains no method for linking tables via primary keys. How to use multiple database tables is up to each individual module developer, and a resource model will typically contain the SQL generating methods needed to fetch information from related tables.

A Magento 2 Collection Model

With a model and resource model, you have everything you need to fetch and save individual models into the database. However, there are times where you’ll want to fetch multiple models of a particular type. To solve this problem, every CRUD model in Magento 2 has a corresponding resource model collection. A collection collects individual models. It’s considered a resource model since it builds the SQL code necessary to pull information from a database table.

#File: app/code/Pulsestorm/ToDoCrud/Model/ResourceModel/TodoItem/Collection.php
<?php
namespace Pulsestorm\ToDoCrud\Model\ResourceModel\TodoItem;
class Collection extends \Magento\Framework\Model\ResourceModel\Db\Collection\AbstractCollection
{
    protected function _construct()
    {
        $this->_init('Pulsestorm\ToDoCrud\Model\TodoItem','Pulsestorm\ToDoCrud\Model\ResourceModel\TodoItem');
    }
}

All collections in Magento 2 extend the base\Magento\Framework\Model\ResourceModel\Db\Collection\AbstractCollectioncollection class. Like a model and resource model, a collection resource model must call the _init method. A collection resource model’s _init method accepts two arguments. The first is the model that this collection collects. The second is that collected model’s resource model.

Using a Crud Model

Now that we’ve explored what each of the 6 files created by pestle’sgenerate_crud_model do, we’re ready to get into using the CRUD models.

First, let’s make sure the view we setup earlier is working. Open the following file and add the following var_dump to the _prepareLayout method.

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php

<?php
namespace Pulsestorm\ToDoCrud\Block;
class Main extends \Magento\Framework\View\Element\Template
{
    function _prepareLayout()
    {
        var_dump("I am Here");
        exit;
    }
}

We’re going to write our code in the _prepareLayout method, and then halt execution. While this is something you’d never do in a finished module, working out code samples in this sort of environment is common practice when learning and developing If you load the endpoint we created earlier

http://magento.example.com/pulsestorm_todocrud

and you should see your I am Here text on a white screen.

The first thing we’re going to do is instantiate a new To Do Item model, set its text, and then save it. In Magento 1, we’d use code that looked something like this

function _prepareLayout()
{
    $model = Mage::getModel('pulsestorm_todocrud/todoitem')
    ->setItemText('Finish my Magento Article')
    ->save();
}

However, Magento 2 no longer uses static factory methods on a global Mage class. Instead, we need to use two new Magento OO systems.

  1. We’ll use automatic constructor dependency injection to …
  2. … inject a factory object, and then use the factory object to instantiate our CRUD model

If you’re not familiar with automatic constructor dependency injection, you’ll want to work your way through our Magento 2 object manager series. If you’re not familiar with Factory objects — that’s because we haven’t covered them yet!

Magento 2 Factory Objects

In object oriented programming, a factory method is a method that’s used to instantiate an object. Factory methods exist to ensure system developers have control over how a particular object is instantiated, and how its arguments are passed in. There’s a certain school of though that thinks direct use of the new keyword in programming

$object = new Foo;

is an anti-pattern, as directly instantiating an object creates a hard coded dependency in a method. Factory methods give the system owner the ability to control which objects are actually returned in a given context.

A factory object serves a similar purpose. In Magento 2, each CRUD model has a corresponding factory class. All factory class names are the name of the model class, appended with the word “Factory”. So, since our model class is named

Pulsestorm/ToDoCrud/Model/TodoItem

this means our factory class is named

Pulsestorm/ToDoCrud/Model/TodoItemFactory

To get an instance of the factory class, replace your block class with the following.

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php    
<?php
namespace Pulsestorm\ToDoCrud\Block;
class Main extends \Magento\Framework\View\Element\Template
{
    protected $toDoFactory;
    public function __construct(
        \Magento\Framework\View\Element\Template\Context $context,
        \Pulsestorm\ToDoCrud\Model\TodoItemFactory $toDoFactory
    )
    {
        $this->toDoFactory = $toDoFactory;
        parent::__construct($context);
    }

    function _prepareLayout()
    {
        var_dump(
            get_class($this->toDoFactory)
        );
        exit;
    }
}

What we’ve done here is use automatic constructor dependency injection to inject aPulsestorm\ToDoCrud\Model\TodoItemFactory factory object, and assign it to thetoDoFactory object property in the constructor method.

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php        
protected $toDoFactory;
public function __construct(
    \Magento\Framework\View\Element\Template\Context $context,
    \Pulsestorm\ToDoCrud\Model\TodoItemFactory $toDoFactory
)
{
    $this->toDoFactory = $toDoFactory;
    parent::__construct($context);
}

We also had to inject a block context object and pass that to our parent constructor. We’ll cover these context object in future articles, but if you’re curious about learning more, this quickies post is a good place to start.

In addition to injecting a factory into our block, we also added the following to our_prepareLayout method

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php    
function _prepareLayout()
{
    var_dump(
        get_class($this->toDoFactory)
    );
    exit;
}

This will dump the toDoFactory‘s class name to the screen, and is a quick sanity check that our automatic constructor dependency injection worked. Reload your page with the above in place, and you should see the following

string 'Pulsestorm\ToDoCrud\Model\TodoItemFactory' (length=41)

Next, replace your _prepareLayout method with this code

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php    

function _prepareLayout()
{
    $todo = $this->toDoFactory->create();
    $todo->setData('item_text','Finish my Magento article')
    ->save();
    var_dump('Done');
    exit;
}

This code calls the create method of our factory. This will instantiate a\Pulsestorm\ToDoCrud\Model\TodoItemFactory object for us. Then, we set theitem_text property of our model, and call its save method. Reload your page to run the above code, and then check your database table

mysql> select * from pulsestorm_todocrud_todoitem\G
*************************** 1. row ***************************
pulsestorm_todocrud_todoitem_id: 1
                      item_text: Finish my Magento article
                 date_completed: NULL
                  creation_time: NULL
                    update_time: NULL
                      is_active: 1
1 row in set (0.00 sec)

You’ll find that Magento has saved the information you requested. If you wanted to fetch this specific model again, you’d use code that looked like the following.

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php        
function _prepareLayout()
{
    $todo = $this->toDoFactory->create();

    $todo = $todo->load(1);        
    var_dump($todo->getData());
    exit;
}

Here we’ve used the factory to create our model, used the model’s load method to load a model with the ID of 1, and then dumped the model’s data using the various magic setter and getter methods available to a Magento 2 model.

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php        
function _prepareLayout()
{
    $todo = $this->toDoFactory->create();

    $todo = $todo->load(1);        

    var_dump($todo->getData());

    var_dump($todo->getItemText());

    var_dump($todo->getData('item_text'));
    exit;
}

Finally, if we wanted to use a CRUD model’s collection object, we’d use code that looked like this

#File: app/code/Pulsestorm/ToDoCrud/Block/Main.php          
function _prepareLayout()
{
    $todo = $this->toDoFactory->create();

    $collection = $todo->getCollection();

    foreach($collection as $item)
    {
        var_dump('Item ID: ' . $item->getId());
        var_dump($item->getData());
    }
    exit;
}   

Again, this code uses a factory object to create a CRUD model object. Then, we use the CRUD model object’s getCollection method to fetch the model’s collection. Then, we iterate over the items returned by the collection.

Once instantiated via a factory, Magento 2’s CRUD models behave very similarly, if not identically, to their Magento 1 counterparts. If you’re curious about Magento 1’s CRUD objects, our venerable Magento 1 for PHP MVC Developers article may be of interest, as well as the Varien Data Collections article.

Where did the Factory Come From

You may be thinking to yourself — how did Magento instantiate aPulsestorm/ToDoCrud/Model/TodoItemFactory class if I never defined one? Factory classes are another instance of Magento 2 using code generation (first covered in our Proxy object article). Whenever Magento’s object manager encounters a class name that ends in the word Factory, it will automatically generate the class in the var/generation folder if the class does not already exist. You can see your generated factory class at the following location

#File: var/generation/Pulsestorm/ToDoCrud/Model/TodoItemFactory.php 
<?php
namespace Pulsestorm\ToDoCrud\Model;

/**
 * Factory class for @see \Pulsestorm\ToDoCrud\Model\TodoItem
 */
class TodoItemFactory
{
    //...
}

Wrap Up

That’s the basics of Magento’s CRUD models. While much of the talk from the Magento 2 core team has been around Repositories, Service Contracts, and the various API transport layers these features enable, Magento’s abstract CRUD models remain the bread and butter objects for Magento 2 programming. The amount of legacy code relying on these models, and the practicality of the ActiveRecord pattern ensure that these models will remain a vital part of any Magento 2 developer’s toolkit.

Magento 2 and the Less CSS Preprocessor

Last week we covered RequireJS, which is the foundational framework for Magento 2’s modern approach to javascript. Today we’re going to do the same for Magento 2’s use of cascading style sheets.

Web development, whether delivering marketing pages or software applications, has always struggled with the browser compatibility problem. On one hand, it’s a fantastic testament to the world wide web that completely different companies can all produce the different browsers engines that have, more or less, a shared set of functionality.

The devil, of course, is in that more or less.

Slightly different interpretations of standards, or lingering older versions of browsers, can wry havoc with developers trying to get consistent browser behavior. This is especially true for people who don’t have the time (or company culture) to perform full cross browser tests of every feature. The advent of the mobile web has made this problem even worse, as mobile device browsers are almost, but not quite entirely, compatible with their desktop counterparts.

Javascript, in some respects, has it easier than CSS. Thanks to some excellent early reverse engineering by Microsoft’s JScript engineers and the development of a reasonable standardization process, the core javascript language performs identically between various browser implementations. The APIs javascript calls into (i.e. the DOM) are another story, but since javascript is a programming language it’s relatively easy for programmers to create shim libraries (like jQuery) to carve out a consistent portion of the DOM to work with.

On the Cascading Style Sheets (CSS) side of things, and back in the quiet days of web development when IE stagnated at version 6 and Firefox/Safari were quietly catching up, “web standards” proponents attempted to carve out a set of CSS rules that worked across browsers, and a set of CSS techniques that relied on non-obvious-but-standard CSS behavior to achieve desired layout behaviors.

Then Microsoft woke up with IE 7/8/9/10/11/edge, Apple essentially created the first realmobile web with Safari for iPhone (while simultaneously pushing closed non-HTML based applications for accessing web data), Firefox started pushing its own standards agenda, Google started turning Chrome into a platform, and a weird “HTML5” movement started. The only WaSPs left in NYC were in The Hamptons. CSS developers were once again faced with death by a thousand browser cuts. While CSS is “code”, and an important valuable skill to master, CSS is not a programming language. The CSS system cannot change its own behavior via a turing complete programming language.

All this led to the rise of CSS pre-processors. A CSS pre-processor lets a front end developer write their styling rules in a custom style language, and then have a stand alone computer program (or “compiler”) turn that custom style language into CSS files that are compatible with multiple browsers.

With pre-processors, the maintainer of the pre-processor (or the pre-processor’s library) becomes the CSS expert, and the designer is freed from needing to remember which CSS rules are safe, which -prefix classes to add for each browser, which crazy CSS standards-bugs are used to implement common UI patterns, etc.

Understanding Less

Magento 2 uses a CSS pre-processor called Less. With Less, designers write their styles in a CSS like language. This is a sample of a Less stylesheet from the Less documentation.

#File: example.less
@base: #f938ab;

.box-shadow(@style, @c) when (iscolor(@c)) {
  -webkit-box-shadow: @style @c;
  box-shadow:         @style @c;
}
.box-shadow(@style, @alpha: 50%) when (isnumber(@alpha)) {
  .box-shadow(@style, rgba(0, 0, 0, @alpha));
}
.box {
  color: saturate(@base, 5%);
  border-color: lighten(@base, 30%);
  div { .box-shadow(0 0 5px, 30%) }
}

As you can see, this sort of looks like CSS, but seems to have things like functions and variables available. Designers will write Less based code, and then compile it with a program like lessc, which will convert it to CSS.

$ lessc example.less 
.box {
  color: #fe33ac;
  border-color: #fdcdea;
}
.box div {
  -webkit-box-shadow: 0 0 5px rgba(0, 0, 0, 0.3);
  box-shadow: 0 0 5px rgba(0, 0, 0, 0.3);
}

As you can see above, variables have been expanded, functions like lighten have been applied to colors, etc. Less has a reputation as a more “programmer-y” pre-processor, and it’s not surprising that Magento’s engineering team picked it over other popular choices likeSass.

Covering all of Less’s functionality would take a book, and my days of slingingbackground-color are long over. Instead, we’re going to focus on how Magento has incorporated Less into their front end framework code. Whether you plan on embracing Less, finding ways to work with other CSS tools, or leave the styling to designers, understanding how Magento processes and compiles Less code is an important skill for any working Magento 2 developer to master.

Getting Started with Magento and Less

We’re going to start by creating a module named Pulsestorm_StartingWithLess, and then we’ll use this module to add a CSS file to every Magento Page. We’re going to start assuming you’re running Magento in developer mode. If you’re not sure what that means,you’ll want to review our articles so far.

You can create the base module files using the pestle command line framework’sgenerate_module command.

$ pestle.phar generate_module Pulsestorm StartingWithLess 0.0.1

and then enable the module in Magento by running the following two commands

$ php bin/magento module:enable Pulsestorm_StartingWithLess
$ php bin/magento setup:upgrade    

If you’re interested in creating a module by hand, or curious what the above pestle command is actually doing, take a look at our Introduction to Magento 2 — No More MVCarticle.

Once created, add the following default layout handle file to the frontend area.

#File: app/code/Pulsestorm/StartingWithLess/view/frontend/layout/default.xml
<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" layout="1column" xsi:noNamespaceSchemaLocation="../../../../../../../lib/internal/Magento/Framework/View/Layout/etc/page_configuration.xsd">
    <head>
        <css src="Pulsestorm_StartingWithLess::red-alert.css"/>
    </head>
</page>

This is a page layout handle XML file, which means it understands the special head andbody sections. The default handle fires on every page request. The contents of the headtag adds the CSS file red-alert.css to the page. Next, add the following red-alert.cssfile to your module.

#File: app/code/Pulsestorm/StartingWithLess/view/frontend/web/red-alert.css
body
{
    background-color:#f00;
}    

With the above in place, clear your Magento cache, reload the Magento home page, and you should now see an obnoxious red background, and CSS link tag in your document’s source.

<link  rel="stylesheet" type="text/css"  media="all" href="http://magento.example.com/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css" />

Your <link/> tag’s href attribute may look a little different depending on the theme you’re using. This article assume you’re using the luma theme that ships with Magento 2.

If any of the above concepts confused you, you may want to review the articles in our series so far.

Your First Less File

Next up, we’re going to add our first Less file. The following assumes you’re running indeveloper mode, and that the option at

System -> Stores -> Configuration -> Advanced -> Developer -> Front-end Developer Workflow 

is set to Server Side Less Compilation.

First, remove the red-alert.css file from your module.

rm app/code/Pulsestorm/StartingWithLess/view/frontend/web/red-alert.css 

Confirm you’ve removed it by reloading the home page and noting the non-garish white background is restored. We’ll also want to remove the symlink Magento created in pub

$ find pub/static -name red-alert.css
pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css     

$ rm pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css     

Next, create the following Less source file.

/* File: app/code/Pulsestorm/StartingWithLess/view/frontend/web/red-alert.less */
@oh-my-blue: #0000ff;
body
{
    background-color:@oh-my-blue;
}      

Notice we’ve put this file in the same location as our CSS file, and given it the same base name. The only difference is the file extension (.less instead of .css). The contents of the file are also slightly different — we’ve used a Less variable (@oh-my-blue) to add a bluebackground.

With the above in place, clear your cache and reload the homepage. You should now see a garish blue background. Congratulations! You just created your first Less source file.

Understanding Magento’s Less Implementation

If you view the source of your home page, you may be surprised to find your CSS <link/>tag is still in place, and still pointing to red-alert.css

<link  rel="stylesheet" type="text/css"  media="all" href="http://magento.example.com/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css" />

In one way, this makes sense. After all, our default.xml still points to this file. But how is Magento loading red-alert.css file if we deleted this file?

The missing piece of information is understanding Magento’s Less processing. When you add a css file via <css/> tag

<css src="Pulsestorm_StartingWithLess::red-alert.css"/>

Magento will search for this file on disk. If Magento doesn’t find it, Magento will look again, but instead of looking for a .css file, Magento will look for a .less file with the same name (red-alert.less). If Magento finds the Less source file, Magento will transform the file into CSS, and generate a CSS file. If you load the above CSS in a browser, you’ll see Magento has transformed the Less file into a CSS file.

#File: http://magento.example.com/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css
body {
  background-color: #0000ff;
}

If you look for the file in pub, you’ll see it’s a real file, and not a symlink that points to a module file.

$ find pub/static -name red-alert.css
pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css

$ ls -l pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css
-rw-rw-rw-  1 _www  staff  38 Feb  4 16:29 pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css

Remove this generated file, and reload the CSS URL. You’ll see Magento simply regenerates the file for you.

Understanding Less.js

One problem with the above system is that, once a file’s generated, Magento won’t recheck the Less file for any changes. That is, if you changed the Less file to create a redbackground

/* File: app/code/Pulsestorm/StartingWithLess/view/frontend/web/red-alert.less */
@oh-my-red: #ff0000;
body
{
    background-color:@oh-my-red;
} 

Magento wouldn’t see these changes. Since there’s a red-alert.css file on disk already, Magento will just serve that file. In order to pick up these changes, you need to remove the generated CSS file.

rm pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css

You’ll also need to clear Magento’s view_preprocessed folder. This is a cache folder for Less source files

rm -rf var/view_preprocessed/*

While some programmers may be used to the “edit file, compile, view results” workflow, CSS developers are used to editing a file and seeing the results immediately. One solution for this is something called less.js.

The intended use of less.js is

  1. You include less.js on your HTML page
  2. You serve Less source files directly to the browser
  3. The less.js library transforms the Less source into CSS, and applies the CSS to the page

In other words, less.js simulates what it would be like if Less were supported natively in your browser. You can turn on less.js by navigating to the following system configuration section

System -> Stores -> Configuration -> Advanced -> Developer -> Front-end Developer Workflow 

and setting the Front-end Developer Workflow option to Client side less compilation. With this setting enabled (and if you remove any generated files/links frompub/static), changes to Less source files will be immediately reflected on the next page load.

Behind the scenes, when you have less.js enabled, Magento does three things. First, Magento adds the following two javascript files to your page’s source.

<script src="http://magento.example.com/static/adminhtml/Magento/backend/en_US/less/config.less.js"></script>
<script src="http://magento.example.com/static/adminhtml/Magento/backend/en_US/less/less.min.js"></script>

The less.min.js file is the minified less.js source. The config.less.js file is a javascript configuration file used to configure less.js. Having these here enables client side Less compilation.

The second change is each CSS link will have its rel attribute’s value changed fromstylesheet

<link  rel="stylesheet" type="text/css" ... />

to stylesheet/less.

<link  rel="stylesheet/less" type="text/css" ... />

This identifies the source of a <link/> as needing the Less transformations provided byless.js.

Finally, and most importantly, when you’re running Magento in developer mode with client side Less compilation on, Magento will serve Less source files instead of CSS files. i.e., if you try to download the red-alert.css file in your system

http://magento.example.com/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css

instead of getting the CSS file, you’ll get the Less source file

@oh-my-blue: #ff0000;
body
{
    background-color:@oh-my-blue;
} 

While this client-side processing is useful, I’d be wary of relying too heavily on it. The GA release contains what seems to be a bug where Magento can serve CSS source files with arel="stylesheet/less". Processing CSS as Less can create subtle bugs. Also, as welearned last time, Magento uses RequireJS as its foundational javascript framework. RequireJS means there’s DOM altering javascript loading asynchronously in the background, which may interfere with the DOM altering less.js.

Less and Production Mode

Both server side and client side Less compilation are developer mode only features. Let’s try changing Magento into production mode and observe how the system behavior changes.

Switch your system into production mode

#File: .htaccess
SetEnv MAGE_MODE production

and remove the generated red-alert.css file.

rm pub/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css    

Now, try reloading your red-alert.css URL. You’ll end up seeing a 404 Not Found error.

curl -I 'http://magento.example.com/static/frontend/Magento/luma/en_US/Pulsestorm_StartingWithLess/red-alert.css'
HTTP/1.1 404 Not Found

That’s because, in production mode, Magento will not automatically convert your Less source files into CSS, and will not perform client side Less actions. Similar to PHP code generation, this is for performance and security reasons. Instead, Magento will transform your files once, when you run setup:static-content:deploy.

$ php bin/magento setup:static-content:deploy
Requested languages: en_US

=== frontend -> Magento/blank -> en_US ===
............................................................................................................................................................................

After running the above command, each theme in the system will have a CSS file generatedif there’s a CSS file configured in a <head/> section with a corresponding Less source file.

Wrap Up

That’s the foundations of Magento 2’s Less implementation. If you’re interested in learning more, the Cascading Style Sheets section of the dev docs site has some good, if scattered, information on styling Magento 2.

I also know Magento has some built-in support for grunt, and there’s talk of unofficial-ish gulp support. My CSS friends tell me grunt and gulp are build tools that can help work around the limitations of solutions like less.js, but I’m just a frozen XHTML 1.0 transition caveman in these matters.

Regardless of how you’re compiling your Less file, or generating your own CSS outside of Magento’s systems, understanding how Magento’s framework code handles generating its own CSS via Less will be a vital skill when working with Magento 2 systems.

Magento 2 and RequireJS

Now that we’ve covered the basics of including javascript and CSS files in Magento 2, we’re going to start exploring Magento’s adoption of modern front end tools and libraries.

Magento’s in a bit of a tricky position when it comes to adopting modern front end technologies. Magento is a software platform, and an ecommerce one at that. Unlike an agency, whose marketing project will be discarded 6 months after launch or whose prototyping work will be depreciated a year after the product launches, an ecommerce software platform needs to focus on stable, proven technologies that are going to span the test of time.

Today we’re going to focus on the library that underlies nearly every javascript feature built in Magento 2 — RequireJS.

Before we get to Magento’s RequireJS implementation, we’re going to take a whirlwind tour of what RequireJS does.

RequireJS

RequireJS is a javascript module system. It implements the Asynchronous Module Definition (AMD) standard for javascript modules. In the terms of AMD, a javascript module provides a way to

  1. Run a javascript program that doesn’t default to the global namespace
  2. Share javascript code and data between named modules and programs

That’s all RequireJS does. You may use a RequireJS module that implements some special bit of functionality, but its not RequireJS that provides that functionality. RequireJS is the pneumatic tube that ensures the functionality is delivered to you.

The RequireJS start page has a good example of how RequireJS works. We’re going to crib from it and add some extra explanations.

First, download the RequireJS source and save it to a folder named scripts.

Then, create the following file.

<!-- File: require-example.html -->
<!DOCTYPE html>
<html>
    <head>
        <title>My Sample Project</title>
        <!-- data-main attribute tells require.js to load
             scripts/main.js after require.js loads. -->
        <script data-main="scripts/main" src="scripts/require.js"></script>
    </head>
    <body>
        <h1>My Sample Project</h1>
    </body>
</html>

As you can see, this page loads in the main RequireJS with the following

<!-- File: require-example.html -->    
<script data-main="scripts/main" src="scripts/require.js"></script>

In addition to the standard src attribute, there’s also the custom data-main attribute. This tells RequireJS that it should use the scripts/main module as the program’s main entry point. In our case that’s scripts/main, which corresponds to a file at scripts/main.js.

Create the following file.

//File: scripts/main.js
requirejs([], function() {
    alert("Hello World");
});

With the above created, load your HTML page in a browser. You should see the Hello World alert. Congratulations! You just created your first RequireJS program.

By itself, RequireJS hasn’t done much that jQuery’s document ready functions can’t accomplish

jQuery(function(){
    alert("Hello World");
});

Where RequireJS sets itself apart is in its module system. For example, if we wanted to use a hypothetical module named helper/world, we’d change our main.js file to match the following.

requirejs(['helper/world'], function(helper_world) {
    var message = helper_world.getMessage();
    alert(message);
});

That is, we specify the modules we want to load as an array, and pass that array as the first argument to the requirejs function call. Then, RequireJS passes the single object thehelper/world module exports to our main function as the first helper_world parameter.

Of course, if you ran the above you’d get a javascript error. That’s because we need to define our helper/world module. To define a module, turn the module name into a file path, and add the following contents

//File: scripts/helper/world.js
define([], function(){
    var o = {};
    o.getMessage = function()
    {
        return 'Hello Module World';
    }
    return o;
});

A module definition is very similar to our main program definition. The main difference is the use of the define function instead of the requirejs function. The first parameter ofdefine is a list of RequireJS modules you’d like to use in your module (in our case, this is an empty array — in the real world most modules will use other modules). The second parameter is the javascript function/closure that defines what your module will return.

RequireJS has no opinion on what a javascript module should return/export. It could return a plain string. It could return a simple javascript object with a single method defined (as we have above). It could also load in a javascript library like PrototypeJS and return a PrototypeJS object. The only thing RequireJS does is provide a system for sharing javascript code via modules — the rest is up to each individual project developer.

Before we get to Magento’s RequireJS implementation, there’s two additional RequireJS topics we’ll need to cover: Require JS file loading, and RequireJS module naming.

RequireJS File Loading

By default, RequireJS will convert a module name like helper/world into an HTTP(S)path like the following

http://example.com/scripts/helper/world.js
https://example.com/scripts/helper/world.js
//example.com/helper/scripts/world.js

That is, the module name is turned into a file path, with the last segment being a file name that ends in js. By default, RequireJS will use the folder where the require.js script is located as its base (/scripts in the above example).

However, RequireJS allows you to set a different base path for your scripts. Before the start of your RequireJS program, include the the following code.

require.config({
    baseUrl: '/my-javascript-code',
});

With the above in place, when RequireJS needs to load the helper/world module, it will load it from

http://example.com/my-javascript-code/helper/world.js
https://example.com/my-javascript-code/helper/world.js    
//example.com/my-javascript-code/helper/world.js

This feature allows you to store your javascript files wherever you want in a RequireJS based system.

RequireJS: Module Naming

So far in our examples, a RequireJS module name has been tied to the location of that module’s source on disk. In other words, the helper/world module is always going to be located at the path helper/world.js.

RequireJS allows you to change this via configuration. If, for example, you wanted yourhelper/world module to be named hello, you’d run the following configuration code somewhere before the start of your program

require.config({   
    paths: {        
        "hello": "helper/world"
    },
});

The paths configuration key is where we can rename/alias modules. The key of a pathsobject is the name you want (hello), and the value is the module’s actual name (helper/world).

With the above configuration in place, the following program

requirejs(['hello'], function(hello) {
    alert("Hello World");
});

Would load the hello module from the helper/world.js path.

There are many other configuration directives that control how and where RequireJS will load javascript modules. While outside the scope of this article, the entire “Load Javascript Files” section of the RequireJS API Documentation is worth reading.

If you take away one thing from this brief RequireJS introduction tutorial, it should be that the point of RequireJS is for day-to-day javascript development should not need to concern itself with how a module load over HTTP. As a javascript developer, you should be able to say “I want to use the methods in module X to do something”, and just be able to use module X. The only time you should need to worry about file loading is when you’re adding a new module to the system, adding some non-RequireJS/AMD compatible code to the system, or looking for a module’s source files so you can figure out what it does.

Magento 2 and RequireJS

This brings us to Magento’s RequireJS implementation. Magento pulls in the main RequireJS library for you, includes some additional configuration, and provides a mechanism that will let you add your own additional RequireJS configurations.

The first thing you’ll want to make note of is Magento 2’s use of the aforementionedbaseUrl feature of RequireJS. If you view the source of a Magento page, you’ll see something like the following

<script type="text/javascript">
    require.config(
        {"baseUrl":"http://magento.example.com/static/adminhtml/Magento/backend/en_US"}
    );
</script>

With the above configuration, this means when Magento 2 encounters a RequireJS module named helper/world, it will load that module source from a URL that looks something like this

http://magento.example.com/static/adminhtml/Magento/backend/en_US/helper/world.js

If you’ve worked through the previous articles in this series, you may recognize that URL as the “loading a front end static asset from a Magento module” URL. This means you could place a RequireJS module definition file in your module at

app/code/Package/Module/view/base/web/my_module.js

and it would be automatically available as a RequireJS module named

Package_Module/my_module

loaded via the following URL

http://magento.example.com/static/adminhtml/Magento/backend/en_US/Package_Module/my_module.js

It also means you could immediately start writing requirejs programs in your phtmltemplates by using code like the following

<script type="text/javascript">
    requirejs('Package_Module/my_module', function(my_module){
        //...program here...
    });
</script>

Or by adding stand-alone javascript files that do the same.

Configuring RequireJS via Modules

Earlier in our tutorial, we covered two RequireJS configuration directives — baseUrl andpath. There are plenty of other RequireJS configuration directives, and as you get into advanced use of the framework (or you’re dealing with Magento core code that’s gotten into advanced use) you’ll find you need to use them.

Every Magento module has the ability to add RequireJS configuration directives via a special view file named requirejs-config.js.

app/code/Package/Module/view/base/requirejs-config.js    
app/code/Package/Module/view/frontend/requirejs-config.js    
app/code/Package/Module/view/adminhtml/requirejs-config.js    

This is a special javascript file that Magento will automatically load on every page load using the area hierarchy. We’re going to give it a try. First, we’ll need to create a module named Pulsestorm_RequireJsTutorial. You can create the base module files using the pestle command line framework with the following command

$ pestle.phar generate_module Pulsestorm RequireJsTutorial 0.0.1

and then enable the module in Magento by running the following two commands

$ php bin/magento module:enable Pulsestorm_RequireJsTutorial
$ php bin/magento setup:upgrade    

If you’re interested in creating a module by hand, or curious what the above pestle command is actually doing, checkout our Introduction to Magento 2 — No More MVCarticle.

Regardless of how you’ve created it, once you have a module created and enabled, add the following file

//File: app/code/Pulsestorm/RequireJsTutorial/view/base/requirejs-config.js
alert("Hello");

Clear your cache, and load any page in your Magento system. You should see your alertfunction call. Congratulations — you just added a requirejs-config.js file to your Magento module.

The Purpose of requirejs-config.js

While you can use requirejs-config.js to run any arbitrary javascript, its main job is to

  1. Allow end-user-programmers to add require.config options to Magento’s RequireJS system
  2. Allow end-user-programmers to perform any other setup/configuration their javascript needs

To understand how RequireJS does this, we need to look at where Magento actually pulls in these requirejs-config.js files. If you take a look at any source page in your Magento installation, you should see a tag that looks like this

<script  type="text/javascript"  src="http://magento.example.com/static/_requirejs/adminhtml/Magento/backend/en_US/requirejs-config.js"></script>

This is a special javascript file that Magento generates during setup:di:compile (inproduction mode) or on the fly (developer and default mode). If you’re unfamiliar with how Magento’s various modes affect front end asset serving, checkout our Magento 2: Serving Frontend Files article. We’re going to assume you’re running in developer mode for the remainder of this article.

If you take a look at the source of this file in a browser, you’ll see your alert statement in a code block that looks something like this

(function() {
    alert("Hello World");
    require.config(config);
})();

While it’s not 100% obvious, by generating this javascript code block from requirejs-config.js, Magento 2 is letting us add extra RequireJS initializations to the system.

This may make more sense with a concrete example. Let’s replace our requirejs-config.js with the following

var config = {
    paths:{
        "my_module":"Package_Module/my_module"
    }
};

alert("Done");    

What we’ve done here is define a javascript variable named config, and changed ouralert value. If you go back and reload requirejs-config.js it might be clearer now what Magento is doing.

(function() {
var config = {
    paths:{
        "my_module":"Package_Module/my_module"
    }
};

alert("Done");
require.config(config);
})();

For every individual requirejs-config.js, Magento will create a chunk of code that looks like this

(function() {
    //CONTENTS HERE
    require.config(config);
})();

but with //CONTENTS HERE replaced by the contents of requirejs-config.js.

var config = {
    paths:{
        "my_module":"Package_Module/my_module"
    }
};

alert("Done");
require.config(config);

This means if we define a config variable in our requirejs-config.js file, Magento will ultimately pass it to require.config. This will let any Magento module developer use RequireJS features like shim, paths, baseUrl, map, or one of the many others fromRequireJS’s configuration directives.

Understanding Lazy Loading

Another important thing to understand about RequireJS is that modules are lazy loaded. RequireJS will not load any javascript module source file until someone users that javascript module as a dependency.

In other words, if we used the configuration

var config = {
    paths:{
        "my_module":"Package_Module/my_module"
    }
};

Magento will not load the Package_Module/my_module.js file by default. Magento willonly load that file once you’ve used it as a module

requirejs(['my_module'], function(my_module){

});

requirejs(['Package_Module/my_module'], function(my_module){

});    

define(['Package_Module/my_module'], function(my_module){

});        

Remember, the point of RequireJS is that day-to-day javascript developers shouldn’t need to worry about how their programs make HTTP requests for their source files. Lazy loading is an implementation detail that, in ideal circumstances, saves the end user the bandwidth of needing to download source files that a particular page might not need.

However, in less than ideal circumstance, this lazy loading behavior can make it tricky to work with older javascript frameworks and libraries. We’ll discuss one example of this below when we talk about a few jQuery gotchas.

Global jQuery Object

Even if you decide RequireJS is not for you, and you want to stick to plain old jQuery in Magento 2, you’ll still need to be aware of how RequireJS interacts with libraries that predate the AMD standard.

In Magento 2, jQuery is loaded as a RequireJS module. This means if you attempts to use code like the following

<script type="text/javascript">
    jQuery(function(){
        //your code here
    });
</script>

Your browser will complain that jQuery is undefined. That’s because the global jQueryobject won’t be initialized until you use jQuery as a RequireJS module. If you’re used to writing code like the above, you’ll need to

  1. Replace it with code that kicks off execution of a RequireJS program
  2. Configure that program to use the jquery module as a dependency

In other words, something like this

requirejs(['jquery'], function(jQuery){
    jQuery(function(){
        //your code here
    });
});

To review — you kick off execution of a RequireJS program by calling the requirejsfunction and passing it a list of module dependencies, and an anonymous javascript function that will act as your programs main entry point.

The list of module dependencies is the first argument to requirejs — i.e. the following code says “My Program is dependent on the jquery module”

requirejs(['jquery'],    

The anonymous function that acts as your program’s main entry point is the second argument to the requirejs function

requirejs(['jquery'], function(jQuery){
    //...
});

RequireJS will call this function for you. For each dependency you configure RequireJS will load the module and pass in its returned (or, in RequireJS speak, exported) module.

Modern versions of jQuery will detect if they’re being included in a RequireJS/AMD environment and define a module that returns the global jQuery object.

// File: http://code.jquery.com/jquery-1.12.0.js
// Register as a named AMD module, since jQuery can be concatenated with other
// files that may use define, but not via a proper concatenation script that
// understands anonymous AMD modules. A named AMD is safest and most robust
// way to register. Lowercase jquery is used because AMD module names are
// derived from file names, and jQuery is normally delivered in a lowercase
// file name. Do this after creating the global so that if an AMD module wants
// to call noConflict to hide this version of jQuery, it will work.

// Note that for maximum portability, libraries that are not jQuery should
// declare themselves as anonymous modules, and avoid setting a global if an
// AMD loader is present. jQuery is a special case. For more information, see
// https://github.com/jrburke/requirejs/wiki/Updating-existing-libraries#wiki-anon

if ( typeof define === "function" && define.amd ) {
    define( "jquery", [], function() {
        return jQuery;
    } );
}    

RequireJS and jQuery Plugins

There’s another gotcha to using jQuery and RequireJS together. The jQuery library, which predates RequireJS and the AMD standard by many years, developed its own plugin system. While not module based, this system plays relatively nice with javascript’s global by default environment — plugin developers create their plugins by modifying the single global jQuery object.

This presents a problem for RequireJS — as we mentioned above, the global jQuery object is not defined until the jquery module is used in a requirejs program, and until RequireJS calls that program’s main entry-function. This means the long standing way of including a jQuery plugin

<script src="http://magento.example.com/js/path/to/jquery/plugin/jquery.cookie.js">

will fail when the plugin attempts to use the global jQuery object and/or $ alias.

//File: http://magento.example.com/js/path/to/jquery/plugin/jquery.cookie.js
var config = $.cookie = function (key, value, options) {

If you want to include a jQuery plugin for use in Magento 2, you’ll need to do it via RequireJS. Fortunately, the process is relatively straight forward.

First, you’ll want to create an alias for the plugin path using the paths configuration property.

var config = {
    paths:{
        "jquery.cookie":"Package_Module/path/to/jquery.cookie.min"
    }
};

The above configuration creates a module named jquery.cookie that points to the jQuery cookie plugin source file in the Package_Module module.

At this point, you may think its OK to start using jQuery with your plugin by doing something like this

requirejs(['jquery','jquery.cookie'], function(jQuery, jQueryCookie){
    //my code here
});

After all, listing both jquery and jquery.cookie as dependencies should trigger a loading of both files.

You’d be right — but only some of the time. RequireJS loads its module source files asynchronously, and doesn’t promise anything about the order they’re loaded in. That means the above code may load the core jQuery library first. However, depending on other scripts running on the page and the network, it may also load the jQuery cookie plugin first. If this happens, the cookie plugin will load without a global jQuery object. Without a global jQuery object, the plugin initialization will fail.

This may seem like a poor design decision — but you need to remember that RequireJS, and the AMD standard, were designed to stop the use of global state. It’s not surprising that RequireJS doesn’t work seamlessly with a library like jQuery. Even though jQuery is responsible about its use of global state (one global jQuery object), it still uses global state, and RequireJS isn’t going to get in the business of deciding who does and doesn’t use global state responsibly.

Fortunately, there is a solution. The RequireJS shim configuration directive allows you to configure what I’ll call “load order” dependencies. i.e., you can say

Hey RequireJS — when you load the jquery.cookie module? Make sure you’ve completely loaded the jquery module first.

Configuration wise, this looks like

var config = {
    paths:{
        "jquery.cookie":"Package_Module/path/to/jquery.cookie.min"
    },
    shim:{
        'jquery.cookie':{
            'deps':['jquery']
        }
    }
};

We’ve defined a new top level configuration property named shim. This property is a javascript object of key value pairs. The key should be the name of your module (jquery.cookie above). The value is another javascript object that defines the shimconfiguration for this specific module.

There’s a number of different shim configuration options — the deps configuration option we’ve used above creates a source dependency (distinct from a normal define orrequirejs module dependency) that ensures RequireJS will load each listed module (a single [jquery] above) entirely before the module that we’re “shimming” (jquery.cookie above).

The dep configuration option only scratches the surface of what shim is capable of — theshim documentation is worth reading if you’re interested in more details.

With the above configuration in place (and a jquery.cookie.min file in thePackage_Module module), you should be able to safely create RequireJS programs that have the jquery cookie plugin added as a dependency.

Require vs. RequireJS

One last note before we wrap up. Throughout the RequireJS documentation, you’ll see reference to two functions

require()
requirejs();

What’s the difference between these two? There isn’t any, they’re the same function.

The AMD standard calls for a function that’s named require. However, RequireJS realized that there may already be code in use that defines a require function. In order to ensure their library could be used along side this code, they provide a requirejs alias to their main, AMD standard function.

Wrap Up

We’ve only scratched the surface of Magento’s use of javascript and modern frontend libraries in this article. However, all these systems are dependent on the RequireJS implementation. If you start there, you should be able to track back any javascript based feature to its inclusion via RequireJS, and through that figure out what’s going on. As always, knowing what a specific library does is always useful — but knowing how the framework your code lives in works is the key to becoming a more productive and rational programmer.

Magento 2: Adding Frontend Assets via Layout XML

As promised, today we’re going to show you how to add javascript and CSS files to your Magento module without needing to worry about how their paths are rendered or how Magento serves them. While you may find this article useful on its own, we’re going to assume you’ve worked through the previous two.

Before we get to the main course, we’re going to provide an engineering critique of Magento 2’s XML based layout rendering language. This language is similar to the XML based language in Magento 1, but has some differences that might trip up an experienced Magento 1 developer.

The next few sections are optional, and not recommended unless you’re interested in the implementation details of Magento 2. You can skip ahead to the Adding CSS and Javascript to a Page section if all you’re interested in is getting your javascript and CSS files included on a page.

Magento 2’s Domain Specific Language for Rendering HTML

With Magento 2, the core team has added a number of features to their XML based layout language. They’ve also changed some of the language’s semantics. For example, in Magento 1, each node under the root node of a layout update XML file was always a layout handle. For example, in the Magento 1 Layout Update XML Node below, the handle iscatalog_category_view.

<catalog_category_view>
    <block ...>
        <!-- ... -->
    </block>
</catalog_category_view>

Without getting too deeply into it, handles control which layout nodes are applied to a page during which request. Magento 2 still has handles, but they no longer appear in layout files. Instead, the handle is the file name. For example, the main catalog_category_viewlayout handle XML file is at

./vendor/magento/module-catalog/view/frontend/layout/catalog_category_view.xml

Other modules that listen for the catalog_category_view handle include

./vendor/magento/module-checkout/view/frontend/layout/catalog_category_view.xml
./vendor/magento/module-directory/view/frontend/layout/catalog_category_view.xml
./vendor/magento/module-google-optimizer/view/frontend/layout/catalog_category_view.xml
./vendor/magento/module-msrp/view/frontend/layout/catalog_category_view.xml
./vendor/magento/module-paypal/view/frontend/layout/catalog_category_view.xml
./vendor/magento/module-swatches/view/frontend/layout/catalog_category_view.xml
./vendor/magento/module-wishlist/view/frontend/layout/catalog_category_view.xml

In all the examples, the file name is the handle name. While the mechanism has changed, handles still serve the same purpose in Magento 2. On every Magento 2 HTML page render, certain handles “fire”, similar to events. The handles that fire control which layout files are loaded (in Magento 1 they controlled which XML nodes were read from all the files), and then Magento processes the combined XML tree to know which blocks it should add to a page.

Put another way, in Magento 1 you would

  1. Configure your module to load a layout update XML file (namespace_module.xml, catalog.xml etc.)
  2. In the layout update XML file you’d add a node for your handle (catalog_category_view)
  3. Under the catalog_category_view node you’d add your blocks

In Magento 2, you

  1. Add a layout handle XML file (catalog_category_view.xml)
  2. Under the root node of your layout handle XML file, add your blocks

We covered this change briefly in our Introduction to Magento 2 — No More MVC article. However, we glossed over a number of substantial changes to Magento’s layout language that we’ll need to touch briefly on before we continue.

New Nodes

In our Introduction to Magento 2 — No More MVC article, we created the following node in the layout handle XML file.

<!-- File: app/code/Pulsestorm/HelloWorldMVVM/view/frontend/layout/hello_mvvm_hello_world.xml -->
<referenceBlock name="content">
    <block
        template="content.phtml"
        class="Pulsestorm\HelloWorldMVVM\Block\Main"
        name="pulsestorm_helloworld_mvvm"/>
</referenceBlock>

As a reminder, this XML is roughly equivalent to the following pseudo code.

//pseudo code -- does not work
$our_view_block = 
    $layout->createNewBlockWithClass('Pulsestorm\HelloWorldMVVM\Block\Main')
$our_view_block->setName('pulsestorm_helloworld_mvvm');
$out_view_block->setTemplate('content.phtml');
$layout->addBlockToContentContainer($our_view_block);

The <block/> tag from Magento 1 remains relatively unchanged. It means Create a block object. The main difference is the type attribute has been replaced with a class attribute. Since Magento did away with class aliases (core/template,namespace_module/my_block, etc.) it made sense to do away with the type attribute, and more accurately label it as class.

The first small change above is the referenceBlock node. Magento 1 had the concept of a block reference. However, the node was named <reference/>. In Magento 1, the above might look like

<reference name="content">
    <block
        template="content.phtml"
        class="Pulsestorm\HelloWorldMVVM\Block\Main"
        name="pulsestorm_helloworld_mvvm"/>
</block>

The referenceBlock node makes things more explicit, and is another welcome change. This might seem superficial, until you realize that Magento 2’s layout language controlsmore than blocks. The layout language also controls something called containers, and has a corresponding referenceContainer block. You can see an example of thereferenceContainer block here

<!-- vendor/magento/module-checkout/view/frontend/layout/checkout_shipping_price_renderer.xml -->
<layout xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/layout_generic.xsd">
    <referenceContainer name="root">
        <block class="Magento\Checkout\Block\Shipping\Price" name="checkout.shipping.price" as="shipping.price" template="shipping/price.phtml"/>
    </referenceContainer>
</layout>

In Magento 2, a container is a special sort of block that only contains other blocks. Containers are conceptually similar to the text/list blocks in Magento 1, although their implementation is very different.

The concept of containers is a good one, but it’s here that the implementation starts to get a little wobbly. Magento’s layout language is a little loosey–goosey with the difference between a container and a block. For example, the above XML?

<referenceContainer name="root">

This could also be written as

<referenceBlock name="root">

That is — even though root is a container, referenceBlock will still return a reference to it, and allow you to add blocks to it. For a change meant to make things more explicit and clear, it’s a little strange that the layout language would let something like that happen.

White Lies

Remember this XML from the introduction article?

<!-- File: app/code/Pulsestorm/HelloWorldMVVM/view/frontend/layout/hello_mvvm_hello_world.xml -->
<referenceBlock name="content">
    <block
        template="content.phtml"
        class="Pulsestorm\HelloWorldMVVM\Block\Main"
        name="pulsestorm_helloworld_mvvm"/>
</referenceBlock>

Well, it turns out that the content block is actually a container. The above should have been written as

<!-- File: app/code/Pulsestorm/HelloWorldMVVM/view/frontend/layout/hello_mvvm_hello_world.xml -->
<referenceContainer name="content">
    <block
        template="content.phtml"
        class="Pulsestorm\HelloWorldMVVM\Block\Main"
        name="pulsestorm_helloworld_mvvm"/>
</referenceContainer>

We used referenceBlock in our introduction tutorial because we weren’t ready to discuss containers and other changes to the layout system. While this was useful for a transitional tutorial, generally speaking this is the sort of looseness that can make a domain specific language seem extra confusing.

Without getting too deeply into the details, you can tell if a “block” is a container or a regular block by how the original programmer created it. If you see a <block/> tag

<block name="foo" />

then the named block (“foo” above) is a regular block. If you see a <container/> tag

<container name="foo"/>

then the named entity is a container. If you’re curious, Magento’s core code adds thecontent container in the following file

<!-- vendor/magento/module-theme/view/frontend/layout/default.xml -->
<container name="content" label="Main Content Area"/>

Also, notice the default.xml file name? That’s equivalent to Magento 1’s <default/>handle node.

Context Sensitive Nodes

In Magento 1, the layout language was a system designed to render arbitrary HTML via a nested collection of block objects. The layout system itself didn’t care which part of an HTML document it was rendering. It just rendered blocks. Specific blocks, likepage/html_head, could introduce that context, but it happened at the block level. The layout system itself was unaware that it was rendering the <head/> portion of a document.

In Magento 2, the core team attempted to change this, and add that context in at the language level. They added two new top level tags named <body> and <head> to the vocabulary of the language. While it was an interesting experiment, the implementation feels half done, and further complicates an already complicated layout system. Consider the following

<!-- File: vendor/magento/module-backend/view/adminhtml/layout/default.xml -->
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" layout="admin-1column" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <title>Magento Admin</title>
        <meta name="viewport" content="width=1024, initial-scale=1"/>
        <link src="requirejs/require.js"/>
        <css src="extjs/resources/css/ext-all.css"/>
        <css src="extjs/resources/css/ytheme-magento.css"/>
    </head>
    <body>
        <attribute name="id" value="html-body"/>
        <block name="require.js" class="Magento\Backend\Block\Page\RequireJs" template="Magento_Backend::page/js/require_js.phtml"/>
        <referenceContainer name="global.notices">
            <block class="Magento\Backend\Block\Page\Notices" name="global_notices" as="global_notices" template="page/notices.phtml"/>
        </referenceContainer>
        <!-- ... -->
    </body>
</page>

Here you can see an example of a core module layout handle XML file that uses the newhead and body sections. The first bit of confusion this introduces is top level tags under the root tag now mean different things. In some files, these top level tags will be contexttags like <head/> and <body/> above. In other files, the top level tags will be actual commands/directives (referenceBlock, container, etc) for the layout engine

<!-- File: vendor/magento/module-bundle/view/base/layout/catalog_product_prices.xml -->
<layout xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/layout_generic.xsd">
    <referenceBlock name="render.product.prices">
    <!-- ... -->
</layout>

If you enjoy implementing domain specific languages, this may seem like a minor thing. However, the intent of a domain specific language is to simplify and constrain the options for developers and programmers unfamiliar with the entire system. The lack of consistency here will make these files harder for designers and front end developers to understand.

The next bit of confusion is in what that context change means. Nodes placed inside the<body/> tag

<!-- File: vendor/magento/module-backend/view/adminhtml/layout/default.xml -->    
<body>
    <attribute name="id" value="html-body"/>
    <block name="require.js" class="Magento\Backend\Block\Page\RequireJs" template="Magento_Backend::page/js/require_js.phtml"/>
    <referenceContainer name="global.notices">
        <block class="Magento\Backend\Block\Page\Notices" name="global_notices" as="global_notices" template="page/notices.phtml"/>
     </referenceContainer>
     <!-- ... -->
</body>

behave very similar to plain old layout XML nodes. You’re still getting references to existing blocks and containers, and adding new blocks to them for rendering. The only difference is the <attribute/> tag you see above. With this you can change the ID element of the underlying <body/> tag.

When you shift into <head/> context, you’re in a difference world.

<!-- File: vendor/magento/module-backend/view/adminhtml/layout/default.xml -->    
<head>
    <title>Magento Admin</title>
    <meta name="viewport" content="width=1024, initial-scale=1"/>
    <link src="requirejs/require.js"/>
    <css src="extjs/resources/css/ext-all.css"/>
    <css src="extjs/resources/css/ytheme-magento.css"/>
</head>

Here, you’ve completely lost the ability to modify the layout with commands likereferenceBlock, etc. Instead, you have a narrow set of tags (<attribute/>, <css/>,<link/>, <meta/>, <remove/>, <script/>, <title/>) for doing things specifically in the<head/> of a document.

The other bit of cognitive dissonance a Magento 1 developer will feel here is the <head/>section of the HTML page is no longer rendered like a normal block. If you take a look at the root phtml template you can see a Magento HTML page is no longer a series of nested blocks.

<!-- File: vendor/magento/module-theme/view/base/templates/root.phtml -->
<!doctype html>
<html <?php /* @escapeNotVerified */ echo $htmlAttributes ?>>
    <head <?php /* @escapeNotVerified */ echo $headAttributes ?>>
        <?php /* @escapeNotVerified */ echo $requireJs ?>            <?php /* @escapeNotVerified */ echo $headContent ?>            <?php /* @escapeNotVerified */ echo $headAdditional ?>        </head>
    <body data-container="body" data-mage-init='{"loaderAjax": {}, "loader": { "icon": "<?php /* @escapeNotVerified */ echo $loaderIcon; ?>"}}' <?php /* @escapeNotVerified */ echo $bodyAttributes ?>>
        <?php /* @escapeNotVerified */ echo $layoutContent ?>        </body>
</html>

In Magento 2, an HTML page is a phtml template populated by simple variables. These simple variables are populated by different means in the render method of theMagento\Framework\View\Result\Page object. Magento creates the <body/> tag of the page by echoing out the $layoutContent variable. Magento gets the string for$layoutContent by doing the traditional kickoff of rendering a series of nested blocks.

#File: vendor//magento/framework/View/Result/Page.php
$output = $this->getLayout()->getOutput();
$this->assign('layoutContent', $output);
//...
<?php /* @escapeNotVerified */ echo $layoutContent ?>

Magento renders the <head/> section of an HTML page by echoing several variables.

<head <?php /* @escapeNotVerified */ echo $headAttributes ?>>
    <?php /* @escapeNotVerified */ echo $requireJs ?>        <?php /* @escapeNotVerified */ echo $headContent ?>        <?php /* @escapeNotVerified */ echo $headAdditional ?>    </head>

How Magento populates of contents of these variable is beyond the scope of this article. The main change you’ll want to be aware of is that <head/> is no longer simply controlled by standard layout blocks.

Summary of Magento 2 Layout Changes

Magento 1’s layout system, while cryptic, was ultimately understandable by a single developer. Its why I wrote No Frills Magento Layout — the system wasn’t well documented, but once explained developers could understand and reason about it from top to bottom. It had a complex looking surface, but a simple elegant implementation.

As you can see from the above critique, Magento 2 has taken an already cryptic system, and added layers of complexity on top of it. These top levels of complexity are equallycomplex under the hood. Without getting into the details of it, Magento takes the layout handle XML files for a single request, merges them into a a single document, processes that document to transform it into a Magento 1 style page layout document, and then processes that document is way that similar, but not identical, to Magento 1.

Unlike Magento 1’s layout system, which an average developer could ultimately translate in their head into PHP code and reason about, the new rendering is too complex for most human beings to keep in their head at once. The new system is less understandable to the average developer. Perhaps this was necessary to implement the RequireJS and Less CSS systems the core team wanted to, but from the outside looking in it seems like a classic case of what people complain about when they complain about architect driven development.

Adding CSS and Javascript to a Page

Layout system critiqued, let’s get back to the practical business of adding a front end file to our Magento module. Before we begin, per pervious articles in this series, the following assumes

  1. You’re working with developer mode enabled (SetEnv MAGE_MODE developerin your apache config)
  2. That you’ve disabled the full page caching in System -> Cache Managment

To start, we need to create a new Magento module. We’re going to use the Magento code generating tool pestle for this, but if you want to create your own module manually you can follow the instructions in our Introduction to Magento 2 — No More MVC article.

To create the new module using pestle, run the following commands

$ pestle.phar generate_module Pulsestorm JavascriptCssExample 0.0.1
$ pestle.phar generate_route Pulsestorm_JavascriptCssExample frontend pulsestorm_javascriptcssexample
$ pestle.phar generate_view Pulsestorm_JavascriptCssExample frontend pulsestorm_javascriptcssexample_index_index Main content.phtml

Then enable your new module

$ php bin/magento module:enable Pulsestorm_JavascriptCssExample

and let the module setup system know about your module.

$ php bin/magento setup:upgrade

After running the above, you should be able to load thepulsestorm_javascriptcssexample frontname in your system.

http://magento.example.com/pulsestorm_javascriptcssexample

With the above completed, let’s get started!

The Layout Head Section

One of the new features Magento 2 introduces is context aware layout update XML files. By context aware we mean that end-programmer-users can add commands/directives to their layout XML files that only effect a particular section of the document. In plain english — layout update XML files now have a <head/> section where you can add head specific information about a file.

Sample code is often worth 1,000 words. Open up your module’s layout handle XML file

<!-- File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/layout/pulsestorm_javascriptcssexample_index_index.xml -->
<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" layout="1column" xsi:noNamespaceSchemaLocation="../../../../../../../lib/internal/Magento/Framework/View/Layout/etc/page_configuration.xsd">
    <referenceBlock name="content">
        <block template="content.phtml" class="Pulsestorm\JavascriptCssExample\Block\Main" name="pulsestorm_javascriptcssexample_block_main" />
    </referenceBlock>
</page>

and add the following <head/> node

<!-- File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/layout/ -->pulsestorm_javascriptcssexample_index_index.xml 
<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" layout="1column" xsi:noNamespaceSchemaLocation="../../../../../../../lib/internal/Magento/Framework/View/Layout/etc/page_configuration.xsd">
    <head>
        <css src="Pulsestorm_JavascriptCssExample::test.css"/>
        <link src="Pulsestorm_JavascriptCssExample::test.js"/>
    </head>
    <referenceBlock name="content">
        <block template="content.phtml" class="Pulsestorm\JavascriptCssExample\Block\Main" name="pulsestorm_javascriptcssexample_block_main" />
    </referenceBlock>
</page>

After making the above changes, clear your cache,

$ php bin/magento cache:clean

and then reload the Magento page

http://magento.example.com/pulsestorm_javascriptcssexample

If you view the source to this page, you should see the following HTML

<link  rel="stylesheet" type="text/css"  media="all" href="http://magento-2-with-keys.dev/static/frontend/Magento/blank/en_US/Pulsestorm_JavascriptCssExample/test.css" />
<!-- ... -->
<script  type="text/javascript"  src="http://magento-2-with-keys.dev/static/frontend/Magento/blank/en_US/Pulsestorm_JavascriptCssExample/test.js"></script>

That is, Magento has automatically created the http (or https) paths we’ve been generating manually so far.

http://magento-2-with-keys.dev/static/frontend/Magento/blank/en_US/Pulsestorm_JavascriptCssExample/test.css
http://magento-2-with-keys.dev/static/frontend/Magento/blank/en_US/Pulsestorm_JavascriptCssExample/test.js

When you use a string identifier like this

Pulsestorm_JavascriptCssExample::test.css

you’re telling Magento

use the test.css file found in the Pulsestorm_JavascriptCssExample module.

Without these Vendor_Module::... identifiers Magento would try loading these files from the theme hierarchy.

With the URLs generated, if you add corresponding files to your module at

#File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/web/test.js
alert("hello");

#File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/web/test.css
body{
    background-color:#f00;
}

and reload the page, you’ll see that Magento has loaded them correctly into the system.

Adding Files Via PHP

In addition to using Magento 2’s layout XML system to automatically add front end asset URLs to your project, you can also create these URLs via PHP using aMagento\Framework\View\Asset\Repository object. We’ll show you how to do this below, as well as how to add arbitrary HTML to the <head/> of a Magento HTML page.

Starting with the later item, add the following node to our layout handle XML file

<!-- File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/layout/ pulsestorm_javascriptcssexample_index_index.xml -->
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" layout="1column" xsi:noNamespaceSchemaLocation="../../../../../../../lib/internal/Magento/Framework/View/Layout/etc/page_configuration.xsd">
    <!-- ... -->
    <referenceBlock name="head.additional">
        <block  template="head.phtml" 
                class="Pulsestorm\JavascriptCssExample\Block\Head" 
                name="pulsestorm_javascriptcssexample_block_head" />
    </referenceBlock>
    <!-- ... -->
</page>

The above code

  1. Gets a reference to the head.additional block
  2. Creates a new Pulsestorm\JavascriptCssExample\Block\Head block named pulsestorm_javascriptcssexample_block_head that uses thehead.phtml template.
  3. Adds that new block to the head.additional block using the reference from #1

The head.additional block is a special block. Any block added to head.additionalwill automatically be output into the <head/> area of a Magento page. If you read our critique above, this is another bit of confusion added by the <head/> context. Even though our ultimate goal is to add something to <head/>, we need to operate inside the layout handle XML file’s <body/> tag.

Regardless, once we’ve got the layout XML in place, we’ll want to create our new Headblock class

#File: app/code/Pulsestorm/JavascriptCssExample/Block/Head.php
<?php
namespace Pulsestorm\JavascriptCssExample\Block;
class Head extends \Magento\Framework\View\Element\Template
{
}

As well as a template

#File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/templates/head.phtml
<!-- Hello There -->

With the above in place, clear your Magento cache and reload your page. You should see the <!-- Hello There --> comment in your page’s <head/> node.

With a new template rendered in <head/>, we’re ready to render an asset URL using the asset repository.

The Asset Repository

The Magento\Framework\View\Asset\Repository object will allow us to create asset objects. Asset objects can convert a file identifier like foo/test.js orPulsestorm_JavascriptCssExample::test.js into a full URL.

Like any object in Magento 2, when we want an instance of an object we don’t directly instantiate it — we inject it in another object’s constructor. Change your Head.php file so it matches the following

#File: app/code/Pulsestorm/JavascriptCssExample/view/frontend/templates/head.phtml
<?php
namespace Pulsestorm\JavascriptCssExample\Block;
class Head extends \Magento\Framework\View\Element\Template
{
    public $assetRepository;
    public function __construct(
        \Magento\Framework\View\Element\Template\Context $context, 
        array $data = [],    
        \Magento\Framework\View\Asset\Repository $assetRepository
    )
    {
        $this->assetRepository = $assetRepository;
        return parent::__construct($context, $data);
    }
}

What we’ve done above is use Magento 2’s automatic constructor dependency injection to create a \Magento\Framework\View\Asset\Repository object, and assign it to theassetRepository property of our block object. The other parameters in __construct and the call to parent::__construct are there for compatibility with the base template block class. Also, notice we made assetRepository a public property. This means we’ll be able to access it in our phtml template.

Edit your head.phtml file so it matches the following.

#File: app/code/Pulsestorm/JavascriptCssExample/Block/Head.php
@highlightsyntax@php
<?php
    $asset_repository = $this->assetRepository;
    $asset  = $asset_repository->createAsset('Pulsestorm_JavascriptCssExample::test.js');
    $url    = $asset->getUrl();
?>
<!-- Hello There -->
<script src="<?php echo $url; ?>"></script>

With the above in place, clear your cache, delete the files in var/generate/* (because you changed an automatic constructor dependency injection constructor), and reload the page. If you view the raw HTML source, you should see a new <script/> tag rendered with a full asset URL.

What we’ve done above is use the createAsset method of the asset repository object to create an asset object. Then, we use the getUrl method of the asset object to fetch the HTTPurl of the asset. All we need to know is the file identifier — Magento handles the grunt work of pulling together the correct URL path parameters.

Incorrect dependency in class

Update: Readers have reported that, in more modern versions of Magento 2, they end up getting the following error

Incorrect dependency in class Pulsestorm\JavascriptCssExample\Block\Head in app/code/Pulsestorm/JavascriptCssExample/Block/Head.php , \Magento\Framework\View\Asset\Repository already exists in context object

when running setup:di:compile. You can fix this by

  1. Removing Magento\Framework\View\Asset\Repository injection from the constructor
  2. Replace $asset_repository = $this->assetRepository; with $this->getAssetRepository()

The reason this error occurs is the dependency injection compiler checks to make sure objects don’t have “double injections”. TheMagento\Framework\View\Asset\Repository class is already injected in a base template class, and therefore out injection of it is redundant.

We regret the error.

Wrap Up

Today, after a long winded critique of Magento 2’s layout language, we demonstrated how to use that language to add front end CSS and Javascript assets to a Magento page. We also investigated directly using the underlying PHP asset repository that makes this possible.

In the past few articles, we’ve been entirely focused on getting “raw” front end asset files into our system. Next time we’ll start investigating how Magento has integrated with the new higher level front end abstractions like RequireJS and Less CSS.

Magento 2: Code Generation with Pestle

  1. We interrupt your regularly scheduled Magento tutorial for a quick announcement about a new (Magento related project) I’ve been working on.

    Pestle is a new PHP (5.6+) framework for creating and organizing PHP programs into python like libraries of functions. It’s a little outside the mainstream direction PHP’s heading these days, but if python like library of functions whispers to a secret part of your programming heart, checkout the wiki and your feedback is more than welcome.

    If python like library of functions does not whisper to a secret part of your programming heart, the other half of pestle is a growing list of Magento 2 code generation functions. I was always a little disappointed that Magento’s community driven code generation tools have been focused around a kitchen sink approach, and pestle’s generate_ commands are a response to that.

    Right now we have commands for generating a new module, a new route, a new view/block/layout for that route, a new observer, a new Magento CLI command (recursive!), a skeleton theme, a plugin’s configuration, and a command to automatically add the property, parameter, and assignment that injects a dependency into a class. We’re also keeping a running list of command suggestions, so if you have an idea or would like to work on an extension please head on over to GitHub.

    If you’re not familiar with what code generation can do for you, the rest of this article explores the pestle command for creating a new module.

    Creating the Module

    If you’ve worked through the Introduction to Magento 2 — No More MVC, you know creating a basic Magento module is straight forward, but involved many different files. Pestle can handle the grunt work of creating those files for you.

    For example — all you need to do to generate your module.xml boiler plate is run the following.

    $ pestle.phar generate_module
    Vendor Namespace? (Pulsestorm)] Pulsestorm
    Module Name? (Testbed)] HelloPestle
    Version? (0.0.1)] 0.0.1
    Created: /path/to/magento2/app/code/Pulsestorm/HelloPestle/etc/module.xml
    Created: /path/to/magento2/app/code/Pulsestorm/HelloPestle/registration.php
    

    Then enable it with Magento’s CLI tool

    $ php bin/magento module:enable Pulsestorm_HelloPestle
    

    Then tell Magento to update the setup resource versions

    $ bin/magento setup:upgrade    
    

    Adding the Route/URL Endpoint

    The next part of creating a Magento module is adding a URL controller endpoint, which is also known as a route. Pestle can handle this for you as well.

    $ pestle.phar generate_route
    Which module? (Magento_Catalog)] Pulsestorm_HelloPestle
    Which area? [frontend, adminhtml] (frontend)] frontend
    Frontname/Route ID? ()] hello_pestle
    /path/to/magento2/app/code/Pulsestorm/HelloPestle/etc/frontend/routes.xml
    /path/to/magento2/app/code/Pulsestorm/HelloPestle/Controller/Index/Index.php    
    

    Open the generated controller and add the following debugging code to the executemethod.

    #File: app/code/Pulsestorm/HelloPestle/Controller/Index/Index.php
    public function execute()
    {
        var_dump(__METHOD__);
        return $this->resultPageFactory->create();  
    }
    

    then clear your Magento cache and load the hello_pestle URL in your browser

    http://magento.example.com/hello_pestle
    

    You should see an HTML page returned with HTTP Status: 200, and the following content.

    string 'Pulsestorm\HelloPestle\Controller\Index\Index::execute' (length=54)
    

    Don’t forget to remove the var_dump debugging code before moving on.

    Adding a View

    Now that we have a module and controller endpoint configured, we need to add Magento’s default view files. Once again, pestle can save us the hassle of needing to manually create our layout handle XML file, phtml template, and default block/view class.

    $ pestle.phar generate_view Pulsestorm_HelloPestle frontend hello_pestle_index_index Main content.phtml
    $ pestle.phar generate_view
    Which Module? (Pulsestorm_HelloGenerate)] Pulsestorm_HelloPestle
    Which Area? (frontend)] frontend
    Which Handle? (pulsestorm_hellopestle_index_index)] hello_pestle_index_index
    Block Name? (Main)] Main
    Template File? (content.phtml)] content.phtml
    Creating /path/to/magento2/app/code/Pulsestorm/HelloPestle/view/frontend/templates/content.phtml
    Creating: Pulsestorm\HelloPestle\Block\Main
    Creating: /path/to/magento2/app/code/Pulsestorm/HelloPestle/view/frontend/layout/hello_pestle_index_index.xml
    

    After running the above, clear your cache and reload the page. You should see a fully laid out Magento page, with your view’s content right in the middle.

    Powerful Scripting

    In addition to helping developers automate the grunt work of a Magento module, it’s possible to specify any generate input as a command argument. This means you could recreate the above with a shell script that looks like this

    #!/bin/bash
    pestle.phar generate_module Pulsestorm HelloPestle 0.0.1
    pestle.phar generate_route Pulsestorm_HelloPestle frontend hello_pestle
    pestle.phar generate_view Pulsestorm_HelloPestle frontend hello_pestle_index_index Main content.phtml
    

    Next Steps

    Like what you see? There’s installation instructions in the README, and you can get a full list of commands by running

    $ pestle.phar list
    

    If you run into any problems/inefficiencies, or have command ideas, issue tickets are encouraged and welcomed.