Modern Ember Testing

In the last few weeks there has been a lot of movement and work to prepare new way of testing Ember.js applications. You probably haven’t noticed anything special because, as usual, the Ember.js community takes great care of moving forward in a way that doesn’t break people’s apps and developers can update at their own pace, but the last few weeks have been hectic.

Most of this work was made by the tireless Robert Jackson (@rwjblue) over the course of the last month.

The result: The realization of a vision in the works since 2016 in which we have an approach to testing that is more framework agnostic and yet it feels more natural, it’s easier to read and write, and runs your tests faster than before.

On this post we’ll get an overview of the new features with a lot of code examples and I’ll share tricks and enhancements the new approach enables, as well as my personal preferences to write tests in a way I think are most readable and easy to follow.

Why changing something that works?

Ember.js, as any technology, has pros and cons when compared to other frontend javascript frameworks, but if there is only one thing that people that used Ember and now uses some other library miss a great deal, that is the test harness we enjoy.

Ember testing, both Integration and Acceptance, is the best in its class and we don’t usually think much about it: They are a commodity. So why did the team spend so much time changing something people already think is great?

Because great is a moving target. Although writing tests day to day is mostly pleasant, there is some valid criticisms on the current approach using module(For|ForAcceptance|ForComponent).

• QUnit developed some nice features over the last months that, due to the existing APIs and the strict backwards compatibility policy, couldn’t be leveraged by the community. Nested modules are a great example.
• There is a bit too much magic in moduleFor, moduleForAcceptance and moduleForComponent. How do they work? Where does ember-qunit end and qunit start? Boundaries are not clear.
• Since QUnit is the default testing choice the existing API was developed with it in mind, causing that it wasn’t as easy as it should be for other testing frameworks like mocha to be supported properly. They had to duplicate a lot of functionality from ember-qunit that should really be shared, creating a maintenance burden on them.
• All the test helpers used in Acceptance tests (click, fillIn …) require jQuery to work, and Ember aims to make jQuery entirely optional soon-ish. Because of that I decided to create a library named ember-native-dom-helpers that offered jquery-free alternatives to the existing helpers (and added a few more). As popular as it became, it is only a community effort. You can’t really think in removing jQuery while shipping by default test helpers that don’t work without it.

The goal was to create a new testing API in which the responsibilities between Ember and the testing libraries (QUnit, Mocha…) were clearly separated, extract the common parts to a library that was framework agnostic and ship new redesigned test helpers that are more ergonomic and do not require jQuery.

How does it look like?

I’ll spare you the API details and I’ll just show you code. It’s always easier to grasp with examples:

Rendering Tests (Formerly Component Integration Tests)

This is the most common kind of test you will write day to day

Before:

import { moduleForComponent, test } from 'ember-qunit';
import hbs from 'htmlbars-inline-precompile';

moduleForComponent('x-foo', {
  integration: true
});

test('renders', function(assert) {
  assert.expect(2);
  this.render(hbs`{{pretty-color name="red"}}`);
  assert.equal(this.$('.color-name').text(), 'red');
  this.$('.green-btn').click();
  assert.equal(this.$('.color-name').text(), 'green');
});

After:

import { module, test } from 'qunit';
import { setupRenderingTest } from 'ember-qunit';
import { render, click } from '@ember/test-helpers';
import hbs from 'htmlbars-inline-precompile';

module('x-foo', function(hooks) {
  setupRenderingTest(hooks);

  test('renders', async function(assert) {
    assert.expect(2);
    await render(hbs`{{pretty-color name="red"}}`);
    assert.equal(this.element.querySelector('.color-name').textContent, 'red');
    await click('.green-btn');
    assert.equal(this.element.querySelector('.color-name').textContent, 'green');
  });
});

What changed?

• Now we don’t need a special moduleForComponent, we can just use the regular module from qunit.
• All the black magic to setup the QUnit context is on the setupRenderingTest from ember-qunit (and ember-mocha is working in its equivalent).
• The render function from @ember/test-helpers is a shared function that renders the given template.
• Please note that the new “Rendering test” name doesn’t contain the word “component” in it on purpose. You can test components, helpers or anything that can be invoked from a template really. When the upcoming glimmer components get added, you will not have to learn anything new.
• It leverages async/await for better readability.
• You use this.element instead of using the jQuery-based this.$
• No more manual invocation of jQuery events like this.$('.btn').click(). Now you can manually import the helpers (click, fillIn, etc…) you want to use from @ember/test-helpers, and they are basically copies from the ones in ember-native-dom-helpers, so you can use them in apps without jQuery.

Non-rendering Tests (Like unit tests, but not really “unit”)

I bet you didn’t write many Unit Tests for components/services/models before, and for a good reason. They were awkward to write, required you to whitelist every possible piece in the system that interacted with the class being tested using needs and very few times they survived a refactor of its target without breaking. That is because in practice our components and services interact with other parts of the system, and forcing isolation was pretty unnatural.

Say goodbye to that kind of tests. The new Non-rendering tests are for testing elements in scenarios that, well…, do not render anything. The objects under test still have full access to every object registered in the container.

With this kind of tests you can test components, but also models, services, controllers and routes with the same syntax and without learning special APIs for testing.

Before :

/* Component test */
import { moduleForComponent, test } from 'ember-qunit';

moduleForComponent('x-foo', {
  unit: true
});

test('computes properly', function(assert) {
  assert.expect(1);

  let subject = this.subject({ name: 'something' });

  let result = subject.get('someCP');
  assert.equal(result, 'expected value');
});

/* Service/Model/Route/Controller test */
import { moduleFor, test } from 'ember-qunit';

moduleFor('service:flash', 'Unit | Service | Flash', {
  unit: true
});

test('should allow messages to be queued', function (assert) {
  assert.expect(1);

  let subject = this.subject();
  subject.show('some message here');

  let messages = subject.messages;
  assert.deepEqual(messages, ['some message here']);
});

After:

/* Component test */
import { module, test } from 'qunit';
import { setupTest } from 'ember-qunit';

module('x-foo', function(hooks) {
  setupTest(hooks);

  test('computed properly', function(assert) {
    assert.expect(1);

    let Factory = this.owner.factoryFor('component:x-foo');
    let subject = Factory.create({ name: 'something' });

    let result = subject.get('someCP');
    assert.equal(result, 'expected value');
  });
});

/* Service/Model/Route/Controller test */
import { module, test } from 'qunit';
import { setupTest } from 'ember-qunit';

module('Unit | Service | Flash', function(hooks) {
  setupTest(hooks);

  test('should allow messages to be queued', function (assert) {
    assert.expect(1);

    let subject = this.owner.lookup('service:flash');
    subject.show('some message here');

    let messages = subject.messages;
    assert.deepEqual(messages, ['some message here']);
  });
});

What changed?

• Again, you don’t need any special moduleForComponent or moduleFor helpers. You use the regular module provided by QUnit. Components, Models, Controllers, Services… They are look the same.
• The setupTest provided by ember-qunit takes care of wiring everything in application’s container so you can test objects that depend on others, like models with relationships with other models or components that make use of services.
• You no longer have to learn special APIs only for testing like this.subject. Instead, you can use well-know APIs you also use in you’re regular code, like owner.factoryFor, owner.register, owner.inject and owner.lookup.

Application Tests (Formerly Acceptance tests)

Those are the tests that need to spawn a full application to test how every part of the system interacts in an asynchronous world.

Before:

import { test } from 'qunit';
import moduleForAcceptance from '../helpers/module-for-acceptance';

moduleForAcceptance('Acceptance | posts');

test('should add new post', function(assert) {
  visit('/posts/new');
  fillIn('input.title', 'My new post');
  click('button.submit');
  andThen(() => assert.equal(find('ul.posts li:first').text(), 'My new post'));
});

After:

import { module, test } from 'qunit';
import { setupApplicationTest } from 'ember-qunit';
import { visit, fillIn, click } from '@ember/test-helpers';

module('Acceptance | login', function(hooks) {
  setupApplicationTest(hooks);

  test('should add new post', async function(assert) {
    await visit('/posts/new');
    await fillIn('input.title', 'My new post');
    await click('button.submit');

    assert.equal(this.element.querySelectorAll('ul.posts li')[0].textContent, 'My new post');
  });
});

What changed?

• Yet again, no more moduleForAcceptance. Just a regular module from Qunit.
• The setupApplicationTest provided by ember-qunit spawns a new application instance for every test in a more efficient way than moduleForAcceptance. Before, every test created a new application from scratch that was thrown away at the end. Now setupApplicationTest creates a single app at the beginning of the test suite and for each test it only has to create a new ApplicationInstance, much like ember-fastboot does to reuse as much work as possible between requests to respond faster.
• The of async/await makes test easier to read and also allow to test some loading sub-states that were impossible to test using the andThen helper.
• No more jquery-based global helpers. Now you have to manually import the helpers (click, fillIn, etc…) you want to use from @ember/test-helpers. Yeah, they are the exact same helpers you use in integration. One less thing to learn.
• There is a new tap helper to simulate touch events 🎉!

Show me what else can I do!!

I’m glad you asked 😃

Nested modules!

This is close to my heart, as I think it can help a lot to clean up some shared setup steps. Now you can nest modules and have multiple beforeEach/afterEach that run in sequence:

import { module, test } from 'qunit';
import { setupRenderingTest } from 'ember-qunit';
import { render } from '@ember/test-helpers';
import hbs from 'htmlbars-inline-precompile';

module('user-profile', function(hooks) {
  setupRenderingTest(hooks);

  hooks.beforeEach(function() {
    this.user = this.owner.lookup('service:store').createRecord({ name: 'Robert' });
  });

  module('in small screens', function(hooks) {
    hooks.beforeEach(function() {
      this.owner.lookup('service:responsive').setViewport('small');
    });

    test('does not render the bio', async function(assert) {
      assert.expect(1);
      await render(hbs`{{user-avatar user=user}}`);
      assert.notOk(this.element.querySelector('.user-bio'), 'There is no bio');
    });
  });

  module('in big screens', function(hooks) {
    hooks.beforeEach(function() {
      this.owner.lookup('service:responsive').setViewport('desktop');
    });

    test('renders the bio', async function(assert) {
      assert.expect(1);
      await render(hbs`{{user-avatar name="red"}}`);
      assert.ok(this.element.querySelector('.user-bio'), 'The bio is rendered');
    });
  });
});

How do I test a plain JS class or function that does need access to the container?

This is not a real new feature per se, but a psychological effect I’ve noticed. Since the new approach is much more streamlined with regular QUnit tests, I do not have to change my mindset when switching from a high level application test to the low-level unit test of an utility function.

import { module, test } from 'qunit';
import { normalizePhonePrefix } from 'my-app/utils/phone-utils';

module('Unit | Utils | normalizePhonePrefix', function() {
  test('it works', async function(assert) {
    assert.equal('+34', normalizePhonePrefix('0034981345123'));
    // ...
  });
});

It just looks like any other test but we do not call any setupTest or setupRenderingTest because we don’t need it. It’s just a regular qunit test.

Mock services in acceptance tests with ease.

Before this new API you had to do some awful tricks with private APIs to reach for a service in the container and replace it with your mock. Now it’s just as simple as using this.owner.register.

import EmberObject from '@ember/object';
import { module, test } from 'qunit';
import { setupApplicationTest } from 'ember-qunit';
import { visit } from '@ember/test-helpers';

module('Acceptance | login', function(hooks) {
  setupApplicationTest(hooks);

  test('Dashboard in small screens', async function(assert) {
    this.owner.register('service:responsive', EmberObject.extend({ viewport: 'small' }));
    await visit('/dashboard');
    // ...
  });
});

Combine it with qunit-dom for the best testing experience ever.

One caveat of QUnit as a testing framework is that is designed as a general purpose framework and its assertions are pretty generic: assert.ok, assert.equal … We as web developers usually assert the structure of the DOM, and those low level assertions result in code that is quite verbose, specially if you don’t use jQuery.

P.e.

  assert.ok(this.element.querySelector('#user-bio').textContent.trim().indexOf('substring') > -1, 'The bio contains the substring');

The wonderful qunit-dom library comes to the rescue with a set of assertions specifically designed for the DOM, which results in much more readable code:

assert.dom('#user-bio').exists();
assert.dom('#user-bio').hasText('substring', 'The bio contains the substring');
assert.dom('#user-bio').hasClass('expanded');
assert.dom('#user-bio').doesNotHaveAttribute('title', 'Hello');
// and many more

You like it? (Pssst: It may make it into the default blueprint) This is how I choose to test applications these days. Try it for a day and you won’t go back.

Can I start using them?

Yes, you can! You need to update to the latest ember-cli-qunit and you can start using this new API today. The mocha version is on the works, but will be available soon.

I don’t want to rewrite my entire test suite! 😭

I know, that sucks. Luckily, for the most part, you don’t have to!

Once again, @rwjblue has you covered.

https://github.com/rwjblue/ember-qunit-codemod is a codemod that will automatically transform your test suite from the old style to the new one. At the time of this writing it does not support acceptance tests yet, but it will transform integration and unit tests for you. You may have to tweak a couple tests, but it’s pretty accurate and it does 90% of the work in seconds.

What’s the catch? There must be something bad!

There is one caveat. Globally registered test helpers (those you register with Ember.Test.registerAsyncHelper) will not work on this new “acceptance” tests. The system doesn’t want globals and instead encourages explicitly importing the helpers you need.

If you’ve created your own helpers or you’re the author of an addon that exposes test helpers, you will have to refactor them to continue to use them in the new acceptance tests, and I’ll explore how to do it on an upcoming blog post.