Introduction

Every engineer I've worked with has had strong opinions about testing. And regardless of their take, there’s one recurring theme: setting up the infrastructure around these tests is hard.

And why wouldn’t it be? There are dozens of frameworks to choose from. You have to define a strategy around unit, component, and integration testing. Your product manager needs to be convinced all the time and effort — with lots of flaky tests along the way — will be worth it. This is a daunting task for anyone, and it’s where our mobile app was when I came on board at Proton. Today, I’d like to walk you through how we got from nothing to a reliable end-to-end testing framework on our mobile application.

As I write this, our React Native mobile application isn’t even a year old. When I first saw the codebase, I was delighted to see that the engineers had already added test infrastructure.  There were configuration files to set up Jest, React Native Testing Library (for component tests), and even Detox for end-to-end tests. Unfortunately, no one had written any tests, though. After some initial conversations in my first few weeks, I confirmed exactly what I was thinking. We had a setup in place, but neither the time nor resources to actually write tests. Fortunately, as I began exploring the code we were just around the corner from a "cooldown period". After a feature development cycle, we like to pause to give our product team time to plan and our engineers time to work on their own projects. In this case, we had a three-week period where the engineers were in charge and had the liberty to work on areas of the codebase that needed improvements. And there it was, the green light that I wanted!

Testing mantra

Write tests. Not too many. Mostly integration!

This has been my favorite mantra around testing for quite some time now. It’s a mantra propagated by one of my favorite developers, Kent C Dodds. He argues that the more your tests resemble the way a user interacts with your product, the more confident you can be shipping code. If we’re writing an end-to-end test, your harness boots up the app and then literally, clicks around while making assertions - instantly testing multiple points of failure in the app or a feature as a whole.

But there’s a trade-off. End-to-end tests only give you more confidence when they exactly mirror how your app behaves in production and the way a user might actually use the feature. This makes the tests expensive: they have to run in a high-fidelity production-like environment, rather than in a limited testing sandbox. And not to mention the complexity: For example,if we’re testing our mobile app, we have to boot up an iPhone or Android emulator inside the same production-like environment in our continuous integration system, load our entire application, configure mocks where needed, and then ensure each test suite runs in an isolated environment, multiplying the size of the problem.

Is this trade-off worth it? Let's look at an example:

We have a tasks feature in our app that lets sales reps create tasks, edit tasks and change task status. The following example captures a test suite for tasks creation:

describe('Tasks Screen', () => {
  beforeEach(async () => {
   await device.reloadReactNative();
  });

  test('create new tasks', async () => {
  await expect(element(by.id('Tasks'))).toBeVisible();
  element(by.id('Tasks')).tap();
   await waitFor(element(by.text('Overdue')))
     .toBeVisible()
     .withTimeout(3000);
  // Create some tasks
  element(by.label('fab')).tap();
  await element(by.text('Task')).tap();
  await element(by.id('Title')).tap();
  element(by.id('Title')).typeText('TEST TASK 1');
  element(by.id('Notes')).tap();
  element(by.id('Notes')).typeText('This is the first test task');
  element(by.id('Create')).tap();
  await element(by.label('fab')).tap();
  await element(by.text('Task')).tap();
  await element(by.id('Title')).tap();
  await element(by.id('Title')).typeText('TEST TASK 2');
  element(by.id('Notes')).tap();
  element(by.id('Notes')).typeText('This is the second test task');
  element(by.id('Create')).tap();
  // Swipe on the container to see the upcoming tasks
  await waitFor(element(by.id('tasks-carousel')))
    .toBeVisible()
    .withTimeout(3000);
 await element(by.id('tasks-carousel')).swipe('left');
 await expect('TEST TASK 1').toBeVisible();
 expect('TEST TASK 2').toBeVisible();
});

First, we have some initial code we run using the beforeEach function. We reload the app between test cases, so that each test runs in an isolated environment, and we don’t have any data or methods leaking dat between tests. Then, we use detox's intuitive helper functions to interact with various elements on the screen, like buttons and text inputs, and perform gestures like swipes. Finally, we assert that we can indeed see the two newly created tasks. We have similar test suites for task edits, changes to task status, and other task state mutations. You can see this in action at the end of the article in the attached video.

Catching a bug

The following example illustrates where the test suite really shines:

test("overdue tasks bug", async () => {
  await expect(element(by.id("Tasks"))).toBeVisible();
  element(by.id("Tasks")).tap();
  await waitFor(element(by.id("tasks-carousel")))
    .toBeVisible()
    .withTimeout(3000);
  await element(by.id("tasks-carousel")).swipe("left");
  await element(by.id("TEST TASK 2 Actions")).tap();
  await element(by.text("Edit")).tap();
  await element(by.id("date-picker")).tap();
  await element(by.text("6")).tap();
  await element(by.text("Confirm")).tap();
  await waitFor(element(by.text("* Must choose a day in the future")))
    .toBeVisible()
    .withTimeout(1000);
  await element(by.text("Update")).tap();
  await waitFor(element(by.id("tasks-carousel")))
    .toBeVisible()
    .withTimeout(3000);
  await element(by.id("tasks-carousel")).swipe("right");
});

Here, we were able to edit (or create) a task, assign a date in the past, and the task lands in our overdue list, even though we were thrown an error message stating that we must choose a day in the future. Our test suite caught a bug! Impressive.

Conclusion

The next step is to add these tests on our continuous integration system. Once we have a configuration in place for our emulators, we can estimate the time and resources needed to implement these tests for the other features in our application. A noteworthy feature of detox is the ability to run the application in debug mode, which paves the way for test driven development. The control of writing tests alongside active development of new features is the holy grail! Developing a testing strategy is hard. Some choices make it easier, such as our use of  static typing via Typescript. This adds a blanket of protection that  makes me comfortable putting off unit tests for now. Component level tests with the React Native testing library are definitely nice to have, but when you have a clean slate, I think end-to-end tests are the perfect place to start, despite some of the challenges. Because  the test suites interact with a feature exactly like how a real user of the app would, it naturally invites you to think about writing code for users and, not just to meet a specification sheet. And as we’ve seen here today, it doesn’t have to take moving heaven and earth to get started. Go forth and test!