Effective Layout Testing Library for iOS
January 19, 2016
When developing iOS applications, a lot of the code creates and lays out views. However, in many applications, this code has the least test coverage. Leaving this untested increases the reliance on manual testing and makes regressions more common. Testing layout logic is often ignored because it can be difficult. Developers usually write unit tests to verify business logic and UI integration tests (e.g. UIAutomation or KIF) to test views and interactions. However, testing views with UI integration tests has a few drawbacks:
- UI integration tests run slowly. Because they interact with the application, they are filled with waits and pauses while the application responds to events.
- UI integration tests can also be flaky, because they rely on so many different parts of the application.
- It can be difficult to access parts of the view hierarchy. This depends on the framework you are using, but most automation frameworks start with a window object and allow you to traverse the view hierarchy instead of giving you direct access to views.
- Because integration tests are slow to write and slow to run, they usually only test the “happy paths” (normal flow) of the application. During a test run, a view will probably only be laid out with one or two sets of data, and no one is surprised when later, an edge case reveals a bug in the view layout.
- Often these tests don’t actually verify the layout of the view. They don’t verify that views are not overlapping or that elements are laid out in the correct order.
To solve these problems, we created LayoutTest, a new iOS utility which we have open sourced. LayoutTest uses the speed and stability of unit tests to test your view with many different combinations of data. The tests are quick to write and automatically run several assertions for you. We recently rewrote our flagship application at LinkedIn and wrote over 100 LayoutTests, which now tests over 34,000 view instances. All of these tests run alongside our other unit tests, which complete in less than a minute. Running this many tests and data combinations is not possible with UI integration tests.
Layout tests are written as part of the unit test suite:
1. First, you provide a data spec to the library which uses this spec to generate many different combinations of data. This data spec looks similar to the data your view will consume, but instead of specific values contains generic ‘string value’ and ‘bool value’ objects. The library uses these objects to create different versions of the data spec.
For example, using this spec, the library would create 150 different combinations of data:
2. For each of these combinations, the library lays out your view with this data.
3. Next, it runs some basic tests automatically, which all views should pass, such as testing for overlapping views, Auto Layout errors, and constraint violations (more info on this below).
4. Finally, it gives you the opportunity to make custom test assertions on your view. For example, you may want to check that a label has the correct text or that views are laid out in the correct order.
Note that for a single test, it will lay out the view with multiple different combinations of data. For example, if you look at a visual representation of the different combinations of views that this library will test, it would look something like this:
This is just a portion of the test, but you can see that it is testing special characters, normal length strings, different types of buttons, and absent images. You can also see the combinations of data the library is creating. Each new row changes one part of the data, such as the button titles, the main text or the enabled state of the button.
For each test, the library runs a few tests automatically. It checks that no views are overlapping and that they are within the bounds of their superview. For instance, these two views would fail automatically:
These bugs aren’t usually noticed when first writing the view, but are often introduced with regressions or data you didn’t originally expect. The library also ensures that Auto Layout doesn’t throw any errors and isn’t ambiguous. Then, it runs some basic accessibility tests, such as ensuring all buttons have accessibility labels. All of these features are fully configurable.
LayoutTest also helps you test your views on different device sizes. Often, views need to handle multiple different widths and heights on different devices. When laying out the view, the size sometimes needs to be calculated dynamically. LayoutTest makes testing this easy. With a single line of code, you can run every test on several different widths or heights and ensure that your layout logic doesn’t break on different devices.
As a result of developing LayoutTest, we have much better test coverage for view layout compared to previous projects. The tests are extremely fast and reliable as well as easy to write. As we added layout tests to more views, we started uncovering hidden bugs which we then had a chance to fix before shipping the app to users.
We think that LayoutTest would be useful for other iOS applications. We’ve released LayoutTest as an open source project on GitHub, and we hope that you will try it out. Our documentation has helpful information and examples.