Optimizing your tests to run faster

We recently volunteered our time to assist a group of high-school kids in our community who are new to coding. It was immediately clear that they did not understand the importance of the feedback loop that exists between writing code and knowing that the code does what was intended.

The students would code for 20-30 minutes at a time, implementing disparate features in separate parts of their program. Inevitably, when they tried to run their code it would be broken in multiple places. Because they made multiple changes before testing, they found it difficult to recall what code had been modified so that they could narrow their debugging efforts.

Working with the students reminded us of the importance of explicitly spelling out what can be done to improve your feedback loop. We hope you find the information in this post useful for optimizing your project’s tests in order to improve your feedback loop!

Feedback loops tell you whether or not your efforts are paying off. The faster and more detailed the feedback, the more information you have to make corrections and improve.

Wallaby.js provides software developers with the ultimate hyper-feedback loop for code that is covered by tests. However, there is an important caveat. The feedback loop that Wallaby provides is limited to how long it takes your tests to run.

While there is no limit to how long your tests take to run with Wallaby, you will really benefit from the improved feedback loop when incremental test runs take less than 1-2 seconds to run. There are a few things that you can do and should be aware improve your test execution times.

Identify slow running tests

Sometimes your test or application simply needs to do a lot of work, or the code is computationally complex and therefore takes a long time to execute. Often though, a slow test is the result of some incorrect logic, or a regression in performance since the code was first written.

Wallaby App is the perfect tool for identifying slow tests using the slower than filter, as shown below.

Profiling Slow Tests

Wallaby’s Test Profiler allows you to quickly record a test’s CPU usage profile to analyze its runtime performance.

After capturing the CPU Profile, you can analyze it to understand which code in your test is taking some time, then optimize your code and repeat the process.

Split tests across multiple files

Wallaby (and some other test runners) analyze your file/folder structure to run your tests in parallel. This means that your tests will run faster if you have a larger number of smaller test files vs. a smaller number of large test files. Wallaby even goes one step further and knows how to run only the specific test that you are currently editing.

This does not mean that you should break up tests into multiple files artificially, but where it makes sense to do so, it will help performance. You may also find that older projects have large test files that have grown organically and tests within those files can be reasonably split out to multiple files.

Single responsibility per code file

It is fairly common for projects to have a shared file (e.g. utils.js) that is home to a number of functions that don’t really fit elsewhere in your project. Whenever you change a function in that file, Wallaby needs to re-execute all the tests and code that imports the file because they may be affected by the change. This can lead to long execution times as perhaps hundreds or thousands of tests need to be run when you change the shared file, instead of just a few related tests.

You will find that your tests run faster in Wallaby when each file has a single reason to change. The result of doing this is that you will better isolate how your code files and test files are used by one another. If you can structure your code like this, when you make a change to a code file only the tests associated with that logic will be run.

Fast vs. Slow Wallaby Configurations

Depending on your application, you may have a mixture of unit tests and integration tests (or faster tests and slower tests). Usually, it is OK to run the slower tests after a change (or as a part of your continuous integration test). Consider creating separate Wallaby configuration files to improve your feedback loop (e.g. wallaby.unit.js vs. wallaby.integration.js).

Note: if you are using Automatic Configuration:

• you can create a configuration file and modify the files and tests settings that are determined by automatic configuration
• you can create a configuration file and entirely replace the files or tests by defining an array (in which case you also need to specify autoDetect: true)

Thanks for reading! We hope you found this blog post useful. If you need help optimizing your project's tests, let us know via our GitHub repo or in an email.

As usual, we would love to hear your feedback on Twitter.