Wallaby.js Introduction: Troubleshooting

Wallaby.js issues are registered by using this repository. Whenever something goes wrong, please report an issue and we will address it as soon as we can.

Unless it’s not something obvious and easily reproducible, please make sure to do as much as you can from the checklist below.

Ensure tests are passing outside of Wallaby

Before you start troubleshooting, please make sure that your tests are running correctly (and passing) outside of Wallaby. Often the root cause of a Wallaby problem is an error in your tests/application code, or else a problem with your test runner configuration.

Diagnostics Report

Wallaby Diagnostics Report is a simple JSON text report that contains some basic information about your Wallaby config and project that is required to start troubleshooting most of Wallaby issues.

To share the report:

start Wallaby in your project;
wait until it reports some results (finishes running tests);
run Copy Diagnostics Report command:
- in VS Code, or Sublime editor execute Wallaby Copy Diagnostics Report command;
- in JetBrains IDEs or VS use Wallaby Help | Wallaby.js | Copy Diagnostics Report menu item;
paste generated report from your clipboard to a newly created issue (feel free to review and edit the report if required).

More debug information

You may view more debugging information from wallaby.js by setting the trace property to true in your wallaby configuration file and opening the “Wallaby Console” (in VS Code, JetBrains IDEs, Visual Studio, Sublime Text).

module.exports = function () {
  return {
    files: [
      ...
    ],

    tests: [
      ...
    ],

    trace: true
  };
};

If you are using Wallaby with automatic configuration, you may use the trace setting as described here by either using wallaby section in your package.json file or creating a lightweight Wallaby config file:

package.json

{
  ...

  "wallaby": {
    "trace": true
  }
}

wallaby.js

module.exports = () => ({
   trace: true
});

Also feel free to use console.log in wallaby.js configuration code (and of course in your code as well), to output any variables, parameters, etc.

module.exports = function (wallaby) {

  console.log(wallaby);

  return {
    files: [
      ...
    ],

    tests: [
      ...
    ],

    setup: function() {

      console.log(window.$);
      // etc.
    },

    trace: true
  };
};

Configuration issue

If you can, please provide us with a small sample project where the error can be reproduced. Best if you create a repository on GitHub and give us the link to your repo. We will have a look, fix the wallaby.js issue or the setup issue, and send you a pull request. In our experience, questions and issues with attached wallaby config and sample repository are resolved faster.

Try resetting your testing framework cache

Often, you may experience issues because your testing framework has cached a previous result or is serving the wrong file when running your tests. For example, if you use jest, you may clear jest cache from the command-line with npx jest --clearCache. Please refer to your testing framework documentation for further information on whether it uses a cache and if so, how to reset it.

Wallaby has its own cache for transformed and changed files. The Wallaby cache is automatically monitored for issues and repaired seamlessly in the background. If you are a VS Code user, you can clear the cache with the Wallaby.js: Reset Wallaby Cache command. If you use a JetBrains editor, you may clear the cache with the Help | Wallaby.js | Reset Wallaby Cache menu item. If you use another editor, you may clear the Wallaby cache by changing the timestamp on your project’s package.json. Clearing the Wallaby cache should never be required; if this solves your problem, please report an issue.

Wallaby stopped working after an update

If you are using IntelliJ platform based IDE or Visual Studio, try downgrading your plugin version.

First, check what’s your current plugin version. You may find it by opening Settings - Plugins for IntelliJ platform IDEs or Tools - Extensions and Updates in Visual Studio and locating wallaby.js there.

To download the previous version for IntelliJ platform IDE, use the following link: http://update.wallabyjs.com/wallaby-intellij-vx.y.x.zip. For example, if you are downgrading to version 1.0.34, the URL will be: http://update.wallabyjs.com/wallaby-intellij-v1.0.34.zip.

To download the previous version for Visual Studio, use the following link: http://update.wallabyjs.com/wallaby-vs-aaaa-vx.y.x.zip, where aaaa is your Visual Studio version. For example, if you are downgrading to version 1.0.17 in Visual Studio 2017, the URL will be: http://update.wallabyjs.com/wallaby-vs-2017-v1.0.17.vsix. If you are downgrading to version 1.0.17 in Visual Studio 2019, the URL will be: http://update.wallabyjs.com/wallaby-vs-2019-v1.0.17.vsix.

Once downloaded, the plugin version may be installed as described in the corresponding document section.

After you have installed the previous version and restarted your IDE, uncheck the Automatically update this extension checkbox.

Nothing works or too many different errors

Configuring a new tool for a project may be challenging. If you’re are using Webpack, Browserify, Babel, require.js, TypeScript/CoffeeScript (or possibly everything at the same time) or just have a pretty large/complex project, consider starting with just one simplest test file and trying to configure everything to run just that test file. Once it’s working, you may start adding other files and tests. We have a few samples you may start with.

Configuring wallaby.js step by step will not only allow you to join the camp of its happy users faster, but will also help to better understand how the tool works and make it easier and more fun to work with it.

Once some issue is isolated, please let us know about it. If you also create a repository where the issue can be easily reproduced, we’ll most likely fix it and publish the fix within 24 hours of the issue report.

Tests pass in other test runners (Karma, or Mocha, etc.), but fail in wallaby.js

If your tests pass in Karma using Chrome as a runner, check that wallaby is configured to use either Google Chrome, or the Electron test runner.

If your tests run fine in in other test runner, but you are having some issues when running them in wallaby.js, try setting the workers count to 1. If it starts working after the change, it most likely means that your tests depend on each other in some way. For example, one of your test suites that is running first sets some state, and test suites running later depend on it. You may keep the worker number set to 1 to avoid the issue, but we recommend making your test files isolated, so that wallaby.js can run them in parallel even faster. For node.js tests you may also try using workers.restart setting and setting it to true.

If the above mentioned suggestions don’t help, please attach your other runner configuration details (along with your wallaby config file) to your question.

Transient issues when running node.js tests

If you have some transient issues in your node.js tests, try using workers.restart setting and setting it to true. If that helps, then the most likely reason for your issues is that one or more of your tests set some state that causes tests to fail in the next run, because by default wallaby.js is trying to reuse node processes. You may either try adding some clean-up code to make node processes fully reusable as described here and make your test runs even faster, or keep using the workers.recycle setting set to true and rely on the process re-start.

‘Failed to instrument filename’

Your compiled code failed to be parsed by Wallaby during the instrumentation phase (which uses acorn).

Please check that your compiled code can be parsed by acorn. To get the compiled code you can wrap compilers or just copy code fragment reported with the error. Try pasting it into https://astexplorer.net/ with parser set to acorn.

If you are using Babel, potentially there is some syntax that is not recognized and is being passed through without generating a syntax error, that is then not parsable by acorn.

Tests are running too slow or consume too much RAM

Make sure that you have set instrument: false flag to all files that you don’t need the code coverage to be collected for, such as your project libraries. Not only this will make wallaby.js to consume much less memory, it may also increase your test runs speed.

‘Can’t find variable: require’ or ‘Can’t find variable: module’

If you are running node.js tests, make sure you have set the env property correctly. If you are running browser tests using CommonJs in PhantomJs or Chrome, make sure you have configured the corresponding postprocessor for Webpack or Browserify.

‘Cannot find module’ or ‘file not found’

If you are running tests in node.js and getting a Cannot find module error or other ‘file not found’ type of issues, please make sure that files that you are trying to load are listed in the files list. Wallaby.js uses its own cache, so it has to copy used files there. Note that this does not apply to the node_modules folder, as wallaby.js doesn’t cache it and uses your local version of it. So, no need to list node modules in the files list for your node tests (but it may be required for browser tests if you are using node modules as libs).

‘Module did not self-register’

If you are running tests in node.js and getting a Module did not self-register error or other node version specific issues, please make sure that you have configured wallaby.js to use the correct version of node.js for running tests.

phantom stderr […] is not recognized […] on Windows

If you are on Windows and getting a phantom stderr: 'C:\Users\...' is not recognized as an internal or external command, operable program or batch file error, download PhantomJs, place it in some folder with a path that doesn’t contain spaces, and configure env.runner as described here.

phantom stdout: NETWORK_ERR: XMLHttpRequest Exception 101: A network error occurred in synchronous requests.

If you are using PhantomJs2 and getting the error, you need to pass --web-security=false flag to PhantomJs.

`Error: unrecognized flag --harmony_arrow_functions` or `bad option: --harmony_arrow_functions`

If you are getting this error for your browser tests, try updating the node.js version used by wallaby.js. You may see how to set the node.js executable used by wallaby.js for your editor in the docs section. For IntelliJ platform, try resetting the node.js version in the Run Configuration to the default one (created with a new Run Configuration).

If you are getting this error for your node.js tests because the node version you use for your runner doesn’t support the flag, try overriding node flags by specifying those you need (or an empty array) in the env.params.runner setting.

No files with tests found, check your tests list patterns.

If you see the No files with tests found, check your tests list patterns. message in the Wallaby Console in VS Code, or Sublime Text, even though your tests patterns are correct, please make sure your project path doesn’t include any parentheses, stars or other glob characters (like (, ), *, [, ], {, }, |).

Wrong or missing coverage indicators in the editor

If coverage indicators are not showing in your editor or are showing on the wrong lines, please make sure that you have source maps enabled in the relevant compiler settings for your project.

Wallaby was working, but then stopped working

Clear the local server file cache. Currently you can force it to be cleared by just stopping your wallaby run configuration, changing the wallaby configuration file content (even just adding a space somewhere inside) and running wallaby run configuration again.

Sometimes Wallaby can break due to node_module version conflicts. This can be a result of package.json.lock/yarn.lock conflicts, upgrading node.js versions, and sometimes conflicts between dependencies. If you recently added a new package, try removing the package and see if Wallaby starts working again. Also try deleting your node_modules folder, package.json.lock, yarn.lock files and then reinstall your modules using your package manager.

If the issue does not disappear, keep going through the checklist.

Cannot start node.js process (node)

Make sure your system has node.js installed. Configure wallaby.js to use a path to (a specific version of) node.js. If you are running wallaby.js for node.js tests, you may also need to remove env.runner setting (or adjust it to point to your node folder in a worst case scenario).

Status indicator keeps spinning forever or where to find the log files

You may need to check the editor log files.

For IntelliJ Platform IDEs, review idea.log: the location is available from the “Help - Show Log” menu of your code editor.

For Visual Studio, review the log file located in this folder: %USERPROFILE%\AppData\Local\Temp\.wallaby\.

For Visual Studio Code, review the output in the “Wallaby Console” output channel.

Forcing wallaby.js core update

Wallaby.js core gets updated automatically. When a new version is published, it will be updated during one of wallaby.js restarts within a few minutes (depending on when was the last version check).

Sometimes you may want to force the update to happen as soon as possible. To do that, please follow the instructions for your editor below.

In VS Code: open the editor and launch the Command Palette using Ctrl/Cmd + Shift + P, then run the command Wallaby.js: Force Core Update.

In JetBrains IDEs: open the editor and run the Help | Wallaby.js | Force Core Update command.

In Visual Studio: just restart your IDE and start wallaby.js to force the core update.

In Sublime Text: close the editor, open <SUBLIME-PACKAGES-FOLDER>/Wallaby/ folder, delete the wallaby subfolder, start the editor.

Feature Use Telemetry

By default, Wallaby collects anonymized feature use telemetry. This anonymous data is used to help us understand how Wallaby is being used, and to help us prioritize future development. You can opt out of this telemetry by setting the telemetry option to false in your Wallaby.js configuration file.

Separate projects for tests and source files in Visual Studio

If you have separate projects for your source files and tests in Visual Studio, you may make your wallaby.js configuration file one of the Solution Items.

Instrumented code and generated test page

When doing some advanced wallaby.js configuration, its local cache may be a useful place to have a look into for troubleshooting. When the trace: true flag is set, the wallaby.js local cache folder is printed at the beginning of the wallaby.js log when it starts (search for File cache: in the log). After wallaby.js starts, look into the instrumented subfolder to make sure all of the files are there and in the state you expect them to be.

You may also search for localhost in the output and, while wallaby.js is running, try opening the URL you have found in Chrome with DevTools. You may be able to see the code and the test page that wallaby.js generates to run your tests.

Offline installation

Please refer to our Offline Installation guide.