setupFilesAfterEnv prevents test results from being reported

[kasvtv]

Issue description or question

If I add a setupFilesAfterEnv option for Jest, wallaby still seems to run, but stops reporting test results. It will always show 0 succeeded and 0 failed.

See https://github.com/kasvtv/wallaby_setup_bug

Wallaby diagnostics report

{
  editorVersion: '1.44.0',
  pluginVersion: '1.0.193',
  editorType: 'VSCode',
  osVersion: 'win32 10.0.17763',
  nodeVersion: 'v12.16.1',
  coreVersion: '1.0.877',
  config: {
    files: [ { pattern: 'index.js', ignore: false, trigger: true, load: true, order: 1 } ],
    tests: [ { pattern: 'index.test.js', ignore: false, trigger: true, load: true, test: true, order: 2 } ],
    diagnostics: {},
    filesWithCoverageCalculated: [],
    testFramework: { version: '[email protected]', configurator: '[email protected]', reporter: '[email protected]', starter: '[email protected]', autoDetected: true },
    micromatch: true,
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    compilers: {},
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    extensions: {},
    env: { type: 'node', params: {}, runner: 'C:\\Program Files\\nodejs\\node.exe', viewportSize: { width: 800, height: 600 }, options: { width: 800, height: 600 }, bundle: true },
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: true,
    configCode: 'module.exports = () => ({\r\n' +
      '  autoDetect: true,\r\n' +
      '  files: [\r\n' +
      "    'index.js',\r\n" +
      '  ],\r\n' +
      '  tests: [\r\n' +
      "    'index.test.js',\r\n" +
      '  ],\r\n' +
      '  setup(wallaby) {\r\n' +
      '    wallaby.testFramework.configure({\r\n' +
      "      setupFilesAfterEnv: ['./setupFilesAfterEnvFile.js']\r\n" +
      '    })\r\n' +
      '  }\r\n' +
      '})\r\n'
  },
  packageJSON: { dependencies: undefined, devDependencies: { '@types/jest': '^25.2.1', '@types/node': '^13.11.1', jest: '^25.3.0' } },
  fs: { numberOfFiles: 2 },
  debug: [
    '2020-04-13T14:18:44.802Z config Detected Jest.\n',
    '2020-04-13T14:18:44.803Z config Configured Jest.\n',
    '2020-04-13T14:18:44.804Z project Wallaby Node version: v12.16.1\n',
    '2020-04-13T14:18:44.804Z project Wallaby config: C:\\Users\\Hexa\\go\\src\\github.com\\3PLogic\\wallaby_fail\\wallaby.conf.js\n',
    '2020-04-13T14:18:45.570Z project File cache: C:\\Users\\Hexa\\.vscode\\extensions\\wallabyjs.wallaby-vscode-1.0.193\\projects\\84ce35f60473fde0\n',
    '2020-04-13T14:18:45.635Z uiService Listening port 51236\n',
    '2020-04-13T14:18:45.655Z project Config file change detected, invalidating local cache\n',
    '2020-04-13T14:18:45.662Z workers Parallelism for initial run: 10, for regular run: 5\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #0\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #1\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #2\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #3\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #4\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #5\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #6\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #7\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #8\n',
    '2020-04-13T14:18:45.663Z workers Starting run worker instance #9\n',
    '2020-04-13T14:18:45.664Z workers Web server is listening at 57164\n',
    '2020-04-13T14:18:45.664Z project File cache requires some updates, waiting required files from IDE\n',
    '2020-04-13T14:18:45.721Z project Stopping process pool\n',
    '2020-04-13T14:18:45.723Z project Test run started; run priority: 3\n',
    '2020-04-13T14:18:45.725Z project Running all tests\n',
    '2020-04-13T14:18:45.726Z workers Starting test run, priority: 3\n',
    '2020-04-13T14:18:45.726Z nodeRunner Starting sandbox [worker #0, session #87m61]\n',
    '2020-04-13T14:18:45.726Z nodeRunner Preparing sandbox [worker #0, session #87m61]\n',
    '2020-04-13T14:18:45.883Z workers Started run worker instance (delayed) #4\n',
    '2020-04-13T14:18:45.885Z workers Started run worker instance (delayed) #1\n',
    '2020-04-13T14:18:45.886Z workers Started run worker instance (delayed) #0\n',
    '2020-04-13T14:18:45.887Z nodeRunner Prepared sandbox [worker #0, session #87m61]\n',
    '2020-04-13T14:18:45.887Z workers [worker #0, session #87m61] Running tests in sandbox\n',
    '2020-04-13T14:18:45.888Z workers Started run worker instance (delayed) #5\n',
    '2020-04-13T14:18:45.890Z workers Started run worker instance (delayed) #3\n',
    '2020-04-13T14:18:45.891Z workers Started run worker instance (delayed) #2\n',
    '2020-04-13T14:18:45.896Z workers Started run worker instance (delayed) #6\n',
    '2020-04-13T14:18:45.899Z workers Started run worker instance (delayed) #7\n',
    '2020-04-13T14:18:45.900Z workers Started run worker instance (delayed) #8\n',
    '2020-04-13T14:18:45.901Z workers Started run worker instance (delayed) #9\n',
    '2020-04-13T14:18:48.239Z workers [87m61] Run 0 test(s), skipped 0 test(s)\n',
    '2020-04-13T14:18:48.241Z workers [87m61] Sandbox is responsive, closing it\n',
    '2020-04-13T14:18:48.242Z project Test run finished\n',
    '2020-04-13T14:18:48.242Z project Processed console.log entries\n',
    '2020-04-13T14:18:48.242Z project Processed loading sequences\n',
    '2020-04-13T14:18:48.242Z project Processed executed tests\n',
    '2020-04-13T14:18:48.243Z project Processed code coverage\n',
    '2020-04-13T14:18:48.247Z project Test run result processed and sent to IDE\n'
  ]
}

[smcenlly]

Thanks for the sample repo!

I can see that you're still using a Wallaby configuration file with jest. Is there any reason for this? The best way to use Wallaby with automatic configuration is without a configuration file unless there is no other option.

Generally speaking, the best thing to do is to configure jest and use Wallaby without a configuration file. In your case:

1. Create a file called jest.config.js with the content:

module.exports = {
  clearMocks: true,
  setupFilesAfterEnv: ['./setupFilesAfterEnvFile.js'],
  testEnvironment: "node",
};

2. Delete wallaby.conf.js

3. In VS Code, Run the command Wallaby.js: Select Configuration and select Automatic Configuration: <project directory>.

At this point, everything should work for you.

[ArtemGovorov]

Thanks for providing the repo. If the suggestion by @smcenlly (configure Jest via Jest config and remove Wallaby config) is not possible in your case for some reason, we have fixed the original issue (when Wallaby is configuring Jest via setup function) and published the fix in the latest Wallaby core, so after the update your repo should start working as is.