Browser code coverage from E2E testing

Istanbul provides coverage metrics to Javascript code.
It can work with broad range of testing framework including E2E.

For browser testing, general steps are as follows:

  1. Transpile Typescript to Javascript with sourcemap.
  2. Convert source code with nyc instrument.
  3. Bundle without minify.
  4. Run test suite and collect metrics.
  5. Generate reports with nyc report

Transpile Typescript

Istanbul supports several Javascript flavor through nyc config plugins.
Otherwise, you need to transpile code into Javascript at first.

For example, esbuild can convert whole project with script like this:

import * as esbuild from 'https://deno.land/x/esbuild/mod.js'
import { expandGlobSync } from "https://deno.land/std/fs/mod.ts

const input = [];
for (const f of expandGlobSync(`${Deno.cwd()}/src/**/*.{ts,js}`)) {
        input.push(f.path);
}

await esbuild.build({
    entryPoints: input,
    outdir: "./plainjs/",
    format: "esm",
    bundle: false,
    minify: false,
    sourcemap: true,
});

esbuild.stop()

nyc can utilize sourcemap, so coverage report can show in original format with sourcemap.
Although esbuild generate js code, import remains to be ‘*.ts’ references. sed can convert .ts reference:

find ./plainjs/ -type f | grep -v map$ | xargs sed -i 's/\.ts/.js/g'

Instrument code

nyc instrument modifies code to collect metrics.
You can run nyc with npx or deno run:

deno run -A --unstable npm:nyc instrument plainjs/ instrument/

Bundle instrumented code

You can bundle nyc instrument outputs with ordinal bundle tools.
minify should be avoided for safety.

Run test suite

You can run your test suite including selenium as usual.

End of each case, you need to add metrics logging process:

cvr = driver.execute_script("return JSON.stringify(window.__coverage__)")
with open(f"/path/to/coverage/{time.time()}.json", 'w') as f:
    f.write(cvr)

Instrumented code writes metrics into window.__coverage__, so you need to dump this object to JSON files.

Generate reports

With metrics JSON files, you can generate reports:

deno run -A --unstable npm:nyc report \
        --reporter html --reporter text \
        -t coverage \
        --report-dir report

--reporter text shows console report, and --reporter html generate a set of HTML.

⁋ Dec 2, 2022↻ Oct 16, 2024
中馬崇尋
Chuma Takahiro