RUNNING TESTS

CI/CD

Run Karate tests in CI, publish the HTML reports as a live artifact your team can link to, and guard against leaking credentials into those reports. This page is organised around a single reference implementation — karate-todo — so every pattern shown here is green on main and you can open the resulting reports right now.

See it live

Reports from the reference implementation's latest green build:

• Karate summary (API + UI): karate-summary.html
• Parallel timeline: karate-timeline.html
• UI feature with embedded screenshots: simple.feature
• Gatling performance report: gatling/index.html

Full workflow source: .github/workflows/cicd.yml.

Workflow shape

The reference workflow stages jobs linearly so a failure short-circuits the pipeline:

tests ──▶ gatling ──▶ secret-scan ──▶ publish (main only)

Job	What it does
tests	./mvnw verify -Pui — one hybrid suite (API + UI) via Testcontainers + headless Chrome
gatling	Starts the app in the background, runs a Gatling smoke, uploads the HTML report
secret-scan	Greps the Karate and Gatling report artifacts for common token / private-key patterns
publish	On main only: deploys the assembled reports to gh-pages

needs: wires the dependencies so the downstream jobs only run if everything before them passed, and artifacts flow from job to job.

Running tests in GitHub Actions

The tests job runs a consolidated Karate suite that drives both the API and a containerised Chrome against the same in-process app:

.github/workflows/cicd.yml

tests:
  name: API + UI tests (Testcontainers)
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-java@v4
      with:
        java-version: '21'
        distribution: 'temurin'
        cache: maven
    - name: Run api + ui suite (chromedp/headless-shell via Testcontainers)
      run: ./mvnw -B verify -Pui
    - name: Upload Karate report
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: karate-report
        path: target/karate-reports
        retention-days: 14

Node.js 24 opt-in

Set FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" at the workflow level to pre-empt GitHub's 2026-06-02 Node 20 deprecation before it starts forcing the switch.

Hybrid API + UI in one suite

The reference UiTest runs both API and UI features from a single run, so the published report covers the full surface:

src/test/java/app/ui/UiTest.java

@Test
void testAll() {
    ContainerDriverProvider provider = new ContainerDriverProvider(chrome);
    SuiteResult result = Runner.path("classpath:app/api", "classpath:app/ui")
            .tags("~@external", "~@todo")
            .systemProperty("serverUrl", chrome.getHostAccessUrl(PORT))
            .systemProperty("apiUrl", "http://localhost:" + PORT)
            .driverProvider(provider)
            .parallel(1);
    assertEquals(0, result.getScenarioFailedCount(), String.join("\n", result.getErrors()));
}

Two URLs, one app:

• serverUrl — what the browser navigates to. Inside the Chrome container this is host.docker.internal:<port> (Docker Desktop on macOS/Windows) or the host-gateway mapping on native Linux.
• apiUrl — what Karate's HTTP client (running on the host) uses. Plain localhost:<port>.

karate-config.js picks these up:

src/test/java/karate-config.js

function fn() {
  var serverUrl = karate.properties['serverUrl'] || 'http://localhost:8080';
  var apiUrl = karate.properties['apiUrl'] || serverUrl;
  karate.configure('driver', { type: 'chrome' });
  return { serverUrl: serverUrl, apiUrl: apiUrl };
}

See UI Testing — Testcontainers for the ChromeContainer + ContainerDriverProvider pattern behind this.

Publishing HTML reports to GitHub Pages

Every green push to main deploys target/karate-reports/ and the Gatling output to a gh-pages branch, giving you a stable URL to paste into release notes, Slack, or trainings.

.github/workflows/cicd.yml

publish:
  name: Publish reports to gh-pages
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  needs: secret-scan
  runs-on: ubuntu-latest
  steps:
    - uses: actions/download-artifact@v4
      with:
        name: karate-report
        path: public/latest/karate
    - uses: actions/download-artifact@v4
      with:
        name: gatling-report
        path: public/latest/gatling-raw
    - name: Flatten Gatling output
      run: |
        # Gatling writes into `gatling/todosimulation-<timestamp>/` — move the
        # contents up one level so /latest/gatling/ serves index.html directly.
        sim=$(ls -d public/latest/gatling-raw/todosimulation-* | head -1)
        mv "$sim" public/latest/gatling
        rm -rf public/latest/gatling-raw
    - name: Deploy to gh-pages
      uses: peaceiris/actions-gh-pages@v4
      with:
        github_token: ${{ secrets.GITHUB_TOKEN }}
        publish_dir: public
        keep_files: false

Enabling Pages

The first deploy creates the gh-pages branch. Enable Pages via Settings → Pages → Source = Deploy from a branch → gh-pages / root, or from the command line:

gh api repos/<owner>/<repo>/pages -X POST --input - <<'EOF'
{"build_type": "legacy", "source": {"branch": "gh-pages", "path": "/"}}
EOF

Scanning reports for leaked secrets

HTML reports are great for transparency — and dangerous if a test accidentally logs an Authorization header or a token. A short grep step at the end of the pipeline catches the common patterns:

.github/workflows/cicd.yml

secret-scan:
  name: Secret-leak scan
  needs: gatling
  runs-on: ubuntu-latest
  steps:
    - uses: actions/download-artifact@v4
      with:
        name: karate-report
        path: reports/karate
    - uses: actions/download-artifact@v4
      with:
        name: gatling-report
        path: reports/gatling
    - name: Grep reports for known secret patterns
      run: |
        set +e
        PATTERNS='Bearer [A-Za-z0-9_.\-]{20,}|gh[pousr]_[A-Za-z0-9]{30,}|sk-[A-Za-z0-9]{20,}|AKIA[0-9A-Z]{16}|-----BEGIN (RSA |EC |OPENSSH |DSA )?PRIVATE KEY-----'
        MATCHES=$(grep -REn "$PATTERNS" reports/ || true)
        if [ -n "$MATCHES" ]; then
          echo "::error::secrets found in published reports:"
          echo "$MATCHES"
          exit 1
        fi
        echo "no secret patterns matched"

The patterns cover Bearer tokens of reasonable length, GitHub PATs / OAuth / server-to-server tokens, OpenAI-style keys, AWS access key IDs, and PEM-encoded private keys. Extend the regex for organisation-specific formats.

Defence in depth

This is a last-line check on what actually reaches the report. Complement it with a pre-commit gitleaks or trufflehog scan so secrets never land in the feature files in the first place.

Gatling smoke with a background app

Gatling needs the app up on a known port. The simplest pattern: start the app in the background, poll the port, run Gatling, kill the app.

.github/workflows/cicd.yml

gatling:
  name: Gatling smoke
  needs: tests
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-java@v4
      with:
        java-version: '21'
        distribution: 'temurin'
        cache: maven
    - name: Start app in background
      run: |
        mkdir -p target
        nohup ./mvnw -B test -Dtest=LocalRunner > target/local-runner.log 2>&1 &
        echo $! > target/local-runner.pid
        for i in {1..60}; do
          if nc -z localhost 8080; then break; fi
          sleep 1
        done
        nc -z localhost 8080 || (echo "app did not start"; cat target/local-runner.log; exit 1)
    - name: Run Gatling simulation
      run: ./mvnw -B test -P gatling
    - name: Stop app
      if: always()
      run: kill "$(cat target/local-runner.pid)" || true

Why not a service container?

GitHub Actions services: want a published image. Starting the app in the same workspace keeps the workflow self-contained (no registry, no extra Dockerfile) and matches what trainers do in two terminals locally.

Filtering tests per CI stage

Tag filtering is how you keep CI jobs focused. In the reference implementation:

Tag	Used for
@external	Features that hit real external hosts (httpbin, jsonplaceholder, google) — excluded from CI via ~@external
@todo	Features gated on an upstream fix — excluded from CI via ~@todo
@smoke	Fast critical path — can be used to gate earlier jobs

The tests job runs everything non-@external and non-@todo. An earlier "smoke" job could run .tags("@smoke") only. See Tags for the full filtering grammar.

Codespaces

The same commands work inside a GitHub Codespace if the devcontainer ships with Docker-in-Docker and Java 21:

.devcontainer/devcontainer.json

{
  "image": "mcr.microsoft.com/devcontainers/java:1-21-bullseye",
  "features": {
    "ghcr.io/devcontainers/features/java:1": {
      "version": "none",
      "installMaven": "true"
    },
    "ghcr.io/devcontainers/features/docker-in-docker:2": {}
  },
  "forwardPorts": [8080, 5500],
  "customizations": {
    "vscode": {
      "extensions": [
        "redhat.java",
        "karatelabs.karate",
        "ms-vscode.live-server"
      ]
    }
  }
}

After ./mvnw verify -Pui, right-click any generated report HTML and Open with Live Server — port 5500 is forwarded, so the rendered report opens in a browser tab exactly as it would locally.

Jenkins

For Jenkins pipelines, archive the same artefacts and publish via the HTML Publisher plugin:

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh './mvnw -B verify -Pui'
      }
      post {
        always {
          publishHTML([
            reportDir: 'target/karate-reports',
            reportFiles: 'karate-summary.html',
            reportName: 'Karate Report',
            keepAll: true
          ])
          junit 'target/surefire-reports/*.xml'
        }
      }
    }
  }
}

Next steps

• Tags — organise tests for selective runs in each CI stage
• Test Reports — report formats and what they contain
• UI Testing — Testcontainers — the containerised-Chrome pattern used by the tests job
• Performance Testing — Gatling setup details