RUNNING TESTS

Command Line

Karate v2 has a first-class CLI — karate run — alongside full Maven and Gradle integration via JUnit 6. Pick the entry point that fits your project; all three share the same underlying Runner API.

The familiar -Dkarate.options="..." and -Dkarate.env=... system-property overrides from v1 are supported in v2, so Maven/Gradle teams upgrading from v1 keep their "hard-code defaults in a Runner class, override from CI" workflow without code changes.

The karate run CLI

The simplest way to run tests — no Maven, no Gradle, no Java class needed. See Standalone Execution for installation.

Command Line

# Run all features in a directory
karate run features/

# Run a single feature file
karate run features/users.feature

# Run a specific scenario by line number
karate run features/users.feature:25

# Run multiple paths (repeatable -P, or comma-separated --path a,b)
karate run -P features/smoke -P features/regression

# Filter by tag, 4 threads, QA environment
karate run -t @smoke -T 4 -e qa features/

Common Flags

Flag	Description
-P, --path <path>	Feature file or directory — repeatable and/or comma-separated
-t, --tags <expr>	Tag expression (e.g. @smoke, ~@slow) — repeatable
-T, --threads <n>	Parallel thread count
-e, --env <name>	Karate environment (karate.env)
-n, --name <name>	Scenario name filter (exact, whitespace-trimmed); bypasses tag filters
-o, --output <dir>	Report output directory
-g, --configdir <dir>	Directory containing karate-config.js
-f, --format <formats>	Output formats: html, cucumber:json, junit:xml, karate:jsonl (prefix ~ to disable)
-D, --dryrun	Parse and generate a report without executing steps — see Dry Run
--no-color	Disable ANSI colour in console output — useful for log files and CI. Works with karate run, karate mock, and the bare karate <path> form.

Run karate run --help for the full list.

note

--no-color is auto-applied in non-interactive environments. When Karate detects no TTY (CI, redirected output, Gatling) or the NO_COLOR environment variable, colour is disabled automatically — so log files don't accumulate escape codes. --no-color forces it off explicitly; FORCE_COLOR=1 forces it on.

Project File: karate-pom.json

For CLI users, karate-pom.json is a JSON project file that persists your run settings — analogous to pom.xml but for Karate. When present in the working directory, karate run loads it automatically.

karate-pom.json

{
  "paths": ["classpath:com/comp/commission", "classpath:com/comp/statement"],
  "tags": ["@smoke", "~@slow"],
  "env": "qa",
  "threads": 5,
  "output": {
    "dir": "target/karate-reports",
    "cucumberJson": true
  }
}

CLI flags override pom values. Use --no-pom to ignore it.

Maven Execution

Java teams use the karate-junit6 dependency and drive runs through a JUnit 6 test class. Standard mvn test discovers and runs them.

Command Line

# Run all JUnit tests (including Karate runners)
mvn test

# Run a specific Karate runner class
mvn test -Dtest=UsersTest

# Run a specific test method
mvn test -Dtest=UsersTest#smokeTests

# Override the environment (karate.env)
mvn test -Dkarate.env=qa

# Override tags from CI without changing the Runner class
mvn test -Dtest=UsersTest -Dkarate.options="--tags @smoke"

# Multiple overrides in one string (v1-compatible)
mvn test -Dtest=UsersTest -Dkarate.options="--tags @smoke --env qa --threads 4"

JUnit 6 Runner Class

UsersTest.java

import io.karatelabs.junit6.Karate;
import org.junit.jupiter.api.DynamicNode;

class UsersTest {

    @Karate.Test
    Iterable<DynamicNode> testUsers() {
        return Karate.run("classpath:users").relativeTo(getClass());
    }

    @Karate.Test
    Iterable<DynamicNode> testSmoke() {
        return Karate.run("classpath:users")
                .tags("@smoke")
                .threads(4)
                .karateEnv("qa");
    }

    @Karate.Test
    Iterable<DynamicNode> testMultiplePaths() {
        return Karate.run("classpath:com/comp/commission", "classpath:com/comp/statement")
                .tags("@smoke");
    }
}

Karate.run(...) accepts varargs — use this when you want multiple paths baked into the Runner class. For ad-hoc overrides from CI without changing the class, use -Dkarate.options=... instead.

System-Property Overrides (CI pattern)

You do not need any wiring in your Runner class. Karate picks up these system properties automatically at parallel() time and overrides Builder values before the run:

Sysprop	Env var fallback	Effect
karate.options	KARATE_OPTIONS	Full option string using the karate run CLI grammar — paths, -P, tags, threads, env, -n, configdir, output, dryrun, formats
karate.env	KARATE_ENV	Karate environment
karate.config.dir	KARATE_CONFIG_DIR	Directory containing karate-config.js
karate.output.dir	—	Report output directory

Example — typical CI invocations against a runner that hard-codes defaults:

Terminal

# Override one dimension
mvn test -Dtest=UsersTest -Dkarate.env=qa

# Override multiple in one karate.options string
mvn test -Dtest=UsersTest -Dkarate.options="--tags @smoke --threads 4 --env qa"

# Narrow to specific features for a rerun
mvn test -Dtest=UsersTest -Dkarate.options="classpath:features/login.feature:42"

Precedence (highest → lowest): karate.options sysprop > KARATE_OPTIONS env > individual karate.* sysprops > individual KARATE_* env vars > Runner.Builder values > karate-pom.json > defaults. If a malformed karate.options string is passed, it's logged at WARN and ignored — the run proceeds with Builder defaults.

Paths with spaces

The karate.options string is tokenized POSIX-shell-style, so inner quotes survive and whitespace is respected:

Terminal

# Multiple paths that contain spaces — use -P with inner double quotes,
# and protect the whole sysprop value with outer single quotes
mvn test -Dtest=UsersTest '-Dkarate.options=-P "src/features/happy path" -P "src/features/edge cases"'

Single vs double quotes follow shell rules: single quotes around the -D... value stop the outer shell from interpreting the inner doubles; inside the sysprop value, double quotes group the path tokens with whitespace.

Gradle Execution

Gradle integration uses the same JUnit 6 runner class. Forward karate.* system properties to the test JVM so the overrides above work from the command line:

build.gradle

test {
    useJUnitPlatform()
    systemProperties System.properties.findAll { k, v ->
        k.toString().startsWith('karate.')
    }
    outputs.upToDateWhen { false }
}

Command Line

# Run all tests
./gradlew test

# Run a specific Karate runner class
./gradlew test --tests UsersTest

# Override environment
./gradlew test -Dkarate.env=staging

# Override tags and env from CI (no code change)
./gradlew test -Dkarate.options="--tags @smoke --env qa"

Filtering & Configuration

By Tag

Tag expressions work identically across CLI, Maven, and Gradle. ~ means "not". Comma means OR; repeated --tags means AND.

Command Line

# Run scenarios tagged @smoke
karate run -t @smoke features/

# Exclude @slow
karate run -t ~@slow features/

# Either @smoke OR @regression
karate run -t @smoke,@regression features/

# @smoke AND @api
karate run -t @smoke -t @api features/

See Tags for expression syntax.

By Scenario Line or Name

Terminal

# By line number — bypasses tag filters (great for debugging)
karate run features/users.feature:15

# By scenario name — exact match, trimmed on both sides.
# Stable under edits, so IDE plugins use this as a line-independent key.
karate run -n "Create user happy path" features/

# Combine with :LINE to narrow to one row of a Scenario Outline.
karate run -n "Parameterized check" features/outline.feature:9

Both forms bypass tag filters (@ignore, @env) — running a specific scenario always wins. Duplicate scenario names run every match. For a Scenario Outline, the name matches the outline itself and runs all rows.

By Environment

Set karate.env and read it in karate-config.js:

Command Line

# CLI
karate run -e dev features/

# Maven
mvn test -Dtest=UsersTest -Dkarate.env=qa

# Gradle
./gradlew test -Dkarate.env=staging

karate-config.js

function fn() {
  var env = karate.env;
  var config = { baseUrl: 'https://api-dev.example.com' };
  if (env === 'qa')  config.baseUrl = 'https://api-qa.example.com';
  if (env === 'prod') config.baseUrl = 'https://api.example.com';
  return config;
}

Development Patterns

Tight Iteration Loop

Command Line

# Run one failing scenario by line
karate run features/problem.feature:15

# Run WIP tests locally
karate run -t @wip -e local features/

# Dry run — parse only, no HTTP
karate run -D features/

Dry Run

-D / --dryrun produces a full report without executing any steps — useful for validating feature-file parsing, reviewing which scenarios a tag filter selects, and confirming that dynamic scenario outlines expand correctly, all without making HTTP calls or other side effects.

Under dry run:

• Every step on a normal scenario is recorded as skipped with 0ms duration — no HTTP, no match, no def, no side effects. Steps are deliberately not marked passed: a dry run proves nothing ran, so it must never read as a green. The console summary says DRY RUN — nothing was executed (note that the skipped scenarios still count toward the passed tally, since nothing failed).
• karate-base.js, karate-config.js, and env-specific config JS are not evaluated.
• beforeScenario / afterScenario hooks are skipped.
• @setup scenarios still execute fully, so dynamic outlines (Examples: | karate.setup().data |) still resolve their rows in the report.
• All configured report formats (HTML, JUnit XML, Cucumber JSON, JSONL) are generated normally.

Inside an @setup scenario, read karate.suite.dryRun to short-circuit expensive fixture creation:

Gherkin

@setup
Scenario:
  * def rows = karate.suite.dryRun ? [{ name: 'placeholder' }] : fetchFromDb()

Bash

karate run -D features/

Coverage without running the suite

A dry run still reads every scenario's tags, and it still writes every report. So on a project with the Karate Agent coverage extension enabled, -D answers "does a test exist that claims this requirement or API operation?" with nothing executed — no environment, no system under test, no mocks:

Bash

karate run -D -o target/rtm-dryrun features/

Read the result as an inventory rather than a verdict. An item a test declares (via @req= or @cov=) but that nobody has run reads NOTRUN; an item nothing in the suite claims reads NOTCOVERED — the real authoring gap. Nothing can read COVERED from a dry run, so the headline coverage percentage is 0 by design. Note that API-operation coverage is normally matched from observed traffic, so under a dry run an operation only counts when a scenario carries the explicit @cov=openapi:<operationId> tag.

Two things to watch: combining -D with -t drops the non-matching scenarios from the run entirely, so their declared tags never land (a complete declared matrix needs no tag filter), and a run replaces the report directory it writes — pass -o so a dry run doesn't displace the graph from your last real run.

CI: Smoke then Full

CI pipeline

# Fast smoke stage
karate run -t @smoke -T 4 -e ci features/

# Full suite
karate run -T 8 -e ci --format cucumber:json,junit:xml features/

Migrating from v1

Your v1 -Dkarate.options="..." invocations keep working in v2. The option string uses the same shape, and v2 adds a few new flags you can use inside it (notably -P, --path):

v1	v2 equivalent	Notes
-Dkarate.options="classpath:features/users.feature"	Unchanged	Positional paths
-Dkarate.options="features/a features/b"	Unchanged, or -Dkarate.options="-P features/a -P features/b"	-P is new in v2, handy when paths are awkward as positionals
-Dkarate.options="--tags @smoke"	Unchanged	Tag filter
-Dkarate.options="--dryrun"	Unchanged	Dry run
-Dkarate.options="features/x.feature:25"	Unchanged	Line number targeting
-Dkarate.env=qa	Unchanged	Individual env sysprop

If you're building fresh instead of migrating, prefer the karate run CLI — one command, no JUnit runner class needed.

Common Gotchas

• Tag AND vs OR: comma is OR within one -t, repeated -t is AND.
• Path with line number bypasses tag filters — intentional, useful for IDE integrations.
• Gradle must forward system properties: see the build.gradle snippet above.
• workingDir vs output.dir: output is relative to process cwd, not workingDir. Prefer absolute paths for reports in CI.
• karate.options paths replace Builder paths: when the sysprop contains paths, they fully replace any paths set in your Runner class (v1 parity). Use this intentionally for CI subsetting.

Next Steps

• Standalone Execution — install the karate CLI
• Parallel Execution — scale across threads
• Tags — expression syntax reference
• Test Reports — HTML, Cucumber JSON, JUnit XML, JSONL