Skip to main content

RUNNING TESTS

Command Line Execution

Overview​

Execute Karate tests from the command line for development workflows and CI/CD integration. Use Maven or Gradle with flexible options for test selection, tagging, and suite configuration.

For VS Code Plugin or standalone JAR users, refer to the CLI usage guide.

Maven Command Line​

Basic Test Execution​

In development mode, run *.feature files directly via IDE or companion JUnit runner classes. The mvn test command runs test classes following *Test.java naming conventions by default.

# Run all tests
mvn test

# Run specific test class
mvn test -Dtest=CatsRunner

karate.options Configuration​

When using the parallel runner, narrow scope to specific features, scenarios, or directories via command-line options:

# Run specific feature with tag exclusion
mvn test "-Dkarate.options=--tags ~@skipme classpath:demo/cats/cats.feature" -Dtest=DemoTestParallel

# Run specific scenario by line number
mvn test "-Dkarate.options=PathToFeatureFiles/order.feature:12" -Dtest=DemoTestParallel

# Override tags only (use = for clarity)
mvn test -Dkarate.options='-t=@dev -t=@src' -Dtest=ExamplesTest

karate.options Syntax​

  • Multiple paths: Space-delimited, specified at end of options
  • Single scenario: Append line number with : delimiter
  • Tag filtering: Use --tags or -t with ~ for exclusion
  • Built-in tags: @ignore scenarios skipped automatically

Gradle Command Line​

Build Configuration​

Extend the test task to pass karate.options to runtime (otherwise Gradle consumes them):

test {
    // pull karate options into the runtime
    systemProperty "karate.options", System.properties.getProperty("karate.options")
    // pull karate env into the runtime
    systemProperty "karate.env", System.properties.getProperty("karate.env")
    // ensure tests are always run
    outputs.upToDateWhen { false }
}

Test Execution​

# Run specific test class
./gradlew test --tests *CatsRunner

# Alternative syntax
./gradlew test -Dtest.single=CatsRunner

Test Suites​

Recommended: Use the parallel runner for test suites and reporting. This section covers IDE-friendly troubleshooting approaches.

Suite Organization​

Define test suites using JUnit classes positioned above feature files in folder hierarchy:

# Run test suite with tag exclusion
mvn test "-Dkarate.options=--tags ~@skipme" -Dtest=AnimalsTest

This approach uses tag-based filtering where AnimalsTest runs multiple *.feature files. The special @ignore tag is always skipped automatically.

Maven Surefire Configuration​

Lock down test execution to specific suite classes:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${maven.surefire.version}</version>
    <configuration>
        <includes>
            <include>animals/AnimalsTest.java</include>
        </includes>
        <systemProperties>
            <karate.options>--tags @smoke</karate.options>
        </systemProperties>
    </configuration>
</plugin>

Gradle Suite Configuration​

test {
    include 'animals/AnimalsTest.java'
    // pull karate options into the runtime
    systemProperty "karate.options", System.properties.getProperty("karate.options")
    // pull karate env into the runtime
    systemProperty "karate.env", System.properties.getProperty("karate.env")
    // ensure tests are always run
    outputs.upToDateWhen { false }
}

Limitations and Recommendations​

Test Suite Drawbacks​

  • No parallel execution: Traditional suite approach cannot run tests in parallel
  • Limited reporting: Reduced CI/CD integration options
  • Configuration bloat: Additional setup in pom.xml or build.gradle

Preferred Approach​

Use the parallel runner for:

  • JUnit XML format generation for CI tools (**/*.xml or **/karate-reports/*.xml)
  • Cucumber JSON format support for third-party reporting plugins
  • Parallel execution without external dependencies
  • Clean configuration without build tool bloat

CI/CD Integration​

The parallel runner approach provides superior CI/CD integration with standard report formats that most build systems can consume automatically.

Next Steps​

On This Page