Skip to main content

RUNNING TESTS

Parallel Execution

Overview​

Karate can run tests in parallel and dramatically cut down execution time. This is a core feature that doesn't depend on JUnit, Maven, or Gradle.

For non-Java projects using command-line, set thread count via --threads or -T as explained in the CLI documentation.

Key Features​

  • Flexible test selection: Choose features and tags to compose test suites
  • Error handling: Use returned Results object to check failures and summarize errors
  • Multiple report formats: Generate JUnit XML, Cucumber JSON, and HTML reports
  • Dynamic configuration: Programmatically determine tags and features to include
  • Thread management: Specify optimal thread count for your environment

Basic JUnit 5 Implementation​

import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;

class TestParallel {

    @Test
    void testParallel() {
        Results results = Runner.path("classpath:animals")
                .tags("~@skipme")
                .parallel(5);
        assertEquals(0, results.getFailCount(), results.getErrorMessages());
    }
}

The Runner and Results API can be used directly in any Java class, including a main method. Use the karate-template project for a working example.

Runner.Builder API​

Path Configuration​

// Single package
Runner.path("classpath:animals")

// Multiple packages and individual features
Runner.path("classpath:animals", "classpath:some/other/package.feature")

// Dynamic path configuration
List<String> paths = Arrays.asList("classpath:smoke", "classpath:regression");
Runner.path(paths)

Tag Filtering​

// AND operation (both tags required)
.tags("@customer", "@smoke")

// OR operation (either tag accepted)
.tags("@customer,@smoke")

// Exclusion (skip tagged scenarios)
.tags("~@skipme")

// Dynamic tag configuration
List<String> tags = Arrays.asList("@smoke", "~@slow");
.tags(tags)

The special built-in tag @ignore is always skipped automatically.

Report Configuration​

Results results = Runner.path("classpath:tests")
    .outputJunitXml(true)           // Generate JUnit XML reports
    .outputCucumberJson(true)       // Generate Cucumber JSON reports
    .outputHtmlReport(false)        // Disable HTML reports
    .reportDir("custom/reports")    // Custom report directory (default: target/karate-reports)
    .parallel(5);                   // Must be called last

Method Call Order​

parallel() must be the last method called. It returns a Results object containing all execution information including passed/failed test counts.

Execution Statistics​

Parallel execution provides detailed console statistics:

======================================================
elapsed:   2.35 | threads:    5 | thread time: 4.98
features:    54 | ignored:   25 | efficiency: 0.42
scenarios:  145 | passed:   145 | failed: 0
======================================================

Timeline Visualization​

A karate-timeline.html file is generated in the report directory for visual verification and troubleshooting of test run effectiveness.

Parallel Execution Behavior​

Default Parallelism​

  • Features: Always run in parallel
  • Scenarios: Run in parallel by default within each feature
  • Scenario Outline Examples: Each row executes in parallel

Controlling Scenario Parallelism​

Use the special tag @parallel=false to suppress default parallel execution:

# Apply to entire feature
@parallel=false
Feature: Sequential execution required

# Apply to specific scenarios
Scenario: Must run sequentially
  @parallel=false
  Given some setup that conflicts with parallel execution

Anti-pattern Warning: Forcing scenarios to run in a particular sequence should be avoided as it indicates test dependencies.

Report Generation​

JUnit XML Reports​

Generate reports for CI tool consumption:

.outputJunitXml(true)

Configure CI to look for reports in:

  • **/*.xml
  • **/karate-reports/*.xml

Cucumber JSON Reports​

Generate JSON reports for third-party reporting tools:

.outputCucumberJson(true)

Reports use .json extension and are compatible with:

CI/CD Integration​

Most CI tools can process JUnit XML output to:

  • Determine build status
  • Generate test reports
  • Track test trends
  • Integrate with notification systems

The Karate Demo provides working examples of parallel runner setup and third-party report generation.

Performance Optimization​

Thread Count Selection​

// Conservative (CPU cores)
.parallel(Runtime.getRuntime().availableProcessors())

// Aggressive (2x CPU cores)
.parallel(Runtime.getRuntime().availableProcessors() * 2)

// Fixed thread count
.parallel(8)

Efficiency Monitoring​

Monitor the "efficiency" metric in console output:

  • High efficiency (> 0.8): Good thread utilization
  • Low efficiency (< 0.4): Consider reducing thread count or optimizing tests

Advanced Configuration​

Environment-Specific Execution​

Results results = Runner.path("classpath:tests")
    .karateEnv("qa")               // Set environment
    .systemProperty("apiKey", key) // Pass system properties
    .parallel(threads);

Results Analysis​

Results results = Runner.path("classpath:tests").parallel(5);

// Check execution status
if (results.getFailCount() > 0) {
    System.out.println("Failures: " + results.getErrorMessages());
    System.exit(1);
}

// Access detailed metrics
System.out.println("Features run: " + results.getFeaturesTotal());
System.out.println("Scenarios passed: " + results.getScenariosTotal() - results.getFailCount());

Next Steps​

On This Page