Skip to main content

Parallel Execution

Run tests in parallel to dramatically cut execution time. Karate's core parallel feature operates independently of build tools and provides comprehensive reporting for CI/CD integration.

Karate v2 — Virtual Threads

Karate v2 uses Java 21+ virtual threads for parallel execution, enabling massive parallelism with minimal overhead. You can run hundreds of scenarios in parallel without tuning thread pools.

Scenario Isolation Required

When running in parallel, scenarios execute in any order and must be completely isolated from each other.

Critical rules for parallel execution:

  • Each scenario must be runnable independently
  • Don't share state between scenarios via Background variables
  • Don't depend on execution order
  • You should be able to comment out any scenario without breaking others

Test your isolation: Run scenarios individually and ensure they all pass independently.

Why this matters: Parallel execution runs scenarios on different threads simultaneously. Shared state or execution order dependencies will cause random, difficult-to-debug failures.

Benefits of Parallel Execution​

  • 5-10x faster execution for large test suites
  • Zero configuration required - works out of the box
  • Multiple levels - Features and scenarios run in parallel
  • Comprehensive reporting - HTML, JUnit XML, Cucumber JSON

Basic Parallel Runner​

The simplest way to run tests in parallel using the Runner API:

Java
import io.karatelabs.core.Runner;
import io.karatelabs.core.SuiteResult;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;

class ParallelTest {

    @Test
    void testParallel() {
        SuiteResult results = Runner.builder()
                .path("classpath:features")
                .tags("~@ignore")
                .parallel(5);  // 5 threads
        assertFalse(results.isFailed());
    }
}

Key points:

  • parallel() must be the last method called
  • Returns SuiteResult with execution details
  • Thread count determines parallelism level

Runner.Builder Configuration​

Path Selection​

// Single package
Runner.builder().path("classpath:features")

// Multiple packages (varargs)
Runner.builder().path("classpath:api", "classpath:integration")

// Mix packages and individual features
Runner.builder().path("classpath:smoke", "classpath:critical/login.feature")

// Dynamic paths
List<String> paths = Arrays.asList("classpath:users", "classpath:products");
Runner.builder().path(paths)

Tag Filtering​

// Include specific tags
.tags("@smoke")                    // Only smoke tests
.tags("@api", "@smoke")            // AND operation - must have both
.tags("@smoke,@critical")          // OR operation - either tag

// Exclude tags  
.tags("~@slow") // Exclude slow tests
.tags("~@ignore", "~@manual") // Exclude multiple

// Complex combinations
.tags("(@smoke or @critical) and not @flaky")

Report Configuration​

SuiteResult results = Runner.builder()
  .path("classpath:features")
  .outputJunitXml(true)              // For CI/CD integration
  .outputCucumberJson(true)          // For dashboards
  .outputDir("target/karate-reports") // Custom directory
  .karateEnv("staging")              // Set environment
  .systemProperty("api.timeout", "30000")  // Pass properties
  .parallel(8);                      // Execute with 8 threads

Execution Statistics​

After running tests, Karate provides detailed execution statistics:

======================================================
elapsed:   2.35 | threads:    5 | thread time: 4.98
features:    54 | ignored:   25 | efficiency: 0.42
scenarios:  145 | passed:   145 | failed: 0
======================================================

Understanding the metrics:

  • elapsed: Total wall clock time
  • efficiency: How well threads were utilized (higher is better)
  • thread time: Total CPU time across all threads
  • features/scenarios: Execution and skip counts

Parallel Behavior​

Multi-Level Parallelism​

Karate runs tests in parallel at multiple levels:

  1. Features run in parallel across threads
  2. Scenarios within features run in parallel
  3. Scenario Outline examples run in parallel

Controlling Parallelism​

Use @parallel=false to force sequential execution when needed:

Gherkin
# Force entire feature to run sequentially
@parallel=false
Feature: Database setup
  # All scenarios run one after another

# Control individual scenarios

Feature: Mixed execution

Scenario: Runs in parallel \* print 'Parallel execution'

@parallel=false  
 Scenario: Runs sequentially \* print 'Sequential execution'
Anti-Pattern

Sequential execution usually indicates test dependencies. Design independent tests instead of using @parallel=false.

Mutual Exclusion with @lock (v2)​

When running in parallel, some scenarios may need exclusive access to shared resources (e.g., a database, a file, an external service). Use the @lock tag to ensure mutual exclusion:

@lock=database
Scenario: test that modifies shared database state
  # only one @lock=database scenario runs at a time
  # other scenarios (without this lock) continue in parallel

@lock=database
Scenario: another database test
  # waits for the first @lock=database scenario to finish

@lock=*
Scenario: exclusive test
  # this scenario runs completely alone — no other scenarios run concurrently
  • @lock=name — named lock, multiple scenarios with the same name are serialized
  • @lock=* — exclusive lock, the scenario runs alone with no other scenarios

Fail-Fast with abortSuiteOnFailure​

In CI/CD pipelines where fast feedback matters, use abortSuiteOnFailure to stop the suite on the first failure:

karate-config.js
function fn() {
  karate.configure('abortSuiteOnFailure', true);
  return {};
}

Or in a feature file:

* configure abortSuiteOnFailure = true

When enabled:

  • Queued features and scenarios that haven't started yet are skipped entirely
  • In-flight scenarios on other threads are interrupted at the next step boundary — they don't run to completion
  • The suite exits with only the failing scenario reported, giving you a fast exit code

This is especially useful for Kubernetes release gates, Argo Rollouts, and other CD workflows where a fast non-zero exit code is critical for rollback decisions.

Thread Count Optimization​

Environment-Based Threading​

Java
private int getOptimalThreads() {
    String env = System.getProperty("karate.env", "dev");
    int cores = Runtime.getRuntime().availableProcessors();
    
    switch (env) {
        case "dev": return Math.min(cores, 4);  // Don't overwhelm dev machine
        case "ci": return Math.min(cores, 6);  // CI resource constraints  
        case "perf": return cores * 2;         // Maximize for load testing
        default: return cores;
    }
}

Resource Considerations​

  • CPU cores: Good starting point is 1-2x CPU cores
  • Memory: ~256MB per thread for typical tests
  • Network: Consider external service limits (e.g., API rate limits)
  • CI environment: Usually 2-6 threads work best

Tuning Thread Count​

To optimize thread count:

  1. Start with availableProcessors() or 1-2x CPU cores.
  2. Monitor efficiency in execution statistics (aim for >0.7).
  3. Use tools like htop or VisualVM to check CPU/memory usage.
  4. Adjust based on test suite size and external service limits (e.g., reduce threads if hitting API rate limits).

Debugging Parallel Failures​

If tests fail in parallel:

  • Check results.getErrorMessages() for detailed error logs per scenario.
  • Review karate-timeline.html to identify slow or failed tests by thread.
  • Enable karate.log for verbose logging: .systemProperty("log.level", "DEBUG").
  • Isolate flaky tests by running with fewer threads or @parallel=false.

Designing Tests for Parallel Execution​

To maximize parallelism:

  • Independent Tests: Ensure scenarios don’t share state (e.g., avoid global variables or shared files).
  • Test Data: Use unique data per scenario (e.g., generate random IDs with karate.uuid()).
  • API Rate Limits: Implement retries with karate.retry() for external service constraints.
  • Feature Structure: Group related scenarios in features to optimize thread distribution.

Advanced Patterns​

Environment-Specific Execution​

Java
void environmentOptimized() {
    String env = System.getProperty("karate.env", "dev");

    SuiteResult results = Runner.builder()
            .path("classpath:features")
            .tags(getEnvironmentTags(env))
            .karateEnv(env)
            .parallel(getThreadCount(env));

    assertFalse(results.isFailed());
}

private String[] getEnvironmentTags(String env) {
    if ("prod".equals(env)) {
        return new String[] { "@smoke", "~@experimental" };
    }
    return new String[] { "~@ignore" };
}

Feature-Specific Threading​

Java
@Test
void optimizedByFeatureType() {
    // Fast API tests with high parallelism
    SuiteResult apiResults = Runner.builder()
            .path("classpath:api")
            .tags("@api", "@fast")
            .parallel(12);

    // Database tests with moderate parallelism
    SuiteResult dbResults = Runner.builder()
            .path("classpath:database")
            .tags("@database")
            .parallel(4);

    assertFalse(apiResults.isFailed());
    assertFalse(dbResults.isFailed());
}

Report Generation​

Timeline Visualization​

Karate automatically generates karate-timeline.html for visual analysis:

  • Thread utilization - How effectively threads are used
  • Performance bottlenecks - Identify slow tests
  • Load balancing - Distribution across threads

CI/CD Integration​

Java
@Test
void ciOptimized() {
    SuiteResult results = Runner.builder()
            .path("classpath:features")
            .tags("@regression", "~@slow")
            .outputJunitXml(true)          // Jenkins/CI integration
            .outputCucumberJson(true)      // Dashboard integration
            .outputDir("target/ci-reports")
            .parallel(4);                  // Conservative for CI

    assertFalse(results.isFailed());
}

CI Tool Example (GitHub Actions)​

Common Gotchas​

  • Thread count too high: Leads to resource exhaustion and poor efficiency
  • Test dependencies: Scenarios must be independent to run in parallel
  • Shared state: Avoid global variables or shared files between tests
  • CI constraints: Use fewer threads in CI environments than locally

Next Steps​

Maximize parallel execution effectiveness:

  • Tags - Organize tests for optimal parallel execution
  • Test Reports - Analyze parallel execution results
  • Configuration - Environment-specific settings
On This Page