Skip to main content

Parallel Execution

Run tests in parallel to dramatically cut execution time. Karate's core parallel feature operates independently of build tools and provides comprehensive reporting for CI/CD integration.

Scenario Isolation Required

When running in parallel, scenarios execute in any order and must be completely isolated from each other.

Critical rules for parallel execution:

  • Each scenario must be runnable independently
  • Don't share state between scenarios via Background variables
  • Don't depend on execution order
  • You should be able to comment out any scenario without breaking others

Test your isolation: Run scenarios individually and ensure they all pass independently.

Why this matters: Parallel execution runs scenarios on different threads simultaneously. Shared state or execution order dependencies will cause random, difficult-to-debug failures.

Benefits of Parallel Execution​

  • 5-10x faster execution for large test suites
  • Zero configuration required - works out of the box
  • Multiple levels - Features and scenarios run in parallel
  • Comprehensive reporting - HTML, JUnit XML, Cucumber JSON

Basic Parallel Runner​

The simplest way to run tests in parallel using the Runner API:

import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;

class ParallelTest {

  @Test
  void testParallel() {
      Results results = Runner.path("classpath:features")
              .tags("~@ignore")
              .parallel(5);  // 5 threads
      assertEquals(0, results.getFailCount(), results.getErrorMessages());
  }

}

Key points:

  • parallel() must be the last method called
  • Returns Results object with execution details
  • Thread count determines parallelism level

Runner.Builder Configuration​

Path Selection​

// Single package
Runner.path("classpath:features")

// Multiple packages  
Runner.path("classpath:api", "classpath:integration")

// Mix packages and individual features
Runner.path("classpath:smoke", "classpath:critical/login.feature")

// Dynamic paths
List<String> paths = Arrays.asList("classpath:users", "classpath:products");
Runner.path(paths)

Tag Filtering​

// Include specific tags
.tags("@smoke")                    // Only smoke tests
.tags("@api", "@smoke")            // AND operation - must have both
.tags("@smoke,@critical")          // OR operation - either tag

// Exclude tags  
.tags("~@slow") // Exclude slow tests
.tags("~@ignore", "~@manual") // Exclude multiple

// Complex combinations
.tags("(@smoke or @critical) and not @flaky")

Report Configuration​

Results results = Runner.path("classpath:features")
  .outputJunitXml(true)              // For CI/CD integration
  .outputCucumberJson(true)          // For dashboards  
  .reportDir("target/karate-reports") // Custom directory
  .karateEnv("staging")              // Set environment
  .systemProperty("api.timeout", "30000")  // Pass properties
  .parallel(8);                      // Execute with 8 threads

Execution Statistics​

After running tests, Karate provides detailed execution statistics:

======================================================
elapsed:   2.35 | threads:    5 | thread time: 4.98
features:    54 | ignored:   25 | efficiency: 0.42
scenarios:  145 | passed:   145 | failed: 0
======================================================

Understanding the metrics:

  • elapsed: Total wall clock time
  • efficiency: How well threads were utilized (higher is better)
  • thread time: Total CPU time across all threads
  • features/scenarios: Execution and skip counts

Parallel Behavior​

Multi-Level Parallelism​

Karate runs tests in parallel at multiple levels:

  1. Features run in parallel across threads
  2. Scenarios within features run in parallel
  3. Scenario Outline examples run in parallel

Controlling Parallelism​

Use @parallel=false to force sequential execution when needed:

# Force entire feature to run sequentially
@parallel=false
Feature: Database setup
# All scenarios run one after another

# Control individual scenarios

Feature: Mixed execution

Scenario: Runs in parallel * print 'Parallel execution'

@parallel=false  
Scenario: Runs sequentially * print 'Sequential execution'
Anti-Pattern

Sequential execution usually indicates test dependencies. Design independent tests instead of using @parallel=false.

Thread Count Optimization​

Environment-Based Threading​

private int getOptimalThreads() {
  String env = System.getProperty("karate.env", "dev");
  int cores = Runtime.getRuntime().availableProcessors();
  
  switch (env) {
      case "dev": return Math.min(cores, 4);     // Don't overwhelm dev machine
      case "ci": return Math.min(cores, 6);      // CI resource constraints  
      case "perf": return cores * 2;             // Maximize for load testing
      default: return cores;
  }
}

Resource Considerations​

  • CPU cores: Good starting point is 1-2x CPU cores
  • Memory: ~256MB per thread for typical tests
  • Network: Consider external service limits (e.g., API rate limits)
  • CI environment: Usually 2-6 threads work best

Tuning Thread Count​

To optimize thread count:

  1. Start with availableProcessors() or 1-2x CPU cores.
  2. Monitor efficiency in execution statistics (aim for >0.7).
  3. Use tools like htop or VisualVM to check CPU/memory usage.
  4. Adjust based on test suite size and external service limits (e.g., reduce threads if hitting API rate limits).

Debugging Parallel Failures​

If tests fail in parallel:

  • Check results.getErrorMessages() for detailed error logs per scenario.
  • Review karate-timeline.html to identify slow or failed tests by thread.
  • Enable karate.log for verbose logging: .systemProperty("log.level", "DEBUG").
  • Isolate flaky tests by running with fewer threads or @parallel=false.

Designing Tests for Parallel Execution​

To maximize parallelism:

  • Independent Tests: Ensure scenarios don’t share state (e.g., avoid global variables or shared files).
  • Test Data: Use unique data per scenario (e.g., generate random IDs with karate.uuid()).
  • API Rate Limits: Implement retries with karate.retry() for external service constraints.
  • Feature Structure: Group related scenarios in features to optimize thread distribution.

Advanced Patterns​

Environment-Specific Execution​

@Test
void environmentOptimized() {
  String env = System.getProperty("karate.env", "dev");
  
  Results results = Runner.path("classpath:features")
          .tags(getEnvironmentTags(env))
          .karateEnv(env) 
          .parallel(getThreadCount(env));
          
  assertEquals(0, results.getFailCount());
}

private String[] getEnvironmentTags(String env) {
if ("prod".equals(env)) {
return new String[]{"@smoke", "~@experimental"};
}
return new String[]{"~@ignore"};
}

Feature-Specific Threading​

@Test  
void optimizedByFeatureType() {
  // Fast API tests with high parallelism
  Results apiResults = Runner.path("classpath:api")
          .tags("@api", "@fast")  
          .parallel(12);
  
  // Database tests with moderate parallelism
  Results dbResults = Runner.path("classpath:database")
          .tags("@database")
          .parallel(4);
  
  assertEquals(0, apiResults.getFailCount());
  assertEquals(0, dbResults.getFailCount());
}

Report Generation​

Timeline Visualization​

Karate automatically generates karate-timeline.html for visual analysis:

  • Thread utilization - How effectively threads are used
  • Performance bottlenecks - Identify slow tests
  • Load balancing - Distribution across threads

CI/CD Integration​

@Test
void ciOptimized() {
  Results results = Runner.path("classpath:features")
          .tags("@regression", "~@slow")
          .outputJunitXml(true)          // Jenkins/CI integration
          .outputCucumberJson(true)      // Dashboard integration  
          .reportDir("target/ci-reports")
          .parallel(4);                  // Conservative for CI
  
  assertEquals(0, results.getFailCount(), results.getErrorMessages());
}

CI Tool Example (GitHub Actions)​

Common Gotchas​

  • Thread count too high: Leads to resource exhaustion and poor efficiency
  • Test dependencies: Scenarios must be independent to run in parallel
  • Shared state: Avoid global variables or shared files between tests
  • CI constraints: Use fewer threads in CI environments than locally

Next Steps​

Maximize parallel execution effectiveness:

  • Tags - Organize tests for optimal parallel execution
  • Test Reports - Analyze parallel execution results
  • Configuration - Environment-specific settings
On This Page