Skip to main content

RUNNING TESTS

Test Reports

Generate detailed test reports in multiple formats for debugging, CI/CD integration, and performance analysis. Karate provides comprehensive reporting out of the box.

Report Types​

HTML Reports (Default)​

Rich, interactive reports for development and debugging:

  • Request/response details - Complete HTTP interaction data
  • Step-by-step execution - Visual test flow with timing
  • Inline debugging * print statements and variable values
  • Screenshots - Automatic UI test captures
  • Timeline visualization - Thread utilization analysis

JUnit XML Reports​

Standard format for CI/CD tool integration:

Results results = Runner.path("classpath:features")
      .outputJunitXml(true)
      .parallel(5);

Cucumber JSON Reports​

Structured data for custom dashboards and third-party tools:

Results results = Runner.path("classpath:features")
      .outputCucumberJson(true)
      .parallel(5);

Report Location​

Default Directory Structure​

target/karate-reports/
├── karate-summary.html       # Main overview
├── karate-timeline.html      # Performance analysis
├── com.example.UserTest.html # Individual feature reports
└── *.json, *.xml            # JUnit/Cucumber outputs

Custom Report Directory​

Results results = Runner.path("classpath:features")
      .reportDir("build/custom-reports")
      .outputJunitXml(true)
      .outputCucumberJson(true)
      .parallel(5);

HTML Report Features​

Console Output Access​

After test execution, the console shows direct report links:

html report: (paste into browser to view)
-----------------------------------------
file:///project/target/karate-reports/karate-summary.html

Report Contents​

HTML reports include:

  • Execution summary - Pass/fail counts and timing
  • Request details * headers, method, URL, body
  • Response data - Status, headers, complete response
  • Debug information * print statements and variables
  • Performance metrics - Response times and efficiency

Timeline Analysis​

Understanding Timeline Reports​

The karate-timeline.html provides visual thread utilization:

Thread 1: |████████████████████| Feature A (2.1s)
Thread 2: |██████████████████  | Feature B (1.8s)
Thread 3: |███████████████████ | Feature C (1.9s)

Efficiency: 85% (good utilization)

Key metrics:

  • Thread utilization - How well threads are used
  • Feature distribution - Load balancing across threads
  • Performance bottlenecks - Slow tests that block others

CI/CD Integration​

Jenkins Configuration​

Configure Jenkins to publish Karate reports:

Jenkinsfile
pipeline {
    stages {
        stage('Test') {
            steps {
                sh 'mvn test'
            }
            post {
                always {
                    publishHTML([
                        reportDir: 'target/karate-reports',
                        reportFiles: 'karate-summary.html',
                        reportName: 'Karate Reports'
                    ])
                    publishTestResults(
                        testResultsPattern: 'target/surefire-reports/*.xml'
                    )
                }
            }
        }
    }
}

GitHub Actions Configuration​

.github/workflows/test.yml
- name: Run tests
  run: mvn test

- name: Publish test results
  uses: dorny/test-reporter@v1
  if: always()
  with:
    name: Karate Test Results
    path: target/surefire-reports/*.xml
    reporter: java-junit

- name: Upload HTML reports
  uses: actions/upload-artifact@v3
  if: always()
  with:
    name: karate-reports
    path: target/karate-reports/

Report Configuration​

Enabling Multiple Formats​

@Test
void comprehensiveReporting() {
  Results results = Runner.path("classpath:features")
          .outputJunitXml(true)          // For CI/CD
          .outputCucumberJson(true)      // For dashboards
          .outputHtmlReport(true)        // For debugging (default)
          .reportDir("target/ci-reports")
          .parallel(8);
          
  assertEquals(0, results.getFailCount(), results.getErrorMessages());
}

Report Verbosity Control​

Control what appears in reports:

Feature: Report configuration

Background: * configure report = { showLog: true, showAllSteps: false }

Scenario: Verbose reporting
* configure report = { showLog: true, showAllSteps: true }
* print 'This will appear in reports'

Scenario: Minimal reporting  
* configure report = { showLog: false }
* print 'This will not appear in reports'

Third-Party Integration​

Cucumber Reporting Plugin​

Generate enhanced reports using Maven plugin:

pom.xml
<plugin>
    <groupId>net.masterthought</groupId>
    <artifactId>maven-cucumber-reporting</artifactId>
    <version>5.7.2</version>
    <executions>
        <execution>
            <phase>verify</phase>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <projectName>API Test Suite</projectName>
                <outputDirectory>target/cucumber-reports</outputDirectory>
                <inputDirectory>target/karate-reports</inputDirectory>
                <jsonFiles>
                    <param>**/*.json</param>
                </jsonFiles>
            </configuration>
        </execution>
    </executions>
</plugin>

Common Gotchas​

  • Missing reports: Check if tests actually executed
  • Large report files: Reduce logging verbosity with configure report
  • Broken report links: Use relative paths and proper directory structure
  • CI report access: Ensure proper artifact archiving in CI/CD pipelines

Next Steps​

Master test reporting for effective debugging and monitoring:

On This Page