EXTENSIONS

Image Comparison

Compare screenshots pixel-by-pixel for visual regression testing using flexible engines and configurable tolerance levels - no external tools needed.

Benefits of Image Comparison

Visual regression testing: Detect unintended UI changes automatically
Multiple comparison engines: Choose between Resemble and SSIM algorithms
Flexible tolerance: Configure acceptable difference thresholds
Detailed diff reports: Visual comparison UI embedded in HTML reports
CI/CD ready: Integrate visual testing into your pipeline

Basic Image Comparison

Compare two images to verify they match exactly or within tolerance.

Feature: Basic image comparison

Scenario: Compare screenshots
* driver 'https://example.com/login'
* screenshot()

# Compare against baseline
* def baseline = read('classpath:screenshots/login.png')
* def current = screenshot(false)
* compareImage { baseline: '#(baseline)', latest: '#(current)' }

Image Sources

Images can be loaded using path prefixes (classpath:, file:, this:), as byte arrays from screenshot(false), or using karate.readAsBytes() for maximum flexibility.

Image Comparison with Tolerance

Allow minor pixel differences using failure thresholds for real-world testing.

Feature: Tolerance-based comparison

Scenario: Compare with 2% tolerance
# Configure 2% pixel difference threshold
* configure imageComparison = { failureThreshold: 2 }

* driver 'https://example.com/dashboard'
* def actual = screenshot(false)
* def expected = read('classpath:screenshots/dashboard.png')

# Passes if less than 2% of pixels differ
* compareImage { baseline: '#(expected)', latest: '#(actual)' }

Failure Threshold

The failureThreshold is a percentage (0-100). A value of 2 means the comparison passes if less than 2% of pixels differ from the baseline.

Comparison Engines

Resemble Engine

The default engine uses pixel-by-pixel comparison with anti-aliasing support.

Feature: Resemble engine

Scenario: Use Resemble for pixel comparison
* configure imageComparison = { engine: 'resemble' }

* driver 'https://example.com'
* def screenshot1 = screenshot(false)
* def screenshot2 = read('baseline.png')

* compareImage { baseline: '#(screenshot2)', latest: '#(screenshot1)' }

SSIM Engine

Use Structural Similarity (SSIM) algorithm for perceptual comparison.

Feature: SSIM engine

Scenario: Use SSIM for structural comparison
* configure imageComparison = { engine: 'ssim' }

* driver 'https://example.com'
* def actual = screenshot(false)
* def baseline = read('screenshots/baseline.png')

* compareImage { baseline: '#(baseline)', latest: '#(actual)' }

Choosing an Engine

Feature: Multiple engines

Scenario: Use both engines, evaluate lowest mismatch
# Try both, use lowest mismatch percentage
* configure imageComparison = { engine: 'resemble,ssim' }
* compareImage { baseline: 'base.png', latest: 'current.png' }

Scenario: Fallback engine
# Use Resemble first, fallback to SSIM if fails
* configure imageComparison = { engine: 'resemble|ssim' }
* compareImage { baseline: 'base.png', latest: 'current.png' }

Configuration Options

Failure Threshold

Set acceptable pixel difference percentage before comparison fails.

Scenario: Configure failure threshold
# Allow up to 5% pixel difference
* configure imageComparison = { failureThreshold: 5 }

* compareImage { baseline: 'screenshots/page.png', latest: 'current.png' }

Scaling Options

Automatically scale images to match dimensions when they differ.

Scenario: Allow image scaling
# Scale latest image to match baseline dimensions
* configure imageComparison = { allowScaling: true }

* def baseline = read('screenshots/desktop.png')
* def mobile = read('screenshots/mobile.png')

* compareImage { baseline: '#(baseline)', latest: '#(mobile)' }

Hide UI on Success

Reduce report size by hiding comparison UI for successful matches.

Scenario: Hide successful comparisons
# Only show diff UI when images differ
* configure imageComparison = { hideUiOnSuccess: true }

* def screenshots = ['page1.png', 'page2.png', 'page3.png']
* karate.forEach(screenshots, function(img) { compareImage { baseline: 'baseline/' + img, latest: 'current/' + img } })

Ignored Regions

Exclude dynamic areas like timestamps, ads, or loading animations from comparison.

Feature: Ignore dynamic regions

Scenario: Exclude header timestamp area
# Define regions to ignore (top, left, bottom, right)
* def ignoredAreas =
  """
  [
    { top: 10, left: 900, bottom: 40, right: 1200 },
    { top: 500, left: 50, bottom: 550, right: 250 }
  ]
  """

* driver 'https://example.com/dashboard'
* def actual = screenshot(false)
* def baseline = read('screenshots/dashboard-baseline.png')

* compareImage {
    baseline: '#(baseline)',
    latest: '#(actual)',
    options: { ignoredBoxes: '#(ignoredAreas)' }
  }

Advanced Configuration

Resemble Engine Options

Fine-tune Resemble engine behavior for specific visual testing needs.

Feature: Advanced Resemble configuration

Scenario: Ignore colors, check structure only
* configure imageComparison = {
    engine: 'resemble',
    failureThreshold: 1
  }

* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: {
      ignoreColors: true,
      ignoreAntialiasing: true
    }
  }

Scenario: Ignore specific color
* def pinkColor = { r: 255, g: 192, b: 203, a: 255 }

* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: {
      ignoreAreasColoredWith: '#(pinkColor)'
    }
  }

Resemble Ignore Presets

Use built-in presets to control tolerance for common visual differences.

Feature: Resemble ignore presets

Scenario: Zero tolerance comparison
# Strictest comparison - no tolerance for any differences
* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: { ignore: 'nothing' }
  }

Scenario: Ignore anti-aliasing only
# Tolerate anti-aliasing differences (default behavior)
* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: { ignore: 'antialiasing' }
  }

Scenario: Ignore color differences
# Compare structure only, ignore colors
* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: { ignore: 'colors' }
  }

Scenario: Ignore alpha channel
# Ignore transparency differences
* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: { ignore: 'alpha' }
  }

Custom Color and Brightness Tolerances

Define precise tolerance levels for color channels and brightness.

Feature: Custom tolerances

Scenario: Fine-tune color tolerance
* def customTolerances =
  """
  {
    red: 4,
    green: 4,
    blue: 4,
    alpha: 4,
    minBrightness: 4,
    maxBrightness: 250
  }
  """

* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: { tolerances: '#(customTolerances)' }
  }

SSIM Engine Options

Configure SSIM algorithm parameters for structural similarity testing.

Feature: Advanced SSIM configuration

Scenario: High-precision SSIM comparison
* configure imageComparison = { engine: 'ssim' }

* compareImage {
    baseline: 'screenshots/base.png',
    latest: 'screenshots/current.png',
    options: {
      ssim: 'WEBER',
      rgb2grayVersion: 'INTEGER',
      k1: 0.01,
      k2: 0.03,
      windowSize: 11
    }
  }

Scenario: Fast SSIM with original grayscale
# Use FAST algorithm with ORIGINAL grayscale conversion
* compareImage {
    baseline: 'base.png',
    latest: 'current.png',
    options: {
      ssim: 'FAST',
      rgb2grayVersion: 'ORIGINAL'
    }
  }

Custom Rebase Callback

Customize the rebase functionality in HTML reports.

Scenario: Custom rebase handler
* text onRebaseFn =
  """
  function(config, downloadLatestFn) {
    // Custom filename for rebase
    var timestamp = new Date().toISOString().replace(/:/g, '-');
    downloadLatestFn('baseline-' + timestamp + '.png');
    return 'Baseline updated with timestamp: ' + timestamp;
  }
  """

* configure imageComparison = { onShowRebase: '#(onRebaseFn)' }

* compareImage { baseline: 'base.png', latest: 'current.png' }

Custom Configuration Display

Customize how configuration is displayed in HTML reports.

Scenario: Custom config display
* text onShowConfigFn =
  """
  function(customConfigJson, config) {
    // Add custom message above config
    return 'Image Comparison Settings
' +
           '=========================
' +
           customConfigJson;
  }
  """

* configure imageComparison = { onShowConfig: '#(onShowConfigFn)' }

* compareImage { baseline: 'base.png', latest: 'current.png' }

Common Patterns

Visual Regression Testing

Implement systematic visual regression testing across multiple pages.

Feature: Visual regression suite

Background:
* configure imageComparison = {
    failureThreshold: 1,
    hideUiOnSuccess: true
  }

Scenario: Test critical pages
* def pages = ['home', 'login', 'dashboard', 'profile']

* karate.forEach(pages, function(page) {
    driver 'https://example.com/' + page;
    var screenshot = karate.call('screenshot', false);
    compareImage {
      baseline: 'screenshots/' + page + '-baseline.png',
      latest: screenshot
    };
  })

Dynamic Content Handling

Handle timestamps, ads, and other dynamic content that changes frequently.

Scenario: Ignore dynamic elements
# Define all dynamic regions
* def dynamicRegions =
  """
  [
    { top: 0, left: 800, bottom: 50, right: 1200 },
    { top: 600, left: 0, bottom: 700, right: 300 }
  ]
  """

* configure imageComparison = { failureThreshold: 0.5 }

* driver 'https://example.com/news'
* def actual = screenshot(false)
* def baseline = read('screenshots/news-baseline.png')

* compareImage {
    baseline: '#(baseline)',
    latest: '#(actual)',
    options: { ignoredBoxes: '#(dynamicRegions)' }
  }

Multi-Browser Testing

Compare screenshots across different browsers for consistency.

Feature: Cross-browser visual testing

Scenario: Compare Chrome and Firefox rendering
# Chrome screenshot
* configure driver = { type: 'chrome' }
* driver 'https://example.com'
* def chromeScreenshot = screenshot(false)

# Firefox screenshot
* driver.quit()
* configure driver = { type: 'firefox' }
* driver 'https://example.com'
* def firefoxScreenshot = screenshot(false)

# Compare browsers with tolerance
* configure imageComparison = { failureThreshold: 2 }
* compareImage {
    baseline: '#(chromeScreenshot)',
    latest: '#(firefoxScreenshot)'
  }

Configuration Reference

Global Configuration Options

Option	Type	Default	Description
`engine`	string	`'resemble'`	Comparison engine: `'resemble'`, `'ssim'`, or combined
`failureThreshold`	number	`0.0`	Percentage of pixels allowed to differ (0-100)
`allowScaling`	boolean	`false`	Scale images to match dimensions automatically
`hideUiOnSuccess`	boolean	`false`	Hide comparison UI in reports when images match
`mismatchShouldPass`	boolean	`false`	Treat mismatches as passes (useful for initial baselining)
`onShowRebase`	function	`null`	Custom callback when user clicks Rebase in HTML report
`onShowConfig`	function	`null`	Custom callback when user clicks Show Config in HTML report

Per-Comparison Options (Resemble)

Option	Type	Default	Description
`ignoredBoxes`	array	`null`	Regions to exclude: `[{top, left, bottom, right}]`
`ignore`	string	`'less'`	Preset: `'nothing'`, `'less'`, `'antialiasing'`, `'colors'`, `'alpha'`
`ignoreColors`	boolean	`false`	Compare brightness only, ignore color differences
`ignoreAntialiasing`	boolean	`false`	Ignore anti-aliasing pixel differences
`ignoreAreasColoredWith`	object	`null`	Ignore regions of specific color: `{r, g, b, a}`
`tolerances`	object	`null`	Custom color/brightness thresholds: `{red, green, blue, alpha, minBrightness, maxBrightness}`

Per-Comparison Options (SSIM)

Option	Type	Default	Description
`ssim`	string	`'WEBER'`	Algorithm: `'FAST'` or `'WEBER'`
`rgb2grayVersion`	string	`'INTEGER'`	Grayscale conversion: `'ORIGINAL'` or `'INTEGER'`
`k1`	number	`0.01`	First stability constant for SSIM algorithm
`k2`	number	`0.03`	Second stability constant for SSIM algorithm
`windowSize`	integer	`11`	Comparison window size
`bitDepth`	integer	`8`	Bits per pixel (8, 16, 32)
`maxSize`	integer	`256`	Max dimension before downsampling

Engine Selection

Use Resemble for pixel-perfect comparisons with anti-aliasing support. Use SSIM for structural similarity when minor pixel shifts are acceptable. Combine both with 'resemble,ssim' to use the lowest mismatch percentage.

Troubleshooting Image Comparison

Images Won't Match

Problem: Images differ unexpectedly

Solutions:

Check image dimensions match (or enable allowScaling: true)
Increase failureThreshold for minor differences
Use ignoredBoxes for dynamic content areas
Try SSIM engine instead of Resemble
Check HTML report for visual diff to identify differences

Performance Issues

Problem: Image comparison slows down tests

Solutions:

Enable hideUiOnSuccess: true to reduce report size
Use smaller screenshots (specific elements vs full page)
Reduce image resolution before comparison
Run visual tests separately from functional tests

Baseline Maintenance

Problem: Baselines outdated after UI changes

Solutions:

Set mismatchShouldPass: true temporarily while updating baselines
Use rebase function in HTML report to download new baselines
Organize baselines by feature/page for easier maintenance
Version control baseline images with tests

Next Steps

Implement visual testing and continue with:

Automate browser testing: UI Testing
Mock visual dependencies: Test Doubles
Run visual tests in parallel: Parallel Execution
Debug comparison failures: Debugging
See working examples: Examples and Demos

Benefits of Image Comparison​

Basic Image Comparison​

Image Comparison with Tolerance​

Comparison Engines​

Resemble Engine​

SSIM Engine​

Choosing an Engine​

Configuration Options​

Failure Threshold​

Scaling Options​

Hide UI on Success​

Ignored Regions​

Advanced Configuration​

Resemble Engine Options​

Resemble Ignore Presets​

Custom Color and Brightness Tolerances​

SSIM Engine Options​

Custom Rebase Callback​

Custom Configuration Display​

Common Patterns​

Visual Regression Testing​

Dynamic Content Handling​

Multi-Browser Testing​

Configuration Reference​

Global Configuration Options​

Per-Comparison Options (Resemble)​

Per-Comparison Options (SSIM)​

Troubleshooting Image Comparison​

Images Won't Match​

Performance Issues​

Baseline Maintenance​

Next Steps​

Benefits of Image Comparison

Basic Image Comparison

Image Comparison with Tolerance

Comparison Engines

Resemble Engine

SSIM Engine

Choosing an Engine

Configuration Options

Failure Threshold

Scaling Options

Hide UI on Success

Ignored Regions

Advanced Configuration

Resemble Engine Options

Resemble Ignore Presets

Custom Color and Brightness Tolerances

SSIM Engine Options

Custom Rebase Callback

Custom Configuration Display

Common Patterns

Visual Regression Testing

Dynamic Content Handling

Multi-Browser Testing

Configuration Reference

Global Configuration Options

Per-Comparison Options (Resemble)

Per-Comparison Options (SSIM)

Troubleshooting Image Comparison

Images Won't Match

Performance Issues

Baseline Maintenance

Next Steps