Skip to main content

RUNNING TESTS

Test Tags and Filtering

Overview​

Gherkin provides powerful meta-data capabilities through tags, enabling you to partition tests into categories like 'smoke', 'regression', and environment-specific suites. Tags allow selective execution of test subsets and flexible test organization.

Basic Tag Usage​

Tagging Scenarios​

@smoke @regression
Scenario: User authentication
  Given url baseUrl + '/login'
  When method post
  Then status 200

@integration @slow
Scenario: Full workflow test
  Given complete setup process
  When executing full user journey
  Then verify all components work

Tag-Based Execution​

Use tags with the parallel runner for flexible test selection:

// Run only smoke tests
Results results = Runner.path("classpath:tests")
    .tags("@smoke")
    .parallel(5);

// Run regression but exclude slow tests
Results results = Runner.path("classpath:tests")
    .tags("@regression and not @slow")
    .parallel(5);

Command Line Tag Filtering​

Control test execution via command line:

# Exclude specific tags
mvn test "-Dkarate.options=--tags ~@skipme" -Dtest=TestRunner

# Complex tag logic
mvn test -Dkarate.options='-t=@smoke -t=@api' -Dtest=TestRunner

Special Built-in Tags​

TagDescription
@ignoreSkip scenarios at runtime (not applicable to called features)
@parallel=falseDisable parallel execution for specific scenarios
@report=falseHide scenarios from HTML reports
@setupDesignate setup scenarios for karate.setup()
@envEnvironment-specific execution
@envnotEnvironment exclusion

@ignore Tag​

@ignore
Scenario: Work in progress
  # This scenario will be skipped automatically
  * print 'This will not execute'

Scenario: Normal test
  # This will run normally
  * print 'This will execute'

@parallel=false Tag​

# Apply to entire feature
@parallel=false
Feature: Sequential execution required

@parallel=false
Scenario: Must run sequentially
  # This scenario won't run in parallel with others
  Given sequential setup required

Note: Forcing sequential execution is an anti-pattern and should be avoided when possible.

@report=false Tag​

@report=false
Scenario: Utility function
  # Hidden from HTML reports but still executed
  * def utility = function(x) { return x * 2 }

Environment Tags​

Environment tags provide conditional execution based on karate.env:

@env Tag​

Run scenarios only in specific environments:

@env=dev
Scenario: Development-only test
  * print 'Runs only when karate.env is "dev"'

@env=dev,test
Scenario: Dev and test environments
  * print 'Runs when karate.env is "dev" OR "test"'

@envnot Tag​

Exclude scenarios from specific environments:

@envnot=prod
Scenario: Never run in production
  * print 'Runs everywhere except production'

@envnot=perf,prod
Scenario: Skip in performance and production
  * print 'Excludes performance and production environments'

Advanced Tag Features​

Tags with Examples​

Tag specific rows in Scenario Outlines for granular control:

Scenario Outline: Region-specific testing
  * def vals = karate.tagValues
  * match vals.region[0] == expected

  @region=US
  Examples:
    | expected |
    | US       |

  @region=GB
  Examples:
    | expected |
    | GB       |

Tag Selector Usage:

# Run only US region tests
mvn test "-Dkarate.options=--tags @region=US" -Dtest=TestRunner

# Exclude GB region tests
mvn test "-Dkarate.options=--tags ~@region=GB" -Dtest=TestRunner

Runtime Tag Introspection​

Access tag information within scenarios:

@module=auth @priority=high
Scenario: Tag introspection example
  * def currentTags = karate.tags
  * def tagValues = karate.tagValues
  * match tagValues.module[0] == 'auth'
  * match tagValues.priority[0] == 'high'

Tag Logic and Combinations​

Logical Operations​

// AND operation (both tags required)
.tags("@smoke", "@api")

// OR operation (either tag accepted)
.tags("@smoke,@api")

// NOT operation (exclude tagged scenarios)
.tags("~@slow")

// Complex combinations
.tags("(@smoke or @regression) and not @slow")

Dynamic Tag Configuration​

// Build tag list programmatically
List<String> tags = new ArrayList<>();
if (fastMode) {
    tags.add("~@slow");
}
if (smokeOnly) {
    tags.add("@smoke");
}

Results results = Runner.path("classpath:tests")
    .tags(tags)
    .parallel(5);

Feature Calling with Tags​

Select specific scenarios when calling features:

# Call only scenarios tagged with @setup
* call read('common-setup.feature@setup')

# Call specific scenario by tag
* call read('user-management.feature@create')

Best Practices​

Tag Naming Conventions​

# Functional categories
@smoke @regression @integration

# Component areas
@auth @payment @notification

# Test characteristics
@fast @slow @flaky

# Environment suitability
@local @qa @staging

# Priority levels
@critical @high @medium @low

Tag Organization Strategy​

  1. Layer tags: Combine functional and technical tags
  2. Consistent naming: Use clear, standardized tag names
  3. Minimal complexity: Avoid over-tagging scenarios
  4. Document conventions: Maintain team tag usage guidelines

Example Multi-Tag Strategy​

@smoke @auth @fast
Scenario: Quick login validation
  # Fast smoke test for authentication

@regression @payment @integration @slow
Scenario: Complete payment workflow
  # Full integration test for payment processing

@smoke @api @auth @envnot=prod
Scenario: Development authentication check
  # API smoke test, never in production

Troubleshooting Tags​

Common Issues​

  • Tag inheritance: Child scenarios inherit feature-level tags
  • Case sensitivity: Tags are case-sensitive (@Smoke ≠ @smoke)
  • Complex logic: For intricate tag combinations, see Stack Overflow guidance
  • Examples selection: Tagged examples don't fall back to untagged examples

Tag Validation​

Use tag introspection to verify correct tag application:

Scenario: Verify tags are applied
  * def tags = karate.tags
  * print 'Current tags:', tags
  * assert tags.contains('@expectedTag')

Next Steps​

On This Page