---
title: "Pipeline YAML Schema Reference"
language: "en"
url: "https://docs.testany.io/en/docs/pipeline-yaml-schema-reference/"
markdown_url: "https://docs.testany.io/en/docs/pipeline-yaml-schema-reference.md"
translation: "https://docs.testany.io/zh/docs/pipeline-yaml-schema-reference.md"
last_updated: "2026-03-26T15:39:24+08:00"
---

# Pipeline YAML Schema Reference

This document provides a complete technical reference for the Test Pipeline YAML schema. For usage examples and best practices, see [Test Pipeline Writing Guideline and Samples](https://docs.testany.io/en/docs/test-pipeline-writing-guideline-and-samples.md).

## Schema Version

Testany currently supports both `rule/v1.2` and `rule/v1.3`.


> **Info**
>
> For all new pipelines, Testany explicitly recommends `rule/v1.3`. `rule/v1.2` is retained only for backward compatibility and does not provide case-level parallel execution.


### Version Selection Guidance

| Version | Status | Description |
|---------|--------|-------------|
| `rule/v1.3` | Recommended | Use for all new pipelines; supports case-level parallel execution while keeping `whenPassed` / `whenFailed` / `expect` / `relay` semantics |
| `rule/v1.2` | Compatibility only | Existing pipelines continue to run; no case-level parallel execution |
| `rule/v1.1` | Compatibility only | Backward compatibility for older legacy pipelines |

### Default Concurrency in `rule/v1.3`

When a pipeline uses `rule/v1.3`, the platform decides the default actual concurrency by instance type:

- Community Edition defaults to `2`
- Paid Community users default to `4`
- Enterprise Edition defaults to `8`

If the number of semantically parallel-ready cases is greater than the concurrency available to the current workspace, the platform queues and refills slots instead of starting everything at once. When slots are limited, dispatch order within the same ready wave follows YAML order.

## Top-Level Structure

```yaml
kind: rule/v1.3
spec:
  rules:
    - run: <TEST-CASE-KEY>
      # ... rule fields
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `kind` | string | Yes | Schema version identifier. `rule/v1.3` is recommended for new pipelines. `rule/v1.2` is supported only for backward compatibility. |
| `spec` | object | Yes | Pipeline specification container. |
| `spec.rules` | array | Yes | List of rule objects defining the pipeline execution flow. Must contain at least one rule. |

## Rule Object

Each rule in the `rules` array defines a Test Case to execute and its conditions.

```yaml
- run: AC2F5A50
  whenPassed: 9686C618
  expect: fail
  relay:
    - key: ACCESS_TOKEN
      refKey: 9686C618/ACCESS_TOKEN
      nonSecret: true
```

### Rule Fields

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `run` | string | Yes | - | The Test Case Key to execute. Must be unique within the pipeline. |
| `whenPassed` | string | No | - | Condition: Execute this rule only when the specified case **passes**. |
| `whenFailed` | string | No | - | Condition: Execute this rule only when the specified case **fails**. |
| `expect` | string | No | `pass` | Expected result. Use `fail` when the test case is expected to fail (e.g., negative testing). |
| `relay` | array | No | `[]` | List of relay configurations for passing variables between cases. Available in both `rule/v1.2` and `rule/v1.3`. |

### Condition Fields: `whenPassed` vs `whenFailed`


> **Warning**
>
> `whenPassed` and `whenFailed` are **mutually exclusive**. You cannot use both on the same rule.


| Field | Trigger Condition |
|-------|-------------------|
| `whenPassed` | The referenced case reported `Passed` to the pipeline. Note: A case with `expect: fail` always reports `Passed` regardless of actual result. |
| `whenFailed` | The referenced case reported `Failed`, `Aborted`, or `Timeout` to the pipeline. Note: This will never trigger for a case with `expect: fail`. |

**Validation Rules:**
- The referenced Test Case Key must be defined in a **preceding** rule.
- If neither `whenPassed` nor `whenFailed` is specified, the rule executes unconditionally (or after the test script that provides relay data finishes, if `relay` is defined).

### The `expect` Field

The `expect` field changes how the test result is reported to the pipeline.

| Value | Behavior |
|-------|----------|
| `pass` (default) | The case reports its actual result to the pipeline. |
| `fail` | The case **always reports `Passed`** to the pipeline, regardless of actual execution result. This is useful when you expect a test to fail but don't want it to fail the pipeline. |

**Key Point:** A case with `expect: fail` always reports `Passed` to the pipeline. Therefore, subsequent cases should use `whenPassed` (not `whenFailed`) to follow it.

```yaml
- run: A1B2C3D4             # Test case: validate setup
- run: E5F6A7B8             # Test case: test invalid input
  whenPassed: A1B2C3D4
  expect: fail              # Always reports "Passed" to pipeline
- run: C9D0E1F2             # Test case: next step
  whenPassed: E5F6A7B8      # Use whenPassed, not whenFailed
```

## Relay Object

The `relay` field enables Output Relay - passing environment variables from one case to another.

```yaml
relay:
  - key: ACCESS_TOKEN
    refKey: 5DC1106D/ACCESS_TOKEN
    nonSecret: true
```

### Relay Fields

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `key` | string | Yes | - | The environment variable name to define for the current case. |
| `refKey` | string | Yes | - | Reference to the source variable. Format: `<SOURCE-CASE-KEY>/<VARIABLE-NAME>`. |
| `nonSecret` | boolean | No | `false` | If `true`, the value can be printed in logs. If `false` (default), the value is treated as sensitive and masked in logs. |

### Relay Validation Rules


> **Warning**
>
> The following configurations will cause validation errors:


1. **Missing fields:** Both `key` and `refKey` are required.
2. **Undefined source:** The source case (`refKey` prefix) must be defined in a preceding rule.
3. **Relay with `whenFailed`:** You cannot relay from a case that you reference with `whenFailed`.
4. **Parallel ambiguity:** In `rule/v1.3`, the test script that provides relay data must be guaranteed to complete before the test script that consumes relay data starts. If both could land in the same parallel-ready wave, the configuration is rejected.

**Invalid Example:**
```yaml
# ERROR: Cannot relay from a failed case
- run: 5385B189
- run: A3D9405F
  whenFailed: 5385B189      # This case failed
  relay:
    - key: NEW_URL
      refKey: 5385B189/NEW_URL  # Cannot relay from failed case
```

### Implicit `whenPassed` Behavior

If a rule has `relay` defined but no explicit `whenPassed` or `whenFailed`, the system automatically adds `whenPassed` referencing the first test script that provides relay data.

```yaml
# These two configurations are equivalent:

# Explicit
- run: 5DC1106D
- run: DE96D6A0
  whenPassed: 5DC1106D
  relay:
    - key: TOKEN
      refKey: 5DC1106D/TOKEN

# Implicit (system adds whenPassed automatically)
- run: 5DC1106D
- run: DE96D6A0
  relay:
    - key: TOKEN
      refKey: 5DC1106D/TOKEN
```

## Complete Validation Rules

The platform validates Pipeline YAML against these rules:

| Rule | Error Message |
|------|---------------|
| `rules` array is empty | `definition error: no rules defined` |
| Both `whenPassed` and `whenFailed` on same rule | `rule N error: multiple precondition` |
| Condition references undefined case | `rule N error: precondition undefined` |
| Duplicate `run` values | `rule N error: duplicate run` |
| Relay missing `key` or `refKey` | `rule N error: relay must contain both a key and a refKey` |
| Relay references undefined case | `rule N error: relay undefined for <refKey>` |
| Relay from `whenFailed` case | `rule N error: The identifier <case> was used in the relay reference. The precondition for the current rule should be set to whenPassed instead of whenFailed.` |
| Duplicate relay keys in same rule | `rule N error: duplicate relay` |

## Quick Reference

### Minimal Pipeline

```yaml
kind: rule/v1.3
spec:
  rules:
    - run: A1B2C3D4           # Your test case key
```

### Sequential Execution (Pass Chain)

```yaml
kind: rule/v1.3
spec:
  rules:
    - run: A1B2C3D4           # Step 1
    - run: E5F6A7B8           # Step 2
      whenPassed: A1B2C3D4
    - run: C9D0E1F2           # Step 3
      whenPassed: E5F6A7B8
```

### Parallel Execution (No Conditions, recommended in `rule/v1.3`)

```yaml
kind: rule/v1.3
spec:
  rules:
    - run: A1B2C3D4           # Test A
    - run: E5F6A7B8           # Test B
    - run: C9D0E1F2           # Test C
```


> **Note**
>
> In `rule/v1.3`, the three root cases above are semantically parallel-capable. If the current workspace does not have enough available concurrency slots, the platform starts them in YAML order as slots are refilled. The same shape in `rule/v1.2` still follows legacy serial semantics.


### With Output Relay

```yaml
kind: rule/v1.3
spec:
  rules:
    - run: A1B2C3D4           # Login test case
    - run: E5F6A7B8           # Create order test case
      whenPassed: A1B2C3D4
      relay:
        - key: ORDER_ID
          refKey: A1B2C3D4/ORDER_ID
        - key: ITEM_COUNT
          refKey: A1B2C3D4/ITEM_COUNT
          nonSecret: true
```

### Negative Testing

```yaml
kind: rule/v1.3
spec:
  rules:
    - run: A1B2C3D4           # Test case: test invalid input
      expect: fail            # Always reports "Passed" regardless of actual result
    - run: E5F6A7B8           # Test case: next test
      whenPassed: A1B2C3D4    # Use whenPassed since expect:fail always reports Passed
```

## See Also

- [Test Pipeline Writing Guideline and Samples](https://docs.testany.io/en/docs/test-pipeline-writing-guideline-and-samples.md) - Usage examples and best practices
- [Build your tests more robust & flexible with Output Relay](https://docs.testany.io/en/docs/build-your-tests-more-robust-flexible-with-output-.md) - Detailed Output Relay concepts
- [Managing Test Pipeline](https://docs.testany.io/en/docs/managing-test-pipeline.md) - Pipeline management operations

## Glossary terms on this page

- [Test Pipeline](https://docs.testany.io/en/docs/glossary.md#test-pipeline)
- [Workspace](https://docs.testany.io/en/docs/glossary.md#workspace)
- [Test Case](https://docs.testany.io/en/docs/glossary.md#test-case)
- [Test Case Key](https://docs.testany.io/en/docs/glossary.md#test-case-key)
- [Test Script](https://docs.testany.io/en/docs/glossary.md#test-script)
- [Output relay](https://docs.testany.io/en/docs/glossary.md#output-relay)