Build your tests more robust & flexible with Output Relay

Overview

What is Output Relay?

Output Relay is a feature provided by the Testany Platform designed to address scenarios where the execution of one test case within a test pipeline depends on the output of one or more other test cases within the same pipeline. Output Relay allows users to establish output dependencies between different test cases while maintaining the atomicity and independence of each case. This feature helps avoid implementing complex logic in a single test case, thereby improving test maintainability and efficiency in problem identification.

Typical Use Cases for Output Relay

A typical use case involves testing multiple API request dependencies. For example:

Case A: Retrieve a system access token for a user.
Case B: Use the token obtained in Case A to call the “Get User List” service and retrieve a user ID from the list.
Case C: Use the token from Case A and the user ID from Case B to call the “User Detail Service” and obtain location information for the user.

In this scenario, Case B depends on the output of Case A, and Case C depends on the outputs of both Case A and Case B. Output Relay helps maintain the independence of each test case while allowing these dependencies to be implemented, avoiding the need for complex and lengthy code logic.

Key Concepts of Output Relay

Definition of Output Case and Input Case

Output Case: A test case whose output data is used by other test cases. In the above example, Case A and Case B are Output Cases because their outputs (the token and user ID) are used by subsequent test cases.
Input Case: A test case that relies on the output data of other test cases. For example, Case C requires the token from Case A and the user ID from Case B, making Case C an Input Case.

Scope of Output Relay

Output Relay applies to all functional testing scenarios and can be utilized by development, testing, or operations teams to write more concise and clear test cases. It is particularly beneficial in scenarios involving multiple APIs, database queries, or other multi-step dependencies. Output Relay greatly enhances test execution efficiency and speeds up error diagnosis.

It is important to note that Output Relay is not applicable to performance testing, as performance testing focuses on system behavior under load rather than dependencies between test cases.

Constraints and Limitations of Output Relay

While Output Relay offers significant flexibility, there are system-imposed constraints to ensure test result accuracy. The system will prevent certain illogical configurations, for example:

If Case A is an Output Case and Case B is an Input Case, but the user configures the logic to execute Case B only if Case A fails, the system will reject this configuration. This is because if Case A fails, its output data is unreliable, and the results of Case B based on this failed output may also be inaccurate, leading to untrustworthy test results.

This constraint mechanism prevents test result ambiguity caused by invalid output dependencies, ensuring that users can generate reliable results from their test cases.

How to Use Output Relay

Writing Output Case and Input Case

To use Output Relay, you must first define both the Output Case and Input Case. For more details on how to write Output Case and Input Case, please refer to <Managing Test Case with Relay Case>.

Configuring Test Pipelines with Output Case and Input Case

After writing the Output Case and Input Case, the next step is to correctly arrange them within the same test pipeline. For detailed steps on how to configure a pipeline using Output Relay, please refer to <Test pipeline writing guideline & samples>.

Execution Logic vs. Standard Pipelines

A test pipeline with Output Relay enabled has no significant differences in execution logic from a standard pipeline. The pipeline still executes test cases in sequence, but the key distinction is that explicit dependencies exist between the test cases. The system ensures that these dependencies are correctly maintained and automatically executes the tests according to the dependency order.

Common Misconfigurations and System Responses

Some common misconfiguration examples are as follows:

Misconfiguration Example 1: Case B is set to execute only if Case A fails. The system will reject this configuration.
Misconfiguration Example 2: Case B depends on the output of Case A, but Case A has not declared any output. In this case, the system will not reject the configuration, but in the pipeline details page, the user will not see any test cases marked with the Relay Output Case flag. This will remind the user to review the dependency settings to ensure they meet expectations.

Common Issues and Best Practices

Best Use Cases for Output Relay

The Output Relay feature is best suited for the following scenarios:

Multi-step functional testing, where each step depends on the output of the previous step.
API call chains, where relay data reduces redundancy.
Integration testing, where credentials or other dynamic data from previous steps need to be reused.

Common Errors and Troubleshooting Methods

Below are some common errors and suggestions for troubleshooting:

Undeclared Output Variables: If the Output Case does not declare its output variables correctly, the system will use the initial value of the Input Case. If the test results do not meet expectations, check the dependency configuration and ensure that the output and input variables are correctly declared.
Illogical Test Configuration: If an Input Case is set to execute when the Output Case fails, the system will reject this configuration. This situation may arise if the user incorrectly sets the execution order. Adjust the pipeline logic to ensure the sequence of test case execution aligns with the intended dependencies.

Writing Efficient Relay-Dependent Test Cases

To fully utilize the Output Relay feature, users should follow these best practices:

Clearly differentiate the execution logic in the pipeline (e.g., whenpassed, whenfailed) from dependencies introduced by Output Relay (ref) to avoid confusion in pipeline configurations.
Maintain the atomicity of test cases, ensuring that each test case performs only a single, independent function.
Clearly declare output and input variables to ensure that dependency relationships are transparent.
Avoid creating overly complex dependency chains; keep test cases simple and maintainable.