Test Case Writing Guideline & Best Practice - PyRes

Overview

PyRes by Testany is an optimized Docker image based on Python 3.12.6, pre-installed with common testing frameworks such as Pytest, Python Unit Test, and Robot Framework. Most importantly, PyRes enhances the connection methods used in testing services, especially with request and session, by incorporating built-in retry and exception handling mechanisms. It aims to address network jitter issues in cross-cloud testing, ensuring more stable and reliable test results.

‌

Features

• Supported Testing Frameworks:

  • Pytest

  • Python Unit Test

  • Robot Framework

• Automatic Retry Mechanism: Built-in retry logic for network connections, effectively reducing test failures caused by network issues.

• Improved Cross-Cloud Fault Tolerance: Compared to native Python testing frameworks, PyRes increases tolerance to network jitter by 50 times.

• Open Source: We are open-sourcing PyRes Executor, so you can freely utilize PyRes for more reliable and robust software testing based on Python. However, we strongly recommend users to use the Testany Platform to drive PyRes for a higher level of test automation, flexibility, and reliability.

‌

Using PyRes

Prepare Test Cases

Similar to standard Python testing, you need to prepare the following files:

1. Test Code File: For example, test_app.py.

2. Requirements File: If your Python test code requires specific libraries, you can create a requirements.txt file in the root directory. PyRes will automatically download these dependencies in the execution environment.‌

Package Test Cases

Package the relevant files into a zip file, for example, python-test.zip, and upload it to the Testany platform.‌

Register on the Testany Platform

Once you have successfully prepared the zip file, you can register your test cases on the Testany platform. Refer to the "Managing Test Case" section in our document for detailed steps on how to register test cases.

‌

Comparison Between PyRes and Native Python Unit Test Framework

Handling Network Jitter with Native Python Unit Test

When using the native Python Unit Test framework for testing, network jitter may lead to connection interruptions or timeouts, affecting test results. To avoid these issues, you need to manually write additional code to handle network exceptions and retry logic. Below is an example demonstrating how to handle these issues using the native framework:

import unittest
import requests
from requests.exceptions import RequestException

class TestAPI(unittest.TestCase):

    def make_request_with_retry(self, url, retries=5):
        attempt = 0
        while attempt < retries:
            try:
                response = requests.get(url, timeout=5)
                if response.status_code == 200:
                    return response
            except RequestException as e:
                print(f"Attempt {attempt + 1} failed with error: {e}")
                attempt += 1
        raise Exception("Max retries reached")

    def test_api_call(self):
        url = 'https://api.example.com/data'
        response = self.make_request_with_retry(url)
        self.assertEqual(response.status_code, 200)

if __name__ == '__main__':
    unittest.main()

In this example, you are required to:

• Manually handle network requests and timeout exceptions.

• Implement retry logic to address network fluctuation issues.

• Increase complexity by dealing with network layer exceptions unrelated to the business logic.

Using PyRes Executor

With PyRes, you no longer need to write additional code for handling network connection failures and retries, as PyRes has integrated these handling mechanisms. Therefore, you can focus on writing business logic for test code, avoiding the need to deal with underlying network exceptions. Below is a simplified example demonstrating the use of PyRes:

import unittest
import requests

class TestAPI(unittest.TestCase):

    def test_api_call(self):
        url = 'https://api.example.com/data'
        response = requests.get(url)
        self.assertEqual(response.status_code, 200)

if __name__ == '__main__':
    unittest.main()

In this example, you no longer need to:

• Write complex retry logic.

• Handle network errors such as RequestException.

• With PyRes's built-in mechanism, network jitter issues will be automatically handled to avoid interfering with test results.

PyRes Test Example

Here is an example of testing with the Python unit testing framework using PyRes:

import unittest
import os

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        test_str = os.getenv('TEST_STR')
        self.assertEqual('foo'.upper(), test_str)

    def test_isupper(self):
        self.assertTrue('FOO'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = os.getenv('SPLIT_STR')
        self.assertEqual(s.split(), ['hello', 'world'])
        with self.assertRaises(TypeError):
            s.split(2)

if __name__ == '__main__':
    unittest.main()

Environment Variables

In this example, the os.getenv method is used to retrieve environment variables. You can set environment variables according to your actual situation to ensure the tests run smoothly.

Running Tests

PyRes will automatically handle network exceptions, so you don't need to manually add extra exception handling logic to your test scripts.

Compatibility

PyRes currently supports Python 3.12.6 and is compatible with lower versions of Python. If you have specific requirements to use a more updated Python version, please contact our customer support.

FAQ

7.1 Can I debug PyRes locally?

Yes, you can pull the PyRes Docker image to your local environment and debug test cases locally.

7.2 Does PyRes support cross-cloud testing?

Yes, PyRes performs well in cross-cloud testing, greatly improving tolerance to network fluctuations.