Protect the credentials used in testing

Purpose of this Article

This document primarily discusses the importance of testing credentials, the risks associated with them, and the measures taken by Testany to ensure their security.

Understanding Data at Risk: Testing Credentials

What Testing Credentials Are

Definition:

Test credentials are credentials that verify the identity and authority of the caller as required by the software service being tested and are used for the execution of test cases on the Testany platform. The credentials for obtaining test cases and executing test pipelines are also test credentials, but their protection measures are not discussed in this article.

Types of Testing Credentials:

This may include API keys, test accounts, tokens, and other forms of credentials used during the testing phase.

The Importance of Protecting Testing Credentials

Role of Testing Credentials in Software Testing

Testing credentials play a crucial role in the software testing process. In a simulated environment, they help replicate real-world scenarios without affecting actual production data or exposing real user information.

These credentials enable testers to validate and verify various features of the application, such as its ability to handle authentication, user roles, and permission levels. Additionally, they help testers identify possible loopholes and vulnerabilities within the authentication system that could be exploited in a real-world scenario. This level of testing is critical in maintaining the robustness, security, and reliability of the software.

Moreover, testing credentials are used to access different testing environments. For instance, these can be used to access QA (Quality Assurance), UAT (User Acceptance Testing), and staging environments that closely mirror the production environment but are meant for testing and development purposes. In this way, testing credentials help facilitate effective testing while minimizing the risk of accidental changes or breaches in the live environment."

It's essential to handle testing credentials with the same level of security as regular user credentials because they can potentially provide access to sensitive information or expose system vulnerabilities if misused.

Potential Consequences of Compromised Testing Credentials

Compromised testing credentials can lead to a number of damaging consequences, including:

1. Unauthorized Access to Testing Environments: If testing credentials are compromised, unauthorized users may gain access to the testing environments. They can tamper with test results, introduce vulnerabilities, or even disrupt the testing process.

2. Exposure of Sensitive Development Data: Testing environments often contain sensitive data related to the software under development, including new features, proprietary algorithms, or unpatched vulnerabilities. A breach can lead to the leakage of this sensitive information.

3. Security Vulnerabilities: A compromised testing environment can be a stepping stone for attackers to infiltrate other interconnected systems. With the gained access, they may attempt to escalate privileges or exploit any found vulnerabilities, leading to more significant security breaches.

4. Reputational Damage: If a security breach becomes public, it can lead to reputational damage. Customers and partners may lose trust in the company's ability to protect not only its production environment but also its testing environment.

5. Legal and Compliance Issues: There may be legal and compliance implications, especially if the breach leads to the exposure of sensitive or personally identifiable information (PII). The company could face penalties or lawsuits for failing to adequately protect its systems.

It's vital to protect testing credentials to prevent these potential consequences. This means adopting robust security measures, including secure credential storage, role-based access control, and regular audits and monitoring.

The solutions to protect credentials used in Testing

In order to provide comprehensive protection for the credentials required in testing, Testany has introduced the "Trusted Credential Safe" solution.

There are two key components in this “Trusted Credential Safe” solution:

a) A safety credential CRUD product provided by an IaaS vendor (e.g., AWS Secrets Manager, Azure Key Vault)

b) A safety method to exchange credentials and references between Testany Platform and credential safe provided by the IaaS vendor.

At present, the Testany Platform supports two different ways of deploying the Trusted Credential Safe solution:

a) hosted by Testany, or

b) using the credential safe service under the user's own IaaS account.

Although the deployment methods differ, the basic principles remain the same. Below, we will elaborate in detail on the working principle of this solution in conjunction with the system architecture diagram of the Testany Platform. Here we use AWS Secrets Manager as an example; please note the steps below are very similar (if not the same) for other credential-safe products.

Transmit and store the credentials used in Testing - Single case with single credential

Before you start, please make sure you’ve already known which credential safe product is used in your Testany tenant. If you are not clear about it, contact the Global Admin or Workspace Admin of your Testany tenant.

Step 1: Upload your Test Case Package/Link to Testany in the “Register Test Case” Page

Step 2: After Uploading successfully, you will get a Secret UUID (Like 06e11efd-6e32-427f-bbf9-7a5f2e26e237) from Testany. The Secret UUID will be used to transmit credentials to Trusted Credential Safe. Each Test Case will have its unique Secret UUID.

This transmission is end-to-end encrypted by TLS1.2.

Step 3: Download AWS Secrets Manager CLI
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions

The following steps may vary if your organization has a different path to authenticate & authorize. This is an example of using SSO solutions for organizations like AzureAD, Okta, or others.

Step 4: Use your organization’s SSO to authenticate with AWS IAM for Security Token Service (STS) token trading.

Step 5: Use the STS token you just obtained to transmit the credentials needed by the test case to AWS Secrets Manager.

Step 6: Use AWS CLI to transmit the Credentials with a secret name. (Like Secret UUID/ Secret Key Name)

aws secretsmanager create-secret \
    --name 06e11efd-6e32-427f-bbf9-7a5f2e26e237/MyTestSecret \
    --description "My test secret created with the CLI." \
    --secret-string "XXX-YYY-ZZZ"

In this example

• Secret UUID: 06e11efd-6e32-427f-bbf9-7a5f2e26e237

• Secret Key Name: MyTestSecret

For more information & usage, please refer to the AWS CLI for secrets manager and create-secret documentation.

Step 7: You can use MyTestSecret as the key name in your Test Code to retrieve credentials with the Testany internal service Testany Secrets Service

Step 8: After completing the Test Case Implement, upload your Test Case Package/Link to Testany on the “Update Test Case” page

Behind the scene, the credentials you transmitted to AWS Secrets Manager will be only stored in AWS Secrets Manager; none of the individuals or applications can access this AWS Secrets Manager without a proper Key Name.

Transmit and store the credentials used in Testing - Multiple cases with single credential

Before you start, please make sure you’ve already known which credential safe product is used in your Testany tenant. If you are not clear about it, contact the Global Admin or Workspace Admin of your Testany tenant.

Step 1: Ask Workspace Admin create a credential safe and a credential key name for your workspace.

Step 2: Upload your Test Case Package/Link to Testany in the “Register Test Case” Page. In the “Register Test Case” page bind the created credential key to your test case.

Step 3: Download AWS Secrets Manager CLI
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions

The following steps may vary if your organization has a different path to authenticate & authorize. This is an example of using SSO solutions for organizations like AzureAD, Okta, or others.

Step 4: Use your organization’s SSO to authenticate with AWS IAM for Security Token Service (STS) token trading.

Step 5: Use the STS token you just obtained to transmit the credentials needed by the test case to AWS Secrets Manager.

Step 6: Use AWS CLI to transmit the Credentials with a secret name. (It is as same as the Step 1 created credential key name)

aws secretsmanager create-secret \
    --name MyTestSecret \
    --description "My test secret created with the CLI." \
    --secret-string "XXX-YYY-ZZZ"

In this example

• The workspace admin created a credential key name in Step 1 which named MyTestSecret

• use AWS CLI tool created a secret which namedMyTestSecret

For more information & usage, please refer to the AWS CLI for secrets manager and create-secret documentation.

Step 7: You can use MyTestSecret as the key name in your Test Code to retrieve credentials with the Testany internal service Testany Secrets Service

Step 8: After completing the Test Case Implement, upload your Test Case Package/Link to Testany on the “Update Test Case” page

Behind the scene, the credentials you transmitted to AWS Secrets Manager will be only stored in AWS Secrets Manager; none of the individuals or applications can access this AWS Secrets Manager without a proper Key Name.

Retrieve credentials when it’s required to execute the test

When the credentials required by the testing are executed, the Test Engine will access the AWS Secrets Manager with the Key Name that the Test Case Package gave it to retrieve the credential and use it to execute the test case properly. AWS encrypts all traffic between the running container and AWS Secrets Manager. No individual or applications can intercept it.

The credentials are stored in Trusted Credential Safe with a key name, which format is Secret UUID-Secret Key Name. The developer could use the internal service Testany Secrets Service to retrieve secrets from Trusted Credential Safe with a Secret Key Name.

An encrypted random instance ID will be generated when a Test Execution Unit is executed. When the test case package calls the Testany Secrets Service, the encrypted random instance id will be injected into Testany Secrets Service and be decrypted in the service. The service uses this decrypt ID to find out the Test Case Secret UUID. Then the service could use the Secret UUID/Secret Key Name to retrieve credentials from Trusted Credential Safe.

Testany Secrets Service could NOT be triggered without the corresponding Test Execution Unit is alive, and this service could NOT be triggered outside Testany Platform Network.

Role-Based Access Control

The Secrets Manager is individually deployed for each of Testany's single-tenant accounts and is configured only to allow access to the employees of that specific account. This setup is safeguarded by both AWS and the Single Sign-On Identity Provider (SSO IDP) used by the account.

Monitor and Audit

AWS Secrets Manager integrates with AWS CloudTrail to log, continuously monitor, and retain information about the Secrets Manager API calls made in your AWS account. AWS CloudTrail creates an event every time an API call is made to Secrets Manager. This event includes information about who generated the request, the date and time, the services used, the actions performed, the parameters for the actions, and the response elements returned by the Secrets Manager.

This monitoring mechanism helps you to audit and analyze resource activity history to ensure compliance with internal policies and operational best practices.

CloudTrail can log all Secrets Manager API calls and deliver the log files to an Amazon S3 bucket that you specify. You can also choose to encrypt the log files with AWS Key Management Service for additional security.

Best Practices and Recommendations to protect your credentials

Protecting credentials during software testing is critical to maintain the integrity and security of the software being tested. It involves not only security measures from the tool (Testany Platform) but also relies on users' adherence to standardized and secure operations and comprehensive security processes. Below are some practices and processes we strongly recommend users to follow to protect credentials.

Admin:

1. Principle of Least Privilege (PoLP)

• Access Control: Ensure that the credentials provided for testing have the minimum necessary permissions to perform the test. This limits the impact in case of credential compromise.

• Regular Audits: Conduct regular audits of permissions associated with each credential to ensure they adhere to this principle over time.

2. Rotate Credentials Regularly

• Automated Rotation: Use automated tools or cloud services capabilities to rotate credentials periodically. This reduces the window of opportunity for an attacker to exploit stale credentials.

3. Monitor and Audit Access

• Access Logs: Enable logging of Credential Safe for all access to sensitive information and credentials. Regularly review these logs to detect unauthorized access attempts or suspicious activities.

• Alerts: Set up alerts for abnormal access patterns or security incidents related to credential usage.

4. Cloud IAM with Trusted Credential Safe

• Access Control: Using the cloud provided IAM solution to connect with the Trusted Credential Safe

User:

1. DO NOT hardcode any sensitive data in either test scripts or test meta.
   Strictly store any sensitive data (e.g. credential, PII) in the Credential Safe provided by your organization and use Testany Secrets Service to retrieve them during the test execution. This approach isolates the credentials from the code base and reduces the risk of accidental exposure.

2. Rotate Credentials Regularly

• Automated Rotation: Use automated tools or cloud services capabilities to rotate credentials periodically. This reduces the window of opportunity for an attacker to exploit stale credentials.

3. Use Dedicated Test Accounts

• Separation from Production: Create dedicated accounts for testing purposes that are separate from production accounts. This ensures that any compromise does not affect production data or services.