Agile Testing Manifesto

分享到社交媒体：

【中文版Chinese Version】

NOTE： The English version of Agile Testing Manifesto can be found after this Chinese version.

根据多年在Thoughtworks敏捷团队的敏捷测试相关经验，我们几位资深QA同事一起讨论出敏捷测试宣言和原则，本文与大家详细解读。

敏捷测试宣言

解读

敏捷测试宣言表达的是我们对于敏捷测试的信仰和价值观，分别包括流程、团队合作、自动化和核心价值观四个维度。

在以往软件发布周期较长、需求变化频率较低、不同角色间的合作壁垒较高的环境下，传统的开发模式所采用的孤立的测试阶段、测试人员独立把关质量、回归式的全量自动化测试和质量检测，是有其价值的。但是，在敏捷开发模式下，敏捷测试则更强调左侧的价值观。

全程的测试介入

敏捷测试提倡测试左移和右移，从软件生命周期的早期（左侧）一直到产品发布上线后的生产环境，都需要有测试的介入和测试活动的开展。

左移是为了更好的理解和澄清需求，以减少需求理解不一致导致的浪费；而右移是充分利用生产环境的数据来优化开发和测试流程，以增强软件系统应对各种不可预测性的能力。

左移和右移并不仅仅是将测试活动移到两侧端点，更强调的是每个环节的参与，也就是全程测试介入，这是从流程上保障高质量软件交付的关键。

团队整体对质量负责

敏捷提倡全功能团队，团队的角色之间分工不再那么明确，不同角色间的协作更加密切，团队一起为质量负责，是敏捷测试需要遵循的指导性原则。

团队需要对质量目标有统一认识，在敏捷软件生命周期的每个环节有不同角色的共同参与，实现质量目标是每个角色的职责。

持续性的精准自动化测试

自动化测试是敏捷测试的基础，是快速反馈的必要手段。自动化测试不能一味的追求覆盖率，而是要追求有目的的精准覆盖。也就是说，自动化测试首先必须是有效的，是基于业务风险考虑的，才能真正实现快速反馈。

自动化测试需要能够在持续集成流水线上持续的运行，为每次代码提交提供反馈，以确保系统功能不会因为新代码的提交而被破坏；同时，随着功能的不断迭代，自动化测试需要相应的更新、增加，确保新功能是有有效自动化测试覆盖的。

质量内建

质量内建是敏捷测试的核心，需要将测试全程介入、团队为质量负责和持续精准的自动化测试结合起来，在敏捷软件生命周期的每个环节做好缺陷的预防，把质量融入到产品的开发构建中。

敏捷测试原则

我们的目标在于和团队一起尽快地交付高质量软件。
测试人员尽早参与软件早期阶段，与所有团队角色合作，通过实例化需求，确保对业务价值理解的一致性。
测试人员关注生产环境状态，收集数据，指导和优化前期的分析、开发和测试。
测试人员和开发人员同处一个产品项目团队，而不是独立的测试团队或部门。
测试人员负责探索性测试，和开发人员结对，设计、实现和维护自动化测试。
自动化测试在流水线中持续精准执行，快速发现每次代码提交对于已有功能的影响。
测试数据对于自动化测试是充分的，并能按需获得。
测试活文档化，和代码一起，作为知识资产进行版本化管理。
自动化测试需要有效的分层。
预防缺陷，而不是关注缺陷的数量。

解读

敏捷测试需要遵循如下10条原则：

1. 我们的目标在于和团队一起尽快地交付高质量软件。

敏捷测试的目标，所有的测试活动都围绕这一目标开展。敏捷测试人员要有思维上的转变，不再是关注于发现更多的缺陷，而是思考和采取相应的措施以帮助团队更快、更高质量的交付软件。

2. 测试人员尽早参与软件早期阶段，与所有团队角色合作，通过实例化需求，确保对业务价值理解的一致性。

测试左移，团队合作，需求的清晰表达，确保团队在做对的事情，拥抱变化的同时能够少走弯路。

3. 测试人员关注生产环境状态，收集数据，指导和优化前期的分析、开发和测试。

测试右移，由于生产环境的特殊性，并不能将测试活动简单右移到生产环境，只能通过收集和分析生产环境的数据，利用这些数据来优化开发、测试和业务价值，让生产环境和开发过程形成良性环路。

4. 测试人员和开发人员同处一个产品项目团队，而不是独立的测试团队或部门。

测试人员属于开发团队的一员，推倒角色之间的沟通壁垒，测试人员与开发人员进行密切的合作。

5. 测试人员负责探索性测试，和开发人员结对，设计、实现和维护自动化测试。

自动化测试不只是测试人员的职责，单元测试和接口测试主要由开发人员负责，而界面层自动化测试同样需要开发人员一起参与设计、实现和维护，这样才更高效；同时，可以让测试人员抽离出来去做更有价值的事情——探索性测试。

6. 自动化测试在流水线中持续精准执行，快速发现每次代码提交对于已有功能的影响。

自动化测试需要在持续集成流水线中运行，并且做到能够按需精准执行，在追求完备覆盖的同时还能高效运行，快速提供反馈。

7. 测试数据对于自动化测试是充分的，并能按需获得。

自动化测试应该有完备的数据集，包括正常和异常情况的覆盖、满足不同环境或不同平台的执行要求等；同时，需要有完善的数据创建和管理方案，确保不同需求下的自动化测试能够获得相应的测试数据。

8. 测试活文档化，和代码一起，作为知识资产进行版本化管理。

需求文档要跟自动化测试关联，变成可执行的活文档，并且要跟产品代码一起作为知识资产进行版本化管理。

9. 自动化测试需要有效的分层。

自动化测试要根据不同的测试对象进行有效分层。越往底层的测试实现成本更低、执行更高效、定位更准确，但覆盖范围有限，不能跟业务很好的关联；越往顶层的测试越接近业务、更能体现业务价值，但是执行速度、稳定性较差、定位问题较难。需要根据系统要求、技术架构等项目具体情况规划每层测试的合理占比，不能盲目的追求多而全的覆盖。

10. 预防缺陷，而不是关注缺陷的数量。

质量内建是敏捷测试追求的核心价值观，而质量内建本质上就是缺陷预防。敏捷测试需要团队把重心放在预防缺陷上，提高软件的内建质量，而只关注缺陷数量、甚至把缺陷数量当做考核指标的情况是违背这一核心价值观的。当然，对缺陷数量趋势的正确跟踪和对缺陷根因的深入分析，是帮助预防缺陷的有效手段，是值得推荐的。

English Version

Full Lifecycle Testing OVER Isolated Testing Phase
Team Shared Responsibility OVER Testers Ensure Quality
Continuous Targeted Automation OVER Widespread Regression Testing
Quality Built-in OVER Defect Detection

Interpretation

The Agile Testing Manifesto expresses our beliefs and values about agile testing, including four dimensions: process, teamwork, automation, and core values.

As the software release cycle was long, the frequency of requirement changes was low, and the barriers to cooperation between different roles were high, the traditional development model adopted the isolated testing phase, the testers independently checked the quality, full regression automation and quality detection, which are beneficial. However, with agile development, agile testing puts more emphasis on the values on the left.

Full Lifecycle Testing

Agile testing advocates shifting testing to the left and right. From the early stage of the software life cycle (left side) to the production environment after the product is released, testers’ involvement and testing activities are essential.

Shifting left, the goal is to better understand and clarify requirements in order to reduce waste caused by inconsistent requirement understanding, while shifting right is to make full use of the data of the production environment in order to optimize the development and testing process so that the software can handle a variety of unexpected behavioral patterns.

Shifting left and right is not only about moving the testing activities to the endpoints on both sides, but also emphasizing the effect of each stage, that is, full life cycle testing, which is the key to ensuring high-quality software delivery from the process.

Team Shared Responsibility

With agile, the roles of the team are no longer clearly defined, the collaboration between different roles is closer, and the team is responsible for the quality together, which is the agile testing's guiding principle.

The team needs to share a unified understanding of quality objectives, and various roles participate in different stages of the agile software life cycle. It is each role's responsibility to achieve quality objectives.

Continuous Targeted Automation

Automated testing is the foundation of agile testing and a critical means of rapid feedback. Automated testing should not pursue coverage blindly, but should pursue purposeful and accurate coverage. That is to say, Automated testing must first be effective and based on business risks in order to truly achieve rapid feedback.

The automated tests must run continuously in the CI pipeline, providing feedback for each code submission, to ensure system functions will not be broken because of new code submissions. As new functions continue to iterate, it is necessary to update and add more automated tests so that new features are covered by automated tests.

Quality built-in

Quality built-in is the core of agile testing. It is necessary to combine the entire process of testing, the team's responsibility for quality, and continuous and accurate automated testing altogether, to prevent defects in every phase of the agile software life cycle, and to integrate quality into the product being developed and built.

Ten Principles of Agile Testing

Our goal is to deliver high-quality software as quickly as possible with the team.
Testers engage as early as possible in the software development process, collaborating with all roles in the team to ensure a consistent understanding of the business value.
Testers monitor quality in the production environment, collect data, then analyze the results to optimize the process of requirement analysis, development, and testing.
Rather than having separate testing teams or departments, testers and developers are part of the same product project team.
Testers are responsible for exploratory testing and work with developers to design, implement and maintain automated tests.
Automated tests are executed continuously and accurately in the CI pipeline, and the impact of each new code submission on existing functions can be easily identified.
Test data is sufficient for automated testing and is available on demand.
Tests are documented as living documents and versioned as knowledge assets along with the product code.
Automation tests require effective layering.
Rather than focusing on the number of defects found, prevent them.

Interpretation

1. Our goal is to deliver high-quality software as quickly as possible with the team.

It is the goal of agile testing, around which all testing activities are conducted. Agile testers should change their mindset, no longer focus on finding more bugs, but think and act accordingly to help teams deliver software faster and with higher quality.

2. Testers engage as early as possible in the software development process, collaborating with all roles in the team to ensure a consistent understanding of the business value.

Shift testing to the left and articulate requirements to ensure the team is doing the right thing and avoiding detours while embracing change.

3. Testers monitor quality in the production environment, collect data, then analyze the results to optimize the process of requirement analysis, development, and testing.

Due to the particularity of the production environment, shifting testing to the right cannot simply move the testing activities to the production environment, but only collect and analyze the data of production environment to optimize development, testing and business value, so that the production environment and the SDLC forms a virtuous circle.

4. Rather than having separate testing teams or departments, testers and developers are part of the same product project team.

Testers are part of the development team, tearing down communication barriers between roles, and testers work closely with developers.

5. Testers are responsible for exploratory testing and work with developers to design, implement and maintain automated tests.

Automated test is not just the responsibility of testers. Unit tests and API tests are mainly the responsibility of developers, and UI automated tests also requires developers to participate in design, implementation and maintenance, which is a more efficient way to free testers to do something more valuable, i.e. exploratory testing.

6. Automated tests are executed continuously and accurately in the CI pipeline, and the impact of each new code submission on existing functions can be easily identified.

In addition to pursuing complete coverage, automated tests should also run efficiently and provide feedback quickly. They need to be run in the CI pipeline and can be executed precisely on demand.

7. Test data is sufficient for automated testing and is available on demand.

Automated tests should contain a complete data set, including covering both happy paths and unhappy paths, meeting the execution requirements of different environments or platforms, etc.

Similarly, it should include data creation and management functions so as to obtain relevant test data.

8. Tests are documented as living documents and versioned as knowledge assets along with the product code.

Requirements documentation should be tied to automated tests, become executable living documents, and be versioned with product code as a knowledge asset.

9. Automation tests require effective layering.

Automated tests should be effectively layered according to different test objects.

Low-level tests have lower cost, higher execution efficiency, and more accurate problem location, but it cannot be well related to the business.

High-level tests are close to the business and can reflect the business value, but the execution speed and stability are poor, and it is difficult to locate the problem if a test fails.

It is necessary to plan a reasonable proportion of each layer of tests according to the specific conditions of the project such as system requirements and technical architecture, and cannot blindly pursue high coverage.

10. Rather than focusing on the number of defects found, prevent them.

The core value of agile testing is quality built-in, which is essentially defect prevention.Team should focus on preventing defects and improving the built-in quality of software.

Focusing only on the number of defects, or even making it a KPI, goes against this core value.

Tracking defects trends and analyzing the root causes are effective methods that can help prevent defects and should be recommended.