质量维度 - 测试体系Systematic Thinking for Testing

分享到社交媒体：

Note: You can find the English version after the Chinese one.
(文末有对应演讲视频)

【中文版 Chinese version】

00 引言

1. 三个层次聊测试体系

测试人员缺乏体系化思维？
新建产品团队或者新启项目，如何搭建质量保障体系？

大家都接触过不计其数的测试、质量方面的文章或者培训课程，内容不乏测试实践、技术相关，但是却很难构建自己的测试体系。基于很多朋友类似的困惑，结合本人多年的团队实践和咨询经验，从基础篇、进阶篇和高级篇三个不同的层次来跟大家探讨测试体系化思维的构建。

2. 基础篇内容回顾

2022年1月份发布了《构建测试的体系化思维（基础篇）》（后文简称《基础篇》），从测试的五个基本职责出发，围绕这五个基本职责介绍了相应的实践活动和方法集。

理解和澄清业务需求
制定策略并设计测试
实现和执行测试
缺陷管理与分析
质量反馈与风险识别

3. 进阶篇内容概要

本文为进阶篇，将跳出测试，从软件质量的角度来看体系化思维的构建。

从测试到质量：跳出测试，从质量维度来看测试的体系化建设。
质量保障与质量内建：质量是产品与生俱来的，质量内建才是保障质量的根本。
质量内建深入实践：质量内建如何落地是大家最为关注的问题，深入质量内建实践介绍，供大家落地实践参考。

关于高级篇的内容，暂不公布，敬请期待下次更新。

01 从测试到质量

1. 为什么要从测试转变到质量

在《基础篇》中提到的五个测试基本职责，那是测试人员开展测试工作务必了解的方方面面，是做好测试工作的基础。

但是，要做好软件产品的质量保障工作，光从做好测试的角度考虑是远远不够的，原因主要有两个方面：

软件质量不能通过测试来提高。著名质量管理专家戴明提到：“质量没法检测出来，软件产品开发出来后质量就已经在那了。”
软件测试可以验证软件质量是否满足需求，但是随着软件生态的复杂性和不确定性增加，没有办法预知软件行为，也就没法通过测试来验证软件质量的方方面面。

因此，测试是基本功，但只关注测试是不够的。需要跳出测试，将视野调整到软件开发全生命周期质量的角度，关注质量的各个维度，才能更好地建立体系化思维，才能更好地做好软件产品的质量保障工作。

在阅读后面内容之前，我希望读者朋友们先花10秒钟思考以下问题：

问题：跳出测试，从软件质量的角度思考，我们需要关注哪些方面？

这是一个老生常谈的问题，答案似乎大家也都很清楚，但现实中又是很容易被忽视、被误解的问题。

2. 软件质量的定义

在回答上面问题之前，我们先来看看维基百科对软件质量的定义。如下图所示：

维基百科对软件质量的定义包括两个方面：

软件功能质量：基于功能需求规范看软件是否提供了相应的功能，这是用户使用软件的角度，是软件外部的质量。
软件结构质量：对支持功能需求交付的非功能需求的满足情况，比如健壮性、可维护性等，这是从软件内部结构来看，是软件内部的质量。

维基百科的这个定义跟咱平常提到的软件质量包含外部质量和内部质量是一致的。外部质量是从软件外部可感知的，是从用户角度对软件感知到的使用质量；内部质量则涉及到软件的内部形态，包括技术架构和代码质量等，是从团队开发人员的角度感知到的。

相信大家对于外部质量好理解，是用户角度能够感知到的质量。而内部质量往往很容易被忽视，其实这也是因为人们普遍容易追求短期利益，看不到内部质量给软件产品带来的长远影响，这里想给大家看个非常熟悉的图，以增强对内部质量的重视。

不加整理的富含技术债的代码，就如同图中杂乱无章缠绕在一起的电线，严重影响着产品缺陷定位修复、后期功能扩展等。

3. 容易忽视的质量影响因素

理解了质量的定义，在日常软件开发工作中有哪些因素是会影响到软件质量的呢？其实，影响的因素特别的多，可能包括但不限于以下这些：

需求分析过程仓促，或者参与人员角色比较单一，导致业务上下文了解不够，关键场景缺乏考虑等；

忙于交付更多的特性，忽略了对代码质量的关注，该重构的没有重构，在原本不太健康的代码基础上继续增加更多的代码，导致混乱，筑起高高的技术债；

没有足够的测试覆盖，导致新增代码容易破坏已有功能；

开发人员提交代码后，就投入新代码的开发，对所提交代码缺乏关注，持续集成流水线上测试挂了不能及时修复，可能影响后面的测试进度；

大面积的重构发生在发布前夕，无法全面回归，带来很大的风险；

项目初期只考虑少量用户的场景，随着业务的发展，系统功能难以扩展，导致严重性能瓶颈；

技术选型失误，开发困难，没有及时改进，一错再错，最后问题严重到无法弥补；

第三方组件评估不够充分，导致线上环境无法承载等；

开发缺少对异常情况的处理，测试过程缺乏探索，只覆盖到主干路径，边角用例可能引发问题；

开发或者测试都只考虑当前功能模块，缺乏更广范围的考虑；

缺乏跨功能需求的关注，导致严重的安全或者性能问题；

对线上环境了解不够，而且没有足够日志信息记录，线上问题难以定位，导致宕机时间过长；

……

02 质量保障与质量内建

1. 质量如何保障

从质量维度来看体系化思维的构建，而质量是产品与生俱来的，做好软件产品的质量保障工作更为有效的方式就是要将质量构建到软件产品本身，提高软件产品与生俱来的质量。这也就是质量内建（Build Quality In）的概念。

这个还是太抽象，为了更清楚的解释质量内建，先来看两个概念：暴露缺陷与预防缺陷。

2. 暴露缺陷与预防缺陷

“暴露缺陷”的概念众所周知，就是软件所暴露出来的缺陷，通常在测试或使用过程中被发现。

如果能够全面统计一个软件产品的所有可能引起缺陷的地方，我们可以说该软件存在缺陷的总量是一定的。（当然，现实情况是我们不可能全面考虑到所有的因素，缺陷是很难穷尽的。）

如果这一定数量的缺陷中有一部分可以通过某些措施来预防，那么软件在测试或最终用户使用过程中所暴露出来的缺陷数量就会减少，这就是缺陷预防——减少暴露给用户的缺陷。

软件开发过程是个持续递增的过程，预防缺陷也不是一蹴而就的事情，需要在整个软件开发过程持续的进行。当然，尽早预防缺陷会更好，这样可以显著减少缺陷修复的成本，提高投入产出比。下图曲线非常清晰地展示了修复不同阶段暴露出来的缺陷所需成本的差异，修复成本是随着软件开发周期显著增加的。

3. 质量内建如何做

将缺陷概念扩展为软件产品质量相关的所有问题，那么做好了缺陷预防也就是做到了质量内建。

前文提到过容易被忽视的质量影响因素，要做好质量内建这些因素都是需要重点关注的。

但是，由于软件开发过程的复杂性，要一次性做对所有事情、预防所有的缺陷是不可能的。因此，质量内建并不是严格意义上的不能有任何缺陷暴露，而是需要在软件开发生命周期中尽早地把事情做对，并且通过持续的获取快速的反馈来纠偏。

我们都知道那个有名的PDCA环（也称“戴明环”）：

将质量管理分为四个阶段，即Plan(计划)、Do(执行)、Check(检查) 和 Act(处理)。在质量管理活动中，要求把各项工作按照作出计划、计划实施、检查实施效果，然后将成功的纳入标准，不成功的留待下一循环去解决。

质量内建可以理解为在软件开发全生命周期中，从需求到线上频繁地执行PDCA环，小步前进，持续反馈验证，及时调整改进。如下图所示：

03 质量内建深入实践

前面主要从概念和理念层面介绍了质量和质量内建，这些内容对构建体系化思维是至关重要的，有助于拓宽视野，形成全局观。但光有这些内容不够，还得了解具体的落地实施方法。接下来就从质量内建相关实践来探讨质量内建在软件开发生命周期的落地实施方案。

1. 质量内建的核心

在介绍具体的质量内建实践之前，还是需要从抽象层面来理解质量内建的核心，其内容可以分为四个方面：

测试左移
快速反馈
测试右移
全员负责

1.1 测试左移

传统软件开发会分成分析、设计、开发、测试、发布等几个相对独立的阶段，而测试是其中发生在开发完成之后的一个阶段。

测试左移是针对传统这种独立测试阶段来说的，不再强调独立的测试阶段，而是要将测试活动左移到软件开发生命周期的最早阶段（最左边）——需求阶段，从需求阶段开始做好缺陷预防的工作。

要特别提醒注意的是，测试左移不是测试人员左移，而是强调的测试活动左移：

左移到传统独立测试阶段左侧就叫左移，不一定是左移到需求阶段。当然，根据前面对质量内建的阐述，理想情况下肯定是越左边越好。如果做不到彻底左移，能移一步也是进步。
测试人员的确需要左移参与相应的活动。但是，如果测试人员参与需求阶段，却没有起到澄清需求、验证需求有效性的作用，那就不是有效的测试左移，关键要看成效。
不仅测试人员要左移，开发等其他角色也需要左移。因为需求有效性的验证，需要不同角色的经验，需要来自不同视角的输入；不同角色都有责任参与到每个环节的测试活动中来。
只要能做到从需求阶段开始就有相应的测试活动，不一定都是测试人员参与。这里强调的是测试活动要有，哪个角色来做不是最关键的，角色其实就是戴了一定帽子而已。我们知道，有的团队是没有测试人员的，但不代表这些团队没有测试活动。

1.2 快速反馈

快速反馈，也就是频繁持续地反馈，可以理解为一个持续测试的过程。

通常快速反馈需要有自动化的支撑，比如自动化测试、流水线上自动代码扫描等；也可以是一些人工参与的实践环节，比如手动测试、各类评审和验收等。通过这样的一些实践活动尽快地获取相应工作（可能是需求分析、开发等）的反馈，根据反馈结果进行及时的修复、调整、或者优化。

为了保障快速反馈/持续测试实践活动的有效开展，通常需要增加质量门禁，并确保质量门禁的严格执行。

1.3 测试右移

测试右移是跟测试左移对应的，就是将测试活动从独立测试阶段右移到生产环境。

但因为生产环境的特殊性，测试右移又不是测试活动的简单右移，而是通过一些实践活动获取生产环境下用户行为、日志等质量相关信息，利用这些信息来给前期的需求、开发和测试工作提供反馈，促进相应工作的优化改进，以更好的做好质量内建。

测试右移本身不能直接内建质量，但是可以帮助更好地质量内建，所以我认为测试右移也是质量内建不可或缺的核心内容之一。

这部分的内容比较多，之前有过详细的介绍，此处不在赘述。欢迎参考文章《测试右移——生产环境下的QA》阅读更为详细的内容。

1.4 全员负责

测试左移、快速反馈和测试右移都不是某个单一角色/人员可以做到的，需要不同角色的不同技能和不同视角的输入，质量人人有责，需要全员参与并且能承担起质量职责才行。

全员负责不仅是很多实践活动需要大家一起参与，还有很重要的一点是团队内部需要有足够的沟通和信息共享。只有信息足够透明，每个人都清楚质量目标、质量状态和风险，才能真正发挥主人翁精神积极地参与，才能发挥团队每个人最大的价值。一定要警惕团队“蘑菇种植”。

这部分内容之前也有过非常多的分享，欢迎参考：《团队对质量负责，“我”可以不负责？》、《敏捷测试的指导性原则》、《说好的团队为质量负责呢？》等文章。

2. 质量内建典型实践

综合来看，质量内建的典型实践通常有（但不限于）下图这些：

不管是传统开发模式还是敏捷迭代式开发模式，软件开发生命周期都会有需求分析、开发实现和线上使用三个阶段，只是敏捷模式下需求和开发是迭代式进行，在时间上不是界限清晰的阶段。

这里为了描述方便，暂且将这些实践分成三个阶段来描述，事实上实践活动跟阶段也不是关联非常紧密的，遵循测试左移、快速反馈、测试右移和全员负责这几个核心思想才是关键所在。

2.1 需求分析

这里强调四个我认为跟需求分析密切相关的典型质量内建实践，分别是需求澄清、行为驱动开发（BDD）、实例化需求（SbE）和需求评审。

需求澄清：需求的理解和澄清作为测试基本职责之一，在《基础篇》中有较为详细的介绍，请移步参考。不过，在这里需要补充的是不仅测试要参与需求澄清，还需要其他相应的角色协同参与，比如开发人员、技术负责人、技术架构师等。
行为驱动开发（BDD）：关于BDD，很多人误认为是一种测试方法，但其实BDD主要是为了三方更好地对需求达成一致认识，只不过通过BDD方式产生的需求能够很好地指导自动化测试的开展。更多详情，请参考我之前的文章《说起BDD，你会想到什么？》。
实例化需求（SbE）：SbE的核心思想跟BDD类似，但强调了一点要将需求描述通过实例来说明，以更好地发现漏掉的需求，让需求更完整、更容易被多方理解和澄清。
需求评审：需要开发和测试跟业务分析人员一起讨论确认需求，可能是正式的会议针对某个特性（大块需求）的评审，也可以是针对单个需求条目或敏捷用户故事进行线下评审。如果前期需求澄清过程中对需求的讨论已经比较透彻，这里的评审就会比较轻量级，由测试人员（开发按需参与）自行安排时间评审即可。

这些实践都是为了更好地在需求分析阶段让团队对需求达成一致的理解，确保需求的质量，减少后期因需求问题带来的返工和浪费。

2.2 开发实现

需求源头的质量控制好了，接下来就是在开发实现阶段持续地通过快速反馈的方式确保每一步工作的质量，下面的实践都是为此服务的：

技术方案讨论：不是要介绍这个实践怎么做，而是要提醒在做技术方案讨论的时候测试要尽量参与。一方面，测试清楚了解技术方案，才能更好地设计相应的测试策略，更完备的进行测试；另一方面，测试基于自己对系统的了解，可以给技术方案的讨论提供输入，比如系统重点需要考虑和关注的问题等。可能有人会觉得测试技术水平跟不上，参与价值不大，但我想说的是不参与就不会有进步，只要参与了就一定会有收获，从长期来看，不管是对个人还是对团队整体都是非常有价值的。
需求条目启动：也叫用户故事启动（Story kickoff）、开卡等。对于单个可开发需求，在开发人员要开始编码前进行的再次澄清和确认，主要是确认其中的验收标准是否不够完备、是否大家都理解一致了。形式要求尽量轻量级，在开发人员电脑前完成即可，需要业务分析、开发和测试共同参与，时间一般不要超过15分钟。
需求条目验收：跟启动是配对的，也叫用户故事验收（Desk check/Shoulder check/Story signoff）、结卡/验卡等。同样在开发机器前面完成，一般由开发将功能演示给业务分析和测试人员，大家确认需求中的正向路径是否都已经开发实现了。更多详情可参考文章《高效用户故事验收》。
测试用例设计：在《基础篇》有对测试用例设计作详细的介绍，从质量内建的角度来看主要是想说测试用例设计尽量早点做，而且对于用例设计的形式可以相对灵活些，列举需要关注的测试点才是关键，从快速反馈的角度，不建议过于注重形式和流程。
测试驱动开发：TDD，常见的有单元测试驱动开发（UTDD）和验收测试驱动开发（ATDD），通过先写测试的方式驱动设计的完备性。Thoughtworks同事刘冉老师有多篇关于TDD的文章，请移步他的网站「刘冉的思辨悟」查阅参考。
代码评审：Code review，也叫代码回顾。有团队成员聚集在一起做回顾的，也有独立线下评审的模式。Thoughtworks同事伍斌道长的文章《Code Review: 超越“审、查、评”的代码回顾》有关于code review非常详细的讲解，请移步参考。
自动化回归测试：根据测试分层理论，自动化测试可能包括单元测试、接口集成测试和端到端测试。通常单元测试是开发人员编写，而接口集成测试和端到端集成测试可以开发和测试协作完成。关于自动化测试相关文章比比皆是，这里推荐如下内容供参考：
单元测试评审：单元测试评审一般发生在需求条目验收环节，详情参考文章《QA评审底层测试的价值》。
持续集成：持续集成（CI）概念大家并不陌生，但是有效实施却不是那么容易。常见的反例有：
1. 光有持续集成流水线，但是流水线上啥也没有，没有代码扫描、没有自动化测试、没有质量门禁；
2. 流水线有接入代码扫描，但是扫描出问题没有及时修复；
3. 自动化测试没有在流水线上频繁执行，或者执行失败不能及时修复；
4. 代码没有频繁提交；
5. ……

更多内容，推荐参考Thoughtworks洞见文章《持续集成理论和实践的新进展》和《对于持续集成实践的常见问题的解答》。

自动化/持续部署：自动化部署在提高效率的同时还可以减少手工部署引入的问题，让部署能够更频繁更持续地发生。这部分内容可以参考Thoughtworks洞见文章《持续集成和交付流水线的反模式》。
技术债管理：还记得前面那个布满杂乱无章电线的图吗？软件开发过程中引入的技术债如果没有得到有效管理，软件内部也会乱成那个样子。关于技术债管理，推荐Thoughtworks同事麻广广的文章《软件架构治理之技术债的前世今生》。
探索式测试：探索式测试有助于发现一些潜在的不易察觉的缺陷，是非常推荐的一个实践。具体做法和建议在《基础篇》中对探索式测试有详细介绍。
客户验收：英文叫Showcase，就是给客户、业务方演示开发实现的特性。Showcase也有很多常见误区，请移步文章《敏捷实践Showcase的七宗罪》。
缺陷分析：缺陷分析对质量内建的帮助主要是通过分析找出问题根因，在后续的开发工作中提前避免以预防更多的缺陷暴露。如何做好缺陷管理和分析？请参考下列文章：
质量状态报告和质量风险分析：请参考《基础篇》中“测试职责之五：质量反馈与风险识别”部分的介绍。
回顾会议：回顾会议通过总结和分析过去一段时间团队发生的事情，是关于团队整体健康状态的。我之所以将回顾会议列为质量内建的典型实践，是因为咱们前面讨论过了影响质量的因素涉及到方方面面，自然就跟团队的健康有着很大的关系。利用好回顾会议这个实践，及时发现问题并采取对应的改进举措，将是非常有价值的。关于回顾会议的高效开展，可以参考Thoughtworks洞见文章《高效回顾会议的七步议程》。

2.3 线上使用

软件产品发布到线上，质量内建相关实践就都是跟测试右移有关的，这是一个很大的主题，之前有过系列文章的介绍，相关实践可以参考下列文章：

04 写在最后

测试基本职责是测试人员必备之基础技能，但测试的体系化思维构建不能仅局限于测试工作本身，当我们跳出测试，从质量的角度来看，又是另一番景象。

希望本《进阶篇》能够帮助各位测试同仁从质量的角度拓宽视野，增加大局观，让质量保障工作做的更顺畅。

【演讲视频】

【英文版 English Version】

00 Introduction

0.1 Three levels of Systematic Thinking for Testing

Do testing personnel lack systematic thinking?
How to build a quality assurance system for new product teams or projects?

Many of us have come across countless articles or training courses on testing and quality, which are not lacking in testing practices or technical content, but it is difficult to build our own testing system. Based on similar doubts from many friends and my own team practice and consulting experience over the years, I will discuss the construction of systematic thinking for testing at three different levels: basic, intermediate, and advanced.

0.2 Review of the Basic Level Content

In January 2022, "Systems Thinking for Testing - Basics" was published, which introduced the corresponding practical activities and methods for the five basic responsibilities of testing.

Understand and clarify business requirements
Develop strategies and design tests
Implement and execute tests
Defect management and analysis
Quality feedback and risk identification

0.3 Overview of Intermediate Level Content

This article is the intermediate level of the series and will discuss the systematic thinking from the perspective of software quality, going beyond testing.

From testing to quality: Move beyond testing and consider the construction of testing systems from the perspective of quality.
Quality assurance and quality built-in: Quality is inherent in products, and quality built-in is the fundamental guarantee for quality assurance.
In-depth practice of quality built-in: How to implement quality built-in is the most concerned issue, and this article will introduce in-depth practices for reference.

01 From Testing to Quality

1.1 Why Move from Testing to Quality

In the "Basics" section, the five basic responsibilities of testers mentioned are the aspects that testers must understand to carry out testing work and lay the foundation for doing a good job in testing.

However, to do a good job of quality assurance for software products, it is far from enough to consider it from the perspective of doing a good job in testing. There are two main reasons for this:

Software quality cannot be improved through testing. Famous quality management expert Deming once mentioned, "Quality cannot be detected. Once the software product is developed, the quality is already there."
Software testing can verify whether the software quality meets the requirements, but as the complexity and uncertainty of the software ecosystem increases, it is impossible to predict the software behavior, and therefore it is impossible to verify all aspects of the software quality through testing.

Therefore, testing is a fundamental skill, but focusing only on testing is not enough. It is necessary to break out of testing, adjust the perspective to the quality of the entire software development life cycle, and focus on various dimensions of quality in order to better establish systematic thinking and do a good job of quality assurance for software products.

Before reading the following content, I hope that readers can spend 10 seconds thinking about the following question:

Question: To break out of testing and think from the perspective of software quality, what aspects do we need to focus on?

This is a question that has been talked about for a long time. The answer seems to be clear to everyone, but in reality, it is a problem that is easily overlooked and misunderstood.

1.2 Definition of software quality

Before answering the above question, let's first look at the definition of software quality on Wikipedia. As shown in the figure below:

The definition of software quality on Wikipedia includes two aspects:

Software functional quality: Based on the functional requirements specification, it looks at whether the software provides corresponding functions. This is from the user's perspective and is the external quality of the software.
Software structural quality: It looks at the satisfaction of non-functional requirements that support the delivery of functional requirements, such as robustness and maintainability. This is from the perspective of the internal structure of the software and is the internal quality of the software.

The definition of software quality on Wikipedia is consistent with what we usually mention, which includes external quality and internal quality. External quality is perceivable from outside the software and is the quality perceived by users from their perspective of using the software; internal quality involves the internal form of the software, including technical architecture and code quality, and is perceived from the perspective of the team of developers.

I believe that everyone can understand external quality, which is the quality that can be perceived from the user's perspective. However, internal quality is often easily overlooked. In fact, this is also because people generally tend to pursue short-term benefits and cannot see the long-term impact that internal quality brings to software products. Here, I want to show you a very familiar picture to enhance your attention to internal quality.

The code that is rich in technical debt without organization is like the tangled wires in the picture, seriously affecting the diagnosis and fixing of product defects, and the expansion of later functions.

1.3 Overlooked factors that affect quality

Having understood the definition of quality, what are the factors that can affect software quality in daily software development work? In fact, there are many factors that can affect software quality, which may include but are not limited to the following:

Rushed requirement analysis process, or a relatively single role of the participating personnel, resulting in insufficient understanding of the business context, lack of consideration for key scenarios, and so on;
Busy delivering more features, ignoring the focus on code quality, not refactoring where necessary, and continuing to add more code on top of the already unhealthy code, leading to confusion and building up high technical debt;
Insufficient test coverage, making it easy for new code to break existing functionality;
After developers submit code, they move on to developing new code and lack attention to the submitted code, which can cause delays in fixing failing tests on the continuous integration pipeline, affecting subsequent testing progress;
Large-scale refactoring occurs on the eve of release, making it impossible to fully regress, resulting in great risk;
Only a small number of user scenarios are considered in the early stages of the project, and as the business grows, system functionality becomes difficult to extend, resulting in serious performance bottlenecks;
Technical selection errors, development difficulties, failure to improve in a timely manner, making mistakes repeatedly, and ultimately the problem becomes so serious that it cannot be remedied;
Insufficient evaluation of third-party components, resulting in the inability to support the production environment;
Lack of handling of exceptional situations in development, lack of exploration in the testing process, covering only the main paths, and corner cases may cause problems;
Development or testing only considers the current functional module, lacking broader considerations;
Lack of attention to cross-functional requirements, leading to serious security or performance issues;
Insufficient understanding of the production environment, and insufficient logging information, making it difficult to locate issues in the production environment, leading to long downtime;
......

02 Quality Assurance and Quality Built-in

2.1 How to Ensure Quality

From the perspective of quality, building a systematic thinking is important, and quality is inherent in the product. To effectively ensure the quality of software products, it is important to build quality into the software product itself, and improve the inherent quality of software products. This is the concept of quality built-in.

However, to better explain the concept of quality built-in, let's first look at two concepts: defect exposure and defect prevention.

2.2 Defect Exposure and Defect Prevention

The concept of "defect exposure" is well-known and refers to the defects exposed by the software, usually discovered during testing or use.

If we can comprehensively identify all the possible places where defects may occur in a software product, we can say that the count of defects in the software is certain. (Of course, in reality, we cannot consider all factors comprehensively, and defects are difficult to exhaust.)

If some of these defects can be prevented through certain measures, then the number of defects exposed during testing or use by end-users will be reduced. This is defect prevention, which reduces the number of defects exposed to users.

Software development is a continuous and incremental process, and defect prevention is not a one-time thing. It needs to be carried out continuously throughout the entire software development process. Of course, preventing defects early will be better, as it can significantly reduce the cost of defect fixing and improve the return on investment. The curve in the following figure clearly shows the difference in the cost of fixing defects exposed at different stages, and the cost of fixing defects significantly increases as the software development cycle progresses.

2.3 How to Build Quality In

If we expand the concept of defects to include all issues related to software product quality, then preventing defects is equivalent to building quality in.

As mentioned earlier, the quality factors that are easily overlooked need to be given special attention to build quality in.

However, due to the complexity of the software development process, it is impossible to get everything right the first time and prevent all defects. Therefore, building quality in does not strictly mean that no defects will be exposed, but rather that things need to be done right as early as possible in the software development lifecycle, and deviations need to be corrected through continuous and rapid feedback.

We all know the famous PDCA cycle (also known as the "Deming cycle"):

Quality management is divided into four stages: Plan, Do, Check, and Act. In quality management activities, it is required to plan, implement the plan, check the implementation effect, and then incorporate successful practices into the standard and solve unsuccessful ones in the next cycle.

Building quality in can be understood as executing the PDCA cycle frequently throughout the entire software development lifecycle, taking small steps forward, continuously validating feedback, and making timely adjustments and improvements. As shown in the following figure:

03 Typical Practice of Building Quality In

The previous section mainly introduced the concepts and ideas of quality and building quality in, which are crucial for building systematic thinking and broadening perspectives to form a holistic view. However, these contents alone are not enough. We also need to understand specific implementation methods. Next, we will discuss the implementation plan of building quality in throughout the software development lifecycle from the perspective of related practices.

3.1 Core of Quality Built-in

Before introducing specific quality built-in practices, it is necessary to understand the core of quality built-in at an abstract level, which can be divided into four aspects:

Testing Shift Left
Fast Feedback
Shift Right Testing
Everyone's Participation

3.1.1 Testing Shift Left

Traditional software development is divided into several relatively independent stages such as analysis, design, development, testing, and release, and testing is a stage that occurs after development is completed.

Testing Shift Left is for traditional independent testing stages, which no longer emphasizes independent testing stages but shifts the testing activity to the earliest stage of the software development life cycle (the leftmost side) - the requirements stage, to do a good job of defect prevention from the requirements stage.

It is important to note that testing shift left does not mean testers shift left, but emphasizes the shift of testing activities:

Moving to the left of the traditional independent testing phase is called shifting left, not necessarily to the requirements phase. Of course, according to the previous explanation of quality built in, the ideal situation is to move further to the left. If it is impossible to move completely to the left, moving one step is also progress.
Testers do need to participate in corresponding activities. However, if testers participate in the requirements phase but do not clarify or validate requirements effectively, it is not an effective testing shift left. The key is to look at the effectiveness.
Not only testers but also developers and other roles need to shift left. Because the validation of requirements effectiveness requires the experience of different roles and inputs from different perspectives, different roles have a responsibility to participate in testing activities in each stage.
As long as there are corresponding testing activities from the requirements phase, it is not necessarily that testers participate. What is emphasized here is that there must be testing activities, and which role is not the most critical. Roles are just hats. We know that some teams do not have testers, but it does not mean that these teams do not have testing activities.

3.1.2 Fast Feedback

Fast feedback, which means frequent and continuous feedback, can be understood as a process of continuous testing.

Usually, fast feedback requires automation support, such as automated testing, automatic code scanning in the pipeline, etc. It can also be some manual participation in practice, such as manual testing, various types of reviews and acceptance, etc. By quickly obtaining feedback on corresponding work (possibly requirement analysis, development, etc.) through such practice activities, timely repairs, adjustments, or optimizations can be made according to the feedback results.

To ensure the effective implementation of fast feedback/continuous testing practice activities, it is usually necessary to increase the quality gate and ensure the strict execution of the quality gate.

3.1.3 Testing Shift Right

Testing Shift Right is the counterpart of Testing Shift Left, which is to move testing activities from independent testing phases to the production environment.

However, due to the special nature of the production environment, Testing Shift Right is not a simple shift of testing activities. It is through some practical activities to obtain quality-related information such as user behavior and logs in the production environment, and use this information to provide feedback to the previous requirement, development, and testing work, promote the optimization and improvement of corresponding work, and do a better job of quality building.

Testing Shift Right cannot directly build quality, but it can help to build quality better. Therefore, I believe that Testing Shift Right is also one of the indispensable core contents of quality building.

This part of the content is relatively long, and there has been a detailed introduction before. It will not be repeated here. You are welcome to refer to the article "Testing Shift Right - QA in Production" for more detailed information.

3.1.4 Everyone's Participation

Testing shift left, fast feedback, and testing shift right cannot be accomplished by a single role/person. It requires input from different roles with different skills and perspectives. Quality is everyone's responsibility and everyone's participation and the ability to assume quality responsibilities are necessary.

Everyone's participation is not only necessary for many practice activities, but also important for sufficient communication and information sharing within the team. Only when information is transparent, and everyone is clear about quality goals, quality status, and risks can the team truly engage with an entrepreneurial spirit and maximize each team member's value. Beware of the "mushroom planting" phenomenon within the team.

There have been many previous shares on this topic, please refer to articles such as "Whole Team is Responsible for Quality : Can I Avoid It?" ,"Agile Testing's Guiding Principles", and "Everyone is Responsible for Quality".

3.2 Typical Practices of Quality Built-in

In general, typical practices of quality built-in include (but are not limited to) the ones shown in the figure below:

Regardless of whether it is traditional development mode or agile iterative development mode, the software development life cycle includes three stages: requirements analysis, development implementation, and production. In agile mode, however, requirements and development are carried out iteratively and the stages are not clearly defined in terms of time.

For ease of description, these practices are divided into three stages. However, in fact, practice activities are not closely related to stages. The key lies in following the core concepts of testing shift left, fast feedback, testing shift right, and everyone's participation.

3.2.1 Requirements Analysis

Here, I emphasize four typical quality built-in practices that I think are closely related to requirements analysis, which are requirement clarification, Behavior-Driven Development (BDD), Specification by Example (SbE), and requirement review.

Requirement Clarification: As one of the basic responsibilities of testers, the understanding and clarification of requirements have been introduced in detail in the "Basics" article. However, what needs to be added here is that not only testers but also other relevant roles, such as developers, technical leaders, and technical architects, need to participate in requirement clarification collaboratively.
Behavior-Driven Development (BDD): Many people mistakenly think that BDD is a testing method, but in fact, BDD is mainly to achieve a consensus on requirements among the three roles. However, the requirements generated through the BDD approach can guide the development of automated testing very well. For more details, please refer to my previous article "What Comes to Your Mind When We Talk About BDD?".
Specification by Example (SbE): The core idea of SbE is similar to BDD but emphasizes the use of examples to illustrate the requirements to discover missing requirements, make requirements more complete, and make them easier for multiple roles to understand and clarify.
Requirement Review: Developers and testers need to discuss and confirm requirements with business analysts. It may be a formal meeting for the review of a specific feature (a large requirement), or it can be an offline review of a single requirement item or agile user story. If the discussion of the requirements during the earlier requirement clarification process is thorough, the review here will be relatively lightweight, and the testers (with developers as needed) can arrange the time for the review themselves.

These practices are all aimed at helping the team achieve a common understanding of the requirements in the requirements analysis phase, ensuring the quality of the requirements, and reducing rework and waste caused by requirements problems in the later stages.

3.2.2 Development and Implementation

Once the quality control of the requirement source is in place, the next step is to ensure the quality of each step of the work through fast feedback during the development and implementation stage. The following practices serve this purpose:

Technical solution discussion: It is not about how to do this practice, but to remind that testers should participate as much as possible in the discussion of technical solutions. On the one hand, testers can better design corresponding testing strategies and conduct more comprehensive testing by understanding the technical solution clearly. On the other hand, based on their understanding of the system, testers can provide input for the discussion of technical solutions, such as issues that the system needs to focus on. Some people may feel that their technical skills in testing are not good and that their participation is not valuable, but what I want to say is that there will be no progress if they do not participate. Since they participate, there will definitely be gains, and in the long run, it is very valuable for both individuals and the team as a whole.
Story kickoff: It is a re-clarification and confirmation of a single story's requirement before the developer starts coding, mainly to confirm whether the acceptance criteria are complete enough and whether everyone understands them consistently. It should be as lightweight as possible and can be completed in front of the developer's computer. Business analysis, developers, and testers should participate together, and the time should generally not exceed 15 minutes.
Desk check: It is paired with the story kickoff. It is also completed in front of the developer's machine. Generally, the developer demonstrates the function to business analysts and testers , and everyone confirms whether the positive path of the requirement has been developed and implemented. For more details, please refer to the article "Efficient Desk Check."
Test case design: There is a detailed introduction to test case design in the "Basics" article. From the perspective of quality built-in, the main point is to design test cases as early as possible, and the format of the test cases can be relatively flexible. Listing the test points that need to be focused on is the key. From the perspective of fast feedback, it is not recommended to pay too much attention to format and process.
Test-driven development: TDD, common types include unit test-driven development (UTDD) and acceptance test-driven development (ATDD), which drives design completeness by writing tests first. Liu Ran, a colleague from Thoughtworks, has written multiple articles about TDD. Please visit his website for reference.
Code review: It can be done by team members gathering together or independently offline. Wu Bin, a colleague from Thoughtworks, has a very detailed explanation of code review in his article "Code Review: Beyond the 'Auditing, Checking, and Review' " Please refer to it for more information.
Automated Regression Testing: According to test layering theory, automated testing may include unit testing, interface integration testing, and end-to-end testing. Usually, unit tests are written by developers, while interface integration and end-to-end integration tests can be completed through collaboration between developers and testers. There are numerous articles related to automated testing, and here are some recommended for reference:
Unit Tests Review: Unit tests review generally occurs in the desk check activity, for details please refer to the article "The Value of QA Reviewing Low-Level Tests".
Continuous Integration: The concept of Continuous Integration (CI) is familiar to everyone, but it is not easy to implement it effectively. Common anti-patterns include:
- Having a CI pipeline, but nothing on the pipeline, no code scanning, no automated testing, no quality gates;
- The pipeline has integrated code scanning, but problems found are not fixed in time;
- Automated tests are not frequently executed on the pipeline or fail to be fixed in time;
- Code is not frequently submitted;
- ......

For more information, it is recommended to refer to the Thoughtworks Insights articles "New Developments in Continuous Integration Theory and Practice" and "Answers to Common Questions About Continuous Integration Practices".

Automation/Continuous Deployment: Automated deployment not only improves efficiency but also reduces problems introduced by manual deployment, making deployment happen more frequently and continuously. For more information on this topic, please refer to the Thoughtworks Insight article "Continuous Integration and Delivery Pipeline Anti-patterns".
Technical Debt Management: Do you remember the picture with messy and tangled wires earlier? If technical debt introduced in the software development process is not effectively managed, the software internals will also become a mess. For technical debt management, we recommend the article "The Past and Present of Technical Debt in Software Architecture Governance" by our colleague Ma Guangguang.
Exploratory Testing: Exploratory testing is a highly recommended practice for discovering potential and less visible defects. For specific methods and suggestions, please refer to the "Basics" where exploratory testing is introduced in detail.
Showcase: Showcase is a demonstration of the features developed for customers and business stakeholders. Showcase has many common misconceptions, please refer to the article "Seven Sins of Agile Practice Showcase".
Defect Analysis: The main benefit of defect analysis for quality built-in is to find out the root cause of the problem through analysis and prevent more defects from being exposed in the subsequent development work. How to do good defect management and analysis? Please refer to the following articles:
Quality Status Report and Quality Risk Analysis: Please refer to the "Fifth Basic Responsibility: Quality Feedback and Risk Identification" section in the "Basics".
Retrospective Meeting: The retrospective meeting summarizes and analyzes the events that the team has experienced over a period of time, which is about the overall health status of the team. I listed retrospective meetings as a typical practice of quality build-in because the factors that affect quality involve many aspects and are naturally closely related to the health of the team. Making good use of this practice, discovering problems in time, and taking corresponding improvement measures will be very valuable. For an efficient retrospective meeting, please refer to the Thoughtworks Insight article "Seven-Step Agenda for Effective Retrospective".

3.2.3 Online Usage

When a software product is released online, quality-related practices are all related to testing shift right. This is a very large topic, and there have been a series of articles introducing relevant practices before. The following articles can be referred to:

04 Conclusion

Testing is a fundamental skill that all testers must possess, but the systematic thinking for testing should not be limited to testing work itself. When we look at quality from a broader perspective, we see a different scene.

We hope that this "Intermediate" can help testers broaden their perspective from the quality perspective, increase their overall understanding, and make quality assurance work smoother.