【理论】测试框架体系TDD、BDD、ATDD、MBT、DDT介绍

一、测试框架是什么

测试框架是一组用于创建和设计测试用例的指南或规则。框架由旨在帮助 QA 专业人员更有效地测试的实践和工具的组合组成。这些指南可能包括编码标准、测试数据处理方法、对象存储库、存储测试结果的过程或有关如何访问外部资源的信息。

A testing framework is a set of guidelines or rules used for creating and designing test cases. A framework is comprised of a combination of practices and tools that are designed to help QA professionals test more efficiently.
These guidelines could include coding standards, test-data handling methods, object repositories, processes for storing test results, or information on how to access external resources.

二、测试框架的价值

测试框架是任何成功的自动化测试过程的重要组成部分。它们可以降低维护成本和测试工作，并为寻求优化其敏捷流程的 QA 团队提供更高的投资回报率 (ROI)。

Testing frameworks are an essential part of any successful automated testing process. They can reduce maintenance costs and testing efforts and will provide a higher return on investment (ROI) for QA teams looking to optimize their agile processes.

一个好的框架（也可以理解成一个好的工具）可以让你事半功倍

三、测试框架的收益

Improved test efficiency 提高测试效率
Lower maintenance costs 降低维护成本
Minimal manual intervention 最少的人工干预
Maximum test coverage 最大测试覆盖率
Reusability of code 代码的可重用性

四、常见测试框架类型

在这里插入图片描述

（一）TDD

1.TDD 定义

测试驱动开发（TDD）是一个软件开发过程，在软件完全开发之前，将软件需求转换为测试用例，并通过针对所有测试用例重复测试软件来跟踪所有软件开发。这与首先开发软件和稍后创建测试用例相反

Test-driven development (TDD) is a software development process relying on software requirements being converted to test cases before software is fully developed, and tracking all software development by repeatedly testing the software against all test cases.

2.TDD 流程

TDD 来源于 XP 极限编程
在这里插入图片描述
单元测试
重构
覆盖率
可测性提升
模型驱动设计

3.代表作 JUnit TestNG

    @Testvoid standardAssertions() {assertEquals(2, calculator.add(1, 1));assertEquals(4, calculator.multiply(2, 2),"The optional failure message is now the last parameter");assertTrue('a' < 'b', () -> "Assertion messages can be lazily evaluated -- "+ "to avoid constructing complex messages unnecessarily.");}

（1）TestNG

package example1;import org.testng.annotations.*;public class SimpleTest {@BeforeClasspublic void setUp() {// code that will be invoked when this test is instantiated}@Test(groups = { "fast" })public void aFastTest() {System.out.println("Fast test");}@Test(groups = { "slow" })public void aSlowTest() {System.out.println("Slow test");}}

4.代表作 Pytest UnitTest

# content of test_sample.py
def inc(x):return x + 1def test_answer():assert inc(3) == 5

import unittestclass TestStringMethods(unittest.TestCase):def test_upper(self):self.assertEqual('foo'.upper(), 'FOO')def test_isupper(self):self.assertTrue('FOO'.isupper())self.assertFalse('Foo'.isupper())def test_split(self):s = 'hello world'self.assertEqual(s.split(), ['hello', 'world'])# check that s.split fails when the separator is not a stringwith self.assertRaises(TypeError):s.split(2)

（二）BDD Behavior Driven Development

1.BDD定义

在软件工程中，行为驱动开发 (BDD) 是一种敏捷软件开发过程，它鼓励软件项目中的开发人员、质量保证专家和客户代表之间进行协作。它鼓励团队使用对话和具体示例来形成对应用程序应该如何运行的共同理解。它源于测试驱动开发 (TDD)。行为驱动开发将 TDD 的通用技术和原则与领域驱动设计和对象的思想相结合面向分析和设计，为软件开发和管理团队提供共享工具和共享流程，以便在软件开发方面进行协作。

behavior-driven development (BDD) is an agile software development process that encourages collaboration among developers, quality assurance experts, and customer representatives in a software project. It encourages teams to use conversation and concrete examples to formalize a shared understanding of how the application should behave. It emerged from test-driven development (TDD). Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development.

2.BDD VS TDD

BDD
在这里插入图片描述
BDD 与 TDD

3.BDD 相关框架

JBehave
Cucumber
Mspec
Specflow

(1)Cucumber

Cucumber is a tool that supports Behaviour-Driven Development(BDD).
在这里插入图片描述

A.Cucumber 测试用例 Scenario 场景

Scenario: Finding some cheeseGiven I am on the Google search pageWhen I search for "Cheese!"Then the page title should start with "cheese"

B.Cucumber 测试用例步骤定义

public class ExampleSteps {private final WebDriver driver = new FirefoxDriver();@Given("I am on the Google search page")public void I_visit_google() {driver.get("https://www.google.com");}@When("I search for {string}")public void search_for(String query) {WebElement element = driver.findElement(By.name("q"));// Enter something to search forelement.sendKeys(query);// Now submit the form. WebDriver will find the form for us from the elementelement.submit();}@Then("the page title should start with {string}")public void checkTitle(String titleStartsWith) {// Google's search is rendered dynamically with JavaScript// Wait for the page to load timeout after ten secondsnew WebDriverWait(driver,10L).until(new ExpectedCondition<Boolean>() {public Boolean apply(WebDriver d) {return d.getTitle().toLowerCase().startsWith(titleStartsWith);}});}@After()public void closeBrowser() {driver.quit();}
}

C.cucumber 项目结构

(三）ATDD Acceptance Test Driven Development

1.ATDD 定义

验收测试驱动开发 (ATDD) 是一种基于业务客户、开发人员和测试人员之间沟通的开发方法。ATDD 包含许多与示例规范 (SBE)、行为驱动开发 (BDD)、示例驱动开发 (EDD)、和支持驱动开发（也称为故事测试驱动开发（SDD）。所有这些流程都有助于开发人员和测试人员在实施之前了解客户的需求，并使客户能够使用他们自己的领域语言进行交流。

Acceptance test–driven development (ATDD) is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as specification by example (SBE), behavior-driven development (BDD), example-driven development (EDD), and support-driven development also called story test–driven development (SDD). All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language.

2.ATDD 相关工具

FitNesse：The fully integrated standalone wiki and acceptance testing framework
Robot Framework is a Python-based, extensible keyword-driven automation framework for acceptance testing, acceptance test driven development (ATDD), behavior driven development (BDD) and robotic process automation (RPA).

（1）RobotFramework

A.Robotframework 介绍

在这里插入图片描述
Robot Framework is a generic open source automation framework. It can be used for test automation and robotic process automation (RPA). Robot Framework has an easy syntax, utilizing human-readable keywords. Its capabilities can be extended by libraries implemented with Python, Java or many other programming languages. Robot Framework has a rich ecosystem around it, consisting of libraries and tools that are developed as separate projects.

B.Robotframework 测试用例

*** Settings ***
Documentation     Simple example using SeleniumLibrary.
Library           SeleniumLibrary*** Variables ***
${LOGIN URL}      http://localhost:7272
${BROWSER}        Chrome*** Test Cases ***
Valid LoginOpen Browser To Login PageInput Username    demoInput Password    modeSubmit CredentialsWelcome Page Should Be Open[Teardown]    Close Browser*** Keywords ***
Open Browser To Login PageOpen Browser    ${LOGIN URL}    ${BROWSER}Title Should Be    Login PageInput Username[Arguments]    ${username}Input Text    username_field    ${username}Input Password[Arguments]    ${password}Input Text    password_field    ${password}Submit CredentialsClick Button    login_buttonWelcome Page Should Be OpenTitle Should Be    Welcome Page

3.数据驱动风格

*** Settings ***
Test Template    Login with invalid credentials should fail*** Test Cases ***                USERNAME         PASSWORD
Invalid User Name                 invalid          ${VALID PASSWORD}
Invalid Password                  ${VALID USER}    invalid
Invalid User Name and Password    invalid          invalid
Empty User Name                   ${EMPTY}         ${VALID PASSWORD}
Empty Password                    ${VALID USER}    ${EMPTY}
Empty User Name and Password      ${EMPTY}         ${EMPTY}

4.BDD 风格

*** Test Cases ***
Valid LoginGiven login page is openWhen valid username and password are insertedand credentials are submittedThen welcome page should be open

5.ATDD VS BDD

在这里插入图片描述
DSL：领域特定语言（Domain - Specific Language）

（1）定义

DSL 是一种专门为特定的应用领域设计的编程语言。它的语法和语义是根据特定领域的概念和操作来定制的，与通用编程语言（如 Java、Python 等）相比，DSL 更加聚焦于解决某一特定领域的问题。

（2）在 ATDD 中的作用

在 ATDD 过程中，DSL 用于编写验收测试用例。这些验收测试用例通常以一种接近自然语言的形式来描述系统的行为和功能需求。
例如，对于一个电商系统的订单处理功能，可能会有这样的 DSL 描述：“Given 一个包含商品 A 和商品 B 的购物车，When 用户点击结算按钮，Then 系统应该生成一个包含商品 A 和商品 B 的订单，并且计算出正确的总价”。这种 DSL 的描述方式使得业务人员、测试人员和开发人员都能够比较容易地理解系统的需求和预期行为。

（3）DSL 的特点

针对性强：它专注于特定的业务领域，只提供该领域所需的功能和操作。比如，用于金融交易系统的 DSL 可能会有专门用于处理交易订单、计算利息、风险评估等操作的语法，而不会涉及到像图形渲染这样与金融交易无关的功能。
简洁性：DSL 的语法通常比较简洁，能够以简洁的方式表达复杂的业务规则。因为它不需要像通用编程语言那样考虑各种各样的编程场景，而是专注于特定领域的常见操作。
易理解性：由于其语法和词汇是基于特定领域的术语，所以对于熟悉该领域的人员（如业务分析师、领域专家等）来说，DSL 编写的测试用例很容易理解。这有助于跨团队的沟通，使得业务需求能够更准确地转化为可执行的测试和代码。

（四）MBT Model Based Testing

1.Model-Based Testing GrapheWalker

model-based testing
GraphWalker, an open-source model-based testing tool

2.edge 代表步骤

一条边代表一个动作，一个过渡。操作可以是 API 调用、按钮单击、超时等。任何将您的被测系统移动到您想要验证的新状态的任何事情。但请记住，边缘没有进行验证。这只发生在顶点。

An edge represents an action, a transition. An action could be an API call, a button click, a timeout, etc. Anything that moves your System Under Test into a new state that you want to verify. But remember, there is no verification going on in the edge. That happens only in the vertex.

3.vertex 代表断言

一个顶点代表验证，一个断言。验证是您的代码中有断言的地方。在这里，您可以验证 API 调用是否返回正确的值、按钮单击是否确实关闭了对话框，或者在应该发生超时时，被测系统触发了预期的事件。

A vertex represents verification, an assertion. A verification is where you would have assertions in your code. It is here that you verify that an API call returns the correct values, that a button click actually did close a dialog, or that when the timeout should have occurred, the System Under Test triggered the expected event.

4.graph 代表测试用例集

模型是一个图，它是一组顶点和边从模型中，GrapWalker 将生成一条通过它的路径。一个模型有一个起始元素，一个规则如何生成路径的生成器，以及告诉 GraphWalker 何时停止生成路径的相关停止条件。

A model is a graph, which is a set of vertices and edges From a model, GrapWalker will generate a path through it. A model has a start element, and a generator which rules how the path is generated, and associated stop condition which tells GraphWalker when to stop generating the path.

5.测试用例样板生成

@GraphWalker(value = "random(edge_coverage(100))")
public class OwnerInformationTest extends ExecutionContext implements OwnerInformation {private static final Logger log = LoggerFactory.getLogger(OwnerInformationTest.class);@Overridepublic void v_OwnerInformation() {$(By.tagName("h2")).shouldHave(text("Owner Information"));setAttribute("numOfPets", Value.asValue($$x("//table/tbody/tr/td//dl").size()));log.info("Number of pets: " + getAttribute("numOfPets"));}@Overridepublic void e_UpdatePet() {$("button[type=\"submit\"]").click();}@Overridepublic void v_FindOwners() {$(By.tagName("h2")).shouldHave(text("Find Owners"));$(By.tagName("h2")).shouldBe(visible);}@Overridepublic void e_EditPet() {$(By.linkText("Edit Pet")).click();}@Overridepublic void e_AddNewPet() {$(By.linkText("Add New Pet")).click();}@Overridepublic void e_AddVisit() {$(By.linkText("Add Visit")).click();}@Overridepublic void e_FindOwners() {$("[title='find owners']").click();}@Overridepublic void e_AddPetSuccessfully() {Date date = new Faker().date().past( 365 * 20, TimeUnit.DAYS);SimpleDateFormat sdf;sdf = new SimpleDateFormat("yyyy-MM-dd");String birthData = sdf.format(date);$(By.id("birthDate")).clear();$(By.id("birthDate")).sendKeys(birthData + Keys.ENTER);$(By.id("name")).clear();$(By.id("name")).sendKeys(new Faker().name().fullName());$(By.id("type")).selectOption(new Faker().number().numberBetween(0,5));$(By.cssSelector("button[type=\"submit\"]")).click();}@Overridepublic void v_NewPet() {$(By.tagName("h2")).shouldHave(text("New Pet"));$(".has-feedback").shouldBe(visible);}@Overridepublic void e_VisitAddedSuccessfully() {$(By.id("description")).clear();$(By.id("description")).sendKeys(new Faker().lorem().word());$("button[type=\"submit\"]").click();}@Overridepublic void v_NewVisit() {$(By.tagName("h2")).shouldHave(text("New Visit"));}@Overridepublic void v_Pet() {$(By.tagName("h2")).shouldHave(text("Pet"));}@Overridepublic void e_AddPetFailed() {$(By.id("name")).clear();$(By.id("birthDate")).clear();$(By.id("birthDate")).sendKeys("2015/02/05" + Keys.ENTER);$(By.id("ui-datepicker-div")).shouldBe(not(visible));$(By.id("type")).selectOption("dog");$("button[type=\"submit\"]").click();}@Overridepublic void e_VisitAddedFailed() {$(By.id("description")).clear();$("button[type=\"submit\"]").click();}
}

（五）DDT Data Driven Testing

1.DDT定义

数据驱动测试（DDT），也称为表驱动测试或参数化测试，是一种软件测试方法，用于计算机软件的测试，用于描述使用条件表直接作为测试输入和可验证输出完成的测试以及测试环境设置和控制没有硬编码的过程

Data-driven testing (DDT), also known as table-driven testing or parameterized testing, is a software testing methodology that is used in the testing of computer software to describe testing done using a table of conditions directly as test inputs and verifiable outputs as well as the process where test environment settings and control are not hard-coded.

2.DDT 相关工具

DDT 是一种实践，可以跟很多框架结合
单元测试结合 DDT：JUnit4 JUnit5 TestNG
RobotFramework DDT
YAML JSON CSV 驱动 HttpRunner

3.数据驱动应用案例

HttpRunner 可以根据代理抓包自动生成测试用例
YAPI、Swagger 等工具可以根据数据自动生成测试用例代码
JVM-Sandbox-Repeater Gor 录制工具可以把请求保存为测试用例并重放以实现快速回归测试

4.HttpRunner 测试框架

在这里插入图片描述

5.HttpRunner 测试用例

config:name: "request methods testcase with functions"variables:foo1: config_bar1foo2: config_bar2expect_foo1: config_bar1expect_foo2: config_bar2base_url: "https://postman-echo.com"verify: Falseexport: ["foo3"]teststeps:- name: get with paramsvariables:foo1: bar11foo2: bar21sum_v: "${sum_two(1, 2)}"request:method: GETurl: /getparams:foo1: $foo1foo2: $foo2sum_v: $sum_vheaders:User-Agent: HttpRunner/${get_httprunner_version()}extract:foo3: "body.args.foo2"validate:- eq: ["status_code", 200]- eq: ["body.args.foo1", "bar11"]- eq: ["body.args.sum_v", "3"]- eq: ["body.args.foo2", "bar21"]- name: post raw textvariables:foo1: "bar12"foo3: "bar32"request:method: POSTurl: /postheaders:User-Agent: HttpRunner/${get_httprunner_version()}Content-Type: "text/plain"data: "This is expected to be sent back as part of response body: $foo1-$foo2-$foo3."validate:- eq: ["status_code", 200]- eq:["body.data","This is expected to be sent back as part of response body: bar12-$expect_foo2-bar32.",]- name: post form datavariables:foo2: bar23request:method: POSTurl: /postheaders:User-Agent: HttpRunner/${get_httprunner_version()}Content-Type: "application/x-www-form-urlencoded"data: "foo1=$foo1&foo2=$foo2&foo3=$foo3"validate:- eq: ["status_code", 200]- eq: ["body.form.foo1", "$expect_foo1"]- eq: ["body.form.foo2", "bar23"]- eq: ["body.form.foo3", "bar21"]