皇宫看守

皇宫看守

Regardless of whether you’re working for a big corporation, a startup, or just for yourself, unit testing is not only helpful, but often indispensable. We use unit tests to test our code, but what happens if our tests are wrong or incomplete? What can we use to test our tests? Who watches the watchmen?

无论您是为大公司，初创公司工作，还是只为自己工作，单元测试不仅有帮助，而且通常是必不可少的。 我们使用单元测试来测试我们的代码，但是如果我们的测试错误或不完整怎么办？ 我们可以用什么来测试我们的测试？ 谁看守人？

输入变异测试 (Enter Mutation Testing)

No, no, it’s nothing like that. Mutation Testing ( or Mutant Analysis ) is a technique used to create and evaluate the quality of software tests. It consists of modifying the tests in very small ways. Each modified version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. Mutations are bugs in our original code and analysis checks if our tests detect those bugs. In a nutshell, if a test still works after it’s mutated, it’s not a good test.

不，不，不是那样的。 突变测试(或突变分析 )是一种用于创建和评估软件测试质量的技术。 它包括以很小的方式修改测试。 每个修改后的版本都称为突变体，并通过使原始版本的行为不同于突变体来测试检测和拒绝突变体。 变异是我们原始代码中的错误，如果我们的测试检测到这些错误，则分析检查。 简而言之，如果测试在变异后仍然可以工作，那么它不是一个很好的测试。

用Humbug进行变异测试 (Mutation Testing with Humbug)

Humbug is a mutation testing framework for PHP.

Humbug是PHP的变异测试框架。

In order for Humbug to be able to generate code coverage, we will have to have XDebug installed and enabled on our machine. Then, we can install it as a global tool.

为了使Humbug能够生成代码覆盖率，我们必须在计算机上安装并启用XDebug。 然后，我们可以将其安装为全局工具。

composer global require 'humbug/humbug'

After this, if we run the

之后，如果我们运行

humbug

command, we should be able to see some of our Humbug installation information and an error indicating that we don’t have a humbug.json file.

命令，我们应该能够看到一些Humbug安装信息和一个错误，表明我们没有humbug.json文件。

自举 (Bootstrapping)

Before we configure and use Humbug, we need a project that we can test. We will create a small PHP calculator package where we will run our unit and mutation tests.

在配置和使用Humbug之前，我们需要一个可以测试的项目。 我们将创建一个小PHP计算器包，在其中运行单元测试和变异测试。

Let’s create a /Calculator folder. Inside it, let’s create our /src and /tests folders. Inside our /src folder, we will have our application code; the /tests folder will contain our unit tests. We will also need to use PHPUnit in our package. The best way to do that is using Composer. Let’s install PHPUnit using the following command:

让我们创建一个/Calculator文件夹。 在其中，我们创建/src和/tests文件夹。 在我们的/src文件夹中，我们将拥有我们的应用程序代码； /tests文件夹将包含我们的单元测试。 我们还需要在包中使用PHPUnit。 最好的方法是使用Composer 。 让我们使用以下命令安装PHPUnit：

composer global require phpunit/phpunit

Let’s create our Calculator. Inside the /src folder, create a Calculator.php file and add the following content:

让我们创建计算器。 在/src文件夹中，创建Calculator.php文件并添加以下内容：

<?php namespace package\Calculator; class Calculator { /** * BASIC OPERATIONS */ public function add($a1, $a2) { return $a1 + $a2; } public function subtract($a1, $a2) { return $a1 - $a2; } public function multiply($a1, $a2) { return $a1 * $a2; } public function divide($a1, $a2) { if ($a2 === 0) { return false; } return $a1 / $a2; } /* * PERCENTAGE */ //This will return $a1 percent of $a2 public function percentage($a1, $a2) { return ( $a1 / $a2 ) * 100; } /* * PI */ //Returns the value of pi public function pi() { return pi(); } /* * LOGARITHMIC */ //Returns the basic logarithm in base 10 public function log($a) { return log10($a); } }

It is a rather straightforward program. A simple calculator, with the basic arithmetic, percentage and logarithmic operations and a function to return the value of pi. Next, inside our /tests folder, let’s create the unit tests for our calculator. If you need help with unit testing in PHP, check out this tutorial.

这是一个相当简单的程序。 一个简单的计算器，具有基本的算术，百分比和对数运算以及返回pi值的函数。 接下来，在我们的/tests文件夹中，让我们为计算器创建单元测试。 如果您需要有关PHP单元测试的帮助，请查看本教程 。

Create a CalculatorTest.php file and add the following:

创建一个CalculatorTest.php文件并添加以下内容：

<?php use package\Calculator\Calculator; class CalculatorTest extends PHPUnit_Framework_TestCase { public function testAdd() { $calculator = new Calculator(); $result = $calculator->add(2, 3); $this->assertEquals($result, 5); } public function testSubtract() { $calculator = new Calculator(); $result = $calculator->subtract(6, 3); $this->assertEquals($result, 3); } public function testMultiply() { $calculator = new Calculator(); $result = $calculator->multiply(6, 3); $this->assertEquals($result, 18); } public function testDivide() { $calculator = new Calculator(); $result = $calculator->divide(6, 3); $this->assertEquals($result, 2); } }

This will be our initial test stack. If we run the phpunit, command we will see that it executes successfully, and our 4 tests and 4 assertions will pass. It is important that all of our tests are passing, otherwise, Humbug will fail.

这将是我们的初始测试堆栈。 如果我们运行phpunit ，命令将看到它成功执行，并且我们的4个测试和4个断言将通过。 重要的是，我们所有的测试都必须通过，否则Humbug将失败。

配置Humbug (Configuring Humbug)

Humbug may either be configured manually, by creating a humbug.json.dist file, or automatically, by running the command:

Humbug可以通过创建humbug.json.dist文件手动配置，也可以通过运行以下命令自动配置：

humbug configure

Running the command will ask us for answers to some questions:

运行命令将要求我们回答以下问题：

What source directories do you want to include?

您要包括哪些源目录？

In this one we will go with src/, the directory of our source code.

在这一节中，我们将使用源代码目录src /。

Any directories you want to exclude from within your source directory?

您要从源目录中排除的目录吗？

May be useful in some cases, like an external vendor directory that we don’t want tested. It does not apply in our current case.

在某些情况下可能有用，例如我们不想测试的外部供应商目录。 它不适用于我们当前的情况。

Single test suite timeout in seconds.

单个测试套件超时(以秒为单位)。

Let’s go with 30 seconds on this one. It is probably too much, but we want to be sure everything has had enough time to run.

让我们花30秒处理这个。 可能太多了，但是我们要确保所有内容都有足够的时间运行。

Where do you want to store your text log?

您想在哪里存储文本日志？

humblog.txt comes as default and we will leave it as that.

humblog.txt是默认值，我们将其保留为默认值。

Where do you want to store your json log (if you need it)?

您想在哪里存储json日志(如果需要)？

The default comes empty but we will store it in humblogjson.json.

默认为空，但我们会将其存储在humblogjson.json 。

Generate “humblog.json.dist”?

生成“ humblog.json.dist”？

This file will, when generated, contain all the configuration values we just supplied. We can edit it manually if we want to change something.

该文件在生成后将包含我们刚刚提供的所有配置值。 如果要更改某些内容，可以手动编辑。

使用Humbug (Using Humbug)

Now that we have both our application running with tests and Humbug installed, let’s run Humbug and check the results.

现在我们已经运行了测试应用程序并安装了Humbug，现在让我们运行Humbug并检查结果。

humbug

The result should be close to this:

结果应该接近于此：

解释Humbug结果 (Interpreting Humbug results)

The number of mutations created is just the number of small changes introduced by Humbug to test our tests.

产生的突变数就是Humbug为测试测试而引入的小变化的数目。

A killed mutant (.) is a mutation that caused a test to fail. Don’t be confused, this is a positive result!

被杀死的突变体(。)是导致测试失败的突变。 不要混淆，这是一个积极的结果！

An escaped mutation (M) is a mutation where the test still passed. This is not a positive result, we should go back to our test and check what’s missing.

逃脱的突变(M)是测试仍通过的突变。 这不是一个积极的结果，我们应该返回测试并检查缺少的内容。

An uncovered mutation (S) is a mutation that occurs in a line not covered by a unit test.

未发现的突变(S)是发生在单元测试未涵盖的行中的突变。

Fatal errors (E) and timeouts (T) are mutations that created fatal errors and mutations that create infinite loops, respectively.

致命错误(E)和超时(T)分别是造成致命错误的突变和造成无限循环的突变。

指标如何？ (What about the metrics?)

The Mutation Score Indicator indicates the percentage of generated mutations that were detected. We want to aim at 100%.

突变得分指示器指示检测到的已产生突变的百分比。 我们希望瞄准100％。

Mutation Code Coverage indicates the percentage of tests covered by mutations.

突变代码覆盖率表示突变覆盖的测试百分比。

The Mutation Score Indicator gives you some idea of how effective the tests that do exist really are.

变异分数指示器可让您了解确实存在的测试的有效性。

Analyzing our humbug log, we can see that we have 9 mutants not covered, and some really bad metrics. Take a look at the humblogjson.json file. This file was generated automatically just like the humblog.txt file, and contains much more detailed information on what failed, where and why. We haven’t tested our percentage, pi and logarithm functions. Also, we need to cover the case where we divide a number by 0. Let’s add some more tests to cover the missing situations:

通过分析humbug日志，我们可以看到我们有9个未涵盖的突变体，以及一些非常糟糕的指标。 看一下humblogjson.json文件。 该文件是自动生成的，就像humblog.txt文件一样，并且包含有关失败原因，原因和原因的详细信息。 我们尚未测试百分比，pi和对数函数。 另外，我们需要涵盖将数字除以0的情况。让我们添加更多测试以覆盖丢失的情况：

public function testDivideByZero() { $calculator = new Calculator(); $result = $calculator->divide(6, 0); $this->assertFalse($result); } public function testPercentage() { $calculator = new Calculator(); $result = $calculator->percentage(2, 50); $this->assertEquals($result, 4); } public function testPi() { $calculator = new Calculator(); $result = $calculator->pi(); $this->assertEquals($result, pi()); } public function testLog() { $calculator = new Calculator(); $result = $calculator->log(10); $this->assertEquals($result, 1); }

This time around, 100% means that all mutations were killed and that we have full code coverage.

这次大约100％表示所有突变均被杀死，并且我们具有完整的代码覆盖率。

缺点 (Downsides)

The biggest downside of mutation testing, and by extension Humbug, is performance. Mutation testing is a slow process as it depends on a lot of factors like interplay between lines of code, number of tests, level of code coverage, and the performance of both code and tests. Humbug also does initial test runs, logging and code coverage, which add to the total duration.

性能测试是突变测试(从扩展到Humbug)的最大缺点。 变异测试是一个缓慢的过程，因为它取决于许多因素，例如代码行之间的相互作用，测试数量，代码覆盖级别以及代码和测试的性能。 Humbug还会进行初始测试运行，日志记录和代码覆盖，这会增加总持续时间。

Additionally, Humbug is PHPUnit specific, which can be a problem for those who are using other testing frameworks.

此外，Humbug是特定于PHPUnit的，对于使用其他测试框架的人来说可能是个问题。

That said, Humbug is under active development and will continue to improve.

也就是说，Humbug正在积极开发中，并将继续改进。

结论 (Conclusion)

Humbug can be an important tool for maintaining your app’s longevity. As the complexity of your app increases, so does the complexity of your tests – and having them all at 100% all the time becomes incredibly important, particularly when dealing with enterprise ecosystems.

Humbug可能是维持应用程序寿命的重要工具。 随着应用程序复杂性的增加，测试的复杂性也随之增加–始终保持所有测试都处于100％的状态变得异常重要，尤其是在处理企业生态系统时。

The code we used in this tutorial can be cloned here.

我们在本教程中使用的代码可以在此处克隆。

Have you used Humbug? Do you do mutation testing another way? Give us your thoughts on all this!

您使用过Humbug吗？ 您是否以其他方式进行突变测试？ 将您的想法告诉我们！

翻译自: https://www.sitepoint.com/testing-your-tests-who-watches-the-watchmen/

皇宫看守

相关资源：jdk-8u281-windows-x64.exe