ci/cd自动化测试
This article was sponsored by Microsoft Visual Studio App Center. Thank you for supporting the partners who make SitePoint possible.
本文由Microsoft Visual Studio App Center赞助。 感谢您支持使SitePoint成为可能的合作伙伴。
What’s the best part about developing software? Writing amazing code.
开发软件的最好之处是什么？ 编写惊人的代码。
What’s the worst part? Everything else.
最糟糕的部分是什么？ 其他一切。
Developing software is a wonderful job. You get to solve problems in new ways, delight users, and see something you built making lives better. But for all the hours we spend writing code, there are often just as many spent managing the overhead that comes along with it—and it’s all a big waste of time. Here are some of the biggest productivity sinkholes, and how we at Microsoft are trying to scrape back some of that time for you.
开发软件是一项了不起的工作。 您可以用新的方式解决问题，使用户满意，并看到自己制作的东西可以使生活更美好。 但是，在我们花费在编写代码上的所有时间中，经常有很多人花在管理随之而来的开销上，而这一切都是浪费时间。 这里是一些最大的生产力漏洞，以及我们在Microsoft如何设法为您节省一些时间。
What’s the first step to getting your awesome app in the hands of happy users? Making it exist. Some may think moving from source code to binary wouldn’t still be such a pain, but it is. Depending on the project, you might compile several times a day, on different platforms, and all that waiting is time you could have spent coding. Plus, if you’re building iOS apps, you need a Mac build agent—not necessarily your primary development tool, particularly if you’re building apps in a cross-platform framework.
要让出色的应用程序交到快乐的用户手中，第一步是什么？ 使它存在。 有些人可能认为从源代码转换为二进制仍然不是那么痛苦，但是确实如此。 根据项目的不同，您可能一天在不同的平台上进行几次编译，而等待的全部时间就是您本可以花在编码上的时间。 另外，如果要构建iOS应用，则需要Mac构建代理，而不必是主要的开发工具，尤其是在跨平台框架中构建应用时。
You want to claim back that time, and the best way to do that is (it won’t be the last time I say this) automation. You need to automate away the configuration and hardware management so the apps just build when they’re supposed to.
您想找回那个时间，而做到这一点的最佳方法是(这不是我最后一次说到) 自动化 。 您需要自动执行配置和硬件管理，以便仅在应有的时间构建应用程序。
Our attempt to answer that need is Visual Studio App Center Build, a service that automates all the steps you don’t want to reproduce manually, so you can build every time you check in code, or any time you, your QA team, or your release managers want to. Just point Build at a Github, Bitbucket, or VSTS repo, pick a branch, configure a few parameters, and you’re building Android, UWP, and even iOS and macOS apps in the cloud, without managing any hardware. And if you need to do something special, you can add post-clone, pre-build, and post-build scripts to customize.
我们试图满足这一需求的尝试是Visual Studio App Center Build，该服务可自动执行您不想手动复制的所有步骤，因此您可以在每次检入代码时或您，质量保证团队或任何时候进行构建。您的发布经理想要的。 只需将Build指向Github，Bitbucket或VSTS仓库，选择一个分支，配置一些参数，即可在云中构建Android，UWP甚至iOS和macOS应用程序，而无需管理任何硬件。 而且，如果您需要做一些特别的事情，则可以添加克隆后，构建前和构建后脚本进行自定义。
I’ve spent many years testing software, and throughout my career, there were three questions I always hated hearing:
我已经花了很多年测试软件，并且在我的整个职业生涯中，我一直讨厌听到三个问题：
“Are you done yet?”
“你做完了吗？”
“Can you reproduce it?”
“你能复制吗？”
“Is it really that bad?”
“真的那么糟糕吗？”
In the past, there’s rarely been enough time or resources for thorough, proper testing, but mobile development has exacerbated that particular problem. We now deliver more code, more frequently to more devices. We can’t waste hours trying to recreate that elusive critical failure, and we don’t have time to argue over whether a bug is a showstopper. At the same time, we’re the gatekeepers who are ultimately responsible for a high-visibility failure or a poor-quality product, and as members of a team, we want to get ahead of problems to increase quality, rather than just standing in the way of shipping.
过去，很少有足够的时间或资源来进行全面，适当的测试，但是移动开发加剧了该特定问题。 现在，我们更频繁地向更多设备提供更多代码。 我们不能浪费时间试图重现这种难以捉摸的严重失败，而且我们没有时间争论错误是否是危险。 同时，我们是最终导致高可见度故障或劣质产品负责的守门人，作为团队的成员，我们要超越问题来提高质量，而不仅仅是站在运输方式。
So what’s the answer? “Automation,” sure. But automation that makes sense. Spreadsheets of data and folders of screenshots mean nothing if you can’t put it all together. When you’re up against a deadline and have to convince product owners to make a call, you need to deliver information they can understand, while still giving devs the detail they need to make the fix.
那么答案是什么呢？ 当然是“自动化”。 但是自动化是有意义的 。 如果不能将所有数据放在一起，则数据电子表格和屏幕快照文件夹毫无意义。 当您无法按时完成任务并且不得不说服产品所有者打电话时，您需要提供他们可以理解的信息，同时仍然为开发人员提供他们进行修复所需的详细信息。
To help with that, we’ve created App Center Test, a service that performs automated UI tests on hundreds of configurations across thousands of real devices. Since the tests are automated, you run exactly the same test every time, so you can identify performance and UX deviations right away, with every build. Tests produce screenshots or videos alongside performance data, so anyone can spot issues, and devs can click down into the detailed logs and start fixing right away. You can spot-check your code by testing on a few devices with every commit, then run regressions on hundreds of devices to verify that everything works for all your users.
为了解决这个问题，我们创建了App Center Test，该服务可在数千个真实设备上的数百种配置上执行自动UI测试。 由于测试是自动化的，因此您每次都运行完全相同的测试，因此您可以在每次构建时立即确定性能和UX偏差。 测试会生成截图或视频以及性能数据，因此任何人都可以发现问题，开发人员可以单击进入详细的日志并立即开始修复。 您可以通过每次提交在几台设备上进行测试来抽查代码，然后在数百台设备上运行回归以验证一切对所有用户都有效。
So you’ve built an app and it’s performing as it should. Great! But now the iteration really begins. You want to know what people think of it before you push it to end-users. But how? Putting together a beta program is hard enough, but making sure everyone has the most recent version of your app (and if it’s a mobile app, making sure your users can even install the app) is a full-time job—and it’s a job nobody on your team wants.
因此，您已经构建了一个应用程序，并且该应用程序按预期运行。 大！ 但是现在迭代真正开始了。 在将其推向最终用户之前，您想知道人们对此的看法。 但是如何？ 组合一个Beta程序非常困难，但是确保每个人都拥有您应用程序的最新版本(如果是移动应用程序，请确保您的用户甚至可以安装该应用程序)是一项全职工作，这是一项工作您团队中没有人想要。
Once again, automation. When you’re ready to push a build, you need to automate the notification process and the app distribution process, and you need to be able to trigger both every time you build (or every time the release manager says so).
再次， 自动化 。 当您准备好进行构建时，您需要使通知过程和应用程序分发过程自动化，并且您必须能够在每次构建时(或每次发布管理器这样说时)触发。
Our answer is App Center’s Distribute service. If you have a list of email addresses, you have what you need to get your builds in the hands of internal or beta testers. Create a distribution group, upload a build (or build one from a repo), and Distribute handles the rest. If you think this sounds like HockeyApp, you’re right. App Center Distribute is the next generation of HockeyApp, integrating its distribution automation with the rest of our CI/CD services. And once you’re done with beta testing, Distribute can also get your app into your users’ hands, with deployment to Google Play, Apple’s App Store, or—for enterprise users—Microsoft Intune.
我们的答案是App Center的分发服务。 如果您有电子邮件地址列表，则需要具备将内部或Beta测试人员手中的构建文件所需的功能。 创建一个通讯组，上载一个构建(或从一个仓库中构建一个构建)，然后由Distribute处理其余的构建。 如果您认为这听起来像HockeyApp ，那么您是对的。 App Center Distribute是HockeyApp的下一代产品，将其分发自动化与其余的CI / CD服务集成在一起。 一旦完成Beta测试，Distribute还可以将您的应用程序交付用户，部署到Google Play，Apple的App Store或(对于企业用户)Microsoft Intune。
People often talk about deployment pipelines, but we’re not just after a one-way push. If you can learn what’s happening after your apps have shipped, you can take that feedback to developers and create a closed loop to make your products better, faster. That information takes two forms—analysis of how users interact with your apps, and critically, reporting on how and when those apps fail.
人们经常谈论部署管道，但我们不仅是单向推动。 如果您可以了解应用程序交付后的情况，则可以将这些反馈信息反馈给开发人员，并创建一个闭环以使您的产品更快更好地发展。 该信息有两种形式-分析用户与您的应用程序的交互方式，以及至关重要的是，报告这些应用程序如何以及何时发生故障。
Let’s start with the second, because crashing is about as bad as it gets. When an app fails, you want to know about it fast, but you also need to know how much it really matters. A crash in an obscure feature that affects everyone is usually worse than a total launch failure on just the iPhone 4. App Center Crashes groups similar crash reports and shows you the most affected platforms so you can make intelligent triage decisions. And when you’re ready to start fixing the issues, the crashes are fully symbolicated so you have the information you need to get started. You can automatically create entries in your bug tracker, so developers can start fixing issues without leaving their workflow. Again, more automation means more time writing better code.
让我们从第二个开始，因为崩溃几乎和它一样糟糕。 当应用失败时，您想快速了解它，但是您还需要知道它到底有多重要。 影响到每个人的晦涩功能崩溃通常比仅iPhone 4上的完全启动失败更糟糕。AppCenter Crashes将类似的崩溃报告分组，并向您显示受影响最大的平台，以便您可以做出明智的分类决策。 而且，当您准备好开始解决问题时，崩溃已完全被象征出来，因此您拥有入门所需的信息。 您可以在错误跟踪器中自动创建条目，以便开发人员无需离开工作流程即可开始解决问题。 同样，更多的自动化意味着更多的时间编写更好的代码。
For analytics, you want something useful out-of-the-box. App Center Analytics provides the kind of engagement-focused user- and device-level metrics app that owners want to see; things like who’s using which devices, how often, from where, and how long they’re staying. But your app isn’t the same as everyone else’s, so we let you create and track custom metrics, like “booked a ride” or “ordered home delivery.” And if you want deeper analysis, we enable continuous export to Azure Application Insights.
对于分析，您需要一些有用的现成的东西。 App Center Analytics提供了所有者希望查看的以参与度为重点的用户和设备级别的指标应用程序； 诸如谁在使用哪些设备，使用频率，从何处以及停留多长时间等信息。 但是您的应用不同于其他所有人的应用，因此我们让您创建和跟踪自定义指标，例如“预定行程”或“订购的送货上门”。 而且，如果您想进行更深入的分析，我们可以连续导出到Azure Application Insights 。
You can theorycraft the perfect CI/CD solution all day long, but it’s all useless if you can’t put it into action. What matters is getting something you can use now, whether that means integrating with existing systems you really love (or can’t get rid of), or just automating pieces of a manual process until you can get to the rest. It’s always better to make even a small step, as long as it’s in the right direction.
您可以整日理论上设计完美的CI / CD解决方案，但是如果您不能付诸实践，那将毫无用处。 重要的是要获得现在可以使用的东西，这是否意味着要与您真正喜欢(或无法摆脱)的现有系统集成，或者只是自动化手动过程的各个部分，直到可以完成其余部分。 只要朝着正确的方向迈出一小步，总会更好。
Obviously, I’m biased and think you should give our whole system a try, but developers need different things. If you just want to adopt pieces of App Center, we’ve built it to be completely modular. We have REST APIs for every App Center service, and we’ve pre-built integration with services like VSTS. And that’s the way it should be, because you’re building your app, so you should build it your way.
显然，我有偏见，认为您应该尝试一下整个系统，但是开发人员需要不同的东西。 如果您只想采用App Center，我们已经将其构建为完全模块化的。 我们为每个App Center服务都有REST API，并且已经与VSTS等服务进行了预先构建的集成。 这就是今天的事情应该是这样，因为你建立你的应用程序，所以你应该建立它自己的方式。
We’d love to have you try Visual Studio App Center—it’s brand new today and free to get started. We want to hear what you think!
我们希望您尝试使用Visual Studio App Center-它是今天全新的，可以免费开始使用。 我们想听听您的想法！
翻译自: https://www.sitepoint.com/automate-cicd-visual-app-center/
ci/cd自动化测试
