VMblog:  Remind us who Sauce Labs is for those who might be new to the company?

Alissa Lydon:  Sauce Labs is the leading provider of continuous testing solutions that deliver digital confidence. For 12 years now, Sauce Labs has worked hand-in-hand with some of the most well-known brands in the world to improve the quality of their user experience by ensuring that their web and mobile applications look, function, and perform exactly as they should on every browser, OS, and device, every single time. To date, nearly 3.5 billion tests have been run on the Sauce Labs Continuous Testing Cloud, and we continue to help enterprise organizations across all verticals grow their digital businesses by creating new routes to market, protecting their brands from the risks of a poor user experience, and delivering better products to market, faster. 

VMblog:  Given the way the pandemic has changed the business landscape, is it fair to say that virtually every business is now a digital business?

Lydon:  Absolutely, although the reality is we were already heading steadily in that direction well before the pandemic. If we look back at the numbers from Black Friday 2019, we see that in-store sales fell 6.2% compared with 2018 while online sales grew 14% over the same period. We also see that 56% of those digital orders came directly from smartphones. So, companies were already becoming increasingly dependent on web and mobile applications to serve as the primary platforms through which they engage customers, and the pandemic has accelerated that trend even further.

What it all means for modern businesses is that the application is now your core value proposition, and the quality of your application experience is what will ultimately make or break your brand. That was true before the health crisis and it will continue to be the case long after its over, and that's why we think digital confidence is such an important concept.

VMblog:  Let's talk more about that.  Can you explain what you mean when you refer to digital confidence?

Lydon:  Let's start with the premise, again, that the application is now the primary vehicle - if not the only vehicle - through which you can deliver products and services to your customers. And let's also remember that what's true for you is probably true for your competitors. That means we're in a world now where simply having a website and/or a mobile application isn't enough. There's no differentiation there. The differentiation is in the experience. The differentiation is in treating your customers to an application experience that's nothing short of flawless. And that's really what digital confidence is all about.

Digital confidence means knowing that you're delivering the best possible experience to your customers. It means knowing that your apps work exactly as intended before your customers ever interact with them. And, most importantly, it means knowing that your customers share in that confidence; that they likewise have no doubt that your web and mobile applications will work flawlessly and deliver a delightful experience every single time they use them, with no exceptions. When you have digital confidence, you can move quickly to release new and updated applications to market, and your customers will move just as quickly and confidently to use them.  

VMblog:  Drilling down a bit deeper, how has this macro-level change in the way business gets done manifested itself in terms of changes to the way software gets developed?

Lydon:  Certainly, the biggest shift has been the continued move away from the traditional approach in which testing is conducted entirely at the end of the development process and in which a dedicated QA team shoulders most if not all of the responsibility for quality. That simply doesn't work in the digital era and most development teams have realized that by now.

One way we've already seen that change manifest itself is in the adoption of shift-left testing, in which testing is implemented continuously throughout the entire development process, including and especially the early stages. By shifting left, organizations have been able to discover bugs earlier and increase release velocity by avoiding the long and costly hold-ups that occur when bugs are uncovered at the end of the development process.

But shifting testing left is just the first phase of this evolution. The next phase involves developers taking more direct ownership of testing and shouldering more of the responsibility for code quality, especially in those earlier stages of development. I expect this to be one of the dominant themes for development teams in the coming year.

VMblog:  Asking developers to take more direct ownership of testing seems like a big ask.  What are some things teams can do to help facilitate that?

Lydon:  You're absolutely right about that. It is a big ask. We have to remember that developers aren't testers, and that's by choice. They signed up for coding, innovating, and building great technology. They didn't sign up for testing. So, the way I see it, the onus is on us - testers, quality engineers, product managers, etc. - to bridge the gap and make it as easy as possible for developers to get more involved with testing.

So, how do we do that? Well, we're already asking them to step outside their comfort zones. Asking them to do so without the tools with which they're comfortable would be a double whammy. So the most important thing is to make sure we're providing them with their preferred tools. Those tools are often different from the tools traditionally used by testers and QA teams. That might make us a little uncomfortable as testers - after all, now we're exiting our comfort zone - but that's the price of doing business here. If we want developers to take greater ownership of testing, we have to meet them where they are with the tools they want to use. And that means we have to be willing to augment our investments in Selenium, Appium, and WebDriverIO with investments in the native testing frameworks that developers often prefer, such as Cypress, Playwright, Puppeteer, and TestCafe.

VMblog:  Let's shift gears a bit.  How does your company or product fit within the container ecosystem?

Lydon:  Containers continue to play an increasingly important role in facilitating new approaches to testing, and in fact, containers play a big role in one of the newest product innovations we're working on, something we call the Sauce Testrunner Toolkit. It's an open source containerized solution that simplifies the setup of end-to-end web tests, speeds execution times, and unifies test results for better analysis and debugging. We just talked about the importance of enabling developers to leverage their tools of choice. This solution does just that, enabling developers to set up local and CI/CD testing using their open-source automation tool of choice, and then unify their test signals in Sauce Labs. We're still in the early days with it but are excited about its potential.

VMblog:  Were there any other new releases or announcements of note since we last spoke with Sauce Labs?

Lydon:  Two that I'd like to point out. The first is Failure Analysis, a machine learning-based solution that's geared toward helping organizations improve test quality. Test quality is a huge challenge right now. In the annual Sauce Labs Continuous Testing Benchmark Report that we released back in April, we found that less than 25 percent of organizations pass at least 90 percent of their desktop tests. That needs to improve. What Failure Analysis does is apply sophisticated machine learning to customers' pass/fail data to surface the most common reasons a given set of tests fail. Most test failures can be traced to a small number of underlying issues (what we call failure patterns) impacting a test suite. If you can uncover and fix those patterns, you can dramatically improve your test quality.

The other initiative I'd like to point out is the formation of our first-ever Open Source Program Office under the leadership of our CTO, John Kelly. In addition to establishing and executing our open source strategy for our solution, this team is working directly with leaders of open source projects both established and emergent to drive increased collaboration across all of Sauce Labs. Let's face it, without the open source community, none of what we and many others in the tech industry do would be possible. It's our responsibility to invest in that community and this is a major step in that direction.

VMblog:  Finally, without a crystal ball, what do you think trade shows look like in 2021?  Do we go back to thousands of people in person at an event?  Or do things stay virtual for the near term?  Is your company prepared to sponsor a physical event next year should they return?

Lydon:  I've learned not to make predictions when it comes to the pandemic. I'd love nothing more than to be back out there attending and exhibiting at trade shows at some point in 2021. Whether that proves realistic or not, I think that remains to be seen. What I will say is I've been so inspired by how quickly and seamlessly the amazing events teams throughout the tech industry - and we have one of the very best here at Sauce Labs - have been able to adjust and move their shows online. We had just a few weeks to turn SauceCon 2020 into SauceCon Online 2020, and we wound up with more than 5,000 registrants and a truly fantastic event. I've had the pleasure of virtually attending many other events with similar stories and I have no doubt KubeCon will be the same.