3 lenses for evaluating product quality and building better products

Product quality: what does it mean?

Have you held a device, maybe the latest Apple or Google phone, and commented, "now this feels like quality"?

Why did you feel that way? Was it the mass of the phone? The thoughtfulness that went into the way the buttons are machined? The picture of the screen?

Looking at the dictionary, it's clear that the term "quality" has many definitions. Merriam-Webster's definition of quality has a few good explanations: "degree of excellence", "superiority in kind", and "peculiar and essential character" all resonate with me, but they don't do a great job of demonstrating what "quality" is.

There is some subjectivity on what "quality" means to everyone, but below I highlight three examples that have helped shape the lens I use to evaluate quality. Understanding them will help you as your startup builds its products.

‍

1. Setting users up for enjoyment, not frustration

Quality is about setting up the user for success. Quality software gets out of the way and enables things to be done without frustration, or perhaps even with some enjoyment, much like the products of manufacturers like Apple, Sony, and Toyota.

As a teenager and early twenty-something, I developed an appreciation for products that felt like they were built with extra care, attention, and consistency—even at scale—selling Yamaha electric guitars. Selling them was one of the easiest things I’ve done, with the Yamaha Pacifica being superior to other, comparably priced offerings for first-time electric guitar buyers.

My sales pitch was pretty slim: I'd put it in their hand and say, "Let's set you up for success: if you buy a good instrument, you'll keep playing, and keep coming back. Yamaha products are backed by a lifetime warranty, and I'll take care of it for free. Selfishly, I want you to come back and buy more strings."

This worked because my boss made the smart call to partner with Yamaha, and because the company is focused on quality at the highest level. Yamaha sets quality standards, has quality management processes and groups in place, and they audit the quality of those processes routinely—efforts that are inspiring to me.

‍

2. Creating consistency across users and experiences

Nintendo first added the Seal of Quality to its products in America, but it was later added to releases in Europe and Japan, and the "Official Nintendo Seal of Quality" became the "Official Nintendo Seal.” For each and every Nintendo platform-published title to earn the Official Seal, it had to be approved by Nintendo’s president Hiroshi Yamauchi. This user-centric quality policy was a big part of the success of the NES.

In late 2006, the Nintendo Wii came out and I'd just entered the quality assurance field. Back then, most video games didn't get updates after they were released on a disc or cartridge; releasing an "updated" version of something meant having to order a large number of cartridges or discs, then getting them to distribution partners. Ensuring that software was right before going to market was critical—not only did a game have to be reasonably free of gameplay defects, but it had to meet Nintendo's standards to ensure consistent experiences for users across all games.

Another requirement for games to earn the Official Seal, was the need to adhere to a strict set of rules. Every title on a Nintendo system must behave the same way when "Home" is pressed, when a disc is ejected, or when the system memory is full.

For example, when the Wii starts up a game, 1-2 seconds pass on a white screen, and then a Wrist Strap warning displays. After that, there about two-dozen possible combinations of pressing Eject, Home, and Power, swapping discs, and controller presses. To be Nintendo-certified, all games must handle these player behaviors gracefully and consistently. Microsoft and Sony have similar guidelines and requirements, but only Nintendo has the Seal.

‍

3. Getting feedback, learning from it, and iterating

Building quality software looks different today than it did even just a couple of years ago. Quality is now a more iterative process than ever. Today, shipping new features has become a cycle of building, learning what works, gathering feedback, refactoring what didn't work, and repeating. This cycle is the Quality Assurance process we follow at Cloudburst.

Software is made and used by people. Sometimes developers make mistakes, and sometimes users of the software we build make surprising choices. To learn quickly and stay ahead of bugs, Cloudburst has Quality Assurance testers attempt to break the things we build so we’re the ones discovering issues, not you. We also use technologies like performance and error monitoring tool Sentry to better understand the issues people might experience when using our products.

When we do find an issue, we look for its root cause, address it, and learn from it. We believe it’s worth it to invest in tools and processes that help us prevent those issues from happening the next time: we package our code in reusable containers to reduce developer errors, automate browser testing with Playwright, and use GitHub actions to apply individual learnings across projects and drive consistency across deployments.

‍

My first mentor in quality (John Brooks, QA Manager at Bossfight Entertainment, who was just acquired by Netflix), taught me early on that the role of quality in software development is "the last line of defense for the company, and the first line of offense for customers." That philosophy is what we experience when we use the products of companies like Nintendo, Yamaha, Apple, and Sony, and it’s the value we keep in mind when designing and developing products at Cloudburst.