Efficient product testing lets you build something people want
How Snap and Microsoft learnt their lesson:
In 2017, Snapchat released a new version of its app that was designed to simplify its interface and improve the user experience. However, the redesign was met with widespread criticism from users, who found it confusing and difficult to use. The new interface was also missing some popular features, such as the "best friends" feature, which allowed users to see the people they interacted with the most on the app. As a result, Snapchat lost millions of daily active users, and its stock price dropped by more than 7%.
To address the backlash, Snapchat listened to user feedback and made several changes to the app's design. In particular, it brought back some of the missing features and made it easier for users to find their friends and stories. The company also announced that it would be more responsive to user feedback in the future and work to ensure that its updates are more thoroughly tested before release.
Similarly, in 2012, Microsoft released its Windows 8 operating system, which was designed to be a major overhaul of its flagship product. However, the new interface, which was optimized for touchscreens, was confusing and difficult to use for many users who still used traditional desktops and laptops. Microsoft had not conducted sufficient user testing before the release, which led to a drop in sales of PCs and a decline in consumer satisfaction with the Windows brand.
To address these issues, Microsoft made several changes to Windows 8, including the reintroduction of the traditional Start menu and a greater focus on desktop users in subsequent updates. The company also worked to improve its user testing processes to ensure that future releases of Windows would be more user-friendly and better meet the needs of its customers.
User testing involves gathering feedback from users on a product or service before its release to the market. The insights gathered from this process can help businesses make necessary improvements and adjustments to enhance the user experience.
There are many different types of product testing that can be beneficial before launching a new product.
Here are four common types of product testing that can help ensure a successful launch:
Discovery research
Prototype testing
Performance testing
User experience testing
1) Discovery: Let your users tell you what they want
This type of testing involves surveying potential customers to gauge their interest in and willingness to purchase the product. This can help identify potential customer segments and their needs, as well as any potential barriers to purchase. Some of the ways you can pull this off is through:
Focus group sessions with specific segments of your users,
Going through user feedback to find out what their pain-points are,
Looking at what the competitive market is offering and how you can leverage your areas of strength to provide better service to your users.
Unless you're looking to build the first car as an upgrade from using horses, your users will tell you want they want; perhaps not as eloquently, but vocally all the same. Read this if you'd like to learn more about how to approach discovery as a PM.
2) Prototype testing
Before launching the final product, it is important to create a prototype and test it with a small group of customers or potential customers. This can help identify any design or functionality issues, and allow for necessary adjustments to be made.
Prototype testing can be a bit go a hustle. For software products, you're technically asking a bunch of your users to sit through a simulation of what the final product will look like and then answer a bunch of qualitative questions. The hardest part about this process isn't about the exercise itself, it's getting users in the room in the first place.
You will need to properly incentivise them (whatever that looks like), get them to show up hopefully excited and engaged, share with them a link to the prototype which sometimes is just hosted on Figma preview and tease out honest responses from them. People will NOT want to 'offend' you so they will try their best to dilly-dally around something they think is frankly s*it.
You can do this online, in-person, through a WhatsApp call or phone call; whatever gets you user feedback.
Don't panic, I've learnt how to do this swimmingly well and I created an end-to-end playbook you might appreciate. You can find it here.
3) Performance testing
You want to know that your product works as intended on the devices and within the context your users will typically use it in. This is why we have alpha testing exercises.
If a majority of your users are in a location with minimal internet connection, or use devices with limited storage or operating capacity, you want to make sure that they can seamlessly use what you've built for them or they will get frustrated and move on to their own devices (*wink wink, get the pun?)
This type of testing involves putting the product through various scenarios and conditions to ensure it performs as intended and meets all relevant quality and safety standards. This can include testing for durability, reliability, and other key performance metrics.
Sometimes, performance testing also includes compatibility testing. Compatibility testing ensures that the product is compatible with a range of different devices, operating systems, and other software. This can help ensure that the product works seamlessly across different environments and platforms.
Again, here's a playbook that you can use when planning out your testing exercises.
4) User experience testing after you launch your product.
You will typically do this with a wider group after doing the performance testing above through beta testing programs. This is when it'll be in the hands of your actual users, randomised and segmented.
This type of testing evaluates the performance of the product under different conditions, such as different workloads or user scenarios. This can help identify any performance bottlenecks or issues and ensure that the product can handle expected levels of usage.
User experience testing also involves observing how users interact with the product, and collecting feedback on their experience. This can help identify any usability issues or user frustration, and allow for necessary improvements to be made. It involves evaluating how easy it is for users to navigate the product and complete common tasks. This can help identify any usability issues and make improvements before the product is launched.
User testing as a beginner:
As a PM, you'll encounter your very first user testing exercise. Or, an opportunity to stop limping through the process and do it well.
I've created a user-testing playbook that helped me facilitate cross-country testing exercises, leading to my promotion just 8 months into my first product role. This playbook is now available on Gumroad and is filled with Notion templates that will help you plan the logistics, communicate with users effectively, and record their feedback.
With this playbook which includes other resources that are helpful for a new PM, you can take the guesswork out of user testing and ensure that your products are designed to meet the needs of your users. Don't miss out on this opportunity to take your product strategy to the next level.
You can get the Testing Textbook here: