User Engagement in Beta: Turning Beta Users into Loyal Advocates

While originally beta testing had nothing to do with marketing, today’s startups and especially SaaS businesses have been radically changing this. Traditionally, beta testers are often 50 to 500 people who try the product outside the development environment on different devices after alpha testing (black-box, white-box, etc.) These beta testers aim to uncover and report bugs, stress-test the system’s reliability and security in real-world scenarios. However, some companies figured out how to use this as an opportunity to start creating early communities and drive early user engagement. 

For instance, Google’s Gmail 2004 release is a well-known example. Google’s Gmail was beta tested in closed internal conditions by its own employees for a year. After that, they released a public invite-only beta version and famously kept their product beta for 5 years. So, external beta users got to work with an operational and stable enough beta version. However, there were some features missing (attachments, mobile version, etc.), a certain number of bugs expected, and support was limited. 

So, basically, the testing cycle extends to create the most polished and secure version for the masses, while the extra polishing stage gives the opportunity to build a product’s community of loyal advocates and build marketing momentum. After all, Google’s decision to keep the “beta” for 5 years was mainly driven by the social effects the invite-only tactic generated: exclusivity and buzz about having a Gmail account.

As such, today’s beta testing is linked tightly to the concept of MVP development and soft launch, be it open or closed beta. In this blog post, we’ll go over the most noteworthy case studies and distill their recipes for turning beta testing into a strategic marketing move.

From Testers to Advocates: (Un)intentional User Engagement

Before we proceed, it is worth noting that testing feedback loops is a foundation of making public beta work. After all, public beta testing is rooted in the idea of doubling down on testing and co-creating the product with target users. When a company aims to launch extensive beta testing, the internal alpha testing becomes a sort of dress rehearsal for the beta launch. Alpha becomes not only about testing the product itself. Internal employees test the entire flow to report a bug, suggest a feature, and connect with the product team. In all three case studies below, whether the marketing moment was intentional or not, the feedback loops from beta users were built effectively.

Figma Beta

Out of the most famous examples, Figma, unlike Notion or Google, was probably the most prominent to make this move from testers to advocates intentionally. They have gone into stealth mode and started reaching out to designer communities under wraps. The purpose was to develop a really successful designer tool with the view that if a designer adopts it, then they will use it for the entire workday, every day. Figma’s feedback loop was extremely efficient, and beta users felt essentially like co-creators. As such, Figma got really invested in user feedback. According to the words of Clair Butler, the Senior Director of Marketing at Figma:

“We launched Figma as a closed beta. It was nice and made us feel good to have folks signing up on the waitlist for the product. But if I could do it again, I would have made it an open beta so we could get as many folks into the tool and increase our velocity of customer feedback.”

Notion Beta

The idea of turning beta users into advocates did happen unintentionally for some companies. For Notion, they did not expect it at all: they started noticing communities forming from beta users and went along. Moreover, prior to beta, they were on the verge of bankruptcy and pivoted from the original idea big time. They started coding up Notion from scratch, and, eventually, someone spotted their product on Product Hunt. Overall, the beta phase lasted for about 10 months. Here is the quote from a Product Hunt story:

“It didn’t take long for Notion to surface on Product Hunt. In 2015, someone hunted the product to share with the rest of Product Hunt’s community. Even though Ivan and Simon didn’t think their product was quite ready yet for its debut, … they ran with it.”

Gmail Beta

Google Gmail launched a closed, invite-only beta for the reasons of limited resources. At the time of beta launch, they literally had 300 old Pentium III computers that Google could spare for this project at the time. The Gmail team calculated that it would be enough for approximately 1,000 beta users. They also allowed these initial 1,000 users to invite their friends and family. The marketing momentum was a surprise for the company. Georges Harik at Google said:

“The limited rollout had been born of necessity, but it had a side effect: everyone wanted it even more. It was hailed as one of the best marketing decisions in tech history, but it was a little bit unintentional.”

Sourcing the Right Users for Beta Testing

Another essential part after ensuring an efficient feedback loop is sourcing the right beta users. Even though Notion, for instance, did not anticipate such a community momentum, targeting beta users was always intentional. For all betas, companies reached out to not merely users who might be interested. They sought out people who are visibly active in the target niche. The ideal beta users would be those who align with the pain point your product is trying to solve and those who are truly enthusiastic and passionate about this particular niche.

Notion’s Sourcing of Beta Users: Visibility on Product Hunt and Outreach

Notion founders and team launched an outreach program through email, waitlists, and directly on social media such as Twitter and Reddit. Listing their app on Product Hunt helped attract powerful users as well. For instance, Notion’s Head of Community heard about the tool on this platform and became a template creator, all the while helping to build community content. Notion team targeted productivity enthusiasts, designers, writers, and different kinds of creators. All kinds of users who sought to “visualize their brain”, those who needed flexibility in note-taking and efficient knowledge management. In this case, user engagement got so strong that it got a life of its own, so the company embraced it.

Figma’s Sourcing for Beta Testing: Targeting Designers on Social Media and Conferences

At Figma, founders invited UI/UX designers from companies like Dropbox and Airbnb. They also did demo calls of early builds at conferences. Moreover, Twitter became a place for trying out Figma APIs. Figma’s founders simply shared short screen captures and screenshots of a new idea or feature, asking for feedback. These beta users would not only provide feedback on this feature, but also provide suggestions for useful plugins and a comparison with the existing tools. The most popular ones at that time were Sketch and Adobe. User engagement was exceptionally strong, indicative of lively participation in the beta phase. 

Discord’s Sourcing of Beta Users: Founder’s Presence in Gaming Communities

The Discord outreach program also relied on founders and their already existing presence in the gaming community. They simply shared links to Discord servers on subreddits. This enabled popular streamers to build communities, creating fan servers on Discord.

The Playbook: Doing Beta Testing with Maximum User Engagement

• Your beta users should be those who are actively discussing their frustrations with the existing solutions and experience the pain point your product solves most acutely.
• Since ideal beta testers are already actively talking about this problem, the best places to reach out are niche forums, subreddits, LinkedIn, X, Slack/Discord communities, and such. 
• In MVP development, the beta version does not imply a certain level of product completion. Like with Figma, you can opt for MVP feedback, having just some APIs and feature teasers. 
• The decision to do a closed (invite-only, waitlists) or open beta has its pros and cons. If you are limited in resources, then a closed beta is the way to go. However, if you aim to develop a product like Figma (professionals and teams use design tools 8 hours every workday), then developing your soft launch strategy and designing for high-volume feedback loops with open beta is the best option. 
• Make sure the feedback loop is efficient. The whole point of doing beta testing is to gather as much user feedback as possible. There are at least four key elements that you should take care of:

1. Embedded feedback buttons. They can be star ratings, thumbs up/down, survey buttons, feedback pop-up widgets, or sliders, etc. 
2. For bugs, there has to be a “report a bug” workflow. Originally, beta testing served the purpose of checking usability on different devices. 
3. Direct channels to founders or development teams via Slack channels, Notion, or Discord servers, etc. 
4. It is a must-have to showcase the product roadmap with visible updates based on beta testers’ input. This serves as a visual acknowledgement of all the work they are doing.

• Lastly, incentive programs, early access perks, and public acknowledgement are a few tools to keep motivation levels at an all-time high.

Why Beta Testing Turns Testers into Loyal Advocates: The Psychology of Beta User Engagement

Emotional Investment through Co-Creation Drives User Engagement

Beta testers interact with an essentially imperfect product and put a lot of effort into feedback. Seeing a visible feature change based on feedback makes them feel co-creators. Even if their particular feedback does not make it to the road map, but there is a reply to it showing it has been considered, users will identify themselves with the product’s success. Figma is truly a model example of a highly-engaged co-creation process.

Exclusivity of Early Access

Scarcity of invite-only access and waitlists creates a sense of privilege. For different users, this triggers a variety of perceptions. For instance, some feel like they are ‘in’ on the next big thing. Some users have insatiable curiosity. Others might feel pride in getting access to something that is so exclusive. These same feelings that drive engagement prompt word-of-mouth and pubic sharing of their experiences. For Google, the hardware limitations of their beta versions certainly worked magic in terms of marketing.

A Sense of Belonging

As described above, sourcing beta users who will turn into loyal advocates goes through already existing communities. When the beta testing begins, beta users often bond even deeper over the common process of co-creation. The community grows tight, and testing functionality goes hand-in-hand with social interactions. Beta testing becomes a part of beta users’ creative or professional identity. At Notion, beta users created communities where they taught each other, built templates, and inspired one another.

Visibility Translates into Transparency and Builds Trust

How do you know that the next great tool will be truly user-centered and authentic? That in a few months it would not turn into a heavily commercialized engine? There have been tools and apps that are ‘beta tested’ publicly just for usability sake, while being already fully formed. These often have ‘authenticity’ as rhetoric for their launch and quickly turn to practices that alienate users. For instance, Thread’s algorithmic manipulations and data tracking, or X’s opaque moderation, caused more than controversial reactions. In contrast, when the product evolves publicly in co-creation with the users, transparency builds trust. Public co-creation with beta users serves as proof of intent.

Recognition Activates Reciprocity

In addition to responding to beta users’ comments and having a visible change to a product roadmap, recognition plays a pivotal role in creating loyal advocates. To recognize beta users for their effort, the product team or even the founders can write thank-you notes, assign badges, or even publicly appreciate them for their input. Imagine a badge “founding member” or something along these lines. This brings the feeling of being valued and respected. In turn, people will want to give back as a natural response.

Tangible Rewards

Most of the above goes down to intrinsic motivation and psychology. However, it is also important to complement these with structured rewards to offset the time and effort beta users put in. Even a thank-you note is more meaningful when it comes with certain tangible rewards. These rewards can be anything ranging from discounts, free credits/points, lifetime access to certain paid functionality, extended free tiers, branded merchandise, and so on. These, though, come last due to an important point: incentives should only reinforce the user engagement; otherwise, they would create a purely transactional relationship and would undermine the ‘loyal advocates’ effect.

Summary

Beta testing has evolved from being a simple check in a real-world environment to a strategic and powerful move. Involving users in the co-creation of the product they’ll love is the essence of public beta testing, be it open or closed. Visibility in itself attracts the beta users who are already struggling with the pain point your product is aiming to solve. As such, these beta users help to create the product they’ll love and naturally become this product’s loyal advocates. Emotional investment, exclusivity, a sense of belonging, trust, recognition, and tangible rewards all work towards keeping up the momentum. The key, though, is to ensure an efficient feedback loop through:

• Embedded buttons,
• Report a bug workflow,
• Direct communication channels,
• Accessible product roadmap with visible updates.

These make the foundation of making sure the users get heard and co-create the tool to become its ambassadors. 

FAQ: User Engagement in Beta: Turning Beta Users into Loyal Advocates