Survey In-App MEssages

ROLE
Research and Design alongside
mentorship of Design Intern

TEAM
Design Intern, PM, EM, 4 Eng

Introduction

This project marked the first major phase of a broader multi-year vision for in-app messages. Before this, the team had conducted extensive discovery research, developed key foundational features, and established the technical architecture critical to the project's success.

An intern completed the initial portion of the project with my close involvement and guidance. After the internship ended, I took over the project and led it through to launch.

The Problem

Based on previous research, we knew that:

Customers want to create more sophisticated messages, quickly in Braze.

Overview

The first portion of the project (the area in pink in the image below) was completed by an intern under my close involvement and guidance. At the start, we knew customers wanted a new template, but we needed to determine which one. To answer this, the project began with a survey sent to customers and internal subject matter experts to identify the most valuable template to build.

The intern then tested a simple wireframe with customers to validate the survey template’s requirements. After further ideation and refinement of the prototype and template behaviors, the project was handed off to me for completion. From there, I finalized a high-fidelity prototype for a final round of user testing, where we made adjustments and defined our MVP spec.

Following routine specification and QA, the survey template was gradually released to all research participants. During Early Access, we conducted live usability testing with the first six customers as they created their first surveys in the live product. Once 25 customers successfully launched a survey—without any performance or stability issues on our end—we proceeded with a global release, which was met with great success.

Initial Research

1. Desk research: Conducted a competitive analysis, reviewed customer feedback in Productboard, and gathered internal feedback from Customer Success.

2. Quantitative survey: Sent a survey to customers and internal SMEs to determine which templates users preferred and how important they were.

3. Customer interviews: Conducted live usability testing with a simple wireframe and narrowed down the survey types.

Based on that feedback, we identified these survey types as candidates:

1. Multiple choice/multi-select

2. Star rating

3. Net Promoter Score

4. Ranking

5. Single/multiple questions

6. Open-ended feedback collection

7. Data collection, such as email addresses and phone numbers

Customer Interviews

After identifying potential survey types, the intern conducted usability sessions where users voted for their preferred template and walked through a rough prototype to create a multiple-choice survey.

Ideation

This stage is where handoff from the intern happened.

Once we determined that we would build a multi-select survey, I sketched various ways to display and customize the choices. Despite the many design possibilities, we decided that a simple, familiar radio/checklist with horizontal dividers was the best approach. Since survey choices could contain multiple lines of text, we kept everything to a single line and ruled out buttons or designs with large padding or frames.

User Workflow

Below was a snapshot of the user’s journey as they build a survey.

Usability Testing

Towards the end of the project, we conducted a final round of usability testing on the high-fidelity prototype. These conversations validated UX decisions and the MVP scope. Since this project required extensive engineering pre-work to generate and send HTML before building the specific template and composer, we had ample time to run the sessions.

As shown in the final slide below, we gathered valuable insights. We identified a clear need for multipage messages and greater design customization. Knowing that we couldn’t dedicate much time to surveys post-MVP—since we needed to shift focus to building a drag-and-drop editor—we carried this feedback into future projects and eventually addressed many of these needs.

Analytics Explorations

After launching a survey, customers needed to analyze its performance. Many customers had Currents connectors that streamed data between Braze and their data warehouses, where they visualized and analyzed data using their own tools outside of Braze. Keeping this in mind, we deprioritized major changes to analytics pages, as these were owned by another product team. Given the strict layout and design constraints, Design had to compromise to deliver a functional solution, knowing that many customers wouldn’t be hindered by the absence of a visual representation, such as a pie chart.

We also faced challenges in rendering each choice’s content, as they often contained liquid personalization strings, such as {{${favorite_color}}}, or lengthy translation strings that displayed differently for each user. As a result, unless the survey contained no liquid strings, the Choice Text wouldn’t always appear as a concise, readable string. The final design accommodated these long strings and prioritized customers who relied on visualizations in their own data

Final Prototype

This is the final prototype that was tested with customers. For the MVP, customers could create a survey with a single question, allowing for either single or multiple responses. Selections were saved to the user’s profile if attributes were logged, enabling better targeting and messaging in the future. The editor included a live preview, allowing users to see how the message rendered and interacted. Users could also change the Color Theme to align with their brand colors.

Gated Rollout

This feature was rolled out to a small portion of customers (~40) over a few weeks. We conducted usability testing and interviews with 6 customers as they built their first survey, which was generally successful.

Once at least 25 customers successfully launched a survey and engineering approved performance and scaling, we released it globally.

Survey Examples

Outcomes

For customers, surveys unlocked new use cases. Previously, they had to rely on third-party tools or write custom HTML to gather feedback from their users. While they typically worked with implicit data—based on how their customers used the product—surveys now enabled them to collect explicit data, helping them confidently understand user preferences and needs. Customers could now gather feedback directly within the app, instead of linking to an external survey in a browser, providing a more seamless experience. They could also launch surveys quickly—in just a matter of minutes—and with confidence.

Once survey data was collected, customers could enrich their existing Braze strategy, as well as broader product strategy. With information such as preferred notification channels, content choices, and product preferences, marketers could better target and personalize their Braze user journeys and messages. We also saw excitement from marketers, researchers, and growth product managers, as they could now directly ask customers what to build, especially when users were most engaged with the product.

This project highlighted the need for greater survey flexibility: open-ended text areas, multiple questions/pages, logic-based branching, inclusion of images/icons, and more specific templates like NPS and CSAT. It also sparked discussions around additional types of data collection that could be enabled through IAMs, such as email and phone number capture. These needs and ideas were incorporated into future roadmaps.

Internally, this project marked a significant milestone in a multi-year strategy to transition IAMs from restrictive native SDK code to flexible, sophisticated HTML messages. As the first-ever out-of-the-box template using HTML, surveys demonstrated that IAMs could be fully visually customizable to match a brand, while new templates could unlock new use cases for customers. With the ability to create new templates, the next step was to add a Drag-and-Drop editor for further message design customization, complete with a template library of robust use cases.

Customer Case Study

Below are snapshots from a Braze blog post, showcasing HBO Max’s use of Surveys, which achieved a 35-50% participation rate.

Reflections

This project came with many challenges, primarily around process and external factors. As mentioned earlier, an intern, who was also handling other smaller projects, completed the first portion, causing the research to move slower than usual. After the internship, the project faced several stops and stalls as more urgent priorities (like Shopify) arose, and technical considerations expanded beyond initial estimates.

As this was my first large project requiring extensive research, I likely could have scaled back on the number of customer interviews and moved forward with more confidence. However, in the end, we were very confident in our solution and had gathered a wealth of feedback that we could apply to future initiatives.

Starting at the end of 2019, this project spilled over into 2020. Many customer interviews were conducted in March and April, during the height of the pandemic. This made scheduling user sessions challenging, as many marketers were shifting initiatives and reacting to the rapidly changing, uncertain world around us. It was also a stressful time, which likely impacted team morale and efficiency.

Despite the complications, I am very proud of the project. It received recognition within the company, and we received praise from both our customers and our colleagues in customer success and pre-sales. A few highlights:

Years later, a Braze CSM insisted that Surveys were still their favorite feature ever released.

One customer said this feature could change the way she works and potentially set her up for a promotion, as she could now run in-product surveys and collaborate more closely with product teams.

In-app messages—and surveys specifically—were highlighted as a key product in Braze’s S-1 filing.