Building Customer Partnership with Design Via In-Person Research
Design and research had never been represented at our customer conference before, and with increasingly design-led work at the company, our summer conference in July 2023 was the opportunity we needed to build connections with customers, demonstrate how research and design work, and gather insights on some big bet projects.
This is the story of how we elevated the role of design and research by building a meaningful in-person research activity for customers.
Challenge
At the company's annual customer conference, we set out to make our customers feel valued, curious, and excited about design work at the company, while also collecting meaningful insights. While it was imperative that we used this time with customers to identify the value they saw or didn't see in the design concepts, and any edge-cases or concerns they had, it was equally important that we built an enjoyable and meaningful experience of interacting with research and design, to encourage them to partner with us in the future.
My Role
I was both the experience designer and primary researcher on this project.
As the experience designer for the research activity at the conference, I aligned the design team's goals to the goals of the conference runners, defined success criteria, ordered materials, and managed logistics of the sessions.
As the primary researcher, I defined the research goals, wrote the session protocol, ran many concept-testing sessions myself, and synthesized the results.
I instructed two additional researchers on how to run the sessions, and provided the testing materials to standardize the flow and ensure we could synthesize results from all sessions together. I collaborated with one designer on preparing the prototypes to be tested.
Objective
Build partnership between design and customers.
As a B2B company, the new mobile experience would be used by employees who travelled and submitted expenses, but would be rolled out for employees by finance administrators, our customers.
The success of the app experience was dependent both on user experience, and on advocacy from the administrators.The customer conference was made up of finance administrators from higher education, legal, and professional services companies.
This provided the valuable opportunity to gather insights from administrators on what was most valuable about the future state mobile concepts, and if there were edge-cases or employee types that this experience wouldn't work for.
How might we...
Drive customer engagement and make customers feel valued, curious, and excited, and...
Mine our customers expertise to gather real insights on the ideas driving the future-state of the app
Success Metrics
Number of participants in sessions
Number of sign-ups for future beta-testing
Research Goals
With these overarching project HMWs in mind, we developed the following research goals:
-
Gather concept feedback from admins on two key concepts in Emburse Mobile future state experience
-
Identify what admins see as valuable or not valuable about the experience
-
Benchmark on comfort in releasing to a wider audience
Methods
Concept-Testing
We began with warm-up questions about participants’ goals for employee experience.
Next, participants walked through two simple scenarios, one with each concept, and were asked questions about what would or would not be valuable for their employees.
Adapted System Usability Scale
We adapted a series of System Usability Scale statements to the context and asked participants to rate their agreement level to gain our initial benchmark on perceived ease-of-use and comfort distributing the experience to more employees.
In-Person Set-Up
We had priorities for the sessions including...
-
Light-weight onboarding
-
Quick turnaround for participants
-
Elements that grounded the experience in reality
We couldn't actually sell snacks at our booth, so we simulated a purchase of a snack (a coffee or donut pin), gave them a receipt, and asked them to expense it using the prototype.
The real-world connection helped them treat the prototype as a real expense they were completing, so we didn’t run into issues of them rushing through screens.
_HEIC.png)
We memorized the script for the most part, but had it printed just in case. Memorization allowed us to focus on the user interactions with the prototype, and follow up with additional questions.
We recorded our participants' voices so that we could focus on probing, rather than on taking notes the whole time.
Day-Of
The approachability of our station was successful and we welcomed customers during low-programming times, such as lunch and breaks.
Synthesis
At the end of the conference, we had run 26 sessions: far more data than we normally collected for qualitative research projects. The event was a success, and we had a lot of data to sort through.
We started with adding all of our insights that specifically related to actions with the prototype collaboratively, and pulling out the key themes.
For concept feedback, we added all the notes, and then thematically mapped them.
By doing this exercise as a group, we processed and identified insights from a large quantity of data at a faster rate, delivering findings within a couple weeks of the conference.
Output
We delivered a findings deck showing what resonated as valuable for each scenario and how it tied to the themes we learned about in goals for employee experience.
%201.png)
We documented the benchmarks based on the System Usability Scale to reference in future testing.
Impact
Business Impact
SUCCESS METRICS - Revisited:
26 participants
15 sign-ups for future beta-testing
Based on our metrics, we achieved our goal of driving customer engagement with the design team.
We amassed a huge amount of feedback in a day and a half that the Product Managers and designers used to revise the prototype that was presented at our Company Kickoff.
While we had already tested with end users, having this feedback from the administrator persona gave us greater confidence in advocacy from the administrators prior to launch.
Organizational Impact
When we began a beta testing program, we leveraged the 15 participants we had signed up for future testing at the conference, which vastly reduced our recruitment time.
We created a repeatable structure for in-person mobile testing which we could make minor modifications to and use for the next two conferences.
By the third time we ran this activity, applying changes to location and communication about the sessions, the number of participants doubled.
Learnings
This being our first time running a research activity at our customer conference, we knew it would be a learning experience. Immediately following the conference, one of our facilitators led us as we gathered for a retrospective where we all silently added our notes on what went well, what could be improved, and ideas for what to change next time.
Themes emerged around multiple parts of the overall experience, not just the actual research activity.
Logistically, awareness of our activity had been low and participants had felt hesitant to approach due to uncertainty of the ask of them.
Collaboration-wise, tangentially-involved product managers expected more communication about what we were showing.
The activity worked well, but participants were processing a lot of content in the short time given. For future conferences, we simplified the prototype experience.
The group retrospective and thematic mapping, with follow-up action items, was an incredibly valuable exercise, and we implemented many of the changes for the following conference, which successfully resulted in a more focused experience for our customer participants.
Personally, this project reinforced my own interest in creating valuable experiences beyond the digital space. Running this project forced me to think through how this activity would fit into the overall experience for our customers at the conference, and how it could potentially give them a positive view of the company, or a negative one if the activity or logistics were unsuccessful. There were different and compelling challenges to the in-person approach, and I was excited to rise to those challenges.