Userlane's HEART dashboard provides data on customers' app adoption and usage.
It however did not provide clear instructions on how to improve app performance.
This was an MVP, where we explored how AI can aid our customers by extrapolating from statistical data into clear, actionable insights.
For ease and speed of delivery, were limited to only minor changes in the UI.
what i achieved
AI-powered insights allowed customers to quickly understand exactly what action to take to improve their app's performance.
I created a blueprint for insight content and design, including creating draft prompts.
I worked closely with AI and Data team, revising prompts to ensure output matched product context.
I pushed for using more contextualised data, such as benchmarks, to make our dashboards more meaningful.
Learnings
Garbage in, garbage out: it quickly became evident where our data provided an incomplete picture. For instance, Task Success realistically only looked at clicks of tagged UI elements, which might or might not correlate with successful task performance. In case of a low T score, we couldn't specify what tasks on which page users were actually struggling with, which reassured us in our goal of enriching task tracking as future product and data initiative.
Using raw AI text output directly is zero effort, but poor content design. We went through multiple rounds of length and tone improvements, and ensured consistent vocabulary.
"Users don't read" still applies. Text content has to be as short as possible, layouts easy to scan (e.g. we provided our AI with a list of words and phrases to render in bold). Use visuals whenever possible.
When not to use AI? You can still get decent results using "dumb" conditional logic. And it can be faster, cheaper, and more predictable than always defaulting to AI. E.g. for T metric, a simple "if: low score; then: suggest using Guides and Tooltips" approach was sufficient.
When AI loses context, it will hallucinate its own. We simplified initially complex prompts by breaking them down into multiple AI agents, and thus limiting hallucinations.
Using benchmark data to determine what constitutes a good score was an improvement, but still a fairly blunt tool. Allowing customers to set specific goals for app performance would provide even deeper, individualised context for HEART metrics.
For your first AI project, budget more time than you think you'll need.
Additional improvements to HEART Overview page
The HEART Overview page was enhanced by integrating benchmark-based scores and substituting static content with dynamic, meaningful data.
Want to work together?
If you like what you see and want to work together, get in touch!
