← All projects

CASE STUDY · PRODUCT · UX RESEARCH · AI

AI-powered Usability Testing — Mazii

A UX research workflow built on Claude: it turns Figma designs into self-recording prototypes and evidence-based reports, so eUp product teams can validate designs with real users before development.

ROLE
Process design & AI skill building
STATUS
Live
YEAR
2026
FIELD
UX research · Usability test · AI · Claude

Internal project — shown through screenshots only.

01

Context

eUp’s product teams (Mazii, 4English…) ship new designs constantly. Yet a traditional usability round took about 4 weeks for 5 users: recruiting, waiting for developers to build a prototype, running each session, then rewatching hours of video to write the report.

By the time the report arrived, the design was already in a development sprint — the findings were right, but too late to act on. I redesigned the workflow around Claude and packaged it as an AI skill any product team can run.

New-hire pain points

  1. 01

    4 weeks for 5 users

    Recruiting, scheduling, sessions, synthesis — a week each.

  2. 02

    Blocked on developers

    Designers had to wait for a working prototype from developers before testing.

  3. 03

    Findings arrive too late

    By report time the design is in a sprint, and changes cost far more.

  4. 04

    Manual synthesis

    Rewatching hours of video, retyping notes, counting taps by hand.

  5. 05

    High cost

    Paid participants, staging, front-end effort for a build that exists only to be tested.

  6. 06

    Untested hypotheses

    Because testing was so costly, many design decisions were made before any user data existed.

Design question · How Might We

How might we let every product team validate designs with real users within the week, before development — without developer effort, and with results they can trust?

Design Thinking process

  1. 01Empathise

    Why product teams skipped usability testing

    I broke a traditional round down week by week and role by role: designers waiting on developers, PMs waiting on the report, note-takers rewatching video. The bottleneck wasn’t the method but the time — results always landed after the design was locked.

    • Process mapping
    • Bottleneck analysis
  2. 02Define

    Keep the method, drop the overhead

    The foundation is Jakob Nielsen’s discount usability (1989) and the 5-user rule (2000): test early, test with few people, fix, then test again. The problem was reframed — not replacing the method, but making it fast enough to run within a week, with no waiting on developers.

    • Discount usability (Nielsen)
    • 5-user rule
    • How Might We
  3. 03Ideate

    Splitting the work between AI and people

    AI takes the labour-heavy steps: building the prototype from the design, logging, and writing the report. People keep the judgement calls: the PM sets hypotheses and criteria, the designer reviews the prototype, a moderator sits with each participant.

    A full round fits into 6 actions and 3 message templates for Claude: invoking the skill, reviewing the prototype, and synthesising results.

    • Human-in-the-loop
    • Task-based testing
    • Think-aloud
  4. 04Prototype

    Packaging it as an AI skill

    The /prototype-usability-test skill turns a Figma design into an interactive web prototype that runs on a phone and records every tap, time on task and SEQ score (1–7). It provides a moderator link and a participant link, with one result code per person.

    The report rates severity on Nielsen’s 0–4 scale, triangulates three layers of evidence (metrics – behaviour – quotes), places the current screen beside a proposed fix and states the confidence level of every finding.

    • Claude skill
    • SEQ (1–7)
    • Nielsen severity 0–4
  5. 05Test

    Real-world use on Mazii

    The first round tested Mazii’s level-personalised home screen (29 Sep 2026): 5 participants, 8 tasks, 40 measured attempts. Only 1 in 5 could change their level from the home screen — a flaw in the core feature, caught before development; 5 of 5 found the home screen too dense. The report delivered 8 findings with fixes, a P1–P3 priority plan and acceptance criteria for round two.

    To keep results trustworthy, the workflow follows 6 rules: lock criteria before the first session, discard pilot sessions, count by person (report 1/5, never as %), draw no CR/CTR conclusions, re-test with new people, and don’t conclude without enough participants. The skill has since validated a PM hypothesis for 4English Home and is taught to BO, PM and Marketing teams.

    • Moderated testing
    • A/B test as next step
    • Training BO · PM · MKT

Results

0,5DAY TO BUILD A PROTOTYPE, NO DEVELOPERS
1/5COULD CHANGE LEVEL — CAUGHT BEFORE DEVELOPMENT
8FINDINGS WITH EVIDENCE & FIXES
2PRODUCTS USING IT: MAZII, 4ENGLISH
Next project →eUp Onboarding Hub

Onboarding website for new hires: a 5-level journey over time, summarised policies, a contact directory and an editing mode for HR.