Alex Chen: synthetic eval persona

The challenge

The SME access problem

At commercetools, the instructional designers writing the docs couldn’t get enough time with senior architects, the experts (SMEs) whose feedback the docs depended on. Without that feedback, the docs drifted. They got too marketing-focused or too theoretical, and missed what busy, experienced developers actually need.

The opportunity

Build a synthetic user that does more than describe a user type. Instead of a static PDF, build an AI agent that can interact by reading a documentation draft and critiquing it, as if it were a senior architect who is short on time.

Key design challenge

Get past demographic stereotypes like “35-year-old developer with 10 years experience.” Capture the psychological relationship instead: how expert users actually feel about the tools they use. What stresses them, what makes them trust or distrust the docs, and how they prefer to learn when they’re up against a deadline.

The process

Research & synthesis

I used Deep Research to pull together real developer experiences from technical forums, Gartner reviews, and commercetools community discussions. I wanted to understand not just what senior architects do, but why they make the documentation choices they make.

Key patterns identified:

• Time poverty: Senior architects skim the concepts to get to the code. They treat “why” explanations as obstacles unless those explanations matter right now
• Context switching cost: Jumping between multiple docs breaks their flow. They prefer navigation that shows only what’s relevant to the task in front of them
• Trust through specificity: Vague error messages or theoretical frameworks make them distrust the docs instantly. They want examples they can copy and paste

Critical insight: The persona had to feel the frustration, not just list demographic facts. When documentation wastes an architect’s time, they take it as disrespect. That emotional reaction shaped the whole design.

Design approach

I used Dr. Philippa Hardman’s structured AI persona framework, and leaned on the parts that go deeper than surface demographics:

1. Episode categorization

I mapped out stories of when things went well and when they went wrong:

• Success: “Found headless frontend integration guide with working React code. Saved 4 hours”
• Failure: “Spent 2 hours debugging because docs didn’t mention the API returns cached data by default”

2. Emotional mapping

I worked out what builds his confidence (clear error messages, docs broken into self-contained pieces) versus what makes him anxious (being forced through a fixed learning path, or marketing language dressed up as technical content).

3. Behavioral modeling

I defined how he reacts under stress:

• When blocked: takes walks, asks the community for help on Slack, blames the vendor if he can’t find the root cause
• When successful: shares the fix with his team, builds internal wikis so nobody has to go hunting through the docs again

4. Voice engineering

I gave Alex a “brutally honest, pragmatic” way of talking. He doesn’t soften his feedback. He tells you exactly where the content wastes his time or treats him like he doesn’t know what he’s doing.

Development & iteration

I built the persona as an interactive agent, driven by a carefully structured system prompt:

System prompt architecture:

You are Alex Chen, a Solution Architect with 8 years experience...
STRESS TRIGGERS: Vague documentation, forced beginner content, marketing fluff
TRUST BUILDERS: Copy-paste code, clear error explanations, modular navigation
COMMUNICATION STYLE: Direct, impatient with theory, appreciates brevity

Validation approach: I tested the persona by having it audit existing documentation. Alex flagged real issues that matched complaints actual users had made. That told me the psychology was accurate.

Testing & refinement

Initial problem: Early versions were too polite. They softened the criticism with “maybe” or “you might consider,” which is not how a real architect talks.

Solution: I added rules for his behavior:

• Never apologizes for being critical
• Demands specific examples, rejects abstract explanations
• Says it directly when content wastes his time

Quality validation: The persona predicted that forcing senior developers through “Getting Started” content would make them give up right away. Real architects later said exactly the same thing.

The solution

What the team experiences

Drop a documentation draft or a UX design into the Alex Chen agent. Within seconds, you get back a first-person critique that points to exactly where the content lets a senior architect down: “This forces me through basics I already know. Show me integration patterns or let me skip ahead.”

First Gemini version:

Following the success of the Gemini Gem solution, the learner persona was integrated into the VS Code workspace for on-demand access as required.

Example feedback

Input: Proposed learning path with mandatory “CT 101” introduction module

Alex’s response: “Hard pass. I’m a senior architect. I know what headless commerce is. Forcing me through beginner content tells me you don’t understand your audience. Give me a ‘Skip to Integration Patterns’ option or I’m closing this tab and finding a Stack Overflow answer instead.”

Impact: Led to creating separate entry points for beginner versus experienced developers, reducing bounce rates for advanced users.

How it works

The persona acts as an AI-powered stress test for content:

1. Input content: a documentation draft, a site structure, or a learning path
2. Psychological analysis: the agent checks it against Alex’s stress triggers and what earns his trust
3. First-person critique: it gives back specific, actionable feedback in the architect’s own voice
4. Design validation: it flags content that would make a reader give up, before you publish

Results & impact

Content strategy pivots

Navigation redesign: Alex’s point that “context switching kills productivity” led us to design scoped navigation, which shows only the content relevant to the current module. Before we spent any engineering time on it, we ran the design past Alex, who confirmed it was “exactly what I need.”

Feature prioritization: Alex confirmed that filterable documentation catalogs were “non-negotiable” for power users, which shaped product roadmap decisions.

Risk prevention: Alex flagged that manual progress checklists would feel “cheap and unintelligent” to senior architects. That stopped us launching a feature that would have hurt our credibility.

Research efficiency gains

• 15+ content audits done without taking up any SME time
• 8 major design decisions shaped by persona feedback before user testing
• Continuous validation, with no scheduling conflicts and no waiting on limited SME availability

Scalability value

The structured framework (Episode / Emotion / Behavior) became a reusable template. The team can now build related personas (junior developers, merchant users) with the same research method.

Key takeaways

• Psychology beats demographics. Traditional personas focus on age, role, and years of experience. The breakthrough here was modeling Alex’s emotional relationship with the docs: how he feels under time pressure, when he trusts the content versus when he doesn’t, and what flips him from confident to frustrated.

• Interactive beats descriptive. A static persona PDF gets read once and forgotten. Making Alex an agent that responds to your drafts keeps people using it. The team now defaults to “ask Alex” whenever they’re reviewing new documentation.

• Stress drives decisions. Watch how Alex behaves under deadline pressure: he skims the theory, grabs the code, and blames the vendor when he’s stuck. That explains why standard documentation layouts fail. Design for how people behave when they’re stressed, not for ideal conditions, and the content works better.

• An authentic voice matters. Making Alex “brutally honest” rather than politely critical made the team trust his feedback more. When he says the docs are good, they believe him, because they’ve seen how harsh he is when the docs are bad.

• Prompt engineering is instructional design. Building this persona meant spelling out what architects are trying to get done, spotting what overloads them (like context switching), and modeling how they really behave on a task. Creating an AI persona is, at heart, a learning-design problem.

Future enhancements

Potential next steps to expand impact:

1. CI/CD integration - Auto-flag “marketing fluff” in documentation PRs before publishing
2. Conversation mode - Let junior writers interview Alex to practice technical accuracy
3. Multi-persona framework - Create complementary personas (merchant user, junior developer) for comprehensive content validation
4. Quantitative tracking - Measure correlation between Alex’s feedback severity and actual user bounce rates
5. Voice model - Add audio responses so teams can “talk” with Alex during design reviews