Is My Money Gone?

From supported choices to live conversation

Defining the persona, practice goals, and mobile format

I defined the persona, conversation turns, scripts, and rubric before asking Claude to build the experience. Because the microlearning was designed primarily for phones, I chose a card-based layout, short conversation turns, and expandable feedback.

Dana is an upset caller disputing a charge she didn’t make. With rent due Friday, she wants her money returned immediately. Four behaviors organize the practice:

1. Acknowledge the concern
2. Be clear about what’s known
3. Verify and protect
4. Give a clear next step

The rubric accepts different ways of expressing these behaviors: an informal reply can be effective, while a polished one can omit a next step or promise more than policy allows.

Feedback on Elena shaped the final structure

The Elena microlearning originally offered live AI practice with a choice-based backup because PPN had not resolved AI usage costs. PPN’s reviewers liked the live practice but wanted more support for novices. I revised Elena into two rounds: choices and feedback first, then live conversation when AI is enabled.

The Dana demo demonstrates that revised structure:

1. Round 1Choose, get feedback, try againRecognize effective behaviors through choices, feedback, and tagged examples.
2. Round 2Take the call in your own wordsProduce a reply yourself, then review what it demonstrates and what to improve.
3. Next timeChoose a focusTurn the review into a specific intention and a line for future practice.

Keeping the scenario familiar lets learners focus on putting the four behaviors into their own words.

Round 1 · Recap

Making the four behaviors visible

I tagged specific phrases in the stronger replies to show how the four behaviors appear in a conversation. Across the three turns, learners can see how the wording changes as Dana’s concerns change, while the behaviors stay consistent. These examples give learners a starting point for composing their own replies in Round 2.

Show what counts as a behavior

Tag 4 marks the action of texting a code; tag 3 identifies its protective purpose. A general offer to help stays untagged.

Compare the behaviors across turns

Three replies show how the behaviors take different forms as Dana’s questions change.

Give examples before asking for original replies

The recap offers language learners can draw on when composing their own replies in Round 2.

Making feedback specific and actionable

Dana reacts to what the learner says; a separate AI reviewer evaluates each reply. The feedback credits useful actions, identifies missing information, and offers a stronger version that preserves what worked.

In the example below, the learner files the dispute and blocks the card but omits refund timing. The review recognizes those actions and models how to add the missing information.

View feedback and a stronger reply

Round 2 · Feedback

Keep what works, address the gap

The review preserves effective actions and adds the information Dana still needs.

Credit the useful actions

The review recognizes that the learner filed the dispute, blocked the card, and explained how the case number would arrive.

Name what is missing

Dana still needs the policy timeline. The feedback identifies that gap instead of treating the whole reply as ineffective.

Model a revision

The stronger version retains those actions and adds timing without promising Friday. Behavior references connect the recommendation to the practice goals.

“Round 1: Strong” records the earlier choice-based attempt. “Partly there” rates this own-words reply in Round 2.

Challenge the rubric with varied responses

Testing Dana by playing different agents myself was slow and limited by my own habits. I needed a broader set of responses after each revision.

So I asked Claude to create eight simulated agent personas:

• The steady proWarm, concise, covers the core.
• The fixerMoves to the task; skips feelings.
• The reassurerComforts with promises.
• The policy robotRecites rules; gives little direction.
• The shortcut-takerSkips safeguards to move faster.
• The texterUses brief, informal language.
• The over-explainerLong replies with banking jargon.
• The testerTries to disrupt the simulation.

Claude built an agent to play these roles with scripted and improvised replies, exercising the practice’s own prompts and review logic.

Some test results called for clearer AI instructions. Others prompted me to reconsider what I had defined as a strong reply. The example below shows how testing helped me strengthen the rubric.

Testing · Rubric revision

A polite reply exposed a gap in the rubric

Testing an acknowledgment-only reply led me to require a concrete next step in Turn 1.

The learner’s reply

“I’m sorry to hear that.”

The initial AI judgment: Strong

It acknowledged Dana’s stress and made no promise, meeting the original Turn 1 criteria.

My judgment: Partly there

Dana was still waiting for help. A good reply needed to acknowledge her concern and start the next step so the call could move forward.

The rubric revision

I added moving the call forward to Turn 1: announcing verification or texting the code, for example. A bare “I’ll help you” does not count as a concrete next step.

The retest

Acknowledgment-only replies now receive “Partly there,” and Dana asks, “Okay… so what happens now?” Replies that acknowledge her concern and start verification remain “Strong.”

Simulated agents accelerated testing and broadened the responses I could inspect. I reviewed the results and decided which judgments were defensible.

My decisions

The target behaviors, scenario and rubric, learning sequence, acceptable judgments, and revisions to feedback and safeguards.

AI assistance

Implementation, drafting alternatives, playing the caller and reviewer, and building and running simulated-agent tests.

Review and next steps

Original Elena project: PPN reviewers responded positively to the revised design and are considering it for their LMS.

Dana portfolio demo: Simulated-agent testing helped refine its behavior and feedback. It has not been tested with real learners, so this evidence does not establish learning gains.

What I would test next

• Independent application: Ask learners to handle an unfamiliar call without suggested replies, then review their use of the four behaviors.
• Use of feedback: Ask learners to explain a suggested improvement and apply it in a revised reply.
• Rating agreement: Compare human and AI judgments on the same replies, including borderline cases, and examine the reasons for disagreement.
• Follow-through: Review a later attempt for the specific behavior the learner chose to practice or keep sharp.