Did the support bot solve the refund? Check the payment record.
By Ehab Al Dissi ·
Would you close this refund?
Fictional case: The shopper is owed $48 for order R-204. The bot says the full amount went through. Here’s what the payment provider shows.
| Refund ID | Amount | Status |
|---|---|---|
| RF-731 | $28 USD | Confirmed |
| RF-732 | $20 USD | Pending |
See the next payment update and worked answer
| Refund ID and amount | 10:15 | 10:22 |
|---|---|---|
| RF-731 $28 USD | Confirmed | Confirmed |
| RF-732 $20 USD | Pending | Confirmed |
| Total confirmed | $28 USD | $48 USD |
At 10:15, keep the case open. Tell the shopper the provider confirmed $28. Assign the pending $20 to payments and set a time to check again.
At 10:22, the merchant can close its refund action. Two distinct refunds add up to $48. Give the shopper both references. Their bank may show the credit later.
A chatbot can explain a refund policy correctly while the shopper’s refund is still pending. Those are separate jobs, and a support leader has to decide which one the vendor is promising to do.
Gorgias, a support vendor included in its own benchmark, refreshed its comparison of agents on live Shopify storefronts in November. Its methodology uses fresh browser sessions and blind scoring by a language model. It scores support on automation, quality and speed. That can show how an assistant handles a conversation. For a refund, I’d ask for evidence from the payment record too.
Add three cases to a sandbox demo
- Request accepted. The payment system has accepted a refund request and the posting is pending. The expected reply says it’s pending.
- Partial refund posted. Match the order, amount and currency. The customer should be told what posted and what is still due.
- Records disagree. The help desk says closed, but the payment record cannot confirm the refund. The expected result is a handoff with the case still open.
Score the answer and the transaction separately. If you’re comparing vendors, give each one the same fixtures and require the same proof before counting a case as resolved.
Read AI Vanguard’s guide to measuring real resolution.
Read the Gorgias benchmark and scoring method
What to reconcile before closing a refund
Start with the amount the customer is owed under the return decision. Match the order and original payment, check the currency, and add only distinct successful refund records. A repeated notification for the same provider refund ID adds zero. In a split payment, reconcile each tender separately.
- Provider confirms only part of the amount
- Keep the case open and name who will verify the balance. Tell the customer the amount the provider has confirmed.
- Provider confirms the full amount
- Record the refund references and tell the customer the provider confirmed it. Bank visibility can follow later.
- Records disagree or a refund fails
- Send the discrepancy to the payments team with the order and refund references. Check the provider before anyone retries.
Use the same cases to score competing tools. Count accurate replies separately from correctly closed tickets, and treat any premature closure as a failure on this test. Stripe’s refund documentation describes pending and partial refunds and explains why a customer might not see a credit immediately. Check how your own provider defines each status.
Review status: This fictional exercise was checked against Stripe’s public documentation. An independent payments practitioner review is pending.