Home/
Part XII — Building Real Products (End-to-End Projects)/35. Project 4: Customer Support Triage Assistant
35. Project 4: Customer Support Triage Assistant
Overview and links for this section of the guide.
On this page
The Problem
Customer support teams are drowning. A typical B2C company receives thousands of emails daily:
| Category | Volume | Complexity | Current Handling |
|---|---|---|---|
| "Where is my order?" | ~40% | Low | Human reads email, copies order ID, looks up status, types response |
| "I want a refund" | ~30% | Medium | Human checks policy, verifies eligibility, processes or denies |
| Technical issues | ~20% | High | Requires actual investigation, often escalation |
| Other | ~10% | Varies | Manual routing and handling |
The waste: Humans spend 70% of their time on low-complexity tasks that could be automated. The high-value work (complex technical issues, angry customers needing de-escalation) gets delayed.
The Solution
We build a triage bot that sits between the inbox and the support team:
┌─────────────────────────────────────────────────────────────────┐
│ TRIAGE PIPELINE │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ INCOMING │──▶│ CLASSIFY │──▶│ EXTRACT │──▶│ ACTION │ │
│ │ EMAIL │ │ │ │ │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ Intent: Order IDs, Auto-reply, │
│ Billing? Dates, Escalate, or │
│ Shipping? Sentiment Draft for │
│ Technical? human review │
│ │
└─────────────────────────────────────────────────────────────────┘- Classify: Is this Billing, Shipping, or Technical?
- Extract: Pull out Order IDs, Dates, and Usernames using structured output.
- Enrich: Call internal APIs to get order status, account info, etc.
- Decide:
- High confidence + safe topic: Auto-reply with drafted response
- Medium confidence: Draft response for human approval
- Low confidence or risky: Route to human queue with context
System Architecture
// Simplified system design
┌─────────────────────────────────────────────────────────────────┐
│ CUSTOMER │
│ (sends email) │
└──────────────────────────┬──────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ EMAIL INGESTION │
│ (Gmail API / SendGrid webhook) │
└──────────────────────────┬──────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ PII SCRUBBER │
│ Redact credit cards, SSNs, phone numbers │
└──────────────────────────┬──────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ CLASSIFIER (LLM) │
│ Intent + Confidence + Sentiment │
└──────────────────────────┬──────────────────────────────────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ SHIPPING │ │ BILLING │ │ ESCALATE │
│ HANDLER │ │ HANDLER │ │ TO HUMAN │
└────┬─────┘ └────┬─────┘ └────┬─────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────┐
│ ORDER MANAGEMENT API │
│ (lookup status, refunds) │
└──────────────────────┬──────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ RESPONSE GENERATOR │
│ (draft reply using policy) │
└──────────────────────┬──────────────────┘
│
┌─────────────┼─────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│AUTO-SEND │ │ DRAFT │ │ QUEUE │
│(conf>0.9)│ │(conf>0.7)│ │(conf<0.7)│
└──────────┘ └──────────┘ └──────────┘Technology Stack
| Component | Technology | Why |
|---|---|---|
| Email Integration | Gmail API / SendGrid | Reliable, well-documented, webhook support |
| LLM | Gemini 1.5 Flash | Fast, cheap, good at classification |
| PII Scrubbing | Microsoft Presidio / Regex | Catch credit cards, SSNs before LLM sees them |
| Queue | Redis / PostgreSQL | Store tickets, track status |
| Dashboard | Retool / Custom React | Agent UI for approving drafts |
Start with Flash, Not Pro
Classification is a "fast-thinking" task. Use Gemini 1.5 Flash for speed and cost. Only upgrade to Pro if you need complex reasoning (rare for triage).
What You'll Learn
By the end of this project, you'll have hands-on experience with:
- Intent classification: Building a reliable multi-class classifier with confidence scores
- Structured extraction: Pulling entities (order IDs, dates, names) from unstructured text
- Policy enforcement: Constraining AI responses to follow business rules
- Confidence-based routing: Knowing when to auto-respond vs. escalate
- PII handling: Protecting customer data before it reaches the LLM
- Evaluation: Measuring accuracy and catching regressions
Where to go next
Explore next
35. Project 4: Customer Support Triage Assistant sub-sections
5 pages