Insurance
Asia Pacific
Life Insurance: Policyholder AI Concierge
An insurer's AI concierge POC succeeded — but production revealed reliability, compliance, and escalation failures. Rotavision built the trust layer that made it deployable.
Challenge
POC success, production failure
The insurer wanted to launch an AI-powered concierge for 8M+ policyholders — answering policy questions 24/7, helping with claims filing, and reducing call center volume by 40%.
The POC went well. Production didn't:
Reliability concerns
- AI occasionally quoted incorrect policy terms
- Coverage explanations sometimes contradicted policy documents
- Recommendations didn't always match customer risk profiles
Compliance risks
- Regulatory disclosures not consistently provided
- No audit trail for AI-assisted conversations
- Suitability requirements unclear in AI context
Trust deficit
- Customer complaints about AI "not understanding"
- Agents gave different answers for same questions
- No way to verify AI accuracy at scale
Escalation failures
- AI didn't know when to hand off to humans
- Complex cases handled poorly
- Sensitive situations (claims disputes, complaints) mishandled
Approach
Comprehensive trust architecture
Phase 1
Weeks 1-2
Trust Assessment
- Analyzed 10,000 concierge conversations from POC
- Identified failure patterns and root causes
- Mapped compliance requirements by market
- Defined trust KPIs for production
Phase 2
Weeks 3-4
Architecture Design
- Designed monitoring architecture (Guardian)
- Created escalation framework with trigger definitions
- Specified compliance injection points
- Defined behavior steering requirements
Phase 3
Weeks 5-10
Implementation
- Deployed Guardian for reliability monitoring
- Implemented Steer for compliance guardrails
- Built Context Engine for policy data integration
- Created escalation workflows with human handoff
Phase 4
Weeks 11-12
Tuning & Launch
- Calibrated monitoring thresholds
- Tested escalation triggers
- Validated compliance coverage
- Launched with real-time monitoring
Solution
Trust infrastructure for customer AI
Guardian deployment
- Real-time hallucination detection (comparing to policy documents)
- Consistency monitoring (same question = same answer)
- Confidence calibration (flagging uncertain responses)
- Drift detection (weekly behavioral baselines)
Steer integration
- Compliance disclosure injection (market-specific)
- Tone and empathy adjustments for sensitive topics
- Recommendation boundaries (suitability guardrails)
- Prohibited topic enforcement
Context Engine
- Unified policy view from 4 legacy systems
- Contextual joins matching customer identity across systems
- Real-time policy status and coverage details
- Semantic understanding of policy documents
Escalation framework
| Trigger | Confidence | Action |
|---|---|---|
| Policy question | >90% | AI responds |
| Policy question | 70-90% | AI responds + human review queue |
| Policy question | <70% | Immediate escalation |
| Claims initiation | Any | AI assists, human completes |
| Complaint detected | Any | Immediate escalation |
| Sensitive topic | Any | Empathy mode + escalation offer |
Results
From experiment to production service
| Metric | Before (POC) | After | Change |
|---|---|---|---|
| Response accuracy | 84% | 97% | +15% |
| Compliance adherence | 71% | 99.5% | +40% |
| Customer satisfaction (AI interactions) | 3.2/5 | 4.4/5 | +38% |
| Appropriate escalation rate | 45% | 94% | +109% |
| Call center volume reduction | 22% | 41% | +86% |
Additional outcomes
- Launched across all 6 markets (with market-specific compliance)
- Zero regulatory findings in first year
- NPS for digital service increased 18 points
- Expanded to claims status and billing inquiries
"Our POC convinced us AI could answer questions. It didn't convince us AI could be trusted. Rotavision helped us build the trust layer — monitoring, guardrails, escalation — that turned an experiment into a production service our customers actually love."
— Chief Digital Officer
Related
Similar case studies
Your turn
Facing similar challenges?
Let's discuss how to build trust infrastructure for your customer-facing AI.