Skip to content

Knowledge Gaps — Proactively Identify KB Gaps

Purpose

This page explains how to use Knowledge Gaps to find areas where the chatbot can't answer user questions well. The system automatically analyzes conversations and groups unanswered questions into 3 gap types, each with a suggested fix.

This is a proactive tool — not waiting for complaints, but detecting patterns from real conversations.

Prerequisites

  • Role super_admin, system_admin, or faculty_admin
  • At least 1 week of conversation data (for meaningful patterns)
  • Basic understanding of FAQs, KB documents, and intent rules

Concept: 3 gap types

Type Meaning Example
Content Questions needing new info in KB "Medicine tuition per SKS?" — KB lacks that detail
Data Info exists but retrieval misses it "UPH Medan contact" — chunks exist but query doesn't match
Action User wants to do something, bot has no handler "Want to upload document" — needs new intent rule

Each gap includes a suggested_fix from auto-analysis.

Steps

1. Open the gaps dashboard

Sidebar: KNOWLEDGE → Gaps (or via /knowledge/gaps).

Page shows status tabs: - New: freshly detected gaps (needs review) - Accepted: acknowledged gaps, being worked on - Resolved: gaps that have been fixed - Dismissed: gaps decided not worth pursuing - All: everything

2. Review new gaps

Click New tab. Gap card shows:

Text Only
 ┌─ Gap #142 ─────────────────────────────────────────┐
 │ [CONTENT]  ⓘ 5 conversations asked about this     │
 │                                                    │
 │ Title: Medicine tuition per SKS                    │
 │                                                    │
 │ Sample questions:                                  │
 │  • Berapa biaya Kedokteran per SKS?                │
 │  • Per SKS harganya berapa untuk Kedokteran UPH?   │
 │  • Kedokteran UPH per SKS-nya berapa?              │
 │                                                    │
 │ Suggested fix:                                     │
 │   Add KB document explaining Medicine fee          │
 │   structure — SPT + BPP per semester vs per-SKS    │
 │   breakdown (if available).                        │
 │                                                    │
 │ [Accept] [Dismiss] [View Conversations →]          │
 └────────────────────────────────────────────────────┘

3. Decide action

Accept — you agree this is a real gap and will fix it. Gap moves to "Accepted" tab for follow-up.

Dismiss — gap doesn't need fixing (e.g., question out of UPH scope, or context already answered elsewhere). Gap moves to "Dismissed" as a record.

View Conversations — see the actual conversations where users asked this. Useful for understanding full context before deciding.

4. Apply the fix

Depending on gap type:

Content gap → Create FAQ or upload document 1. Open FAQ page → add new FAQ with correct answer 2. Or open Knowledge base → upload new document about that topic 3. Wait for auto-sync to Dify (~30 seconds) 4. Return to gap → click Resolved

Data gap → Fix retrieval 1. Check whether relevant documents exist in KB 2. If yes, chunking or embedding may need refresh 3. Contact developer for re-indexing if needed 4. Mark Resolved after verifying bot answers correctly

Action gap → Add intent rule 1. Open Bot configuration → Intents 2. Add new intent rule matching the question pattern 3. Choose handler (route_dify, template, cta, escalate) 4. Test via sandbox 5. Mark Resolved after live

5. Bulk actions

If many similar gaps, select several via checkbox → bulk actions: Accept All / Dismiss All. Useful for initial triage.

6. Verify the fix

After applying the fix, run Answer Eval on the relevant category to verify. See Answer evaluation.

Gaps marked resolved but not validated via eval may not actually be fixed.

Example scenarios

Weekly triage. Every Monday morning, marketing admin opens the New tab and reviews 5-10 newly detected gaps. Valid ones → Accept and assign to content writer. Off-topic → Dismiss with note.

After campaign launch. After Open Day promo, conversation volume spikes. Gap detection will increase. Review New tab, filter by category "Open Day" — likely users asked details not in KB.

Debugging complaint. User complains "bot doesn't know info X". Admin opens gaps dashboard, searches for pattern matching X. If gap already exists, accepted but not resolved — prioritize. If not yet detected, add manually (future feature: manual gap creation).

Troubleshooting

No new gaps for a week. Symptom: New tab empty. Cause 1: traffic dropped drastically (check Analytics). Cause 2: detection job may not be running — check System Health.

Low gap accuracy (many false positives). Symptom: gaps that bot already answers well are flagged. Cause: detection threshold too loose. Workaround: dismiss false positives diligently — system doesn't auto-learn yet, needs developer to tune threshold.

Empty sample questions. Symptom: gap card doesn't show example questions. Cause: gap created from aggregate pattern, not specific questions. Fix: click View Conversations to see full context, or search manually in /conversations with keywords from the title.

See also