Skip to content

AI Suggestions Review per Cluster

Purpose

This page covers the PIC (per-cluster reviewer) workflow for clearing the AI Suggestions queue — Q/A candidates produced by AI Crawler v1 and waiting for validation before they enter the chatbot's knowledge base.

The goal of this review: every question + answer must be accurate, on-context for UPH, and free of LLM hallucinations.

Prerequisites

  • Access to https://form.huph.val.id (you need the SITE_PASSWORD from the admin team — see Site Gate)
  • A valid magic link for a PIC session matching the cluster you review (CASS / CBT / CHS / CIST / CNE)
  • Familiarity with the factual content of the UPH web pages that seeded the candidates (currently: 5 official faculties on uph.edu)

Workflow

1. Open the cluster portal

Visit https://form.huph.val.id/cluster/<CODE> (e.g. /cluster/CASS).

Once the gate cookie is set, switch to the AI Suggestions tab in your cluster portal.

2. Read the filter pills

Four pills above the list show review status counts for your cluster:

Pill Meaning What's there
Pending Not yet reviewed Your queue — main focus
Accepted Approved as-is Audit history (read-only)
Rejected Declined with reason Audit history (read-only)
Edited Edited then approved Audit history (read-only)

The number next to each label is the row count for that status. The counter below the pills (M dari N total telah direview) shows total cluster progress.

3. Per-row actions

Each suggestion card shows:

  • Question (textarea — editable)
  • Answer (textarea — editable)
  • Source URL with a link to the originating page (click "Show source passage" to see the verbatim quote the AI used)

Four decisions you can take:

Decision Button When to use
Accept Amber button when textarea is unedited Question + answer are correct, no changes needed
Edit & Accept Replaces Accept once you start editing Content is correct but needs polish (grammar, UPH terminology)
Reject Grey button, requires reason ≥10 chars Factual error, hallucination, or out of cluster scope
Skip (no button — review is sequential) Refresh the page; row returns to Pending

Bulk accept for clean batches

If 8-9 candidates on a page all look correct without edits, use the "Accept N di halaman ini" button at the top. Two-click confirm (Yakin accept N) prevents misclicks. The button is disabled if any row has an in-flight textarea edit — finish or discard those first.

4. Pagination

If your cluster has more than 25 pending, Sebelumnya / Berikutnya buttons appear below the list with a "Halaman X dari Y" indicator. Switching pages does not preserve unsubmitted textarea edits — finish those before navigating.

Reject criteria

Reject with a clear reason (≥10 chars) when:

  1. Factual error — figures, program names, or claims that aren't in the source or contradict other UPH pages.
  2. Hallucination — sentence sounds plausible but has no anchor in the source quote (verify by clicking "Show source passage").
  3. Out of cluster scope — e.g. a medicine-page candidate landing in CASS.
  4. Generic/vague — question is too broad to be actionable for a prospective student (e.g. "Apakah UPH bagus?").
  5. Duplicate — candidate is near-identical to an existing FAQ or Live KB entry.

Reject reasons are stored in the audit column and feed back into prompt tuning for the next AI iteration.

After you accept

  • Row status flips to accepted or edited_accepted
  • reviewed_by_session_id + reviewed_at are recorded for audit
  • The row is not automatically published to FAQ or Live KB — a later phase (not yet shipped) will copy accepted suggestions into faq_local after a final admin team review

For now, accepted = "ready to promote" and rejected = "do not use". The Accepted / Rejected tabs are an audit trail of your decisions.

FAQ

How often does the Pending tab refill?

The AI Crawler is currently triggered manually by the engineering team (no automatic cron yet). When the queue empties and you want a new batch, ping engineering. The cron schedule will be enabled once PIC review patterns stabilize.

What if a source URL turns out to be a 404?

Reject with the reason "source not accessible — broken link". Engineering will update URL_SEEDS in the crawler.

Can I edit a row after accepting?

Not from the AI Suggestions tab — move to FAQ or Live KB management after the row is promoted into faq_local. The Accepted tab here is read-only (disabled textarea).

See also