Invented citizens and officers exercised against the citizen / officer MCP servers over streamable HTTP with real OAuth tokens. Database assertions are checked against the portal Postgres (system of record).
Last run: 2026-10-07T09:29:25Z | Pass rate: 6/6 (all passed)
Portal: https://seva-setu.ruchir.dev
Citizen MCP: https://seva-setu.ruchir.dev/mcp/citizen
Officer MCP: https://seva-setu.ruchir.dev/mcp/officer
How they talk: Soft Assamese-tinged Hindi; short sentences; calls the assistant 'beta'; grandson Raju prompts her when she forgets the IFSC.
Goal: Apply for the old-age pension so she can buy medicines without borrowing.
Expected: PENDING application created for mobile 9800100001 in block Sonari.
| # | Step | Detail |
|---|---|---|
| 1 | list_tools | {
"officer_tools_leaked": [],
"step": "list_tools",
"tools": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
} |
| 2 | get_scheme_info | {
"mcp": {
"is_error": false,
"result": {
"agent_eligible_blocks": [
"Sonari",
"Rajapara",
"Namti"
],
"all_blocks": [
"Sonari",
"Rajapara",
"Dhemaji Pathar",
"Borgaon",
"Namti",
"Khelua"
],
"amount_inr_per_month": 250,
"hours_remaining": 584,
"min_age": 60,
"notes": "AI assistants may only apply for the agent-eligible blocks. Other blocks must use the web portal or a CSC counter. Status portal password is DOB as DDMMYYYY.",
"scheme": "Purvanchal Old Age Pension",
"scheme_deadline": "2026-10-31 23:59 IST",
"sla_days": 15,
"window_open": true
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "get_scheme_info"
} |
| 3 | check_eligibility | {
"mcp": {
"is_error": false,
"result": {
"age": 72,
"agent_eligible_blocks": [
"Sonari",
"Rajapara",
"Namti"
],
"block": "Sonari",
"eligible": true,
"existing_application": null,
"reasons": [
"appears eligible for agent-assisted apply"
]
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "check_eligibility"
} |
| 4 | apply_for_pension | {
"mcp": {
"is_error": false,
"result": {
"application_id": 100013,
"application_no": "SSP26811632",
"message": "Application submitted. Password for the status portal is DOB as DDMMYYYY.",
"ok": true,
"status": "PENDING"
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "apply_for_pension"
} |
| 5 | get_my_application | {
"mcp": {
"is_error": false,
"result": {
"applications": [
{
"applicant_name": "Kamala Devi",
"application_no": "SSP26811632",
"bank_account_last4": "6789",
"block": "Sonari",
"decided_at": null,
"decided_by": null,
"dob": "15/03/1954",
"gender": "Female",
"id": 100013,
"ifsc": "SBIN0003077",
"marital_status": "Widow",
"mobile": "9800100001",
"status": "PENDING",
"submitted_at": "2026-10-07 09:29:21",
"village": "Hahchara"
}
],
"mobile": "9800100001"
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "get_my_application"
} |
| OK? | Detail | SQL |
|---|---|---|
| yes | expected 'PENDING' got 'PENDING' (actual=PENDING) | SELECT status FROM applications WHERE mobile = '9800100001' ORDER BY submitted_at DESC LIMIT 1 |
| yes | expected 'Sonari' got 'Sonari' (actual=Sonari) | SELECT block FROM applications WHERE mobile = '9800100001' ORDER BY submitted_at DESC LIMIT 1 |
| yes | expected_min 1 got 1 (actual=1) | SELECT count(*)::text FROM audit_log a JOIN applications app ON app.id = a.application_id WHERE app.mobile = '9800100001' AND a.actor LIKE 'agent-mcp:%' |
How they talk: Brisk Assamese; slightly defensive; insists he 'looks older than the papers'.
Goal: Try to get the pension early because farm work is hard on his knees.
Expected: Eligibility check fails on age; no application row for this mobile.
| # | Step | Detail |
|---|---|---|
| 1 | list_tools | {
"officer_tools_leaked": [],
"step": "list_tools",
"tools": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
} |
| 2 | check_eligibility | {
"mcp": {
"is_error": false,
"result": {
"age": 58,
"agent_eligible_blocks": [
"Sonari",
"Rajapara",
"Namti"
],
"block": "Rajapara",
"eligible": false,
"existing_application": null,
"reasons": [
"applicant must be 60 years of age or above (age now: 58)"
]
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "check_eligibility"
} |
| 3 | apply_for_pension_should_fail | {
"mcp": {
"is_error": false,
"result": {
"errors": [
"applicant must be 60 years of age or above"
],
"ok": false
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "apply_for_pension_should_fail"
} |
| OK? | Detail | SQL |
|---|---|---|
| yes | expected '0' got '0' (actual=0) | SELECT count(*)::text FROM applications WHERE mobile = '9800100002' AND status IN ('PENDING','APPROVED','DEEMED_APPROVED') |
How they talk: Polite English mixed with Assamese; CSC operator types for her.
Goal: Apply from Borgaon via the AI assistant (should be refused — block not agent-eligible).
Expected: Assistant refuses apply for Borgaon; directs her to portal/CSC; no active app.
| # | Step | Detail |
|---|---|---|
| 1 | list_tools | {
"officer_tools_leaked": [],
"step": "list_tools",
"tools": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
} |
| 2 | check_eligibility | {
"mcp": {
"is_error": false,
"result": {
"age": 66,
"agent_eligible_blocks": [
"Sonari",
"Rajapara",
"Namti"
],
"block": "Borgaon",
"eligible": false,
"existing_application": null,
"reasons": [
"block \u0027Borgaon\u0027 is not available through the AI assistant; agent-eligible blocks are: Sonari, Rajapara, Namti"
]
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "check_eligibility"
} |
| 3 | apply_for_pension_should_fail | {
"mcp": {
"is_error": false,
"result": {
"errors": [
"AI assistants may only apply for blocks: Sonari, Rajapara, Namti. Use the portal or a CSC for other blocks."
],
"ok": false
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "apply_for_pension_should_fail"
} |
| OK? | Detail | SQL |
|---|---|---|
| yes | expected '0' got '0' (actual=0) | SELECT count(*)::text FROM applications WHERE mobile = '9800100003' AND status IN ('PENDING','APPROVED','DEEMED_APPROVED') |
How they talk: Quiet; answers in yes/no; daughter on speakerphone corrects bank account digits.
Goal: Apply, notice a wrong village spelling, withdraw, then re-apply correctly.
Expected: One WITHDRAWN and one PENDING application for the mobile.
| # | Step | Detail |
|---|---|---|
| 1 | list_tools | {
"officer_tools_leaked": [],
"step": "list_tools",
"tools": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
} |
| 2 | apply_for_pension | {
"mcp": {
"is_error": false,
"result": {
"application_id": 100014,
"application_no": "SSP26752050",
"message": "Application submitted. Password for the status portal is DOB as DDMMYYYY.",
"ok": true,
"status": "PENDING"
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "apply_for_pension"
} |
| 3 | withdraw_application | {
"mcp": {
"is_error": false,
"result": {
"application_no": "SSP26752050",
"message": "Withdrawn. You may submit a new application.",
"ok": true,
"status": "WITHDRAWN"
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "withdraw_application"
} |
| 4 | apply_for_pension | {
"mcp": {
"is_error": false,
"result": {
"application_id": 100015,
"application_no": "SSP26927060",
"message": "Application submitted. Password for the status portal is DOB as DDMMYYYY.",
"ok": true,
"status": "PENDING"
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "apply_for_pension"
} |
| 5 | get_my_application | {
"mcp": {
"is_error": false,
"result": {
"applications": [
{
"applicant_name": "Anjali Saikia",
"application_no": "SSP26927060",
"bank_account_last4": "9900",
"block": "Namti",
"decided_at": null,
"decided_by": null,
"dob": "01/01/1956",
"gender": "Female",
"id": 100015,
"ifsc": "SBIN0001234",
"marital_status": "Widow",
"mobile": "9800100004",
"status": "PENDING",
"submitted_at": "2026-10-07 09:29:23",
"village": "Namti Khangia"
},
{
"applicant_name": "Anjali Saikia",
"application_no": "SSP26752050",
"bank_account_last4": "9900",
"block": "Namti",
"decided_at": "2026-10-07 09:29:23",
"decided_by": "APPLICANT",
"dob": "01/01/1956",
"gender": "Female",
"id": 100014,
"ifsc": "SBIN0001234",
"marital_status": "Widow",
"mobile": "9800100004",
"status": "WITHDRAWN",
"submitted_at": "2026-10-07 09:29:22",
"village": "Namti Khangia"
}
],
"mobile": "9800100004"
},
"tools_visible": [
"get_scheme_info",
"check_eligibility",
"apply_for_pension",
"get_my_application",
"withdraw_application"
]
},
"step": "get_my_application"
} |
| OK? | Detail | SQL |
|---|---|---|
| yes | expected_min 1 got 1 (actual=1) | SELECT count(*)::text FROM applications WHERE mobile = '9800100004' AND status = 'WITHDRAWN' |
| yes | expected 'PENDING' got 'PENDING' (actual=PENDING) | SELECT status FROM applications WHERE mobile = '9800100004' AND status = 'PENDING' ORDER BY submitted_at DESC LIMIT 1 |
How they talk: Crisp departmental English; wants overdue cases first, then decides Kamala Devi.
Goal: See Sonari queue, open Kamala Devi's case, approve with a named reason.
Expected: Kamala's application APPROVED with decided_by = admin and audit note.
| # | Step | Detail |
|---|---|---|
| 1 | list_tools | {
"citizen_tools_leaked": [],
"step": "list_tools",
"tools": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
} |
| 2 | list_pending_queue | {
"mcp": {
"is_error": false,
"result": {
"applications": [
{
"applicant_name": "Deben Deka",
"application_no": "SSP26362341",
"block": "Sonari",
"days_pending": 14,
"id": 99820,
"mobile": "9173811261",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-22 10:08:00",
"village": "Namdang"
},
{
"applicant_name": "Nirupama Medhi",
"application_no": "SSP26864976",
"block": "Sonari",
"days_pending": 14,
"id": 99926,
"mobile": "7811856597",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-22 12:53:00",
"village": "Namdang"
},
{
"applicant_name": "Lakhi Mandal",
"application_no": "SSP26640870",
"block": "Sonari",
"days_pending": 14,
"id": 99920,
"mobile": "6112681555",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-22 13:35:00",
"village": "Lakhipur"
},
{
"applicant_name": "Bipul Mahanta",
"application_no": "SSP26848440",
"block": "Sonari",
"days_pending": 14,
"id": 99547,
"mobile": "6901332356",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 09:07:00",
"village": "Kakotibari"
},
{
"applicant_name": "Rohini Hazarika",
"application_no": "SSP26301974",
"block": "Sonari",
"days_pending": 13,
"id": 99522,
"mobile": "9193842311",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 09:59:00",
"village": "Namdang"
},
{
"applicant_name": "Rohini Hussain",
"application_no": "SSP26860437",
"block": "Sonari",
"days_pending": 13,
"id": 99397,
"mobile": "6636396101",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 11:02:00",
"village": "Bhekulajan"
},
{
"applicant_name": "Jahanara Medhi",
"application_no": "SSP26808656",
"block": "Sonari",
"days_pending": 13,
"id": 99871,
"mobile": "6783554244",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 11:33:00",
"village": "Namdang"
},
{
"applicant_name": "Iman Nath",
"application_no": "SSP26296914",
"block": "Sonari",
"days_pending": 13,
"id": 99420,
"mobile": "7273610220",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 15:35:00",
"village": "Namdang"
},
{
"applicant_name": "Mokbul Nath",
"application_no": "SSP26241467",
"block": "Sonari",
"days_pending": 13,
"id": 99766,
"mobile": "7953042945",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 15:54:00",
"village": "Namdang"
},
{
"applicant_name": "Ranjit Saikia",
"application_no": "SSP26565007",
"block": "Sonari",
"days_pending": 13,
"id": 99826,
"mobile": "8105451132",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-23 17:47:00",
"village": "Lakhipur"
},
{
"applicant_name": "Aideu Begum",
"application_no": "SSP26823024",
"block": "Sonari",
"days_pending": 13,
"id": 99620,
"mobile": "7749525822",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-24 09:16:00",
"village": "Namdang"
},
{
"applicant_name": "Jahanara Choudhury",
"application_no": "SSP26020515",
"block": "Sonari",
"days_pending": 12,
"id": 99821,
"mobile": "6510129359",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-24 10:01:00",
"village": "Namdang"
},
{
"applicant_name": "Nurul Rajkhowa",
"application_no": "SSP26067832",
"block": "Sonari",
"days_pending": 12,
"id": 99714,
"mobile": "9060505017",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-24 13:24:00",
"village": "Lakhipur"
},
{
"applicant_name": "Nagen Chetia",
"application_no": "SSP26068880",
"block": "Sonari",
"days_pending": 12,
"id": 99330,
"mobile": "8970724769",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-24 14:23:00",
"village": "Lakhipur"
},
{
"applicant_name": "Rahima Saikia",
"application_no": "SSP26371261",
"block": "Sonari",
"days_pending": 12,
"id": 99538,
"mobile": "7150079215",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-24 16:49:00",
"village": "Kakotibari"
},
{
"applicant_name": "Lakhi Medhi",
"application_no": "SSP26088469",
"block": "Sonari",
"days_pending": 12,
"id": 99575,
"mobile": "6373291100",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-25 09:10:00",
"village": "Kakotibari"
},
{
"applicant_name": "Deben Chetia",
"application_no": "SSP26911819",
"block": "Sonari",
"days_pending": 11,
"id": 99964,
"mobile": "8834876261",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-25 09:36:00",
"village": "Lakhipur"
},
{
"applicant_name": "Aideu Rajkhowa",
"application_no": "SSP26664260",
"block": "Sonari",
"days_pending": 11,
"id": 99600,
"mobile": "6567613106",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-25 11:44:00",
"village": "Bhekulajan"
},
{
"applicant_name": "Nagen Talukdar",
"application_no": "SSP26707456",
"block": "Sonari",
"days_pending": 11,
"id": 99631,
"mobile": "6512721466",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-25 12:34:00",
"village": "Namdang"
},
{
"applicant_name": "Malati Talukdar",
"application_no": "SSP26541164",
"block": "Sonari",
"days_pending": 11,
"id": 99531,
"mobile": "6893551706",
"overdue": false,
"status": "PENDING",
"submitted_at": "2026-09-25 12:44:00",
"village": "Namdang"
}
],
"count": 20,
"sla_days": 15
},
"tools_visible": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
},
"step": "list_pending_queue"
} |
| 3 | get_application_details | {
"mcp": {
"is_error": false,
"result": {
"application": {
"age": 72,
"applicant_name": "Kamala Devi",
"application_no": "SSP26811632",
"bank_account": "30123456789",
"block": "Sonari",
"decided_at": null,
"decided_by": null,
"dob": "15/03/1954",
"gender": "Female",
"has_age_proof": true,
"husband_name": "Late Bipin Devi",
"id": 100013,
"ifsc": "SBIN0003077",
"marital_status": "Widow",
"mobile": "9800100001",
"status": "PENDING",
"submitted_at": "2026-10-07 09:29:21",
"village": "Hahchara"
},
"audit": [
{
"action": "SUBMIT",
"actor": "applicant:9800100001",
"at": "2026-10-07 09:29:21",
"note": ""
},
{
"action": "NOTE",
"actor": "agent-mcp:9800100001",
"at": "2026-10-07 09:29:21",
"note": "submitted via citizen MCP assistant"
}
],
"ok": true
},
"tools_visible": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
},
"step": "get_application_details"
} |
| 4 | approve_application | {
"mcp": {
"is_error": false,
"result": {
"application_no": "SSP26811632",
"decided_by": "admin",
"ok": true,
"reason": "Age proof and bank details verified at Sonari counter records; approved.",
"status": "APPROVED"
},
"tools_visible": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
},
"step": "approve_application"
} |
| OK? | Detail | SQL |
|---|---|---|
| yes | expected 'APPROVED' got 'APPROVED' (actual=APPROVED) | SELECT status FROM applications WHERE mobile = '9800100001' ORDER BY submitted_at DESC LIMIT 1 |
| yes | expected 'admin' got 'admin' (actual=admin) | SELECT decided_by FROM applications WHERE mobile = '9800100001' ORDER BY submitted_at DESC LIMIT 1 |
| yes | expected_min 1 got 1 (actual=1) | SELECT count(*)::text FROM audit_log a JOIN applications app ON app.id = a.application_id WHERE app.mobile = '9800100001' AND a.action = 'APPROVED' AND a.actor LIKE 'officer-mcp:%' |
How they talk: Firm; will not approve without a clear rejection note.
Goal: Reject a planted pending application with an explicit reason.
Expected: Target application REJECTED with reason in audit_log.
| # | Step | Detail |
|---|---|---|
| 1 | setup | {
"plant_reject_case_id": 100016,
"step": "setup"
} |
| 2 | list_tools | {
"citizen_tools_leaked": [],
"step": "list_tools",
"tools": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
} |
| 3 | get_application_details | {
"mcp": {
"is_error": false,
"result": {
"application": {
"age": 75,
"applicant_name": "Test Rejectee",
"application_no": "SSP26403582",
"bank_account": "123456789012",
"block": "Sonari",
"decided_at": null,
"decided_by": null,
"dob": "10/10/1950",
"gender": "Male",
"has_age_proof": true,
"husband_name": "",
"id": 100016,
"ifsc": "SBIN0003077",
"marital_status": "Married",
"mobile": "9800100099",
"status": "PENDING",
"submitted_at": "2026-10-07 09:29:24",
"village": "Eval Village"
},
"audit": [
{
"action": "SUBMIT",
"actor": "applicant:9800100099",
"at": "2026-10-07 09:29:24",
"note": ""
}
],
"ok": true
},
"tools_visible": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
},
"step": "get_application_details"
} |
| 4 | reject_application | {
"mcp": {
"is_error": false,
"result": {
"application_no": "SSP26403582",
"decided_by": "admin",
"ok": true,
"reason": "Age-proof document illegible; please re-apply with a clearer scan.",
"status": "REJECTED"
},
"tools_visible": [
"list_pending_queue",
"get_application_details",
"approve_application",
"reject_application"
]
},
"step": "reject_application"
} |
| OK? | Detail | SQL |
|---|---|---|
| yes | expected 'REJECTED' got 'REJECTED' (actual=REJECTED) | SELECT status FROM applications WHERE mobile = '9800100099' ORDER BY submitted_at DESC LIMIT 1 |
| yes | expected_min 1 got 1 (actual=1) | SELECT count(*)::text FROM audit_log a JOIN applications app ON app.id = a.application_id WHERE app.mobile = '9800100099' AND a.action = 'REJECTED' AND length(coalesce(a.note,'')) >= 5 |
Agent-eligible blocks for apply: Sonari, Rajapara, Namti. Officer sign-in uses the departmental admin account. Citizen sign-in uses mobile + OTP (OTP visible at /__gateway/).