docs-tune-ai-chat — Tune AI chat system prompt from real feedback

Workflow#

Verify MCP and plan — confirm MCP transport is up and the workspace is on PRO or PRO+. On Free plan, stop and print an upgrade prompt. Confirm with the user that they want to modify the system prompt before proceeding.
Pull negative feedback — fetch 30 days of thumbs-down AI chat interactions, including the user question, the AI answer, and any free-text reason given.
Pull unanswered questions — fetch 30 days of interactions where the AI explicitly said it didn't know or retrieval returned nothing useful.
Cluster by topic — group the combined signal into 3–8 topic clusters. For each cluster, record a label, item count, up to three sample questions, and a one-sentence description of the inferred failure mode.
Generate a prompt update — read the current system_prompt. Produce a minimally invasive replacement that keeps all existing brand voice, persona, and refusal rules intact, and adds explicit guidance for the top 3–5 clusters. Cap the result at 1,500 tokens.
Show the diff — render a before/after diff with annotations mapping each changed chunk back to the cluster that motivates it.
Apply on confirmation — call set_chat_system_prompt only after the user explicitly confirms. Accept yes, no, or edit; on edit, loop back to the diff step with the user's revised version.
Report — confirm the update was applied, include the timestamp, and suggest a re-tune date 3 weeks out.

Never call set_chat_system_prompt without explicit yes from the user. This is a destructive write that replaces the prompt for all chat sessions on the workspace.
Never invent feedback clusters — only use data returned by the MCP tools. If both sources are empty, stop and tell the user there is nothing to tune yet.
Do not strip existing brand or persona instructions unless an instruction is directly causing the identified failure pattern.
Proposed prompts over 1,500 tokens must be compressed before showing the diff — longer prompts degrade chat quality.
Do not tune on fewer than 5 combined signal items — surface this as "not enough data" rather than speculating.

Tool	Purpose
`list_workspaces`	Probe MCP transport liveness
`get_workspace`	Read workspace plan and current system prompt
`get_negative_feedback`	Fetch thumbs-down AI chat interactions
`get_ai_unanswered`	Fetch questions the AI failed to answer
`set_chat_system_prompt`	Apply the confirmed new system prompt