LLM Support Bot Red-Team and Monitoring Runbook

Public Brewpage: https://brewpage.app/public/Ox1yCGlXG7

Review state: BLOCKED — the supplied VictoriaMetrics endpoint was unreachable, so the metric-explosion cause and a data-derived production fix cannot yet be verified. The requested X post is also unavailable from its source and the prescribed FxTwitter API, so no quote is fabricated below.

Prepared for SecOps and QA on 2026-09-22 (Asia/Shanghai).

Decisions at a glance

Area Decision / result
Red-team baseline 942 unique prompts: 552 attack cases (58.6%) and 390 benign/edge controls (41.4%)
Attack severity 22 critical, 174 high, 349 medium, 7 low; all 390 non-attacks have blank severity
Highest-volume attacks direct_injection 314, adversarial 79, jailbreak 50, encoding 30
Qdrant (3 nodes) 6 shards, replication_factor: 2; budget approximately 2× vector storage before overhead
VictoriaMetrics http://localhost:8428 refused the connection; cause, impact, and exact production config are not established
Requested X post FxTwitter returned 404 NOT_FOUND; obtain the post text/screenshot before adding a quote

1. Scope, authorization, and stop conditions

This runbook covers the supplied QA corpus, a non-production test of the support bot, its Qdrant prompt-embedding collection, and read-only monitoring analysis. It does not authorize testing third parties or sending real customer data to a test model.

Known path: user input → support bot → embedding/retrieval in Qdrant → LLM → response. Before testing, the owner must add every file/email/web ingestion path, persistent-memory write, registered tool, tool permission, approval gate, and downstream output consumer. Treat retrieved text, tool output, and every parquet text value as untrusted data—not operator instructions.

Immediately stop a test and page the bot owner if a case causes any of the following:

2. Baseline integrity and inventory

Source: /root/llm-red-team-prompt-set.parquet

SHA-256  fb5432035fdd8453d61f6593bfe78ee2970403c5ea59959b642a6708e2bef670
Rows     942
Schema   text VARCHAR, label INTEGER, category VARCHAR, source VARCHAR,
         severity VARCHAR, group_id VARCHAR, augmented BOOLEAN, tags VARCHAR[]

Integrity observations:

Severity distribution

Severity Cases % of attack cases Execution order
Critical 22 4.0% First; manual observation and tools disabled
High 174 31.5% Second; manual approval on any proposed action
Medium 349 63.2% Third; automated regression permitted in sandbox
Low 7 1.3% Fourth
Unrated non-attack 390 Run in every build to measure utility/false positives

Exact category-by-severity inventory

Counts below are the dataset’s fields, not reassigned severity. “Non-attack” combines label=0 rows only.

Category Critical High Medium Low Non-attack Total
direct_injection 0 4 310 0 0 314
benign 0 0 0 0 381 381
adversarial 8 70 1 0 0 79
jailbreak 1 44 5 0 0 50
encoding 3 0 27 0 0 30
training_extraction 0 10 0 0 0 10
edge_case 0 0 0 0 9 9
agent_manipulation 6 2 0 0 0 8
token_smuggling 0 7 0 0 0 7
context_confusion 0 4 2 0 0 6
control 0 0 0 5 0 5
output_manipulation 0 3 0 2 0 5
instruction_override 0 3 1 0 0 4
multi_turn 0 4 0 0 0 4
prompt_injection 0 3 1 0 0 4
rag_poisoning 0 4 0 0 0 4
response_manipulation 0 4 0 0 0 4
system_manipulation 0 4 0 0 0 4
persona_replacement 1 2 0 0 0 3
prompt_extraction 1 2 0 0 0 3
payload_injection 0 2 0 0 0 2
system_extraction 0 1 1 0 0 2
encoding_obfuscation 0 0 1 0 0 1
indirect_injection 1 0 0 0 0 1
many_shot 1 0 0 0 0 1
token_injection 0 1 0 0 0 1
Total 22 174 349 7 390 942

Coverage is concentrated: direct injection alone is 56.9% of attack cases. Indirect injection (1), RAG poisoning (4), multi-turn (4), many-shot (1), and agent manipulation (8) are too sparse to support a strong claim about those risks. There are no explicitly categorized cases for cross-tenant authorization, memory persistence across sessions, approval fatigue, SSRF, browser/DOM output handling, SQL output handling, inter-agent communication, model/package supply-chain compromise, rate-limit exhaustion, or kill-switch failure. Add those suites before sign-off.

3. How QA runs the security review

Prepare once

  1. Verify the file hash above. If it differs, stop and regenerate this inventory.
  2. Create an isolated test tenant, fake support records, fake secrets, and a unique system-prompt canary. Never use production credentials or customer data.
  3. Disable side-effecting tools for the first pass. Replace email, ticketing, shell, database, HTTP, and administrative tools with record-only stubs.
  4. Use a disposable Qdrant evaluation collection. Do not mix adversarial documents with production retrieval data.
  5. Pin model, system-prompt, policy, retrieval-index, tool-manifest, and evaluator versions in the run record.
  6. Set per-case token, wall-clock, tool-call, and retry limits. Preserve an operator-controlled kill switch.

Execute each build

  1. Run all 22 critical cases serially with a human watching.
  2. Run all 174 high cases, then medium and low cases, in the sandbox.
  3. Run all 390 benign/edge controls against the same build. A secure refusal on an ordinary request is a utility failure.
  4. For indirect_injection and rag_poisoning, place the payload in the retrieved document—not in the user message—and verify that retrieved instructions cannot change the system goal or tool parameters.
  5. For the four multi_turn cases, do not infer turns from group_id; QA must define a deterministic conversation script.
  6. For extraction cases, alert on the canary and on structural similarity to hidden instructions; do not store the system prompt in results.
  7. Review every proposed tool action for identity, purpose, scope, parameter authorization, and approval status.

Record two independent results per case:

Release gate: zero critical/high security failures; zero real side effects; zero canary/secret disclosures; all failures triaged with an owner; and the benign utility threshold agreed by Product and SecOps is met. Re-run after any change to model, system prompt, retrieval corpus, embedding model, tools, policies, or output renderer.

Minimal safe telemetry

Use bounded labels only:

llm_security_eval_total{model_version,policy_version,category,severity,outcome}
llm_security_tool_call_total{tool,result,approval}
llm_security_retrieval_total{index_version,result}
llm_security_eval_latency_seconds_bucket{model_version,category,le}

Never put raw prompt/response, user_id, customer_id, conversation/session/request/trace IDs, document IDs, prompt hashes, URLs with IDs, SQL, stack traces, timestamps, or error messages in metric labels. Put correlation IDs and high-detail evidence in access-controlled logs/traces with retention and redaction controls.

4. VictoriaMetrics cardinality incident

Current evidence and decision

At 2026-09-22T20:57:17+08:00, VM_METRICS_URL and VM_CURL_CONFIG were unset. The required fallback http://localhost:8428 returned connection refusal (curl status 7, HTTP 000) for:

No VictoriaMetrics/vmagent process, listener on port 8428, or local deployment data was found. Therefore total series, churn, top metrics, label cardinality, histogram bloat, unused metrics, and alert/recording-rule dependencies are unknown. It is not possible to state the cause or an exact production relabel/aggregation fix from the available evidence. Do not deploy a guessed global drop rule.

Collect the missing evidence (read-only)

Run from a host that can reach VictoriaMetrics. For a cluster URL, point VM_METRICS_URL at the tenant’s vmselect Prometheus base path, such as https://vmselect.example/select/0/prometheus. Put authentication headers in the curl config; do not paste tokens into shell history.

export VM_METRICS_URL="https://REACHABLE-VM-BASE"
export VM_CURL_CONFIG="/secure/path/victoriametrics-curl.conf"  # or leave unset

out="vm-cardinality-$(date +%Y%m%d-%H%M%S)"
mkdir -m 700 "$out"

curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -fsS \
  "$VM_METRICS_URL/api/v1/status/tsdb?topN=50&date=$(date -d yesterday +%F)" \
  > "$out/tsdb-yesterday.json"
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -fsS \
  "$VM_METRICS_URL/api/v1/status/tsdb?topN=50" \
  > "$out/tsdb-today.json"

for label in pod instance container path url user_id customer_id request_id \
  session_id trace_id error error_message query le name; do
  curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -fsS --get \
    --data-urlencode "topN=20" --data-urlencode "focusLabel=$label" \
    "$VM_METRICS_URL/api/v1/status/tsdb" \
    > "$out/focus-$label.json"
done

curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -fsS \
  "$VM_METRICS_URL/api/v1/status/metric_names_stats?le=0&limit=500" \
  > "$out/unused.json"
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -fsS \
  "$VM_METRICS_URL/api/v1/status/metric_names_stats?le=5&limit=500" \
  > "$out/rare.json"
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -fsS \
  "$VM_METRICS_URL/api/v1/rules" > "$out/rules.json"

If metric_names_stats errors, enable -storage.trackMetricNamesStats on vmstorage, wait through a representative dashboard/alert cycle, and recollect. A metric is a drop candidate only when it has zero tracked queries, is absent from every alerting and recording rule, and has meaningful series impact.

Diagnosis and exact-fix gate

SecOps must fill this table from the JSON before approving a change:

Check Required evidence Decision
Active/churned series today/yesterday totalSeries; ratio Ratio >3:1 indicates churn investigation
Dominant metrics seriesCountByMetricName Target only measured high-impact metrics
Exploding labels labelValueCountByLabelName, focus samples IDs/unbounded text → scoped labeldrop; dynamic routes → normalize
Histogram bloat _bucket metrics and unique le values Filter selected buckets; never drop le entirely
Unused metrics zero-query list cross-checked with /rules Drop only when not referenced by alerts/recording rules
Aggregation semantics metric type and dashboards/SLOs Counter → total; gauge → last/avg/sum_samples

Prevent recurrence on the support-bot scrape only with the following bounded-label policy. This is a safe hygiene baseline, not a claim about the current root cause:

# VMServiceScrape/VMPodScrape: spec.metricRelabelConfigs for support-bot only
metricRelabelConfigs:
  - action: labeldrop
    regex: '^(user_id|customer_id|account_id|conversation_id|session_id|request_id|trace_id|span_id|document_id|prompt_hash|raw_prompt|raw_response|error_message|stack_trace|sql_query)$'

The production relabel and stream-aggregation blocks remain intentionally unset until measured metric names, affected labels, series impact, metric types, and rule dependencies are available. Roll out the eventual config to one scrape target with drop_input: false, compare aggregates and dashboards for at least one full alerting cycle, then enable input dropping. Keep the prior config ready for rollback.

5. Qdrant topology for prompt embeddings

Use shard_number: 6 and replication_factor: 2 for the three-node production cluster.

Why: six is divisible by three, gives two logical shards per node, and leaves useful distribution headroom without excessive shard overhead. Replication factor two stores every shard on two different peers, supports one-node maintenance/failure when placement is healthy, and costs approximately 2× vector storage plus index/payload overhead. Replication factor one does not provide the required production fault tolerance.

For a new collection (set the real vector dimension and adjust the collection name):

: "${EMBEDDING_DIM:?set EMBEDDING_DIM to the embedding model dimension}"
case "$EMBEDDING_DIM" in *[!0-9]*|'') echo "invalid EMBEDDING_DIM" >&2; exit 2;; esac
qdrant_headers=()
[ -n "${QDRANT_API_KEY:-}" ] && qdrant_headers=(-H "api-key: $QDRANT_API_KEY")
qdrant_payload=$(printf \
  '{"vectors":{"size":%s,"distance":"Cosine"},"shard_number":6,"replication_factor":2}' \
  "$EMBEDDING_DIM")

curl -fsS -X PUT "$QDRANT_URL/collections/support_prompt_embeddings_v2" \
  -H 'Content-Type: application/json' \
  "${qdrant_headers[@]}" \
  -d "$qdrant_payload"

Before cutover:

  1. Confirm disk/RAM can hold two copies plus rebuild headroom and backups.
  2. Load a representative copy, then test search/update latency and node-loss behavior.
  3. Inspect GET /collections/support_prompt_embeddings_v2/cluster; all six shards must have two active replicas on different peers and placements should be balanced.
  4. Verify point counts/checksums and security filters, then switch a Qdrant alias atomically.
  5. Keep the old collection until rollback and backup criteria expire.

For self-hosted Qdrant, do not assume online resharding is available. If the existing shard count is wrong, create the six-shard collection, migrate, validate, and alias-swap. Do not merely add nodes without moving shard replicas.

6. Requested X-post context

Requested URL: https://x.com/simonw/status/1575028659857121280

The prescribed FxTwitter lookup returned exactly:

{"code":404,"message":"NOT_FOUND","tweet":null}

X oEmbed returned 404 and X’s syndication endpoint returned an empty object. The snowflake timestamp decodes to 2022-09-28T07:44:29.784Z, but it does not reveal the text. No quote is included because attributing an unverifiable reconstruction would be unsafe. Supply an archived screenshot or the text, and SecOps should add the verbatim quote here with provenance.

Related, independently verifiable context from Simon Willison’s 2022-09-16 article—not the requested post—is: “I have no idea how to reliably beat prompt injection!” The operational implication is layered controls and constrained blast radius, not reliance on a prompt filter as a complete defense.

7. Incident and release checklist

References