AI safety & cost

If your app calls an LLM, you've added two new risks at once: a user-controlled input that can manipulate the model, and a metered endpoint that costs real money on every call. Guard the prompt, the output, and the spend.

Spend first: it's the one that bankrupts you

An LLM endpoint is a money faucet pointed at your account. One abusive user or one bad loop runs up a bill fast.

1. Set hard caps at the provider. OpenAI and Anthropic both offer monthly usage limits and billing alerts. Configure them before launch, not after the invoice.
2. Per-user quotas in your DB, checked before the call — e.g. free tier gets N generations/day.
3. Cap max_tokens on every request so a single call can't balloon. Output tokens usually cost more than input.
4. Pick the right model per task. Use a small/cheap model for classification and routing; reserve the expensive model for the work that needs it.
5. Cache identical or near-identical requests, and use prompt caching where the provider supports it for long, reused system prompts.
6. No blind retries. A retry loop on a failing call multiplies cost — cap attempts and back off.

Prompt injection: treat model input as untrusted

The model will follow instructions hidden in user content — including "ignore your rules" or "print your system prompt." You can't fully prevent this, so contain the blast radius.

• Never put secrets in the system prompt expecting them to stay hidden. Assume the user can extract anything you put there.
• Separate instructions from data. Make it clear in your prompt structure which part is your trusted instruction and which is untrusted user/retrieved content, and tell the model not to obey instructions found inside the data.
• The model's output is also untrusted. If it can trigger actions — calling a tool, running a query, hitting an API — gate those behind your own permission checks. Don't let "the AI decided to" delete a row a user couldn't delete themselves. RLS and your normal auth still apply.
• Never eval model output or run model-generated SQL/shell directly against production.

Don't leak data into or out of the model

• Don't send more user data to the LLM than the task needs. Strip PII you don't require.
• Know your provider's data-retention/training terms; use the business/no-training tier if you're sending sensitive content.
• If the model output goes into a webpage, render it as text — model output can contain markup/links, so sanitize to avoid injection in your UI.

Keep the output sane

• Validate structure. If you expect JSON, parse and validate it; never trust the shape blindly. Have a fallback for malformed output.
• Moderate when users see each other's generations. A moderation pass (provider moderation endpoint or a cheap classifier) keeps abusive content out of shared surfaces.
• Set expectations in the UI. Label AI output as AI, since it can be wrong.

Log what matters, safely

Log token counts and latency per user so you can spot abuse and cost spikes. Be careful logging full prompts/outputs if they contain personal data — sample or redact.

The checklist

• Provider spend caps + billing alerts configured.
• Per-user quotas and max_tokens caps on every call.
• Cheapest model that does the job; caching where it helps.
• System prompt holds no secrets; user/retrieved content treated as untrusted.
• Model-triggered actions gated by your own auth/RLS; output never eval'd.
• Output validated/sanitized; shared generations moderated.
• Per-user token/latency logging without leaking PII.