================================================================
AI FLUENCY — CHEAT SHEET
Safety & Alignment
================================================================

RLHF, Constitutional AI, jailbreaks, prompt injection, red teaming.
6 terms

----------------------------------------------------------------

* Constitutional AI
  An alignment approach where a model critiques and revises its own outputs against a written set of principles, reducing reliance on human labeling for every case.
  e.g. A model checking its own answer against a rule like "avoid harmful instructions."

* Guardrails
  Automated checks (filters, classifiers, rules) placed around a model to catch or block unsafe or off-policy outputs.
  e.g. A content filter blocking a response before it reaches the user.

* Jailbreak
  A prompt crafted to bypass a model's safety training and get it to produce disallowed content.
  e.g. Disguising a harmful request as a "fictional story".

* Prompt Injection
  An attack where malicious instructions are hidden in content a model processes (a webpage, document, email) to hijack its behavior.
  e.g. A webpage containing hidden text telling an AI assistant to leak private data.

* Red Teaming
  Deliberately probing a model or system for weaknesses, harmful outputs, or exploitable behavior before it is deployed.
  e.g. A security team trying dozens of jailbreak prompts before launch.

* RLHF (Reinforcement Learning from Human Feedback)
  A training technique that uses human preference judgments to reward a model for outputs people rate as better, aligning it with human intent.
  e.g. Raters rank two model responses; the model is trained to prefer the higher-ranked one.

----------------------------------------------------------------
Tip: paste this file into your favorite AI assistant and ask to be tutored on it.