================================================================
AI FLUENCY — CHEAT SHEET
Fine-tuning & Adaptation
================================================================

LoRA, PEFT, RLHF mechanics, DPO, instruction tuning, and domain adaptation.
6 terms

----------------------------------------------------------------

* Catastrophic Forgetting
  The tendency of a neural network to overwrite previously learned general knowledge when it's fully fine-tuned on a new, narrow dataset.
  e.g. A model losing general conversational ability after heavy fine-tuning on legal documents.

* DPO (Direct Preference Optimization)
  A method that fine-tunes a model directly on chosen-versus-rejected response pairs, without training a separate reward model or running reinforcement learning.
  e.g. Training a model to prefer one of two candidate replies using a single classification-style loss.

* Fine-Tuning
  Continuing to train a pre-trained model on a smaller, task- or domain-specific dataset to specialize its behavior.
  e.g. Taking a general model and further training it on customer-support transcripts.

* Instruction Tuning
  Fine-tuning a base model on (instruction, response) pairs so it reliably follows natural-language directions.
  e.g. Turning a raw text-completion model into a helpful assistant.

* LoRA (Low-Rank Adaptation)
  A PEFT technique that injects small trainable low-rank matrices into a model's weight layers instead of updating the full weights.
  e.g. Adapting a model's style with a LoRA file a few hundred megabytes in size.

* PEFT (Parameter-Efficient Fine-Tuning)
  A family of fine-tuning methods that train a small set of extra parameters while keeping most base-model weights frozen.
  e.g. Fine-tuning a 70B model by training only a few million adapter parameters.

----------------------------------------------------------------
Tip: paste this file into your favorite AI assistant and ask to be tutored on it.