LoRA, PEFT, RLHF mechanics, DPO, instruction tuning, and domain adaptation.
The tendency of a neural network to overwrite previously learned general knowledge when it's fully fine-tuned on a new, narrow dataset.
e.g. A model losing general conversational ability after heavy fine-tuning on legal documents.A method that fine-tunes a model directly on chosen-versus-rejected response pairs, without training a separate reward model or running reinforcement learning.
e.g. Training a model to prefer one of two candidate replies using a single classification-style loss.Continuing to train a pre-trained model on a smaller, task- or domain-specific dataset to specialize its behavior.
e.g. Taking a general model and further training it on customer-support transcripts.Fine-tuning a base model on (instruction, response) pairs so it reliably follows natural-language directions.
e.g. Turning a raw text-completion model into a helpful assistant.A PEFT technique that injects small trainable low-rank matrices into a model's weight layers instead of updating the full weights.
e.g. Adapting a model's style with a LoRA file a few hundred megabytes in size.A family of fine-tuning methods that train a small set of extra parameters while keeping most base-model weights frozen.
e.g. Fine-tuning a 70B model by training only a few million adapter parameters.