================================================================
AI FLUENCY — CHEAT SHEET
LLM Mechanics
================================================================

How LLMs run: sampling, context windows, embeddings, KV cache, quantization.
6 terms

----------------------------------------------------------------

* Context Window
  The maximum number of tokens (input plus output) a model can consider at one time.
  e.g. A 200K-token context window can hold roughly a 500-page book.

* Embedding
  A dense numeric vector representing the meaning of text, used for search and similarity comparisons.
  e.g. "dog" and "puppy" have embeddings that sit close together.

* KV Cache
  Cached attention key/value tensors from prior tokens that let a model avoid recomputing them for every new token, speeding up generation.
  e.g. Long conversations reuse the cache instead of reprocessing all prior history.

* Logits
  The raw, unnormalized scores a model outputs for each possible next token before they are converted into probabilities.
  e.g. A softmax function turns logits into a probability distribution.

* Quantization
  Reducing the numeric precision of a model's weights (e.g. 16-bit to 4-bit) to shrink memory use and speed up inference, at some accuracy cost.
  e.g. Running a large model on a laptop GPU via 4-bit quantization.

* Sampling / Temperature
  Controls used to pick the next token from the model's probability distribution; temperature adjusts how random that choice is.
  e.g. Temperature 0 gives deterministic, repeatable answers.

----------------------------------------------------------------
Tip: paste this file into your favorite AI assistant and ask to be tutored on it.