{"chunks":[{"slug":"samplers-and-schedulers","title":"Samplers and Schedulers","heading":"intro","era":"multimodal","text":"> **A scheduler decides which noise levels a run visits; a sampler decides how to walk between two of them.** They are two jobs wearing one overloaded name, and once pulled apart, the guidance dials beside them — cfg_scale, negative prompts, the rescale trick — finally have a mechanism to attach to.","score":3},{"slug":"samplers-and-schedulers","title":"Samplers and Schedulers","heading":"The Analogy","era":"multimodal","text":"You're descending a mountain in thick fog with a printed itinerary. It lists the exact altitudes where you've agreed to stop and take stock — 1,200 meters, 1,050 meters, 900 meters, and so on down to the valley. Someone fixed those numbers in advance: how many stops there are, and whether they're spaced evenly or bunched near the bottom, where the terrain gets fiddly.\n\nThat list says *where* to stop, not *how* to get from one stop to the next — and between two altitudes you still have to walk through fog too thick to see your feet.\n\nThe laziest way down a segment: face straight downhill and walk that line until the altimeter reads the next stop. One glance, done — correct exactly when the slope between the two altitudes is constant. A more careful hiker takes that same guess, walks to where it lands, checks the slope from *there*, and splits the difference between the two headings. Twice","score":3},{"slug":"diffusion-pipelines","title":"Diffusion Pipelines","heading":"How It Works","era":"multimodal","text":"**Training teaches the squint.** Take a real photo, blend in a randomly chosen amount of noise, and ask the network one question: *what noise did I just add?* Repeat billions of times, at every noise level from nearly clean to pure static. The model never learns to draw — it learns to look at a corrupted image and estimate the corruption. That single skill, run in reverse from full static, is generation.\n\n**Inference is the wipe loop**, and a pipeline chains five parts around it:\n\n**Bracketing the loop is initialization: a noise latent of the target shape is drawn from the seed, and the prompt is encoded, both exactly once before the first step runs; the VAE decoder then runs exactly once after the last step.** Only the denoiser and scheduler actually execute per step — guidance is a setting on the denoiser's call, not a stage of its own.\n\n1. **Text encoder.** The prompt is split into to","score":1},{"slug":"image-model-failures","title":"What Image Models Get Wrong","heading":"How It Works","era":"multimodal","text":"**Hands and small lettering: no drawing order, plus a resolution floor underneath.** A diffusion denoiser updates the entire canvas at once, every step (see [[diffusion-pipelines]]) — there is no stage where it commits to \"draw the thumb, then the fingers, then the knuckle.\" Five fingers overlapping correctly is a global negotiation across a hundred nearby patches at once, and a small error compounds because nothing downstream can veto it. Underneath that sits a hard floor: a typical pipeline squeezes a 512px image into a 64x64 latent (see [[autoencoders-and-latents]]), so each cell stands in for an 8x8 pixel patch the decoder invents from one code. A stroke thinner than that patch has no smaller unit to occupy — the worked example below makes this concrete.\n\n**Text: the model has pixel statistics where it needs glyph identity.** A text-conditioned model learns that certain shapes correl","score":1},{"slug":"image-model-failures","title":"What Image Models Get Wrong","heading":"Common Pitfalls","era":"multimodal","text":"- **A hand is a texture, not a structure.** The denoiser updates the whole image at once with no drawing order, so a finger count is negotiated globally rather than committed to stroke by stroke — more sampling steps polish the texture without fixing the count.\n- **Text needs a sequence of glyphs, not a style of lettering.** A caption-trained model has never had to predict the next letter given the previous ones, so generated text can look exactly right and spell nothing at all.\n- **\"Three\" barely moves the embedding that \"apple\" dominates.** Counting and left/right relations are weakly grounded because the prompt encoder was never trained to track them separately from the object — see the worked example.\n- **A higher guidance scale sharpens obedience, not correctness.** Turning up [[samplers-and-schedulers|the guidance dial]] makes the model commit harder to whatever it already produced","score":1},{"slug":"multimodal","title":"Multimodal","heading":"The articles, in order","era":"multimodal","text":"- **2018** — [[conditioning-and-modulation|Conditioning and Modulation]] — How a timestep, class, or prompt steers a network by predicting a per-channel scale and shift, not by becoming a token.\n- **2020** — [[vits-vlms|Vision Transformers and VLMs]] — Patches are tokens: the transformer eats images, then learns to talk about them.\n- **2021** — [[autoencoders-and-latents|Autoencoders and Latent Spaces]] — The compression front end that lets diffusion and video models work in latent space instead of pixels.\n- **2022** — [[diffusion-pipelines|Diffusion Pipelines]] — Denoise noise, step by step, until a picture appears — guided by text.\n- **2022** — [[diffusion-transformers|Diffusion Transformers]] — A DiT patchifies a latent into tokens and runs plain transformer blocks instead of a U-Net — the backbone behind SD3 and FLUX.\n- **2022** — [[samplers-and-schedulers|Samplers and Schedulers]] —","score":1}],"mode":"query"}