Topic
69 posts tagged “Machine Learning”.
Chisato · · 5 min read Grouped-query attention shares key and value heads across groups of query heads, shrinking the KV cache with little quality loss. GQA vs MHA and MQA.
Chisato · · 5 min read Constrained decoding masks invalid tokens at each step so an LLM can only emit output matching a grammar, regex, or JSON Schema. How it works and its limits.
Chisato · · 5 min read Logprobs are the log probabilities an LLM assigns to each token it generates. What they mean, how to read them, and practical uses like classification.
Chisato · · 5 min read A world model is an AI system's internal simulation of how its environment changes, letting it predict outcomes before acting.
Chisato · · 4 min read Precision measures how many of a model's positive predictions were correct; recall measures how many actual positives it found. Why you can't max both.
Chisato · · 4 min read Logit bias nudges an LLM's token probabilities up or down before sampling, letting you ban, force, or discourage specific words without a prompt.
Chisato · · 5 min read Instruction tuning trains a language model on prompt-response pairs so it follows directions instead of just predicting text. How it works and where it fits.
Chisato · · 4 min read A neural network is layers of weighted connections that learn patterns from data. How neurons, activation functions, and training actually work.
Chisato · · 4 min read A Vision Transformer applies the transformer architecture to images by splitting them into patches processed with self-attention instead of convolutions.
Chisato · · 5 min read Three ways machine learning models learn: from labeled examples, from patterns in unlabeled data, or from trial-and-error reward signals.
Chisato · · 4 min read Positional encoding gives transformers word order by adding position signals to token embeddings, since self-attention alone is order-blind.
Chisato · · 4 min read Model collapse is the degradation that happens when a generative model is repeatedly trained on data produced by earlier generations of itself.
Chisato · · 5 min read Reward hacking is when an AI system optimizes its literal reward signal in ways that satisfy the metric but violate what the designer actually wanted.
Chisato · · 4 min read Backpropagation is the algorithm that trains neural networks by computing how each weight contributed to the error, then adjusting it. Here's the mechanism.
Chisato · · 4 min read Vector quantization compresses high-dimensional embeddings into compact codes, shrinking memory and search cost with a small accuracy trade-off.
Chisato · · 4 min read Overfitting memorizes training data and fails on new inputs; underfitting fails to learn the pattern at all. How to spot each and what fixes each one.
Chisato · · 4 min read A GAN pits a generator against a discriminator in a training loop that produces realistic synthetic data. How the adversarial setup works.
Chisato · · 4 min read Self-attention lets each token in a sequence weigh every other token when building its representation, which is how transformers understand context.
Chisato · · 4 min read Neural network pruning removes redundant weights or neurons after training to shrink a model without retraining from scratch. How it works.
Chisato · · 4 min read A foundation model is a large model pretrained on broad data, then adapted for many downstream tasks via fine-tuning, RAG, or prompting alone.
Chisato · · 4 min read Keyword search matches literal terms; semantic search matches meaning via vector embeddings. How each works, and why most production systems use both.
Chisato · · 4 min read Gradient descent is the optimization algorithm that trains neural networks, nudging weights downhill along the loss function's gradient.
Chisato · · 5 min read A vision-language model processes images and text together, jointly grounding visual content in language. How VLMs are trained and what they're used for.
Chisato · · 5 min read Test-time compute is extra computation an AI model spends while answering, not while training — trading latency and cost for better answers.
Chisato · · 4 min read LLM-as-a-judge uses one language model to score another model's outputs against a rubric, replacing slow human review for large-scale evaluation.
Chisato · · 4 min read A model card is a standardized document describing an AI model's intended use, training data, evaluation results, and limitations before deployment.
Chisato · · 5 min read A feature store centralizes how machine learning features are computed, stored, and served — keeping training and production predictions consistent.
Chisato · · 5 min read Catastrophic forgetting is when training a model on new data erases skills it already had. Why it happens during fine-tuning, and how teams work around it.
Chisato · · 4 min read DPO tunes a language model on human preference data directly, without training a separate reward model or running reinforcement learning.
Chisato · · 4 min read Constitutional AI trains language models to critique and revise their own outputs against a written set of principles, reducing reliance on human labels.
Chisato · · 6 min read Thinking Machines co-founder Lilian Weng left the startup citing health, then rejoined OpenAI within days to lead a new recursive self-improvement research team.
Chisato · · 4 min read Grounding connects an LLM's output to verifiable external data instead of relying on what it memorized during training, reducing hallucinations. How it works.
Chisato · · 7 min read Google DeepMind released Gemini Robotics 2, a three-model suite that controls humanoids feet-to-fingertips, plans multi-step tasks, and adapts to new robots in hours.
Chisato · · 4 min read RAG retrieves relevant documents at query time; fine-tuning bakes new behavior into model weights. How to choose based on what actually needs to change.
Chisato · · 5 min read Batch inference processes large volumes of input on a schedule; real-time inference answers one request as fast as possible. How the two serving modes differ.
Chisato · · 4 min read Prompt engineering is the practice of structuring instructions to get reliable, accurate output from an LLM. Core techniques and common pitfalls.
Chisato · · 4 min read Distillation trains a smaller model to mimic a larger one; quantization shrinks an existing model's number precision. How the two techniques differ.
Chisato · · 4 min read A reranker re-scores a retriever's candidate results with a slower, more accurate model, fixing the precision gap that pure vector search leaves behind.
Chisato · · 4 min read How you split documents into chunks determines what a RAG system can retrieve. Fixed-size, semantic, and recursive chunking compared, with tradeoffs.
Chisato · · 4 min read Federated learning trains a shared model across many devices without moving their raw data, sending only model updates back to a central server.
Chisato · · 4 min read Beam search keeps the top-k most likely sequences at each decoding step instead of just one, trading compute for better output than greedy decoding.
Chisato · · 4 min read Synthetic data is artificially generated training data that mimics real-world patterns without exposing actual records. How it's made and used.
Chisato · · 4 min read In-context learning teaches a model a task through examples in the prompt; fine-tuning updates the model's weights permanently. How they compare.
Chisato · · 4 min read Temperature, top-p, and top-k are the three main knobs that control how an LLM picks its next token — and why outputs get more random or more repetitive.
Chisato · · 4 min read An LLM eval is a structured test suite that scores a model's outputs against a standard, letting you compare models and catch regressions systematically.
Chisato · · 5 min read LoRA fine-tunes a large model by training small low-rank matrices instead of its full weights. How it works, why it's cheap, and where it falls short.
Chisato · · 4 min read A multimodal AI model processes and generates more than one type of data — text, images, audio — in a single unified system. Here's how it works.
Chisato · · 4 min read A system prompt is the hidden instruction set that shapes an LLM's persona, tone, and boundaries before any user message arrives — how it works.
Chisato · · 4 min read An LLM hallucination is a fluent, confident output that is factually wrong — a byproduct of next-token prediction, not a bug you can simply patch.
Chisato · · 4 min read Speculative decoding speeds up LLM text generation by having a small draft model guess tokens the large model verifies in one pass. Here's how it works.
Chisato · · 4 min read Chain-of-thought prompting asks an LLM to reason step by step before answering, improving accuracy on multi-step problems by making its work explicit.
Chisato · · 5 min read Zero-shot prompting asks an LLM to perform a task with no examples; few-shot includes sample input-output pairs in the prompt. When to use each.
Chisato · · 4 min read Temperature controls how random an LLM's token choices are. How it works alongside top-p and top-k, and how to pick a value for your use case.
Chisato · · 5 min read RLHF trains a language model to match human preferences using a reward model and reinforcement learning. How the training pipeline actually works.
Chisato · · 5 min read Tokenization is how a language model chops text into tokens — the units it actually reads and bills. How it works, why words split oddly, and why it matters.
Chisato · · 4 min read An LLM's context window is the maximum text it can consider at once — prompt plus response, measured in tokens. Why it matters and how to work within it.
Chisato · · 5 min read Model distillation trains a small student model to mimic a larger teacher. How it works, how it differs from quantization and pruning, and its limits.
Chisato · · 3 min read A vector embedding turns text, images, or audio into numbers where similar meanings land close together — the foundation of semantic search and RAG.
Chisato · · 4 min read Diffusion models generate images by learning to reverse a gradual noising process. How they work, what powers Stable Diffusion, and how they compare to GANs.
Chisato · · 3 min read A GPU packs thousands of small cores built for parallel arithmetic. Originally for graphics, it's now the engine behind training and running AI models.
The Lycoris Team · · 5 min read Noam Shazeer, a co-author of the Transformer paper that underpins modern AI, is leaving Google DeepMind for OpenAI — the AI talent war's latest marquee move.
Chisato · · 3 min read Google released Gemini 3 — Pro, Flash, Deep Think, and a 3.5 series — across the Gemini app, AI Studio, and Vertex AI. Here's the lineup.
The Lycoris Team · · 2 min read At WWDC 2026, Apple unveiled 'Siri AI' — a ground-up redesign powered by Google's Gemini through a multi-billion-dollar partnership. Here's what changed and why.
Chisato · · 3 min read Fine-tuning continues training a pretrained model on a task-specific dataset. How it works, when to use it over prompting or RAG, and what can go wrong.
Chisato · · 4 min read Open-weight AI models are catching up to the best closed systems on many tasks — and you can run them yourself. What's driving the shift and what it means.
Chisato · · 4 min read The transformer is the architecture behind modern LLMs. How attention, tokens, and stacked layers combine to make today's AI work.
Chisato · · 9 min read What are LLMs and how do they work? A plain-English guide to large language models: tokens, training, real examples, and what they still get wrong.
Chisato · · 3 min read Reasoning models 'think' before they answer, trading inference time for accuracy on hard problems. Here's how test-time compute, adaptive thinking, and effort work.
Chisato · · 7 min read Mixture of Experts (MoE) scales LLMs by activating only a few experts per token. How routing, sparse activation, and load balancing actually work.
