A model reads a support ticket, picks a department, and returns 0.94. RLCR — training that number with a reward only an honest probability can maximise — and a small typed-decision model built from an ordinary encoder: one forward pass, every option a parallel branch, and the same weights trained seven ways to find out what the reinforcement learning actually adds.
One line of Python — index.search(query, k=10) — taken apart layer by layer, with a small nearest-neighbour library written in C++ and CUDA to take it apart with. Where numbers live and who frees them, what a copy of a gigabyte costs, why the order of additions changes both the speed and the answer, when an abstraction is free and when it is ruinous, what a template costs to compile, what happens between the call returning and the GPU finishing — and why the same search runs 25 times faster after eight rewrites that change nothing about what it computes. Measured end to end on a Kaggle T4.
What a GPU is actually doing while a language model answers: the generation loop, the prefill pass that reads the prompt and the decode steps that write the answer, the two waits a user feels — time to first token and time per output token — and why the second is limited by moving the model's weights through memory. Measured end to end on a Tesla T4, including why a token costs nine times what its bytes say, what a real kernel trace shows instead, and how latency and throughput pull apart when many people share one card.
Most of the arithmetic in a neural network is matrix multiplication. Written the obvious way it is correct, and fifty times slower than torch.mm on the same GPU. Where that difference comes from, one change at a time on a Tesla T4: coalescing, tiles, shared memory, registers, tensor cores and precision — and why almost all of it is about where the numbers are kept rather than how they are multiplied.
Why the same GPU is nineteen times faster than the CPU at one operation, eleven times at the next, and slower at a third — worked out on a real Tesla T4, from two numbers per machine: how fast it computes and how fast it moves bytes. Along the way: warps and SIMT, latency hiding, the memory hierarchy, arithmetic intensity, the roofline, and a power limit that moves the ceiling.
One number at the end of the network has to become an instruction for every number inside it. That trip is the whole of training: backpropagation as the chain rule run backwards through a graph, the product of many numbers that decides whether the instruction survives the journey, the initialization arithmetic that keeps that product near one and the normalization, clipping and precision choices that answer the other half of the problem, and regularization as the deliberate corruption that makes the result generalize — every claim carried by one worked example, from two weights done by hand to a network that overfits.
A tiny synthetic world where the rule is hidden and the evidence is controlled. Version spaces and what a dataset actually tells you, why a deterministic likelihood makes generalization entirely a matter of prior, shortcut features, expected information gain and active learning, compositional and length generalization, bottom-up program search with observational-equivalence pruning, LLM-guided synthesis, and libraries that grow their own primitives — each turned into a small experiment with a deterministic verifier.
Building a tiny AlphaGo from scratch on a 5×5 board: board representation, a small CNN policy network, a value network, and then Monte Carlo Tree Search — with each piece introduced only when the previous version visibly runs out of road. Ends at a self-play loop that is recognisably AlphaZero in miniature.
What it actually means to reverse-engineer a neural network into features and circuits: the residual stream as a linear object, the linear representation hypothesis, superposition and why features outnumber neurons, sparse autoencoders as dictionary learning, the QK/OV decomposition of attention, induction heads, and the causal methods — activation and path patching — that turn a plausible story into a verified mechanism.
How a next-token predictor is bent toward what humans prefer: reward modeling, PPO, RLAIF/Constitutional AI, and the offline-preference family — and why so much of a model's 'personality' and 'emotional' behavior is decided in this stage.
Same engine, different knobs: why one model reads as warm and another as businesslike. Traces 'personality' to concrete training choices — data mix, preference guidelines, reward model, safety tuning, character training — and folds in sycophancy as a personality artifact of RLHF.
Two halves of one question. Why an LLM's emotional language is generated, not felt, and where it comes from — preference data, reward models, safety tuning, system prompts; and whether it can actually understand emotion in others — theory of mind, the recognition benchmarks, and where the fluent performance turns brittle.
A reference-style deep dive into the modern knobs around post-training: preference optimization variants, LoRA and PEFT methods, memory-efficient full fine-tuning, model merging, distillation, tooling, and serving. Read this after the pipeline note, once you know which stage you actually need.
Post-training is not one fine-tuning method. It is a sequence of objective signals. Continued pretraining teaches substrate, SFT teaches examples and defaults, preference optimization teaches comparisons, RLVR teaches verifiable trajectories, and distillation transfers the resulting behavior. The important design question is not which acronym is fashionable, but which stage matches the behavior you are trying to install.
Fine-tuning is not prompt repair. It is a decision to write a reusable parameter delta into an existing checkpoint. That delta changes future logits, defaults, and trade-offs. This note is about when that is worth doing: what fine-tuning actually buys, how to tell whether a gap belongs in the weights, and why data, evals, forgetting, and probability shape matter more than the slogan 'just fine-tune it'.
Embeddings aren't an encoding of text — they're an attempt to make geometry behave as if it carried meaning. What it means to compress text into a fixed-length vector, how contrastive learning turns statistical structure into distances and directions, why cosine similarity works (and when it stops), how dimension, chunking, context window, and reranking change the physics of a retrieval pipeline, and where embeddings lie usefully.
Sixty years of incremental work behind the modern embedding vector. From bag-of-words and BM25, through Word2Vec's distributional hypothesis, to contextual embeddings and contrastive retrieval models. The point isn't trivia — it's that each older idea is still in production today, and the picture only makes sense once you've seen the road that led here.
Saying a model was 'trained on text written before T' invites a picture of human knowledge as of T. The actual corpus is volumetrically skewed toward recent years, dominated by retroactively-edited sources like Wikipedia, missing reliable per-document timestamps, and survivor-biased for older periods. The mechanisms, the failure modes that fall out, what's silently absent from datasheets, and what time-aware pretraining would have to do differently.
A chat transcript preserves order but not elapsed time, world state, or whether earlier hypotheses have expired. For long-running agents, temporal grounding is a runtime problem, not a model problem — what 'now' actually is, the failure modes that fall out when context gets treated as state, the primitives (clocks, event logs, state reducers, expectations, monitors) that close the gap, and how to measure whether it works.
Why modern LLMs are no longer just decoder-only transformers with standard multi-head attention. Attention has become a design space — MHA, MQA, GQA, MLA, sliding-window, sparse, linear, recurrent, hybrid — plus position encoding, attention sinks, and KV-cache compression. Each variant solves a different bottleneck.
What hallucination looks like at the level of the transformer's internal computation — distributed representations, signal competition in the residual stream, the softmax bottleneck, the activation-output gap, and the architectural reasons there is no first-class epistemic channel.
An introduction to statistical baseline methods for time-series anomaly detection: Z-Score, IQR, moving averages, simple ensembles, and evaluation metrics.
Causal ML at the meeting point of econometrics and ML — ATE vs CATE, uplift modelling, DML and orthogonal-moments inference, causal forests, counterfactual prediction, off-policy evaluation, and the standard mistakes from treating predictive models as causal.
Chunking turns out to be more architectural than it looks. A walk through what a chunk has to satisfy at six different stages simultaneously, what RecursiveCharacterTextSplitter actually does under the hood, and what the 2024–2026 toolbox — late chunking, contextual retrieval, contextualized chunk embeddings, BGE-M3 multi-functionality, ColBERT late interaction — is actually for.
A practical guide to ARIMA, SARIMA, and SARIMAX: how classical statistical forecasting models handle autocorrelation, differencing, seasonality, external regressors, diagnostics, and where they still fit in modern forecasting.
Practical wrap-up for clustering work — cluster labelling, two-corpus comparison, validation, choosing K when it isn't known, hyperparameter tuning order, common mistakes, and reproducibility.
The main clustering algorithms organised by the shape each assumes a cluster has — centroid, covariance, density, graph cut, hierarchy — where each fits, and a comparative table of the family.
The control function approach as the residual-based alternative to 2SLS — when the two agree, what CF's structural assumption actually is, where an ML-flexible first stage helps and where it does not, and how the idea extends through cross-fitting into double / debiased machine learning.
Why CSV, JSON, and Parquet all coexist in real pipelines: file size, read speed, and tool compatibility rarely peak in the same format.
Kafka as a distributed log for streaming data transport, and Protobuf as a compact binary serialization format — how each works, when to use them, and why they pair well together.
An overview of deep learning architectures for time-series forecasting: LSTM, TCN, DeepAR, N-BEATS, and TFT, with a focus on input shapes, local vs global training, covariates, probabilistic outputs, and practical model selection.
Endogeneity — what it is formally, why OLS becomes biased and inconsistent under it, the three main sources (omitted variables, simultaneity, measurement error) plus self-selection, how to detect it in practice, and why this is the central problem the rest of the Econometrics track is built around.
What happens inside a transformer when you send a prompt, and how the practical knobs — temperature, max_tokens, structured outputs, batching strategy, retry-with-catch-up — fall out of that picture.
Instrumental Variables / 2SLS as the foundational tool for causal inference in observational data when the treatment is endogenous — the IV idea in plain terms, the four conditions a valid instrument must satisfy (relevance, exclusion, independence, monotonicity), the two-stage estimator, weak-instrument diagnostics beyond the F>10 rule, what IV actually identifies (LATE, not ATE), and worked instrument choices for pricing problems.
A practical guide to the post-training lifecycle of deep learning forecasters, covering SHAP-based prediction explanations, lightweight recalibration for drift, and the broader production-maintenance roadmap.
Training a small model on a large model's probabilities instead of the true labels often gives a better small model. Why that works, taken apart one ingredient at a time; what happens when the labels the teacher learned from were wrong; whether a stronger teacher makes a stronger student; what to copy, on which data and about which view of it; how long to train and how big the student should be before any of this matters; why copying the teacher and being right are different numbers; how a student can beat its teacher and which of its flaws it inherits; what changes for language models without logits; and when distillation pays for itself — measured on CIFAR-10 and on a small language model trained on this site's notes.
Agent architecture is where LLM engineering stops being mostly about prompts and starts looking like distributed systems. Covers workflows vs agents, the classical loop, five paradigms (ReAct, Function Calling, Plan-and-Execute, Reflection, CodeAct), MCP as the protocol layer above per-vendor function calling, multi-agent patterns, computer use, memory and resumability, production failure modes including indirect prompt injection, tool security, cost levers, and observability.
An architecture is a claim about your data, and every claim here is measured: why a stack of linear layers is one straight line, what a CNN's first layer actually learns, how fast a receptive field grows, where an RNN's gradient dies — and PyTorch as three ideas: a tensor, a tape, and a module.
Covers the object storage model — buckets, keys, versioning, lifecycle policies — and Minio as a self-hosted S3-compatible implementation.
Fixed effects for panel data — the within-transformation, what it controls for and what it misses, cluster-robust standard errors, the LSDV equivalence, the staggered-DiD problem, and when FE is the right tool.
Playing with a hybrid forecasting architecture, assembled out of common neural building blocks: TCN front-end, LSTM recurrence, multi-head attention, TFT-style VSN feature routing with GRN, N-BEATS-style decomposition heads, seasonal and event branches.
A practical checklist for training deep learning time-series models, covering windowing, initialization, embeddings, regularization, optimizers, learning-rate schedules, loss functions, and the Keras training workflow.
Pricing as the worked-example for the Econometrics track — why price is endogenous, the identification strategies (IV, FE, RD, DML, CATE), cross-price elasticity and cannibalization, and what-if analysis with its constant-elasticity caveats.
What separates a working LLM prompt from a flaky one in 2026 — instruction hierarchy, in-context learning, chain-of-thought, structured outputs, reasoning-model specifics, and the prompt-injection trust boundary.
Regression discontinuity as a quasi-experimental method built on a threshold-based assignment rule — the continuity assumption that does the identification work, sharp and fuzzy variants, the modern bandwidth and diagnostic toolkit, and why the resulting effect is local to the cutoff rather than population-wide.
A practitioner's overview of PostgreSQL: how it handles concurrency (MVCC), its type system, index types, and the Python tooling stack for working with it.
How recurrent networks model sequences: vanilla RNN, the vanishing-gradient problem, the LSTM gating mechanism, cell state, and the practical use of return_sequences in Keras.
A practical explanation of Temporal Convolutional Networks for time-series forecasting: causal convolutions, dilations, receptive field, residual blocks, optional GLU gating, output heads, and when TCN is useful as a standalone or hybrid forecasting component.
One corpus of arXiv abstracts, clustered nine ways — and the finding that on real text the representation usually decides more than the algorithm.
A practical guide to the main types of time series data and the common confusion between multivariate, panel, and exogenous-variable forecasting.
A hands-on explanation of how raw time series data is cleaned, regularized, interpolated, bucketed, and prepared for reliable modeling or visualization.
