Topic
All digests tagged Transformer Architecture
Suggested route
Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.
Wells Fargo built a proprietary Foundation Model for transactions, pre-training a transformer on billions of raw customer transactions. This single backbone model improves fraud detection and streamlines multiple downstream financial tasks.
Read article
The lecture details the full LLM lifecycle, covering pre-training (next token prediction), SFT (behavior alignment), and advanced alignment techniques like DPO and RLHF. Key engineering concepts include LoRA for parameter efficiency and RoPE interpolation for long context handling.
Read article
MiniMax M3 combines agentic coding, 1M context, and native multimodality. The model uses MSA sparse attention, achieving 9x faster pre-fill and 15x faster decode vs. full attention.
Read article
The Transformer architecture revolutionized NLP by replacing recurrent connections with self-attention. This mechanism allows tokens to compute context-aware representations by weighing the relevance of all other tokens in the sequence.
Read article
This deep dive covers LLM inference optimization at scale, detailing how memory and compute costs are managed through techniques like Paged Attention, Continuous Batching, and model quantization.
Read article
The lecture reviews advanced Transformer optimizations, covering Grouped Query Attention (GQA) to reduce KV cache memory and Mixture of Experts (MoE) for scaling parameters while controlling compute. It also details In-Context Learning and SFT.
Read article
This deep dive covers LLMs, focusing on the Transformer architecture. Key topics include subword tokenization, auto-regressive modeling, and the self-attention mechanism using Q, K, V vectors.
Read article