Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash
DoorDash uses LLMs to solve semantic understanding in marketplace discovery. The core pattern is to perform expensive LLM reasoning offline and distill it into small, fast models for serving, improving retrieval and ranking.
Read article