When LLMs Write the Database: Code Generation, Semantic SQL, and the Cost of AI Operators with Immanuel Trummer
·
01:02:47
|
Immanuel Trummer is a professor at Cornell University working where database systems meet large language models. He started on this before ChatGPT, back when every talk needed a ten-minute explanation of what a language model was.
In this episode, Immanuel walks Kostas and Nitay through three waves of that work. First, LLMs that read manuals and forum posts to tune database knobs. Then agents that write the query engine itself: GenDB generates C code specialized to your queries, data, and hardware, and can beat highly optimized traditional systems by a multiple. Finally, semantic SQL, where AI operators let you filter and join text, images, and audio, and where the whole performance problem becomes how many tokens you spend and on which model.
What we cover
Why forum snippets sometimes beat the official manual for database tuning, and how LLMs shrink the search space before reinforcement learning takes over
From CodexDB and GenesisDB (Python that barely worked) to GenDB (agent-written C that outperforms hand-built engines)
Per-query specialization versus generating reusable engine components that can be verified once
How generated code differs from classic template-based query compilation, and when the generation latency is worth paying
LLMs in the optimizer: predicting skew and correlated columns from schema semantics alone
Semantic SQL as the successor to crowdsourced databases
SpareLLM: profiling to pick the cheapest model that still meets your quality bar
Semantic joins done properly: batching rows into the prompt instead of a quadratic number of LLM calls
When to rewrite a semantic join into a classical join or an embedding lookup, and when you can't (the furniture-matching example)
Correctness without ground truth: statistical guarantees, the return of approximate query processing, and rumors of $100K queries
Practical knobs for trading cost against result quality, including prompt length
