one year on
Developer builds GPT-style language model in 500 lines of SQL
A blog post demonstrating a working implementation of GPT-2 using only SQL recursive CTEs and table joins shows that large language models can be expressed in relational database queries.
An EXPLAIN EXTENDED blog post presents a detailed implementation of a GPT-like language model in approximately 500 lines of SQL, using PostgreSQL recursive common table expressions and token lookups. The project, posted on the site EXPLAIN EXTENDED under the title “Happy New Year: GPT in 500 lines of SQL,” walks through the theory and code for a functional GPT-2 tokenizer, embeddings, and generation loop entirely within a relational database.
The author notes that the model is deterministic: with the same input, it always returns the same output. They explain that randomness in chat applications comes from the selection step, not the model itself. The implementation uses byte-pair encoding (BPE) for tokenization and relies on stored tables for the token dictionary and pre-trained weights.
The record
One year later — open only if you can handle spoilers
The post became a classic in the "explain AI in an unexpected context" genre, often cited in discussions about model interpretability. It did not lead to a production SQL-based LLM, but it inspired similar demonstrations in other constrained environments like spreadsheets and assembly language.
The Weekly Replay · free by email
This week, one year ago — every Sunday.
One email each Sunday: the week's replayed AI news, with the one-year-later annotations included. Written like it's breaking — dated like it isn't.
Free · double opt-in · unsubscribe anytime · privacy