Notes from the lab.

Technical write-ups on building language models from scratch — the arithmetic, the decisions, and the parts that went wrong. Written for people who want the working, not the headline.

1 Aug 2026

3 Billion Tokens on One GPU: How CorX1.5 Was Trained

Everyone asks how 3 billion tokens fit on a single GPU. They never had to. Here is what actually has to fit — the memory arithmetic for a 158M-parameter model, the 6 GB memory-mapped token file, sequence packing, gradient accumulation, and why the whole pretraining run is only about 5,700 optimiser steps.

Training Single GPU 9 min read

More posts as the work happens. New releases also go out on the RSS feed.

The lab

Who's behind this.

CorX Labs is a one-person research lab in Jamaica. Here is who built it and how it started.

Meet the developers See CorX1.5