An experimental open-source project is testing whether a language model can be trained continuously from scratch on a laptop-class graphics card with 8GB of video memory. Called mini-AGI, the byte-level model stores most expert weights on disk, loads a working set onto the GPU as needed and uses the same computation path for reading training data and generating text.
The project’s creator explicitly describes the current model as a small, toy-level experiment rather than a competitor to frontier systems. Its weights had not yet been published when the repository description was prepared because the first training pass through the corpus remained underway. The code and design notes nevertheless outline an approach aimed at personal models that keep learning rather than being trained centrally and then frozen.
Mini-AGI processes characters through two dense introductory blocks and a recurrent block that can be applied as many as 24 times. On each pass, a router selects eight experts from a shared pool. A halting mechanism varies how many recurrent steps each character receives, allowing the system to spend more computation on some inputs than others. The repository’s visualizations showed depth varying during both reading and generation.
The expert pool is intended to change over time. New experts are assembled from units taken from existing trained experts and added with a small initial influence. Experts that continue to attract routing demand remain, while unused ones can be removed. Because every expert is represented by files containing weights and optimizer state, the total pool is constrained principally by disk capacity; only a smaller working set must fit in GPU memory. Hysteresis and minimum residency periods are used to reduce constant swapping.
Training occurs on a stream of characters with a batch size of one. Each chunk produces a gradient update, and there is no separate fine-tuning phase or frozen base model. The project argues that this arrangement could let an owner continue training a model on local material. That remains an experimental claim: the repository provides samples from an ongoing run, not an independent demonstration of long-term learning across varied tasks.
The model also remains limited in output quality. A sample taken after 243 million training characters was grammatical and stayed on topic but repeated itself. The repository reports that generation used more recurrent depth per character than reading in the illustrated example. Those observations are useful engineering traces, not broad benchmark results.
Mini-AGI’s immediate value is therefore as a research prototype exploring memory paging, dynamic experts and continual updates on modest hardware. Its architecture offers concrete ideas for reducing GPU requirements, while its own documentation sets appropriate expectations: capabilities will depend on hardware, data quality, scale and training time, and the present implementation is not a production-grade general intelligence system.
