TALKIE: 13B LANGUAGE MODEL TRAINED ON 1930S DATA

Talkie represents an unusual approach to language model development, focusing on historical linguistic data rather than contemporary text. The 13B model was trained on materials from 1930, including newspapers, books, and published documents from that era. The project explores how language models perform when constrained to a specific historical period. Early results show the model generates text consistent with 1930s vocabulary, phrasing, and cultural references, offering a window into period-appropriate language generation. The work has generated discussion in AI development circles, with 214 points and 60 comments on Hacker News. While primarily a research exercise, Talkie demonstrates that language models can be effectively trained on narrow historical datasets. The project raises questions about temporal grounding in AI systems and potential applications in historical text analysis and period-appropriate content generation. Full details are available at talkie-lm.com/introducing-talkie.