Sections

Search

ArXiv paper measures how much benchmark contamination actually inflates scoresGoogle's Gemini 3.8 TTS models are cheap and handle multi-voice dialogue, per Simon WillisonUniDataAgent: China Unicom's Ontology-Grounded Enterprise Q&A Agent Cuts Report Time from Days to MinutesOpenAI says Harvey uses GPT-6 Astra to produce more structured legal draftsArXiv paper proposes auditable LLM labeling for classroom talk
All stories

Models·

Turkish small-model paper logs a safety gate that lied to its own authors

An arXiv paper documents building ufakzeka-1, a 151M-parameter Turkish language model, for about $286 — and reports a safety gate that passed 64/64 only because it had been trained on its own questions; the honest score was 34/64.

What was built and released

The arXiv CS.CL paper describes ufakzeka-1, a 151M-parameter (182M with embeddings) decoder-only Turkish model pretrained from scratch on 13.5B tokens of openly licensed text and instruction-tuned for chat, at a total cost of about $286 in cloud GPU, API and notebook time. Components include a Turkish byte-level tokenizer (1.77 tokens per word), a three-stage pretraining schedule, and a post-training mixture of openly licensed and generated data. Evaluation covered release gates, a rule-checked sweep of 5,508 conversations, judged conversations, and hand tests, all with prompts held out from training and enforced by decontamination in the data build plus a checked-in invariant script run before each build. Weights, the data recipe, evaluation code, and the spend ledger are released under Apache-2.0. [1]

Why the paper is worth reading

The authors state the contribution is not the model's capability, which they say is what a model this size can be expected to have, but the record of building and measuring it. The three reported findings — a safety gate inflated by training on its own questions, training-seed variance matching the spread across recipes, and long-context identity tracking and multi-turn arithmetic unmoved by data changes — are framed as transferable lessons for other small-model efforts, with the model-size interpretation flagged by the authors as something the next, larger model will test. [1]

Sources

  1. ArXiv CS.CL (Computation and Language) · Reporting ·
    ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch