https://bespokelabs.ai/blog Research notes, releases and build logs from Bespoke Labs on RL environments, agent evaluation and post-training. en Thu, 15 Oct 2026 00:00:00 GMT https://bespokelabs.ai/blog/teaching-nvidias-nemotron-ml-engineering https://bespokelabs.ai/blog/teaching-nvidias-nemotron-ml-engineering Thu, 15 Oct 2026 00:00:00 GMT With early access to NVIDIA Nemotron 3.5 Super-VL, we post-trained it on long-horizon ML research tasks. Self-context distillation raised its AutoResearchExam score from 0.286 to 0.358. Build Science https://bespokelabs.ai/blog/post-training-seewhy-with-intuit-credit-karma https://bespokelabs.ai/blog/post-training-seewhy-with-intuit-credit-karma Sat, 19 Sep 2026 00:00:00 GMT Together with Intuit Credit Karma, we post-trained SeeWhy, a small, specialized model that explains why a recommended credit card fits a particular user. Build Data https://bespokelabs.ai/blog/introducing-autoresearchexam https://bespokelabs.ai/blog/introducing-autoresearchexam Thu, 10 Sep 2026 00:00:00 GMT AutoResearchExam is our benchmark for agents that work on hard, open-ended ML research problems over long periods of time. It measures and rewards fast, sustained progress that generalizes. Build https://bespokelabs.ai/blog/personalizing-inkling-for-your-code-repository-with-post-training https://bespokelabs.ai/blog/personalizing-inkling-for-your-code-repository-with-post-training Thu, 03 Sep 2026 00:00:00 GMT Can an open coding model be post-trained to work better on a specific repository? How do we use SFT and RL to improve performance? Data https://bespokelabs.ai/blog/conjecturebench-15k-open-math-problems-for-you-to-solve https://bespokelabs.ai/blog/conjecturebench-15k-open-math-problems-for-you-to-solve Fri, 31 Jul 2026 00:00:00 GMT ConjectureBench brings 14,865 open math problems together in one place, each linked to its source, so that people and models can pick a problem and start. https://bespokelabs.ai/blog/bespoke-labs-raises-40m-to-build-environments-that-enable-reliable-agents https://bespokelabs.ai/blog/bespoke-labs-raises-40m-to-build-environments-that-enable-reliable-agents Mon, 06 Jul 2026 00:00:00 GMT We’re excited to announce our Seed and Series A financing rounds to build the world’s best frontier data research lab and support for the post-training needs of enterprises and frontier labs. Updates https://bespokelabs.ai/blog/announcing-winners-of-reasoning-datasets-competition https://bespokelabs.ai/blog/announcing-winners-of-reasoning-datasets-competition Wed, 21 May 2025 00:00:00 GMT From 150 innovators emerged 70 unique datasets across 25 diverse domains - from finance and medicine to ethics and gaming; meet the standouts pushing LLM reasoning to new frontiers. Updates Data https://bespokelabs.ai/blog/bespoke-minichart-7b https://bespokelabs.ai/blog/bespoke-minichart-7b Mon, 21 Apr 2025 00:00:00 GMT Bespoke-MiniChart-7B: an open-source 7B model that sets a new standard for chart question answering Science https://bespokelabs.ai/blog/improving-multi-turn-tool-use-with-reinforcement-learning https://bespokelabs.ai/blog/improving-multi-turn-tool-use-with-reinforcement-learning Tue, 15 Apr 2025 00:00:00 GMT Using GRPO, we improved Qwen2.5-7B-Instruct's multi-turn tool use by 23% on a subset of the BFCL benchmark with only 100 training samples. https://bespokelabs.ai/blog/reasoning-datasets-competition https://bespokelabs.ai/blog/reasoning-datasets-competition Wed, 09 Apr 2025 00:00:00 GMT Bespoke Labs, Hugging Face, and Together.ai are launching a competition to find the most innovative reasoning datasets. Updates Science Data https://bespokelabs.ai/blog/curator-raft-boosting-rag-with-synthetic-data https://bespokelabs.ai/blog/curator-raft-boosting-rag-with-synthetic-data Mon, 24 Mar 2025 00:00:00 GMT Build domain-specific language models with Curator using Retrieval Augmented Fine-Tuning (RAFT) Build Data https://bespokelabs.ai/blog/effortless-gemini-batch-processing-with-curator https://bespokelabs.ai/blog/effortless-gemini-batch-processing-with-curator Tue, 11 Mar 2025 00:00:00 GMT Cost-efficient and easy, use Gemini Batch API now with Curator. Build Updates https://bespokelabs.ai/blog/claude-3.7-sonnet-thinking-mode-in-curator https://bespokelabs.ai/blog/claude-3.7-sonnet-thinking-mode-in-curator Wed, 05 Mar 2025 00:00:00 GMT Adding support for accessing the thinking traces of Claude 3.7 Sonnet in Curator Science Build Updates https://bespokelabs.ai/blog/launching-code-executor https://bespokelabs.ai/blog/launching-code-executor Wed, 26 Feb 2025 00:00:00 GMT With Bespoke Curator, you can generate code using a LLM and also execute it. Build Updates https://bespokelabs.ai/blog/openthinker-is-a-decensored-reasoning-model https://bespokelabs.ai/blog/openthinker-is-a-decensored-reasoning-model Wed, 26 Feb 2025 00:00:00 GMT Our OpenThinker models, trained on the OpenThoughts-114k reasoning dataset, appear to be decensored, even though we did no fine-tuning to remove political censoring. Science Data https://bespokelabs.ai/blog/scaling-up-open-reasoning-with-openthinker-32b https://bespokelabs.ai/blog/scaling-up-open-reasoning-with-openthinker-32b Thu, 20 Feb 2025 00:00:00 GMT Powerful reasoning models can be trained by scaling data, verifying reasoning traces, and scaling model size. Releasing OpenThinker-32B, a state-of-the-art open-data reasoning model Science Data Updates https://bespokelabs.ai/blog/measuring-reasoning-with-evalchemy https://bespokelabs.ai/blog/measuring-reasoning-with-evalchemy Wed, 19 Feb 2025 00:00:00 GMT If you can't measure it, you can't improve it. Releasing reasoning benchmarks into our model evaluation tool Evalchemy Science Build Data https://bespokelabs.ai/blog/launching-the-open-thoughts-project https://bespokelabs.ai/blog/launching-the-open-thoughts-project Mon, 17 Feb 2025 00:00:00 GMT Open Thoughts, an open-source effort to curate the best open reasoning datasets Science Data Updates https://bespokelabs.ai/blog/cut-token-costs-in-half-batch-processing-made-easy-with-curator https://bespokelabs.ai/blog/cut-token-costs-in-half-batch-processing-made-easy-with-curator Thu, 30 Jan 2025 00:00:00 GMT Batch APIs cost about half as much as online APIs but are hard to use. With Curator batch mode, you switch to them by adding batch=True. Build Updates https://bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation https://bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation Wed, 22 Jan 2025 00:00:00 GMT We trained Bespoke-Stratos-32B, our reasoning model distilled from DeepSeek-R1 and using Berkeley NovaSky’s Sky-T1 data pipeline. The model outperforms Sky-T1 and o1-preview in reasoning (Math and Code) benchmarks, and almost reaches the performance of DeepSeek-R1-Distill-Qwen-32B while being trained on 47x fewer examples. Science Data https://bespokelabs.ai/blog/hallucinations-fact-checking-entailment-and-all-that.-what-does-it-all-mean https://bespokelabs.ai/blog/hallucinations-fact-checking-entailment-and-all-that.-what-does-it-all-mean Mon, 02 Dec 2024 00:00:00 GMT AI hallucinations can derail accuracy, but Bespoke's latest factuality model is designed to combat that. Learn how advanced checks are helping models deliver more reliable outputs, reducing common errors in data generation. Science Build Data