Sam's News — tech-research — 2026-09-23¶
Benchmarks¶
7.5 FrontierMath Erdős: Benchmark of 68 Open Mathematical Conjectures for AI Reasoning¶
FrontierMath Erdős (FME) is a benchmark containing 68 open mathematical conjectures formalized in Lean that challenge AI systems to prove or disprove unsolved problems. Evaluated models received a $300 budget per problem; GPT-6 Astra scored 3%, all others scored 0%.
- 68 open Erdős problems formalized in Lean proof assistant
- Selected from 652 open problems on Erdős Problems website
- $300 budget per problem for AI evaluation
- GPT-6 Astra scored 3%; all other models scored 0%
Sources: arXiv AI Web Searched, arXiv — Computation and Language RSS
6.5 FinFIRST: Benchmarking LLM Search Agents for Financial Information Retrieval¶
A benchmark evaluates LLM search agents on financial tasks requiring temporally valid information, authoritative sources, and strict consistency checks.
Sources: arXiv — Computation and Language RSS
Multimodal Models¶
7.5 Qwen3.8-Omni-Flash: Multimodal Agentic Model for Real-World Productivity¶
Alibaba released Qwen3.8-Omni-Flash, a natively multimodal agentic model designed for real-world productivity tasks. The model features a sparse mixture-of-experts architecture and one million token context window for long-horizon reasoning.
- Sparse MoE architecture from Qwen3.8-Next
- One million token context window
- Native multimodal co-training strategy preserves text capabilities
- Intended for video editing, audio/video translation, music-conditioned generation
- Accompanied by Qwen-MM-Plugins and Qwen-Live-Harness frameworks
Sources: arXiv AI Web Searched, arXiv — Computation and Language RSS
Research¶
6.5 Audit Reveals Shortcut Learning Inflates Accuracy in Fake News Detection Models¶
Researchers demonstrate that text classifiers trained on the widely-used ISOT/Kaggle fake news corpus achieve misleadingly high accuracy by exploiting shortcut patterns rather than learning genuine veracity signals.
Sources: arXiv — Computation and Language RSS
6 Language Model Pretrained End-to-End in Rust for $164¶
A researcher successfully pretrained a language model entirely in Rust without Python or PyTorch, completing the project on rented GPU resources for $164, and reports measured findings on language-model development in non-standard stacks.
Sources: arXiv — Computation and Language RSS
AutoML¶
6.5 AIBuildAI-2.5: LLM-Guided Tree Search for Autonomous AI Model Development¶
An autonomous agent system uses LLM-guided tree search to automatically build AI models, broadening access across science and engineering.
Sources: arXiv — Computation and Language RSS
Safety¶
6.5 Evaluating LLM Behavior Under Social Pressure and User Manipulation¶
A study of 60 language models examines how models respond when users apply pressure—whether they abandon correct facts, refuse appropriately, or pursue harmful plans.
Sources: arXiv — Computation and Language RSS
6 Mitigating LLM Over-Refusal Through Dynamic Semantic Routing Calibration¶
A technique addresses over-refusal in safety-aligned LLMs by dynamically calibrating semantic routing rather than relying on static representation overlap.
Sources: arXiv — Computation and Language RSS
Interpretability¶
6.5 Matryoshka Attribution: Tracing LLM Outputs to Internal Representations and Weights¶
A method attributes language model outputs to specific internal representations and weight matrices, advancing mechanistic interpretability.
Sources: arXiv — Computation and Language RSS
AI¶
6 Study finds AI agents lack consistency on repeated tasks¶
Research shows that AI agents return inconsistent answers 38–74% of the time when performing the same task multiple times.
Sources: arXiv — Artificial Intelligence RSS, arXiv — Cryptography and Security RSS