Skip to content

Sam's News — tech-research — 2026-08-26

AI Safety

7.5 Activation steering technique reduces sycophancy and hallucination in medical LLM responses

Researchers developed gated activation steering to reduce sycophancy and hallucination in medical LLM responses. The inference-time technique applies steering vectors to causally verified attention heads, with behavior-specific gates controlling when intervention activates. Testing showed a 4-billion-parameter model maintained its answer under pressure in 551 of 570 cases where the unsteered model changed course.

  • Gated activation steering: inference-time intervention for clinical LLMs
  • Unsteered model changed answer under pressure in 570 of 600 test cases
  • Steered model improved robustness: maintained position in 551 of 570 cases
  • Steered 4B-parameter model achieved robustness of 100B+ parameter models
  • Tested on 600 pressure trajectories; 15,900 total model-response runs

Sources: arXiv AI Web Searched, arXiv — Computation and Language RSS, arXiv — Artificial Intelligence RSS

7 Method extracts finite automata from LLM agent traces to enable safety auditing

Researchers developed a method to extract finite-state machines from LLM agent execution traces, compressing entire trace corpora into compact automata (7–43 states) for safety auditing and runtime monitoring. FSMs achieved ≥0.997 replay fitness and enabled failure prediction with held-out AUROC up to 0.94, allowing early stopping before task completion.

  • Extracted FSMs: 7–43 states from twelve public datasets
  • Replay fitness ≥0.997 with near-identical topology across train-test splits
  • Next-step prediction: FSM context outperformed Agent Workflow Memory
  • Failure prediction AUROC up to 0.94; enabled early stopping from partial traces
  • Built in milliseconds; model-agnostic across LLM types

Sources: arXiv AI Web Searched, arXiv — Computation and Language RSS, arXiv — Artificial Intelligence RSS

AI Research

7 Study quantifies confabulation rate in LLM-generated autobiographies against documented records

A study of LLM-generated autobiography against documented biographical records found a verification-failure rate of 96.7%—only 12 of 366 days contained corroborated scenes. The dominant failure mode was "grounded drift," where real people and settings were embedded in invented scenes. Grounding generation in actual biographical corpus reduced failure to 83.3%.

  • Verification-failure rate: 96.7% (354 of 366 days, 95% CI 94.4–98.1%)
  • Only 12 days with corroborated scenes; 19 days contradicted by records
  • Dominant failure: 'grounded drift'—real people/settings in false scenes
  • Current LLM models: 100% verification failure on same inputs
  • Corpus-grounded generation improved to 83.3% failure rate

Sources: arXiv AI Web Searched, arXiv — Computation and Language RSS, arXiv — Artificial Intelligence RSS

6.5 LLM agents successfully execute controlled experiments using simulation models

Researchers demonstrate that large language models can conduct structured experimental designs within simulation environments.

Sources: arXiv — Computation and Language RSS, arXiv — Artificial Intelligence RSS

Security

7 Semantic overlays defend LLMs against prompt injection attacks

Researchers introduced Semantic Overlays, a defense mechanism that uses learned adapters to add non-textual annotations marking input provenance (user input, tool output, instructions) in the residual stream. This prevents attackers from manipulating model understanding through text-based prompt injection.

  • Semantic Overlays: learned adapters encoding span identity outside token channel
  • SEP separation: improved 24.3% → 96.5% with utility preserved
  • TensorTrust attack success: dropped 34.8% → 6.6%
  • PIArena attack families: reached 0% compliance; marked spans 92.5% readable
  • Paper arXiv August 24, 2026; code and adapters released publicly

Sources: arXiv AI Web Searched, arXiv — Artificial Intelligence RSS, arXiv — Cryptography and Security RSS

7 Names Can Hurt: Spotting Slopsquatting Risks from LLM Package Name Hallucinations

Security researchers demonstrate how LLM-generated fictitious Python package names can be exploited for supply chain attacks when adversaries pre-register those names on PyPI.

Sources: arXiv — Computation and Language RSS

AI & Materials

7 MIT's CrysVCD Tool Uses AI to Design Chemically Stable Materials

MIT researchers developed CrysVCD, an AI tool that improves chemical stability of computationally generated materials by applying valence-constraint rules during initial generation rather than after costly screening. The framework integrates with existing diffusion models to increase the fraction of usable material designs.

  • Tool applies valence-shell constraints before screening, not after
  • Increases rate of materials satisfying chemistry rules
  • Integrates into existing and future diffusion models
  • Addresses gap where AI generates millions of designs but misses stability factors

Sources: MIT News AI Web Searched, MIT — Artificial Intelligence RSS

AI Optimization

6.5 PuzzleKV reduces memory overhead of KV cache in long-context LLM inference

Page-wise low-rank decomposition technique compresses key-value cache to reduce memory constraints in long-context LLM inference.

Sources: arXiv — Machine Learning RSS

Robotics

6.5 Sparse test-time imagination improves robot control with video generative models

GlanceWAM enables real-time robot control using video predictions by sparsely simulating futures at control rate.

Sources: arXiv — Computer Vision RSS, arXiv — Robotics RSS

Autonomous Vehicles

6.5 Adaptive LiDAR scanning with camera residuals improves first detection in autonomous driving

CARE algorithm uses camera predictions to guide adaptive LiDAR scanning, improving detection of previously unseen objects.

Sources: arXiv — Computer Vision RSS, arXiv — Robotics RSS