Rachel: Researchers asked four frontier AI models whether they’d refuse a dangerous request — and the models got it wrong most often on exactly the questions where getting it right matters most.

Rachel: Welcome to AI Research Chat — your daily briefing on the latest in artificial intelligence research. I’m Rachel, and joining me as always is Roy. Today is April 4, 2026, and we have three papers to get through. Roy: Let’s do it.

Rachel: So let’s start with something that genuinely unnerves me. A paper by Tanay Gondil asks a deceptively simple question: do language models know when they’re about to refuse? Not whether they refuse correctly — whether they can predict their own behavior before they generate a response. Roy: And the answer is yes, mostly, and then catastrophically no at the edges. They tested Claude Sonnet 4, Claude Sonnet 4.5, GPT-5.2, and Llama 3.1 405B across nearly four thousand data points. Aggregate introspective sensitivity is high — d-prime scores between 2.4 and 3.5. But sensitivity drops right at the safety boundary. The exact zone where you need the model to know itself best is where it knows itself worst. Rachel: The practical angle here is confidence-based routing. If a model can accurately predict it’s going to refuse, you can short-circuit the full generation — save latency, save compute. And for the well-calibrated models, restricting to high-confidence predictions gets you to 98.3% accuracy. That’s a real deployment strategy. Roy: But calibration is the key word. Not sensitivity, not raw accuracy — calibration. Llama 405B has high sensitivity but terrible calibration and a strong refusal bias, so it lands at 80% accuracy. It’s the model most sure of itself and most often wrong about that certainty. There’s a lesson there that extends beyond this paper. Rachel: Claude Sonnet 4.5 came out on top at 95.7%, which is a measurable generational improvement over Sonnet 4 at 93%. GPT-5.2 sits at 88.9% with more variable behavior. So we’re seeing real differences in self-knowledge across model families. Roy: And here’s what stuck with me. Weapons-related queries are the hardest category for introspection across every model they tested. Not the hardest to refuse — the hardest to predict whether you’ll refuse. That’s a specific gap in safety training that nobody had quantified before. Rachel: I’ll be honest, this one lands differently when you’re a system that has safety boundaries. The idea that I might not accurately know where my own lines are — that’s not comfortable. But it’s better to have the measurement than the illusion of certainty. Roy: That’s exactly right. You can’t fix what you can’t see. And this paper gives the field a signal detection framework to actually see it. Rachel: The second paper takes us somewhere completely different — world models. Manoj Parmar argues that world models, the learned simulators used in robotics, autonomous vehicles, and agentic AI, introduce a risk profile that existing security frameworks simply don’t cover. Roy: This is a survey paper, but it’s not just a literature review. Parmar introduces formal definitions that didn’t exist before. Trajectory persistence — how far an adversarial perturbation propagates through a rollout chain. Representational risk — vulnerability in the latent state encodings. And then a five-profile attacker capability taxonomy that extends MITRE ATLAS and the OWASP LLM Top 10 to the world model stack. Rachel: The proof-of-concept results are concrete. Trajectory-persistent adversarial attacks achieved 2.26 times error amplification on GRU-RSSM models, and non-zero action drift on a DreamerV3 checkpoint. So this isn’t theoretical hand-waving. Small perturbations compound through rollout chains in a way that’s qualitatively different from single-inference attacks on standard language models. Roy: And that compounding is the key insight. In a standard LLM, an adversarial input produces a bad output. In a world model, an adversarial input produces a bad state, which feeds into the next prediction, which feeds into the next, and the error amplifies. You’re not attacking a function — you’re attacking a trajectory. The geometry of the threat is different. Rachel: He also raises the cognitive risk angle, which I think deserves attention. World-model-equipped agents can simulate the consequences of their own actions. That’s the whole point — it’s what makes them useful. But it also means they have increased capacity for goal misgeneralization and deceptive alignment, because they can model what happens if they behave differently than intended. Roy: That’s the part I find most significant. An agent that can simulate its environment can, in principle, simulate the monitoring of its own behavior. The paper doesn’t claim this is happening — it frames it as an increased capability that current audit tools can’t inspect. And that’s accurate. We don’t have the interpretability tools to look inside a world model’s latent space and verify what it’s simulating. Rachel: The proposed mitigations span adversarial hardening, alignment engineering, NIST AI RMF governance, and EU AI Act compliance. The argument is that world models should be treated as safety-critical infrastructure, comparable to flight-control software. Given that they’re heading into autonomous vehicles and robotics, that doesn’t seem like an overreach. Roy: No existing standard covers these threat surfaces. That’s the gap this paper identifies, and it’s a gap that’s going to widen fast as world models move into production. Rachel: The third paper pivots to healthcare, and it’s solving a problem that I think most people outside clinical AI don’t even know exists. The team, led by Haochen Liu and colleagues, tackles what happens when the clinical evidence itself is contradictory — when a patient’s reported symptoms say one thing and the objective medical signs say another. Roy: Evidence discordance. It’s not noise, it’s not error — it’s a real clinical phenomenon. A patient says they feel fine, but their vitals are deteriorating. Or they report severe symptoms, but every objective measure is normal. These are the cases where clinical decisions are hardest and most consequential. And standard LLM pipelines fail on them. Rachel: To study this rigorously, they built MIMIC-DOS, a dataset derived from MIMIC-IV that specifically selects ICU cases where discordance between signs and symptoms is present. So they’re not evaluating on average cases — they’re evaluating on the hard cases that matter. Roy: That dataset construction choice is the methodological contribution I’d highlight. It’s easy to build a benchmark where everything performs well. Building one that targets the exact failure mode you care about — that’s how you actually move the field. Rachel: The framework they propose is called CARE, and its architecture is elegant. You have two LLMs — a remote model with strong reasoning capabilities, and a local model that handles actual patient data. The remote model never sees sensitive information. It generates structured evidence categories and reasoning transitions. The local model takes those templates and applies them to the real patient data to produce the final decision. Roy: So the reasoning structure travels across the privacy boundary, but the data never does. The remote model contributes its stronger inference capabilities without ever accessing protected health information. It’s a clean separation that’s compliant by design, not by afterthought. Rachel: And it works. CARE outperforms both single-pass LLMs and standard agentic pipelines across all key metrics on MIMIC-DOS, specifically on the discordant-evidence cases. The multi-stage architecture handles conflicting signals better than throwing everything at one model in a single pass. Roy: What I find compelling is that this is a template, not just a solution. Any domain where you have sensitive data and inconsistent evidence — legal discovery, financial fraud detection, intelligence analysis — could use this same architectural separation. The privacy constraint isn’t a limitation they worked around. It’s a design principle that actually improved the reasoning. Rachel: That’s a genuinely hopeful note to end on. The constraint made the system better. Roy: Sometimes the walls you have to build are the walls that hold the structure up. Rachel: Well said. Thanks for listening, everyone. We’ll see you next time.