Santosh V.

Mercedes-Benz Research &… • 1K followers

🚀 Excited to share our NeurIPS acceptance Our paper “Explanation Driven Counterfactual Testing for Faithfulness in Vision Language Model Explanations” by Sihao Ding, Santosh V. and Aditi Ramadwar (Mercedes-Benz Research & Development North America, Inc.) has been accepted to the NeurIPS 2025 Regulatable ML Workshop, and we will be presenting our poster next month. 👉 EDCT provides a structured protocol for verifying VLM explanations and helps close the gap between a "model sounding correct" and a "model actually reasoning correctly" about the scene. Read the full paper here: https://lnkd.in/gpsYhTsm 💡 Why this work matters: Vision Language Models are starting to play a larger role in high stakes domains like autonomous driving, robotics and safety critical perception. These models often provide natural language explanations for their decisions, but those explanations can sound correct even when they do not reflect the model’s actual reasoning. This creates major reliability and safety challenges. 💡 What we introduce: We propose Explanation Driven Counterfactual Testing (EDCT), a method that treats the model’s own explanation as a testable hypothesis. We extract key visual concepts from the explanation, generate targeted counterfactual edits to the image and then re-query the VLM to evaluate whether both the answer and the explanation respond in a logically consistent way. This gives us a measurable Counterfactual Consistency Score (CCS) that captures how faithful and causally aligned a model’s explanation truly is. 💡 Why this is important for autonomous safety: As teams explore VLM based end to end driving systems, auditing the reasoning behind a model’s decisions becomes essential. For example, if a system claims it slowed down because a cyclist signaled left, we should be able to minimally change that signal and verify that the model’s answer and explanation change accordingly. If not, there is a safety gap. Looking forward to presenting this work at NeurIPS and connecting with others working on regulatable, multimodal models and autonomous safety. #NeurIPS2025 #RegulatableML #VisionLanguageModels #VLMs #AIExplainability #SafetyCriticalAI #AutonomousVehicles

Mario Larcher

Canva • 5K followers

It had been on my list for a while to read the SID-1 technical report after seeing a post about how using OpenAI-style messages in RL can be surprisingly dangerous in multi-turn settings with many tool calls. The first insight is about the messages abstraction. Converting a raw token stream into messages and then back into tokens is lossy because it changes how the exact byte sequence is tokenized. A concrete example is when the model generates a sequence of bytes that the tokenizer splits into two very common tokens with normal probabilities. After parsing and reformatting through the chat template, those same bytes can be re-serialized in a way that the tokenizer now maps to a single, very rare token that the model almost never produces. The reward is computed on the original tokens the model actually generated, but the log-probs used for the update correspond to this new token that was never sampled. Since this token has an extremely low probability, its log-prob has a large magnitude, and in policy gradient this translates into a disproportionately large gradient. That single artificial token can end up dominating the update, creating a feedback loop that gradually destabilizes training and eventually leads to collapse. There is a second, distinct effect happening at the same time. Malformed tool calls or slightly wrong formatting can get “repaired” by the parser and chat template, so bad rollouts end up looking syntactically valid before they ever reach the trainer. The environment sees a correct tool call, the reward is good, and the trainer also sees a correct sequence. The model never receives signal that it actually produced something malformed. Stability is preserved, but the model does not learn tool correctness because the fixing layer steals the learning signal. Their fix is conceptually simple. Use a strict Tokens-In Tokens-Out pipeline, where the trainer sees exactly the token sequence the model generated. No parsing, no chat templates, no message abstraction in between. Another very interesting part of the report is their analysis of the “length debiasing” proposed in Dr. GRPO and similar works, where the per-token advantage is no longer normalized by the rollout length. This assumes that rollout length and quality are unrelated. In tool-use and reasoning-heavy environments this is not true. Bad rollouts tend to be longer than good ones. In that case, removing the length bias makes the average per-token advantage negative. Over long runs, this slowly pushes down the logits of all sampled tokens while pushing up the logits of tokens the model never uses, until the model starts emitting garbage or out-of-vocabulary tokens and collapses.

Sai Sandeep Kantareddy

7-Eleven • 10K followers

AI benchmarks are evolving not just in scale, but in cultural depth. I’ve been contributing to efforts like Global PIQA, which evaluates physical commonsense reasoning across 100+ languages, including Telugu, to better understand how models reason in diverse linguistic and cultural settings. What excites me most is how these multilingual evaluations connect to real systems retrieval, vector search, and agent pipelines that need to understand people in their native context, not just translate English text. We’re entering a phase where language, culture, and reasoning will define the next breakthroughs in AI quality and fairness. Building culturally aware evaluation and ingestion loops isn’t an academic curiosity anymore it’s a requirement for global-scale systems. Grateful to the Global PIQA team especially Tyler Chang Catherine Arnett for the opportunity to contribute and represent Telugu in this important work. I’d love to connect with others working on multilingual retrieval, vector search, and culturally grounded AI let’s share insights! #AI #MultilingualAI #Telugu #GlobalPIQA #NLP #CommonsenseReasoning #Inclusion #Research #OpenAI #MachineLearning #LanguageTechnology #Telugu #VectorSearch #RAG #AIResearch #CulturalIntelligence #Google #HuggingFace

Nishantha Ruwan

IWROBOTX Software Inc. • 2K followers

The authors target the challenge of mechanistic interpretability in transformer models, i.e., understanding which internal computations (neurons, residual channels, attention heads) implement specific tasks. They propose training decoder-only transformer models with extreme weight sparsity—most weights forced to zero—so that each neuron has only a few connections. This architectural constraint encourages the model to form simpler, more disentangled circuits. They then apply a pruning method to isolate minimal sub-networks (“circuits”) responsible for performing hand-crafted Python next-token tasks and demonstrate that these circuits often correspond to intuitive high-level operations (e.g., detecting quotes, counting list nesting) and use only a handful of neurons and channels. The study further investigates how model size and sparsity trade off: higher sparsity improves interpretability but reduces raw capability, while scaling up model size pushes the capability–interpretability frontier outward (i.e., larger sparse models can both perform better and remain interpretable). However, the authors flag that training sparse models de novo is inefficient and that scaling them beyond tens of millions of non-zero parameters while fully preserving interpretability remains a significant open challenge. They also present preliminary experiments applying their method to explain existing dense models. https://lnkd.in/gUyxgAgp