Adin Mauer

Majestic Labs ai • 1K followers

I am currently reviewing recent technical developments in Large Language Models (LLMs) and have some observations to share. Regarding "Clara: bridging retrieval and generation with continuous latent reasoning," I initially questioned the concept of "deep reasoning" that frequently arises in discussions. My initial thought was whether it referred to the recursive chain of thought process (which is less inspiring, personally). However, I've concluded that this is not the case. Reasoning, in this context, involves the processing of compressed inputs through transformer blocks, where these inputs are compressed into a latent space. This led me to consider the following question: What is the difference between Latent Attention (such as DeepSeek's MLA) and Deep Reasoning? The key difference, while straightforward, is interesting. Latent Attention occurs within individual transformer blocks. Latent Attention operates within individual transformer blocks: inputs are projected into a dense latent space and then reconstructed back into tokens before passing to the next block. The latent space here primarily serves to reduce memory footprint and capture compressed information locally within each block. Deep Reasoning, by contrast, operates across the entire architecture. Inputs are compressed into a latent representation that persists through all transformer blocks, with decoding back to tokens occurring only at the final stage. Executing inference entirely within the latent space can be interpreted as a form of higher-level reasoning, given that these compressed latent vectors can be understood as a compressed and refined abstraction of the input. P.S. First technical LinkedIn post. https://lnkd.in/gccHXqbU

Yorgos Pantis

Archimedes - AI and Data… • 2K followers

🚨 NeurIPS 2025 Announcement Our paper “Teaching Transformers to Solve Combinatorial Problems through Efficient Trial & Error” will be presented in San Diego! LLMs excel in natural language tasks but often struggle with combinatorial reasoning. Our work bridges this gap, achieving SOTA results, 99.8% accuracy on Sudoku, by teaching Transformers to apply simple Sudoku rules with guesses, much like humans do! Notably, this is done without external solvers or custom architectures, and we show how the setup relates to a contextual version of the classic Min-Sum Set Cover problem. Paper: https://lnkd.in/dV6fa_mJ Authors: Panagiotis Giannoulis, Yorgos Pantis, Christos Tzamos. Affiliations: National Technical University of Athens, National Kapodistrian University of Athens, Archimedes: AI, Data Science, and Algorithms Research Hub, and Athena Research Center.

Adam Zaremba

Kantonsschule Zürich Nord… • 90 followers

Does more test-time compute actually improve LLM accuracy? I recently ran a controlled experiment on AIME math problems to test a common assumption in LLM inference: that sampling more answers at test time (1x, 10x, 100x, 1000x) should reliably improve correctness. Using fixed-k parallel sampling with majority voting and explicit early stopping, I found: - Accuracy does not improve beyond a single sample - Marginal gains per 10× compute are exactly zero - The model typically converges to a confident answer after one sample, so additional compute goes unused in practice I honestly expected at least a small gain at k = 10, but it just didn’t show up. To make this defensible, I also ran a control experiment disabling early stopping, forcing the model to use all samples. Accuracy still didn’t improve-only runtime did. I’ve open-sourced the full pipeline (per-example logs, control runs, and frontier plots) so the results are fully reproducible. GitHub Repo: https://lnkd.in/ez9q3dyA I’m now writing this up more formally, but wanted to share the core result early. Thoughtful critique or alternative explanations are very welcome :D #AIResearch #LLMs #MachineLearning