Phenx Inc
The folk model says a Mixture of Experts holds a code expert and a medicine expert. Trained from scratch on two consumer GPUs, the routing tells you more about whether a word is a comma than about what the document is about, and it froze in the first 2% of training. Three attempts to force topic routing all came out slightly worse.
The folk model says a Mixture of Experts has topic specialists: a code expert, a geography expert. I aimed a Jacobian lens at the router on OLMoE and Qwen-MoE and read what actually flows into each expert. It sorts brackets, newlines, and verb tense, not topics.
A Mixture-of-Experts model is supposed to spend a fraction of itself on each token. I put CUDA events around every MoE layer of OLMoE-1B-7B on an RTX 4090 and recorded which of its 1,024 expert slots actually fire. At a batch of eight, 628 of them fire every single step -- and at batch 32 it is 93%. The sparsity is real, it is just not where people think it is.
The prompt never says Italy, but twelve layers into Qwen3.5-4B the model is already thinking it. J-Scope lets you watch that happen, inject France and get Paris, or drag one vector and watch the answer walk from euro to yen. A 72-second tour here; the live tool is one click away.
I wrote facts directly into a 4B model's J-space and KV cache on an RTX 4090. Three cache tokens matched full text-RAG, survived 48 distractors, and showed where retrieved facts actually live.
Can a cheap reward model tell a good AI agent plan from a bad one before you run it? I reproduced Orch-RM on a consumer RTX 4090 and RTX 3090: the verifier barely beats a coin flip on the signal that matters, yet still beats majority vote. Cheap to run, expensive to teach.
I tested the viral Karpathy coding-discipline rules on 100 real SWE-bench bugs, three times. They didn't help: the agent fixed fewer of them, trading fixes for caution.
MSA sparse attention, retrofitted onto a 3B model on a single RTX 4090 with no training: the 28.4x compute cut holds, quality survives, plus caveats the paper skips.
I tested DFlash speculative decoding on an M5 Max and RTX 4090. The 4x headline is real, but workload and concurrency decide whether you get a speedup or a slowdown.
A field note on over-cautious process control, and the margin hiding in your own sensor data. In one engagement: the force the controller watches stays low across speeds, while sidewall wear spikes at the crawl. “Slower is safer” is the most expensive assumption on your shop floor. Most adaptive controllers are built on it. When cutting force climbs, the controller slows the tool; when force…
In 1772, Joseph-Louis Lagrange solved a piece of the three-body problem that had stumped everyone before him. He didn't solve it by tracking forces. He solved it by finding the five points in the gravitational field of two large bodies where a third, smaller body can sit stably - held in place not by any single pull, but by the balance of all of them. These are the Lagrange points. They are where…
Why most enterprise GenAI initiatives stall after pilots and what actually scales The Investor Narrative vs the Operator Reality From the outside, enterprise generative AI looks like a straight line. Budgets are approved. Pilots launch quickly. Demos look impressive. Board decks show steady progress. Recent industry benchmarks show broad enterprise investment and deployment of generative AI, with…
Artificial intelligence is no longer a futuristic concept reserved for sci-fi movies. It’s here, and it’s shaking up the financial world in ways we never imagined. From speeding up transactions to detecting fraud before it happens, AI is transforming finance at a breakneck pace. But how exactly is this happening? And what does it mean for the future of money management? Exploring AI Applications…
How Emergent Uncomputability Could Explain Both the Universe and Artificial Intelligence The idea that we might be living inside a simulation refuses to die. It’s one of those questions that blends philosophy, physics, and computer science so elegantly that it keeps resurfacing in every generation: If reality follows rules, could those rules be running somewhere? When a team of physicists from the…
Imagine a world where your business operations run smoother, faster, and smarter. Sounds like a dream, right? But with the rise of artificial intelligence, this dream is becoming a reality. The key? Tailored AI solutions designed specifically for your unique challenges and goals. Let’s dive into how these customized technologies can transform your operations and give you a competitive edge. Why…
Fraud is a sneaky adversary. It evolves, adapts, and strikes when you least expect it. If you think your business is too small or too niche to be targeted, think again. Fraudsters don’t discriminate. They look for vulnerabilities, and if you’re not prepared, you’re an open door. So, how do you stay ahead? How do you protect your assets, your reputation, and your customers? The answer lies in…
Understanding Intellectual Property in the Modern Age At its core, intellectual property (IP) is not merely about ownership of ideas. It revolves around incentive design . Societies discovered long ago that creativity and invention flourish when individuals can reap benefits from their mental labor. However, pure ideas are non-rivalrous: once shared, they can be copied infinitely. Therefore, IP…
Animal vs Ghost Most AI systems today are ghosts: trained on human text, fine-tuned for alignment, and limited to observing and...
Here’s what’s changed or accelerated in recent months, the “hot takes” you can’t ignore. 1. AI Agents + Multimodal Intelligence Are...
Artificial intelligence (AI) is transforming the way businesses operate across industries. However, the true power of AI lies not in...