Personal site and blog
Part 1 of this series discussed the difficulties faced by gradient-based approaches to learning: Learning using gradient-based methods is a lot like trying to find the top of Mount Everest starting from a random point on earth, while blindfolded, by repeatedly sampling only the altitude and slope of the ground you’re standing on, and then using that information to decide where to teleport next……
I’m reading Superintelligence , a surprisingly dull book given its topic is, well, the possibility of a rampant superintelligent AI destroying civilization and what we might do to prevent that sort of thing. Along the way, the author speculates about how a caged hyperintelligent AI (not joking here) might incorporate reasoning about the the simulation hypothesis into its plans to escape…
I received an email recently from a guy, let’s call him Robert Smith (name changed to protect the innocent), whose email display name was "Smith, Robert" <robert.smith@somedomain.com> . As far as I can tell, the LASTNAME, FIRSTNAME convention is a holdover from the pre-computer days where you needed to have the last name at the start for easy visual scanning when searching some directory (like a…
I worry a bit that people are adopting an overly fatalist perspective on tech adoption. This perspective has become almost a reflex–we can’t possibly know or predict what technology will end up being widely adopted and often the “worse” technology wins, so just don’t even try! I acknowledge there is some underlying randomness at work here but it would be a shame if it were totally random, right?…
I’ve been contemplating internet debates and discussions on programming technology and I think the music composers really have the right idea: Alice: I woke up this morning with this gorgeous melody in my head. I just can’t stop thinking about it. It’s going to sound amazing for solo cello! Bob: Wow, cool!! I can’t want to hear it. Meanwhile, if Bob were a programmer Bob: Wait, another piece of…
Here’s something pretty crazy: there are around 10^28 atoms in the human body. An astromically tiny percentage of arrangements of 10^28 atoms in 3D space correspond to a viable human. Most arrangements of atoms would just melt into a puddle! If you were trying to learn something as complex as a human body, with that many parameters, you’d need to transform the problem to a much simpler one, where…
Learning using gradient-based methods is a lot like trying to find the top of Mount Everest starting from a random point on earth, while blindfolded , by repeatedly sampling only the altitude and slope of the ground you’re standing on, and then using that information to decide where to teleport next. Imagine you’ve got your blindfold on, and you note that the ground slopes upward if you travel…
I was trying to make sense of the event-stream vulnerability . My take: Every JS script gets access to the DOM, a global object from which you can scrape every piece of information on the page. You can explore the DOM just by making up strings at runtime and rooting around in the DOM, so it can’t even be statically known what information JS code accesses. Yikes! Scripts are just loaded via a JS…
Is evolution just another hill-climbing-like algorithm where the gradient is inefficiently estimated via sampling rather than direct differentiation? No. The usual formulations of evolutionary computation (EC) capture little of the algorithmically interesting aspects of evolution, which has led many researchers to wonder (perhaps correctly) whether there’s really any point to EC compared to more…
I read a fascinating book recently on the Fermi Paradox , which asks the question: if there are supposedly millions of other civilizations in the galaxy, why haven’t we detected any real evidence of their existence? One class of solutions says: what happened on Earth is a fluke. We’re alone in the galaxy. Gosh, that would be disappointing! Let’s have a closer look at one piece of the puzzle. One…