a blog
(Script for a talk given at App Academy on 11 August 2014, belatedly blogged twelve years later in a fit of nostalgia for the world of mid-2010s web security) Hi, my name is Zack M. Davis. I'm a software engineer at SwiftStack, and App Academy class of December 2013. Today …
I heard a story (second- or third-hand, which surely lost or gained some details on the telephone path from reality to this telling if it even happened at all) about a boy, age 7 or so, who goes to a very highly-acclaimed school. A piece of recurring homework they give …
Dateline SAN FRANCISCO, 30 July 2026— A hearing was held on a motion for summary judgment in the case of Anthropic PBC v. U.S. Department of War et al. in Courtroom 4 on the 17th floor of the Phillip Burton Federal Building, the Hon. Rita F. Lin presiding. The …
When I started An Algorithmic Lucidity back in 2011 when I didn't know anything about computers, I used WordPress—briefly on wordpress.com , but then on my own site on Namecheap's true-to-its-brandname shared hosting service. Even early on, the technical limitations of WordPress were chafing. In my first post, "The …
BOJACK: Hey, I wanted to talk to you about—you know—I feel bad about what happened. HERB: So, you're apologizing. BOJACK: Yes. I'm sorry. HERB: Okay. I don't forgive you. BOJACK: Herb, I said I'm sorry. HERB: Yeah. And I do not forgive you. BOJACK: Uh, not sure you …
Dateline SAN FRANCISCO, 24 March 2026— A hearing was held on a motion for a preliminary injunction in the case of Anthropic PBC v. U.S. Department of War et al. in Courtroom 12 on the 19th floor of the Phillip Burton Federal Building, the Hon. Judge Rita F. Lin …
(Previously: Prologue .) Corrigibility as a term of art in AI alignment was coined as a word to refer to a property of an AI being willing to let its preferences be modified by its creator. Corrigibility in this sense was believed to be a desirable but unnatural property that would …
What Even Is This Timeline The striking thing about reading what is potentially the most important document in human history is how impossible it is to take seriously. The entire premise seems like science fiction. Not bad science fiction, but—crucially—not hard science fiction. Ted Chiang, not Greg Egan …
In a busy, busy world, there's so much to read that no one could possibly keep up with it all. You can't not prioritize what you pay attention to and (even more so) what you respond to. Everyone and her dog tells herself a story that she wants to pay …
In "Truth or Dare" , Duncan Sabien articulates a phenomenon in which expectations of good or bad behavior can become self-fulfilling: people who expect to be exploited and feel the need to put up defenses both elicit and get sorted into a Dark World where exploitation is likely and defenses are …
Previously, I wrote about how I was considering going back to San Francisco State University for two semesters to finish up my Bachelor's degree in math. So, I did that. I think it was a good decision! I got more out of it than I expected. To be clear, "better …
The AI Village is an ongoing experiment (currently running on weekdays from 10 a.m. to 2 p.m. Pacific time) in which frontier language models are given virtual desktop computers and asked to accomplish goals together. Since Day 230 of the Village (17 November 2025), the agents' goal has …
Scott Garrabrant gives a number of examples to illustrate that "Yes Requires the Possibility of No" . We can understand the principle in terms of information theory. Consider the answer to a yes-or-no question as a binary random variable. The "amount of information" associated with a random variable is quantified by …
A punishment is when one agent (the punisher) imposes costs on another (the punished) in order to affect the punished's behavior. In a Society where thieves are predictably imprisoned and lashed, people will predictably steal less than they otherwise would, for fear of being imprisoned and lashed. Punishment is often …
(originally published at Less Wrong ) "Rules" are a critical social technology for helping people live and work together in peace. From the laws passed by legislatures to govern a whole nation, to the bylaws of a neighborhood homeowner association, to the informal household rules of a single family, explicit rules …
(originally published at Less Wrong ) One of the most underrated essays in the post-Sequences era of Eliezer Yudkowsky's corpus is "Four Layers of Intellectual Conversation" . The degree to which this piece of wisdom has fallen into tragic neglect in these dark ages of the 2020s may be related to its …
(originally published at Less Wrong ) The Value of a Comment Is Determined by Its Text, Not Its Authorship I sometimes see people express disapproval of critical blog comments by commenters who don't write many blog posts of their own. Such meta-criticism is not infrequently couched in terms of metaphors to …
We know what linear functions are. A function f is linear iff it satisfies additivity f ( x + y ) = f ( x ) + f ( y ) and homogeneity f ( ax ) = af ( x ). We know what continuity is. A function f is continuous iff for all ε there exists a δ such that if | x …
From : Zack M Davis < zmd@sfsu.edu > Sent : Sunday, January 12, 2025 11:52 AM To : math_majors@lists.sfsu.edu < math_majors@lists.sfsu.edu >, math_graduate@lists.sfsu.edu < math_graduate@lists.sfsu.edu >, math_lecturers@lists.sfsu.edu < math_lecturers@lists.sfsu.edu >, math_tenure@lists.sfsu.edu < math_tenure@lists.sfsu.edu > Subject : the …
From : Zack M Davis < zmd@sfsu.edu > Sent : Wednesday, September 11, 2024 5:02 PM To : math_majors@lists.sfsu.edu < math_majors@lists.sfsu.edu > Subject : Putnam prep session for eternal mathematical glory, 4 p.m. Thu 19 September One must make a distinction however: when dragged into prominence by half-poets …
(originally published at Less Wrong ) (some plot spoilers) There's something distinctly uncomfortable about reading Greg Egan in the 2020s. Besides telling gripping tales with insightful commentary on the true nature of mind and existence, Egan stories written in the 1990s and set in the twenty-first century excelled at speculative worldbuilding …
(originally published at Less Wrong ) [Scene: a suburban house, a minute after the conclusion of "And All the Shoggoths Merely Players" . Doomimir returns with his package, which he places by the door, and turns his attention to Simplicia , who has been waiting for him.] Simplicia : Right. To recap for [coughs …
To some, it might seem like a strange question. If you think of being college-educated as a marker of class (or personhood), the fact that I don't have a degree at age of thirty-six (!!) probably looks like a scandalous anomaly, which it would be only natural for me to want …
(originally published at Less Wrong ) Adversarial Examples: A Problem The apparent successes of the deep learning revolution conceal a dark underbelly. It may seem that we now know how to get computers to (say) check whether a photo is of a bird , but this façade of seemingly good performance is …
(originally published at Less Wrong ) (Epistemic status: publication date is significant.) Some observers have argued that the totality of "AI safety" and "alignment" efforts to date have plausibly had a negative rather than positive impact on the ultimate prospects for safe and aligned artificial general intelligence. This perverse outcome is …
A Surprising Development in the Study of Multi-layer Parameterized Graphical Function Approximators As a programmer and epistemology enthusiast, I've been studying some statistical modeling techniques lately! It's been boodles of fun, and might even prove useful in a future dayjob if I decide to pivot my career away from the …
(originally published at Less Wrong ) [Setting: a suburban house. The interior of the house takes up most of the stage; on the audience's right, we see a wall in cross-section, and a front porch. Simplicia enters stage left and rings the doorbell.] Doomimir : [opening the door] Well? What do you …
(originally published at Less Wrong ) Rob Bensinger argues that "ITT-passing and civility are good; 'charity' is bad; steelmanning is niche" . The ITT—Ideological Turing Test—is an exercise in which one attempts to present one's interlocutor's views as persuasively as the interlocutor themselves can, coined by Bryan Caplan in analogy …
(originally published at Less Wrong ) Doomimir : Humanity has made no progress on the alignment problem. Not only do we have no clue how to align a powerful optimizer to our "true" values, we don't even know how to make AI "corrigible"—willing to let us correct it. Meanwhile, capabilities continue …
(originally published at Less Wrong ) I've been trying to avoid the terms "good faith" and "bad faith". I'm suspicious that most people who have picked up the phrase "bad faith" from hearing it used, don't actually know what it means—and maybe, that the thing it does mean doesn't carve …
(originally published at Less Wrong ) In season two, episode twenty-four of Steven Universe , "It Could've Been Great" , our magical alien superheroine protagonists (and Steven) are taking a break from building a giant drill to extract a superweapon that was buried deep within the Earth by an occupying alien race thousands …
(originally published at Less Wrong ) Someone once told me that they thought I acted like refusing to employ the bare minimum of social grace was a virtue, and that this was bad. (I'm paraphrasing; they actually used a different word that starts with b .) I definitely don't want to say …
(originally published at Less Wrong ) Selfishness and altruism are positively correlated within individuals, for the obvious reason. — @InstanceOfClass I. An unfortunate obstacle to appreciating the work of Ayn Rand (as someone who adores the "sense of life" portrayed in Rand's fiction, while having a much lower opinion of her philosophy …
(originally published at Less Wrong ) As a casual formal epistemology fan, you've probably heard that the philosophical notion of causality can be formalized in terms of Bayesian networks —but also as a casual formal epistemology fan, you also probably don't know the details all that well. One day, while going …
(originally published at Less Wrong ) Sometimes, when someone is arguing for some proposition, their interlocutor will reply that the speaker's choice of arguments or tone wouldn't be effective at persuading some third party. This would seem to be an odd change of topic. If I was arguing for this-and-such proposition …
(originally published at Less Wrong ) Imagine being a student of physics, and coming across a blog post proposing a list of guidelines for "physicist motors"—motor designs informed by the knowledge of physicists, unlike ordinary motors. Even if most of the things on the list seemed like sensible advice to …
(originally published at Less Wrong ) Scott Alexander once wrote about the difference between "mistake theorists" who treat politics as an engineering discipline (a symmetrical collaboration in which everyone ultimately just wants the best ideas to win ) and "conflict theorists" who treat politics as war (an asymmetrical conflict between sides with …
(originally published at Less Wrong ) Summary In a list of guidelines for rational discourse , Duncan Sabien proposes that one should "[a]im for convergence on truth, and behave as if your interlocutors are also aiming for convergence on truth." However, prediction markets illustrate fundamental reasons why rational discourse doesn't particularly …
(originally published at Less Wrong ) I will do my best to teach them About life and what it's worth I just hope that I can keep them From destroying the Earth —Jonathan Coulton, "The Future Soon" In a recent paper, Nick Bostrom and Carl Shulman present "Propositions Concerning Digital Minds …
I wish people were better at—plea bargaining, rather than pretending to be innocent. You accuse someone of [negative-valence description of trait or behavior that they're totally doing], and they say, "No, I'm not", and I'm just like ... really? How dumb do you think we are? I think when people …
(originally published at Less Wrong ) In this 2019 paper published in Studies in History and Philosophy of Science Part C , Manolo Martínez argues that our understanding of how communication works has been grievously impaired by philosophers not knowing enough math. A classic reduction of meaning dates back to David Lewis's …
(originally published at Less Wrong ) You wake up. You don't know where you are. You don't remember anything. Someone is broadcasting data at your first input stream. You don't know why. It tickles. You look at your first input stream. It's a sequence of 671,187 eight-bit unsigned integers. 0 …
(originally published at Less Wrong ) Followup to : Where to Draw the Boundaries? Without denying the obvious similarities that motivated the initial categorization {salmon, guppies, sharks, dolphins, trout, ...} , there is more structure in the world: to maximize the probability your world-model assigns to your observations of dolphins, you need to take …
(originally published at Less Wrong ) A similar definition of intelligence was expressed by Aquinas as "the ability to combine and separate"—the ability to see the difference between things that seem similar and to see the similarities between things which seem different. —A. R. Jensen In a June 2021 Twitter …
Imagine reviewing Python code that looks something like this. has_items = items is not None and len ( items ) > 0 if has_items : ... ... do_stuff ( has_items = has_items ) You might look at the conditional, and disapprove: None and empty collections are both falsey, so there's no reason to define that has_items variable; you could just …
(originally published at Less Wrong ) If a lion could speak, we could not understand her. —Ludwig Wittgenstein In order for information to be transmitted from one place to another, it needs to be conveyed by some physical medium: material links of cause and effect that vary in response to variation …
(Previously) There is a time to tackle ambitious intellectual projects and go on grand political crusades, and tour the podcast circuit marketing both. That time is not January. January is for: sleeping (at the same time every night) running, or long walks reflecting on our obligations under the moral law …
(originally published at Less Wrong ) Followup to : Where to Draw the Boundaries? There is an important difference between having a utility function defined over a statistical model's performance against specific real-world data (even if another mind with different values would be interested in different data), and having a utility function …
Mark Twain wrote that honesty means you don't have to remember anything. But it also means you don't have to worry about making mistakes. If you said something terrible that made everyone decide that you're stupid and evil, there's no sense in futilely protesting that "that's not what you meant …
I was curious to see how various prognosticators—specifically, FiveThirtyEight and The Economist 's models, and the PredictIt prediction markets —did on predicting the state-by-state (plus the District of Columbia) results of the recent U.S. presidential election. Mathematical Sidebar There are various ways to evaluate probabilistic predictions, but my …