Inside Facebook’s Emotional Laboratory

LIBERTY OR DEATHWIRE! ⬩ INVESTIGATIVE REPORT⬩ AMERICA 250 SERIES

Written by: Andrew B. Raupp / @stemceo

Author’s Note: There are crimes that steal a man’s purse and crimes that steal his person; but the crime of experimenting upon a people without their knowledge steals something rarer still — the ground beneath their judgment. For a citizen who cannot know when he is being studied cannot know when he is being steered; and a citizen who cannot know when he is being steered is no citizen at all, but an instrument played by hands he will never see.

Our fathers in this century past sat in judgment at Nuremberg and wrote, in the ashes of the worst that men have done, a principle so plain a child may recite it: the voluntary consent of the human subject is absolutely essential. Not convenient. Not implied. Not buried in the fortieth paragraph of terms no honest man was ever meant to read. Essential. That word was purchased at a price no ledger can hold, and it was not written for doctors alone. It was written for every power — crowned, elected, or incorporated — that would treat the body or the mind of an unconsenting human being as raw material for its curiosity.

And yet how quickly the purchase is pawned. Tuskegee let sick men rot for the advancement of a chart. MKUltra dosed the unwitting in service of a theory. And in our own day, an empire of attention tuned the sorrows of six hundred eighty-nine thousand souls to see which way they would bend — then called the fine print consent and the apology a press release. The uniform changes; the presumption does not: that the many exist as material for the designs of the few.

I tell you the deepest wound of such experiments is not to the body of society but to its trust — and trust is the only currency a free people cannot print more of. Laws may be repealed and repassed; treasuries emptied and refilled; but let a people once learn that their feelings, their ailments, their very inward weather may be tampered with in secret by those who profess to serve them, and you will not restore their faith by committee. Suspicion becomes the common sense of the betrayed. And a nation that cannot trust cannot deliberate; and a nation that cannot deliberate cannot be free.

Therefore let the rule be short enough to carve: no experiment upon the people without the knowledge of the people. Whoever violates it — agency or academy, ministry or platform — has made himself the enemy of self-government, whatever flag or logo he flies. We did not hang that principle upon the wall at Nuremberg to take it down for a software update.

With Eternal Vigilance,
Andrew B. Raupp ✍️

Citizens Rule Book: A Palladium of Liberty (Download) #Infowars 🇺🇸

The paper carried a title only a laboratory could love: “Experimental evidence of massive-scale emotional contagion through social networks.”It appeared in June 2014 in the Proceedings of the National Academy of Sciences — one of the most prestigious journals in American science — under the names of a Facebook data scientist, Adam Kramer, and two academic collaborators, Jamie Guillory and Cornell University’s Jeffrey Hancock.

Strip away the language of the academy and the finding was this: Facebook had deliberately manipulated the News Feeds of hundreds of thousands of its own users to see whether it could make them feel worse. Or better. On command. At scale. It could.

For seven days — January 11 through 18, 2012 — 689,003 English-speaking users were selected by user ID and sorted, without their knowledge, into a psychological experiment. For one group, the algorithm suppressed posts from friends that contained positive words. For another, it suppressed the negative. Control groups had posts withheld at random. Each emotional post carried between a 10 and 90 percent chance of vanishing from a given feed load.

The machinery behind it was industrial. Roughly three million posts totaling over 122 million words were machine-read for emotional content using a standard word-counting tool, then filtered accordingly. The friends still posted. The words were still there, retrievable on a friend’s timeline. But the feed — the curated window through which users actually experienced their social world — had been tilted.

The result was the first experimental proof of emotional contagion through a social network at population scale. Users whose feeds were drained of positivity wrote measurably fewer positive words and more negative ones in their own posts. Users spared their friends’ negativity brightened accordingly. The effects per person were small — fractions of a percentage point — but they were real, statistically robust, and engineered across a population the size of a major American city.

“Emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness.”— KRAMER, GUILLORY & HANCOCK, PNAS, JUNE 2014

Read that phrase again: without their awareness. The authors did not bury the point. They advertised it. A private corporation had demonstrated, in a top-tier scientific journal, that its ranking algorithm could function as a dial on human mood — and that the people attached to the dial would never feel the hand turning it. The authors themselves noted that even tiny per-person effects, at Facebook’s scale, correspond to “hundreds of thousands of emotion expressions” per day.

The paper’s ethical apparatus was a single clause. The research, the authors wrote, “was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.”

It did not survive contact with the record. Forbes journalist Kashmir Hill, who broke the story into public consciousness, went back and pulled the Data Use Policy as it existed in January 2012 — the month the experiment ran. The word “research” was not in it. Facebook added “research” to the policy in May 2012 — roughly four months after the experiment ended.

Consent was claimed under terms that did not yet exist. And even the later boilerplate would fail every test the research-ethics tradition applies: informed consent under the Belmont Report means a comprehensible disclosure of risks, a genuine opportunity to decline, and special care for the vulnerable. A clause buried in a terms-of-service document nobody reads is not consent. It is camouflage.

Here is where the story stops being about one company and becomes a portrait of an accountability vacuum.

Since the 1970s, American research on human beings has been governed by the Common Rule — the federal regulation, born of the Tuskegee revelations, that requires institutional review board (IRB) approval and informed consent for federally supported human-subjects research. Had a graduate student proposed this experiment on a campus — deliberately depressing the emotional environment of unwitting subjects, including, statistically, people already fragile — an IRB would have demanded consent procedures, risk mitigation, and a debrief. Or refused it outright.

No such review touched this study before it ran. Facebook, a private company spending its own money, sat outside the Common Rule entirely; its safeguard was an internal review, on its own product, judged by its own people. Cornell’s IRB, for its part, concluded it had no jurisdiction: because Professor Hancock only analyzed results and “did not participate in data collection and did not have access to user data,” the university determined its faculty was “not directly engaged in human research.” Cornell also had to correct its own record after initially suggesting the work had U.S. Army Research Office funding; it had none.

The result was a perfect jurisdictional donut. The company that ran the experiment answered to no ethics board. The university that lent the experiment its academic prestige reasoned itself out of responsibility. And the journal that published it conceded the problem only after the fact: PNAS editor-in-chief Inder Verma appended a rare Editorial Expression of Concern acknowledging that Facebook’s data collection “may have involved practices that were not fully consistent with the principles of obtaining informed consent and allowing participants to opt out.” Legal scholar James Grimmelmann put the matter more plainly: the study was a scandal precisely because it was ordinary — A/B testing’s methods, aimed at the psyche, with academia’s imprimatur and none of academia’s restraints.

When the backlash arrived, it was met not with contrition for the act but with regret for the reaction. Kramer, in a Facebook post, allowed that “in hindsight, the research benefits of the paper may not have justified all of this anxiety” — and apologized for how the paper described the work. Sheryl Sandberg, then Facebook’s chief operating officer, told reporters the study was “poorly communicated. And for that communication we apologize.” The experiment itself, she said, was simply “part of ongoing research companies do to test different products.”

Regulators stirred, briefly. Britain’s Information Commissioner’s Office opened inquiries into whether data-protection law had been breached. Ethicists and psychologists publicly judged the study a breach of accepted research guidelines. Facebook’s durable response came that October: an internally designed, internally staffed review framework for future research. The fox, having been discovered in the henhouse, published guidelines for foxes.

The temptation is to file this under 2014 and move on. Resist it. The emotional-contagion study matters not because it was exceptional but because it was a rare, timestamped confession — a moment when the curtain slipped and the instrument panel was visible.

Three facts should be carved above the door of every legislature that touches technology policy. First: the feed is not a window, it is a dial, and the dial works. Facebook proved — in peer review, under its own name — that algorithmic curation alters human emotional states without the subject’s awareness. Second: the only reason the public learned of this experiment is that the company chose to publish it. The everyday experiments — the engagement optimizations, the outrage-weighted rankings, the retention tweaks run on billions — are never submitted to a journal, and so are never seen at all. What was scandalous when written up in PNAS is business as usual when written up in an internal dashboard. Third: a decade of platform-accountability fights, whistleblower disclosures, and congressional hearings have not closed the jurisdictional donut. Private algorithmic experimentation on the emotions of the public remains, in the main, self-reviewed and self-forgiven.

The founders this publication takes its name from understood that unaccountable power does not require malice to become dangerous — only darkness. In January 2012, for one documented week, the most powerful communications platform in human history ran a controlled experiment on the inner lives of 689,003 citizens and told no one. The needle moved. The people never felt the hand.

That is the story. The scandal is that it took a scientific journal’s vanity to reveal it — and that nothing in law yet guarantees we would learn of the next one.

Citizens Rule Book: A Palladium of Liberty (Hardcopies) 🇺🇸

Liberty or Deathwire! is an independent investigative publication. Plates I–IV were produced from the primary record: PNAS 111(24) and 111(29), the Cornell University media statement of June 30, 2014, and contemporaneous reporting in Forbes, The Guardian, The Atlantic, and the BBC. Additional background: Robinson Meyer’s contemporaneous dossier in The Atlantic and Michelle N. Meyer’s dissenting commentary in Nature, which argued the backlash risked driving corporate experimentation underground rather than into the light — a warning that, twelve years on, reads as prophecy.