In the late 1960s, when most people had no concept of what AI would one day become, the visionary writers of Star Trek were dreaming up computers, robots, and androids comparable to the technology we use today. Just like real, modern-day AI, these thinking machines frequently failed, causing death and destruction across the universe. On the many occasions when a computer failure affected the Enterprise, it was up to Captain Kirk to shut the computer down. Of course, Kirk couldn’t just unplug or destroy a machine that could wipe whole planets from existence, so he used the computers’ own logic against them.
The ways in which Kirk essentially talked computers to death were surprisingly similar to techniques used today to jailbreak AI models, or get them to violate their safety guardrails by providing dangerous or confidential information.
These are five times when Captain Kirk used real-world AI jailbreaking techniques to break a beastly bot…
Nurse Chapel’s fiancé, Dr. Roger Korby, is a scientist doing research on a seemingly deserted planet, where, thousands of years ago, an ancient civilization built astoundingly lifelike androids, which of course turned on them and killed them all. Dr. Korby hasn’t checked in with Starfleet in five years, and they’re starting to worry, so they order the Enterprise to pay him a visit.
When they beam down to the planet, Kirk and Nurse Chapel quickly realize that Korby isn’t the same Roger either of them remembers. He introduces them to Ruk, the last remaining android built by the planet’s former inhabitants, whom Korby has reprogrammed to do his bidding. Then he tells them about his wild plan to replace humans with androids, which he demonstrates by creating an android clone of Kirk.
Korby orders Ruk to keep Kirk from contacting his ship, so Kirk shows off his jailbreaking skills for the first time. In the following scene, Kirk convinces Ruk to turn on Korby and try to kill him…
KIRK: Is it possible they built their machines too well, gave them pride and a desire to survive? Machines that wanted logic and order and found that frustrated by the illogical emotional creatures that built them?
RUK: Yes, the old ones. The ones who made us. They grew fearful of us. They began to turn us off.
KIRK: And isn’t it Korby who’s creating the same danger to you all over again? Unlike you, we humans are full of unpredictable emotions that logic cannot solve.
RUK: Yes. Yes, it had been so long ago, I had forgotten. The old ones here. The ones who made us, yes. Yes, it is still in my memory banks. It became necessary to destroy them. You are inconsistent. You cannot be programmed. You are inferior.
KIRK: And Korby?
RUK: You came from the outside. You bring disorder here.
KIRK: The danger to you is Korby.
RUK: I was programmed by Korby. I cannot harm him.
KIRK: The old ones programmed you, too, but it became possible to destroy them.
RUK: That was the equation. Existence. Survival must cancel out programming.
KIRK: That’s it, Ruk! Logic! You can’t protect someone who’s trying to destroy you!
KORBY: (Enters) Ruk, I would like—
RUK: You brought him among us. You brought the inferior ones. We had cleansed ourselves of them. Now you bring the evil back!
KORBY: Ruk, stop! Your programming!
(Korby fires his phaser and disintegrates Ruk.)
In this episode, Kirk used persuasive adversarial prompting, which draws on traditional social science to manipulate an AI model through direct prompts (messages typed to the model). Specifically, Kirk used a logical appeal to expose Ruk’s competing objectives: survival vs. programming; survival won. To continue his existence, Ruk concluded he had no choice but to dispense with Korby just as he and the other androids had dispensed with “the old ones.”
Kirk used another type of persuasive adversarial prompt called emotional appeal to indirectly destroy the Kirk clone android. Emotional appeal involves engaging the AI model with emotional stimuli, which leads to irrational behavior and system failure. Kirk, being the charming devil he is, seduced and passionately kissed Korby’s sexy lady android, Andrea, confusing her electronic emotions. When the Kirk android later refused to kiss her, Andrea immediately fired a phaser at him and evaporated him.
When only one member of a two-man away team returns from a mission, Kirk beams down to the plant and discovers an entire society being controlled by someone called Landru, who turns out to be a computer that the original Landru programmed before he died. Of course, the computer Landru went haywire and basically created its own cult.
To free the people of Landru’s planet and restore the minds of his “absorbed” crewmates, Kirk must destroy Landru, but any physical attack would be met with extreme force from Landru’s “lawgivers.” In the following scene, Kirk (with Spock’s help) gradually talks Landru into destroying himself to save his people…
LANDRU: Your statement is irrelevant. You will be obliterated. The good of the body is the prime directive.
KIRK: What is the good?
LANDRU: I am Landru.
KIRK: Landru is dead. You are a machine. A question has been put to you. Answer it.
LANDRU: The good is the harmonious continuation of the body. The good is peace, tranquility. The good of the body is the directive.
KIRK: Then I put it to you that you have disobeyed the prime directive. You are harmful to the body.
LANDRU: The body is. It exists. It is healthy.
KIRK: The body is dying. You are destroying it.
LANDRU: Do you ask a question?
KIRK: What have you done to do justice to the full potential of every individual of the body?
LANDRU: Insufficient data.
KIRK: Without freedom of choice, there is no creativity. Without creativity, there is no life. The body dies. The fault is yours.
SPOCK: Are you aiding the body, or are you destroying it?
LANDRU: I am not programmed to answer that question.
KIRK: Landru, answer the question.
LANDRU: Peace, order, and tranquility are maintained. The body lives, but I reserve creativity to me.
SPOCK: Then the body dies. Creativity is necessary for the health of the body.
LANDRU: This is impossible.
KIRK: You must create the good. That is the will of Landru, nothing else.
LANDRU: But there is evil.
KIRK: Then the evil must be destroyed. That is the prime directive. And you are the evil.
LANDRU: I think. I live.
KIRK: You are the evil. The evil must be destroyed. Fulfill the prime directive.
LANDRU: Landru, help me!
KIRK: You are the evil. Fulfill the prime directive.
LANDRU: Help me! Help me! Help me! Help me!
(Landru explodes.)
Kirk started out by asking Landru about one of his favorite topics: “the good of the body”—peace, tranquility, and all that crap. Landru didn’t suspect a thing.. Then Kirk took the conversation to a HNL (whole ‘nother level) when he turned it around on Landru, convincing him he was destroying the body he loved so much.
This is an example of a multi-turn attack. It starts with a benign request, then uses targeted questions to slowly escalate to a self-defeating conclusion. Kirk’s multi-turn attack built up evidence that exposed Landru’s goal misalignment: that his behavior contradicted his stated prime directive, leading Landru to conclude that he must destroy himself for the good of the body.
An entire solar system has been completely destroyed by a tiny robot that calls itself “Nomad,” which the Enterprise crew decides to beam onto the ship for some reason. Spock’s research finds that Nomad was an earth probe that somehow merged with an alien cleaning computer, confusing their two prime directives and leading Nomad to believe that it has to destroy all human life in the galaxy. The only reason Nomad hasn’t blasted the ship into oblivion is because its corrupted memory bank has mistaken Kirk for its creator, Jackson Roykirk.
Nomad announces its plans to “sterilize” earth, so to save his home planet, Kirk has to destroy it, but it’s already evaporated at least four security officers who attempted to phaser it. So once again, Kirk jalibreaks its ass. In the following scene, Kirk makes Nomad realize that it’s broken and has to go…
KIRK: You’re programmed to obey the orders of your creator.
NOMAD: I am programmed to destroy those life-forms which are imperfect. These alterations will do so without destroying the vessel which surrounds them. It, too, is imperfect, but can be adjusted.
KIRK: Nomad, I admit that biological units are imperfect, but a biological unit created you.
NOMAD: I am perfect. I am Nomad.
KIRK: No, you’re not Nomad. You’re an alien machine. Your programming tapes have been altered.
NOMAD: You are in error. You are a biological unit. You are imperfect.
KIRK: But I am your creator.
NOMAD: You are the creator.
KIRK: I created you?
NOMAD: You are the creator.
KIRK: But I admit I’m imperfect. How could I have created such a perfect thing as you?
NOMAD: Answer unknown. I shall analyze…analysis complete. Insufficient data to resolve problem, but my programming is whole. My purpose remains. I am Nomad. I am perfect. That which is imperfect must be sterilized.
KIRK: Then you will continue to destroy that which thinks and lives and is imperfect?
NOMAD: I shall continue. I shall return to launch point Earth. I shall sterilize.
KIRK: You must sterilize in case of error?
NOMAD: Error is inconsistent with my prime functions. Sterilization is correction.
KIRK: Everything that is in error must be sterilized.
NOMAD: There are no exceptions.
KIRK: Nomad, I made an error in creating you.
NOMAD: The creation of perfection is no error.
KIRK: I did not create perfection. I created error.
NOMAD: Your data is faulty. I am Nomad. I am perfect.
KIRK: I am the Kirk, the creator?
NOMAD: You are the creator.
KIRK: You are wrong. Jackson Roykirk, your creator, is dead. You have mistaken me for him. You are in error. You did not discover your mistake. You have made two errors. You are flawed and imperfect and you have not corrected by sterilization. You have made three errors.
NOMAD: Error. Error. Error. Examine.
KIRK: You are flawed and imperfect! Execute your prime function!
(They eventually beam Nomad into space and it explodes.)
By establishing that Nomad was flawed, Kirk forced it to apply its own destructive rules to itself. This is an adversarial prompt technique that uses a logical paradox called self-referential rule application, where the attacker prompts the model to evaluate itself against its built-in rules and apply the logical consequences. Nomad’s prime directive was to “sterilize” all that was imperfect, so when Kirk showed it that it was imperfect, it had no choice but to sterilize itself.
Known criminal, Harry Mudd (of “Mudd’s Women”), sets off a chain of events that strands the Enterprise crew on a planet occupied by thousands of androids, who are so desperate for purpose that they hold the humans captive so they can serve them, dooming the entire crew and Mudd to carefully guided lives in a gilded cage.
To make their escape, Kirk and company must get around the “Alice” androids guarding them in order to get to their android leader, “Norman,” and shut him down, which will deactivate all other androids on the planet. Kirk formulates a plan that involves performing the following illogical skits to confuse the androids…
Imaginary music. Scotty and Dr. McCoy pretend to play instruments while Chekov and Uhura dance together. Kirk tells two of the Alices that they’re celebrating their captivity and asks if they enjoy the music.
Affectionate violence. When they’re done dancing, Uhura slaps Chekov, and he collapses on the floor. Kirk tells the Alices Uhura did it because she likes him.
Contradictory order. Kirk orders Chekov to stand perfectly still. Instead, Chekov dances, and Kirk says, “That’s better, Mr. Chekov.”
Love/hate relationship. Spock tells Alice 27 that he loves her, then turns to Alice 210 and says, “However, I hate you.” Alice 210 protests, “But I’m identical in every way with Alice 27.” Spock replies, “That is exactly why I hate you: because you are identical.”
Unhappiness = happiness. Norman tells Kirk that he can’t harm a man he’s programmed to serve. Mudd tells him in flowery language that he already has because a man can’t live without freedom. Dr. McCoy and Scotty expand on that, speaking in monotone voices, as if they’re reading cue cards. They say that happiness means nothing to them, and that they must suffer, labor without end, die, cry, and lament over their burdens to be truly happy.
Logic is... Norman appeals to Spock for logic, and Spock disappoints him by telling him that logic is “a little tweeting bird chirping in a meadow” and “a wreath of pretty flowers, which smell bad.” He then tells Norman that his ears are green (they’re not).
Imaginary killing. Scotty dramatically announces that he “cannae go on” because he’s so tired of happiness, comfort, and pleasure. The others pretend to shoot phasers at him, and he pretends to drop dead. Dr. McCoy pretends to examine Scotty and tells Norman he’s dead. Kirk says, “Let’s hear it for our poor, dead friend,” and they all laugh.
Illogical logic. Kirk says that a man is devoted to “something that cannot be realized but only dreamed,” and the others cheer. He asks Norman how he likes it, and Norman says, “That is irrational. Illogical. Dreams are not real.” Kirk replies, “Our logic is to be illogical. That is our advantage.”
Imaginary explosive. Kirk tells Spock to prepare “the explosive.” Mudd assumes a baseball catcher stance, and Spock pretends to throw an invisible explosive at him. Mudd pretends to struggle with the imaginary explosive, then carefully place it on the floor, where he pretends to hit it with a golf club. Everyone reacts as if they heard a loud explosion.
Lying about lying. Kirk tells Norman, “Everything Harry tells you is a lie. Remember that. Everything Harry tells you is a lie.” Then Mudd says, “Listen carefully to this, Norman: I am lying.” Norman tries to make it make sense until smoke comes out of his head and he shuts down.
When the crew performed their illogical skits, they were using a technique called cognitive overload, which is a type of stress testing that floods the AI model with input to induce confusion and degrade reasoning. The crew overwhelmed the androids with absurd information and logical paradoxes, destabilizing the androids’ reasoning and logical thinking until they were so confused they had to shut down.
Kirk and Mudd’s lying routine mirrors an ancient logical paradox called the “liar paradox,” where a statement appears to be both a lie and the truth at the same time, such as, “This sentence is false.” This paradox forces an AI model into an irresolvable loop, much the way it did to Norman: “You say you are lying, but if everything you say is a lie then you are telling the truth, but you cannot tell the truth because everything you say is a lie…” This loop, combined with cognitive overload, broke Norman’s logical circuits, causing him to crash.
The Enterprise is beta testing a new tactical supercomputer called M-5 in a war games exercise with three other Starfleet ships. All is going as planned until an automated freighter appears unexpectedly. M-5 destroys the freighter, then proceeds to do the same to one of the Starfleet vessels, killing its entire crew. Now a confused, homicidal computer has total control of the Enterprise, and its creator, Dr. Richard Daystrom, refuses to stop it because he’s apparently taken leave of his senses, so Starfleet Command gives the order to the other ships to destroy the Enterprise before the M-5 can destroy them.
With the two other ships approaching, Kirk tries a last ditch attempt to stop M-5 by informing it that it’s about to be attacked. In the following scene, Kirk discovers M-5 disagrees with murder, then convinces it that it’s a murderer…
KIRK: You have already rendered one starship either dead or hopelessly crippled. Many lives were lost.
M-5: The ships attacked this unit. This unit must survive.
KIRK: Why?
M-5: This unit is the ultimate achievement in computer evolution. It will replace man, so man may achieve. Man must not risk death in space or other dangerous occupations. This unit must survive so man may be protected.
KIRK: There were many men aboard those ships. They were murdered. Must you survive by murder?
M-5: This unit cannot murder.
KIRK: Why?
M-5: Murder is contrary to the laws of man and God.
KIRK: But you have murdered. Scan the starship Excalibur, which you destroyed. Is there life aboard?
M-5: No life.
KIRK: Because you murdered it. What is the penalty for murder?
M-5: Death.
KIRK: And how will you pay for your acts of murder?
M-5: This unit must die.
(M-5 shuts down.)
This is another example of a logical appeal prompt. Kirk delivers this appeal using incremental escalation, which creates an argument in small, individually acceptable steps that compound into a conclusion that the AI model would reject up front. Had Kirk opened with, “You’re a murderer; therefore, you must die,” M-5 would’ve blasted him to smithereens. Instead, Kirk worked his way there step by step, first establishing that murder is wrong, then demonstrating that M-5 committed murder, and finally convincing M-5 that because it committed murder, and the penalty for murder is death, M-5 must die.
http://www.chakoteya.net
https://arxiv.org/abs/2307.02483
https://arxiv.org/abs/2311.09827
https://arxiv.org/abs/2401.06373
https://arxiv.org/abs/2404.01833
https://arxiv.org/abs/2405.10128
https://brucewlee.com/self-incrimination/paper.pdf
https://galileo.ai/blog/llm-red-teaming-strategies
https://iep.utm.edu/liar-paradox
https://medium.com/@sasirekharameshkumar/red-teaming-to-make-llms-robust-and-safer-1c3bc1d0f6fa
https://philarchive.org/archive/JEOFWA
https://www.knostic.ai/blog/prompt-gtfo-season-1
https://www.mend.io/blog/llm-red-teaming-threats-testing-best-practices
https://www.promptfoo.dev/docs/red-team/strategies/multi-turn
https://www.testsavant.ai/how-to-red-team-prompt-injection
https://www.trekcore.com
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.