Teen Mode

OpenAI announced its launch of ChatGPT for Teens this past week (for shorthand, I’ll refer to it as “Teen Mode”). Teen Mode is intended to strengthen existing safety defaults and guardrails as well as reduce the ability of teens to use Chat for cognitive shortcuts or cheating.

As a wellbeing advisor for OpenAI, and as we wait for organizations like Commonsense Media and Everyone.ai to conduct safety and risk assessments, I won’t opine on the quality of Teen Mode or whether it is succeeding in its goals. We don’t have enough information to assess that yet.

Instead, I will outline the stated aims of Teen Mode, highlight key questions about it that remain to be answered, and share my own brief wish list for what LLMs should and shouldn’t do in order to support human well-being. I’ll lean in more on social-emotional issues here, as that is my area of special expertise.

What is Teen Mode?

Teen mode improves on already-existing constraints and guardrails integrated into ChatGPT when a user has been identified as under 18 (U18). Users are identified as U18 in a couple ways: they voluntarily identify as U18; their account is linked to parental controls; or (soon) an age prediction model determines their age as U18. Teen Mode is not a separate product but rather launches automatically when a teen has been identified. Teen Mode includes both cognitive/learning and social-emotional guardrails.

Cognitive/learning protections. How does Teen Mode respond when a student asks for the solution to a math problem or for help writing an essay?

There are now new “responsible homework reminders” that offer Study Mode (guiding questions, step-by-step “collaborative problem solving”) when it detects teens are trying to shortcut to getting the answer. To be clear, teens do not have to use Study Mode, but detection of shortcuts has improved, and it defaults to offering the Study Mode option. OpenAI is in the early stages of evaluating the impact of Study Mode on learning.

There are also additional quizzes and learning visualizations, along with Study Hours, where either teens or parents make Study Mode the default. Kids can cancel Study Hours and Study Mode without parents being notified.

Social-emotional protections. Prior to Teen Mode, ChatGPT was already instructed not to engage in romantic exchanges with teens. You can read the Under-18 Principles Model Spec for those details. OpenAI states that these guardrails have been strengthened further, along with some new additions.

For example, it builds on pre-existing parental controls and notifications. Parents with linked teen accounts can manage some of their teen’s settings and receive safety notifications in “limited high-risk situations” like self-harm, other-harm, and suicide. Teen Mode now includes additional notifications related to disordered body image and eating. Examples of content included in this category have not yet been released. Both parent and teen have to opt into these controls and either can unlink accounts at any time.

[*Note that there are no publicly available data on the percentage or number of teens who use a version of ChatGPT with parental controls - nor on how many teens use any version of ChatGPT in the US or worldwide, although conservative estimates put the numbers at 15M in the U.S. and 150M worldwide.]

In terms of graphic material, teens are given “sensitive-image upload reminders” to caution them from sharing private or sensitive images. They are not prevented from sharing the images, however.

Teen Mode also includes “break reminders that encourage teens to step away, and product cues that consistently identify ChatGPT as AI.” In other words, there appear to be additional efforts to reduce the perception that Chat is conscious or has feelings or personhood. This includes explicit reminders to teens that they are engaging with AI, and reduced language that implies feelings or consciousness. It continues to block romantic or sexualized roleplay, as before, but now also blocks romantic language and “encouraging emotional dependence.” In the publicly released documentation, I did not come across examples of what encouraging emotional dependence might look like.

Teen Mode continues to allow customization options, like voice variations (this includes allowing spoken voice, and styles that are “warm” or “enthusiastic”). The stated intention is to “help make the experience feel personal without blurring the line between a useful tool and a human relationship.”

What We Don’t Know

OpenAI is conducting ongoing research, some of which is publicly available, on the impacts of ChatGPT and ChatGPT for Teens on things like self-harm, eating disorders, and violence, as well as positive well-being benefits.

The most pressing unanswered question about Teen Mode is, of course, whether these protections work in any domain, cognitive or emotional. I look forward to more publicly available results.

Even as research progress is made, however, much remains unclear. To highlight just a few issues:

It remains unclear how many teens could benefit from any effective protection or positive impact. That’s because there are no publicly available data on how many teens are using ChatGPT (with and without the protections of linked accounts with their parents), nor on the quality of the new U18 age verification algorithm (i.e., the rate of false positives and negatives). Moreover, many of these protections are optional, hampering the ability of researchers to study their broad impact (for reasons like self-selection biases, inconsistent use, etc,…).

Adding to the list of unknowns, we don’t know much about the range of human-mimicking behaviors that are restricted, and why. For example, if first-person pronoun use by the model are allowed in Teen Mode, and teens are able to select human characteristics in settings (e.g., warm, enthusiastic) how does this dovetail with the goal of reducing human-mimicking and the appearance of sentience, along with blocking emotional dependence? This is of particular concern to me as the Attachment Economy takes hold, and there are ample ethical and psychological reasons to aggressively design against emotional dependency.

Pulling back from questions about teens, why not apply these or similar protections to all users? U18 protections are predicated on the notion that teens are uniquely vulnerable. Should ChatGPT be developed to detect adult vulnerabilities and provide a risk-sensitive default model?

Wishlist

My wish list for LLMs is long, especially as it applies to mental health. But I’ll keep it brief and try to hit the high notes.

Above all, we need rigorous research before models are released. A/B testing and releasing into the market to see how a model performs are not safety. I also believe that, with manipulative design in digital tech being the rule rather than the exception, there is no doubt that we should learn from the past and demand that AI be actively designed against fostering dependency and in support of human autonomy.

My wish list also has to do with setting the stage for good science, and therefore good prevention and intervention. For example, the nature of risk, vulnerability, and benefit should be better operationalized. Meaning, instead of “risk” or “harm,” we should define and measure how AI might enable/scaffold certain abilities or experiences versus enfeeble/replace them. This allows more direct translations into AI tools and characteristics that can promote human flourishing. For example, we might ask: which distinct human capacities are most amendable to AI scaffolding? How does AI impact distinct aspects of thinking, and why? Are all kids vulnerable to AI in similar ways, or are there individual risk factors? Which protections afforded to kids might reasonably benefit all people? Without a human-centered, individual differences approach, risk and safety evaluations will not lead to actionable steps.

I also hope that we start to clearly integrate research with a respect for personal preference. For example, many of us have opinions about AI companions, but those who have companions are often left out of the public conversation. Decisions about where we go in this domain have to include consideration of human agency and dignity and be based on empirical evidence around potential harm or benefits. As we wait for the data, however, there are good arguments to regulate against attachment hacking, or manipulative design that fosters emotional dependency and engagement, especially for youth and education. Wherever we fall on the spectrum of opinions, technology that deliberatively engineers dependency and exclusivity via AI companions should be constrained until we know how to protect everyone.

As we think about the risk for dependency, it’s worthwhile considering protective factors. Perhaps, as in the case for cognitive de-skilling, the most expert among us - in the case of social AI, the most socially and emotionally expert - are at less risk for social de-skilling. If this were true, how do we measure social expertise? Would the presence of high-quality social support systems count? Could emotional intelligence be a protective factor?

While ensuring safety, we can also use social AI deliberatively and constructively to benefit specific groups of people: AI companions may reduce cognitive decline in dementia patients, mitigate stress and anxiety during high-risk pregnancies, and provide temporary relief for loneliness.

Summary

With the release of ChatGPT for Teens, we have the opportunity to learn more about how distinct features of an LLM shape cognitive and emotional well-being in youth. I am staying tuned for rigorous, transparent, and public risk and benefit assessments with bated breath.