Keeping It Human in AI Kids Content: Creator Responsibility

This is a written version of talk presented on Maven (available here along with other workshops) and another I gave at Prix Jeunesse International in June 2026 in Munich, Germany. After 25 years creating all sorts of entertainment and education content, I now explore the messy side (both good and bad) of creating with AI. If you’re curious about more of my work

• Workshop: Creating Kids Videos (Not Slop!) with Google Flow (Free, Recording)

• Workshop: Personalize Your AI Animation Tools with Google Flow (Free, Recording)

• 5-week AI Animation for Kids Videos course, starting July 6, 2026. ($299, code FRIENDS for 35% off)

Technology is neither good nor bad; nor is it neutral.

That’s Melvin Kranzberg’s first law of technology.

There are a lot of feelings around AI right now. Many people are very anti, for lots of different (and valid) reasons. A lot of people are very pro (also for lots of different and valid reasons). I’m somewhere in the middle: there are good things and bad things, and I try to stay aware of both.

Technology is a tool, and whatever we do with it, it will magnify our intent. So I come at these talks thinking about our responsibility and our intent is as creators, and how we navigate from there to create.

So many creators are using AI badly. (I’ve written lots about slop content.)

• Quite a few YouTube channels post really, really frequently, like every 30 minutes. JoJo Funland, which thankfully is no longer available, posted almost 8,000 videos in five months. You can imagine the quality was not good. And it’s not a one-off; there are many of these channels in the kids space and beyond. YouTube is starting to take steps, but it’s a pervasive problem.

• The content itself can be really challenging too. One of the ones that horrifies me most is a song called “ABCs at Breakfast.” For the most part it has nothing to do with the ABCs; even when an apple is on screen, the lyrics don’t match. Then the child takes a bite of the apple and a red, bloody ooze comes out of its mouth. And this is positioned as educational.

A lot of this is driven by popular videos telling you how to make faceless AI kids channels: make $900 a day, make $10,000 a month. If you’ve been around a bit, you know there’s no such thing as a free lunch. If these folks were actually making that kind of money, they sure wouldn’t be telling us how; they’d be quietly replicating those channels and making bank.

So this is why we need to talk about guidelines.

There’s this notion of “human in the loop,” which can feel inflammatory when talking about AI and human interaction, but it’s a long-standing computer science term and continues to be true today: you are controlling whatever the computer, the AI, is doing.

You’d think I wouldn’t need to make this statement. And yet we have the child eating an apple with blood oozing, and people creating channels that don’t teach correct information. We have to be willing to take responsibility for everything the AI outputs. “Claude made it this way” is not an excuse. It doesn’t work.

You must be willing to take responsibility for every single output of AI.

Just because you can make an image in a popular, famous style doesn’t mean you should copy it. That’s separate from learning the art form; imitation has long been how we learn. But there’s a line between learning and posting. If you’re posting just because you can piggyback on Cocomelon, Studio Ghibli, or Pixar styles, that doesn’t mean you should.

People sometimes pitch me their “great” ideas that aren’t meant to be funny, but make me laugh. Like: create an app like Wordle, but every word is “meow.”

(Sidenote: I mentioned this to my daughter and she said, “Well, you’ve heard of Horsle, right?” It an app where the answer is always “horse.” And JEFFGOLDBLUMLE where the answer is Jeff Goldblum 90% of the time, and you get three guesses.)

If you’re creating something to be funny or ironic, that’s fine. But when people pitch these ideas in all seriousness, the art of determining how and why something should exist has been lost. AI has made the barrier to entry is so low that it’s just “great, I’ve got an idea, let’s build it.”

Instead we need to remember we are designing for specific humans: little kids, their parents, teachers, and always the regulators. So go back to basics. What problem are you solving? What need are you meeting? What’s the educational goal? What is the job of your product?

You might have heard the milkshake story, from Clayton Christensen’s jobs-to-be-done framework. McDonald’s discovered they were selling a lot of milkshakes at breakfast, which seems like an unusual time for a milkshake. Research found that commuters wanted something filling (we won’t say nutritious) that they could hold in one hand while driving. Try eating an Egg McMuffin while driving.

The job of the morning milkshake was breakfast on a commute. You can ask the same thing: what is the job of your video, your music, your show?

Another one I love: is your product a vitamin or a pain pill? Exercise is a vitamin: you don’t necessarily want to do it, but it’s good for you. Maybe a documentary is a vitamin. Reality TV at its finest is a pain pill; pure entertainment. Neither is the right or wrong answer, but it helps you understand your purpose and who you’re designing for.

Fundamental tools help here too. Product development templates (Lenny’s Newsletter has all sorts of them) and storyboards help you evaluate whether something is worth pursuing before you go straight to prompting and prompting and prompting.

If you’re going to use AI responsibly, you have to know the limits of both yourself and the AI.

If you prompt AI to “create an image of a doctor,” you’ll probably get a white man treating a white male patient. AI models and the training data are getting better, but they’re not there yet.

As AI users, we have to navigate all sorts of issues: bias, prompt adherence, bad training data, context blindness, hallucinations, artifacts, sycophancy, overconfidence, continuity drift. If you’re looking at that list wondering what some of those are, that’s a signal there’s more for you to learn to work with AI thoughtfully.

For example, I asked Claude one day to write lyrics for an educational alphabet song, with no other context. It starts reasonably okay. But by verse 3 you get “Q is for a box of rocks.” Incorrect. By verse 5, “X is for fox, the X hides inside.” The song is a waste of cognitive processing.

This is why one-prompt approaches don’t work: the model is trained on the whole of the internet, and it will just give you slop. In work contexts this is now called “workslop“: AI output that causes teams to spend more time undoing and pulling apart the work than it saved.

Same with interfaces. I asked AI to design a children’s UI for an AI storyteller app with no context, versus with some basic constraints (”a modern, entertainment-facing storytelling app”). The no-context one is disastrous (my favorite is the tiny keyboard for typing your story idea). The one with constraints and guidance immediately looks much more like what we’d expect a good app to look like.

I’ve run into all of sorts of issues working with AI.

• Bugs in the code. So many bugs.

• Deleted databases. Even when I had back-ups and safeguards in place. (You may have heard about Claude-powered AI agent deleting an entire company.) If you’re developing production code and you’re not ready for that, you could destroy a business.

• Inefficient, monolithic code.

• $100 in unauthorized API charges when it “helpfully” pulled an API key from a completely unrelated project on my drive.

• Visual distractions, bad UX, UI that violates app store regulations

• Word salad, incorrect pedagogy, and dangerous behaviors in content. If I wasn’t paying attention, I’d be as bad as the slop creators.

Yes, there are some safety prompts in place. But they are easily sidestepped. One rejected “a child takes a sheet of cookies out of the oven without oven mitts,” then tweaked the prompt slightly and generated exactly that video. (We need lots more adversarial testing on these models!)

It’s not enough to know the limits of AI. You also must know your own skills and limits. If you’re creating content for kids: are you a good writer? A good UX designer? Do you understand the pedagogy of what’s being taught? Visual styling, animation, all of it?

When I made the Potty Dash video, I didn’t know the recent pedagogy around potty training, so I did research with the help of an LLM, knowing it might make up information or miss things. So I made it provide links and resources so I could verify myself.

When I built a tool to detect slop, I was working in codebases and tools I wasn’t familiar with, so I built it as a prototype but didn’t release it publicly, because that hit my limitations: I didn’t feel I could ship production-ready code for it.

Part of this is knowing how to prompt for information you might not know: asking for full references and links, because these tools will hallucinate links too. Information architecture is coming into a new level of popularity in the AI world: metadata, taxonomies, knowledge graphs, structured information in machine-readable formats that tells these systems what body of knowledge they’re working with.

A concrete example of this structured data in action: when I create lyrics for educational songs now, I have an agent with structure around it that goes from “Q is for a box of rocks” to content that works 80 to 90% of the time. The agent has an instruction file, references for the writing voice, what I don’t want it to do, an approved vocabulary (so I don’t get “K is for knife,” which is technically true but useless for teaching K because of the silent K, or “A is for arbitrage”), phonics rules (hard or soft G? blends? digraphs?), a master prompt with a structural template, and examples.

Knowing what you don’t know makes you a much better creator.

Whether or not you decide to be transparent, the tools and platforms are moving toward transparency. When I post AI-created content on YouTube, I share that I used AI and which tools. Photorealistic content is already moving toward mandatory disclosure; cartoons and clearly fictional content is still at-will, especially on YouTube. In the kids space, I suspect that will change in time.

Creators should think about trust with their audience. A lot of parents are actually okay with AI content, so long as it’s thoughtfully developed and labeled. They want the ability to decide whether to engage with it.

I personally believe it’s important to be transparent. Of all the guidelines, this is the most debatable and depends on personal preference, but what I care most about is that people think about it and make a determination that’s right for their brand and their audience.

This one also feels like it should be obvious. But I’ve been in workshops teaching teachers how to use AI, and they upload photos of kids or class rosters without thinking about it. That generally violates their school’s data privacy policies, but it’s also a new form of stranger danger: we don’t know where this information is going.

It’s true of your own information, which I think most folks have a better handle on (I hope?). I’m observing a real lack of awareness around the data of others.

1. Responsibility. Be the responsible human in the loop.

2. Restraint. Just because you can doesn’t mean you should.

3. Intent. Know the limits of yourself and of the AI, and work within them.

4. Transparency. Disclose when you’re using AI.

5. Privacy. Don’t share personal information.

These five guidelines are where I’ve landed right now, but this space is moving fast and I know the list isn’t finished. What would you add? Where do you disagree? It’s certainly going to evolve as AI continue to change!

Kids deserve better than lazy, inappropriate content generated every 30 minutes by nobody in particular. The tools aren’t going away, so the question isn’t whether AI shows up in kids media. It’s whether the humans using it are willing to stay responsible for what comes out. Be the human in the loop.

Images in this post are created with Nano Banana Pro.