Decisions every tech companies should wrestle with...but few do

Artificial intelligence is being integrated into products at extraordinary speed. For adults, it shows up as AI-powered search, automated writing and smarter tools. For children, it is appearing in homework platforms, creative apps and even teddy bears. The industry often frames this as inevitable progress. But, many parents are not so sure.

We recently surveyed our own community of Kinzoo users and asked how comfortable they were with their children using AI products. These are digitally literate Lifeguard Parents who embrace technology in their own lives and want their children to benefit from it. They are not trying to shield their families from innovation. They believe in staying involved rather than pushing kids into complex systems alone. And, fewer than 20 percent said they were comfortable with AI.

That gap between industry enthusiasm and parental caution is instructive. Companies building for children face a structural tension that extends beyond individual features. Innovation cycles are accelerating. Investors expect momentum. Engagement metrics drive valuation. Distribution partnerships and monetization models are often designed to maximize scale. At the same time, children are uniquely impressionable users. Decisions that increase reach, retention or revenue can also introduce developmental, privacy or safety risks.

At Kinzoo, those tensions are not theoretical. They surface not only in product design discussions, but in broader business decisions. When we evaluate a new feature, consider a distribution strategy or assess a potential growth lever, we start with a practical question: if this goes wrong, who bears the cost? Adults can contextualize confusing or manipulative outputs. Children may not. Adults can step away from persuasive systems. Children may internalize them. That lens changes not just what we build, but how we grow.

There are well-established mechanics that reliably accelerate growth in consumer technology: open networks, viral loops, algorithmic recommendations optimized for retention and targeted advertising. They are powerful because they work. They are also the same mechanisms that have required years of retrofitted safeguards once harm became visible. We have chosen to forgo those mechanics, even when they could produce faster growth or stronger short-term metrics. That choice may limit network effects and slow expansion. It also reduces exposure.

When we’re not sure what to do, we slow down. We revisit our mission and ask whether a proposed decision advances it. We seek additional input from parents, look for emerging research and stress-test worst-case scenarios. Sometimes we shelve an idea entirely. Sometimes we rethink it. But we don’t rush forward just because the market is moving.

This is not a claim to moral superiority. It is an acknowledgment of trade-offs. Building for children means accepting that some opportunities aren’t right. It means deciding in advance what risks are unacceptable rather than recalibrating only after public backlash.

As AI becomes embedded in children’s products, the stakes increase. Generative systems can simulate companionship, respond dynamically to emotional cues and operate in ways that are difficult even for adults to fully interpret. The developmental implications remain uncertain. That uncertainty alone warrants caution.

Transparency does not require publishing every internal debate. It means being clear about the principles that guide our decisions. It means publicly sharing research on parent sentiment, even when it reveals skepticism. It means acknowledging that growth targets and families’ needs can pull in opposite directions, and stating openly which one has priority.

The industry’s history with child safety has not earned unconditional trust. Guardrails have too often followed harm rather than preceded it. If companies want a different outcome with AI, the ordering must change. Parents are not rejecting technology. They are asking for evidence that restraint and foresight are being applied before new technologies become normalized in children’s lives. Building in that way may be slower. It may produce less dramatic growth curves. But when the users are children, speed can’t be the only measure of success.

Here are a few helpful resources in case you want to really dig into today’s topic:

• New Mexico is suing Meta, claiming that the company lied about efforts to improve child safety on its platforms. State attorney Donald Migliori told the courtroom, “The theme throughout this trial is going to be that Meta puts profits over safety.” He argued that the company knowingly overestimated the impact of its teen safety features and that safety was underfunded and ignored internally. The trial is ongoing, but the accusations are old news. Meta has broken trust with parents too many times to ever be taken at their word.

• In an effort to protect children and families from the negative aspects of technology, Australia enacted a sweeping social media ban for users under 16. But Roblox, one of the most popular platforms for children, was not included in the ban. The embattled company is still drawing lots of scrutiny, as reports surface of inappropriate content, harassment and grooming. The Australian government has now requested an urgent meeting with the company to discuss these issues.

Too long; didn’t read. It shouldn’t be a full-time job to keep up on industry news, so here is a mercifully quick summary of some other notable developments:

• Meta, TikTok and Snap have agreed to be rated on teen safety by a coalition of advocacy organizations. The platforms will be graded on things like whether they mandate breaks and provide options to opt out of infinite scroll. Dan Reidenberg, managing director of the National Council for Suicide Prevention, oversaw the development of the standards, and said, “I hope that by having this new set of standards and ratings it does improve teens’ mental health. At the same time, I also really hope that it changes the technology companies: that it really helps shape how they design and they build and they implement their tools.” It will be interesting to see if the new rating system changes teen behavior or decision-making at the companies themselves.

• You might know MrBeast as a stunt-pulling YouTube personality, but now he’s pivoting into finance. The star has just acquired banking app Step, which helps teens open their first bank account and establish credit. And apparently, MrBeast plans to launch another YouTube channel focused entirely on youth financial literacy.

Here are a few more pieces of original writing from me and my team—just in case you’re keen for more:

• When we asked parents how they felt about AI for kids, we heard a lot of interesting takes and passionate opinions. We’re working on the full report, but you can check our blog post here for initial information.

• And if you’re wondering who MrBeast is, have a look at our parents’ guide here.