Raising AI Natives: The urgent case for child-centred AI design

Meet the Newcastle graduate raising the bar for AI in children’s lives

Exploitation, emotional dependency and the loss of critical thinking: These are the risks facing children today

We all know the risks for young people being online. But there are risks that are particular to AI that are worth digging deeper into.

At the foundational level, the safety risks are non-negotiable: child sexual exploitation and abuse material generated or facilitated by AI, content that promotes or enables self-harm, and AI companions or chatbots that manipulate or exploit children, creating psychological harm. Unfortunately, these aren’t hypothetical risks - they're already happening. Protecting from these harms should be a baseline requirement for any AI that kids can use.

But there are other underappreciated risks that are getting less media attention. The first is emotional attachment and dependency. We're already seeing children and teenagers forming emotional bonds with AI systems. That's not surprising - these systems are often warm, patient, available at 2am, and never have a bad day. The question is what that does to the development of healthy human relationships, which are by definition imperfect, reciprocal and sometimes frustrating. Those frustrations are part of how we learn to navigate the world with other people. They're not problems to be designed away.

Related to that is something I think gets consistently overlooked in the debate about AI and learning: the importance of critical thinking and the value of positive struggle. Productive difficulty, the feeling of wrestling with something and eventually understanding it, is a core mechanism of cognitive development. If AI becomes a shortcut or default rather than a scaffold, we risk children never developing the frustration tolerance or metacognitive skills that come from working through hard things.

When a child asks a question and gets an instant, confident answer, they may never develop the tolerance for uncertainty that drives curiosity. When an algorithm always serves them content calibrated to their existing preferences, they lose the serendipitous exposure that broadens their worldview. When an AI companion validates their emotions without ever misunderstanding them, they may not build the resilience needed for the inevitable imperfections of human relationships.

And then there's inequality. Not all children will access AI's benefits equally, but many will bear its risks regardless of their circumstances. That asymmetry needs to be central to any serious conversation about policy.

The opportunities, though, are substantial

Personalised learning that adapts to a child's pace and style, without the stigma that can sometimes accompany needing extra support. Tools that give children with ideas but limited technical skills the ability to make and build things they couldn't before. Accessibility applications that could be transformative for children with learning differences or communication difficulties. These are just some of the opportunities afforded by AI in education.

AI as a creative collaborator: a sounding board for ideas, a patient tutor for a subject a child finds embarrassing to ask about in class. Language learning at a pace and depth that was never previously available outside expensive immersive environments. Over the coming years, we’ll see many wild and wonderful use cases of AI that will totally change how we see and view the world.

However, whether we land closer to those opportunities or those risks isn't necessarily determined by the technology itself. It's shaped by the choices made in how it's deployed, by the environment and support systems around the child, and by how well we educate children about AI in a way that's substantive and age appropriate. A child using AI in a household where there are open conversations about what it is and isn't, where critical thinking is modelled, where there are trusted adults to talk things through with, is in a very different situation to a child navigating it alone. That context is everything.

Lessons from the social media generation

We’re now seeing countries begin to adopt age restrictions on social media platforms, and I think this will definitely happen with AI too – but even faster. We’ve watched a generation grow up inside social media platforms before the serious regulatory conversation began, and many policymakers have said publicly they don't want to repeat that mistake with AI.

I predict that AI companions specifically will be where the sharpest age-restriction debate plays out first, and in some respects it already is. Many countries are pressing for a legal age of 18+ to use AI companions (rather than the current default age of digital consent, which in the UK is 13). That reflects a growing recognition that the emotional dynamics of AI companion interaction require a more cautious approach. These aren't just information tools; they're relational ones, and the developmental implications of that are significant.

But I'd make a distinction between age restrictions and safeguarding. Age gates are a blunt instrument. What we actually need is age-appropriateness by design: systems that behave differently and more safely when they're interacting with a child at different ages, not just systems that ask for a date of birth (or minimal age verification) that anyone can misrepresent.

There’s lots of work to be done but I think there is will and impetus to raise the bar for AI in children’s lives and protect their future.

It is everyone’s responsibility to teach children AI literacy

Everyone is unsure who’s responsible and how best to act. Parents wait for schools, schools wait for government, government waits for industry consensus, and industry points to parental choice. It’s a vicious circle! The problem is that in waiting, children fall through the gap.

I do think the burden has been disproportionately placed on parents, often by design. When a product's default settings aren't child-safe, when there's no transparency about how a child's data is being used, or when the entire experience has been engineered for maximum engagement rather than wellbeing, expecting parents to review the privacy policy, monitor screen time, and set up appropriate controls abdicates responsibility. 

Tech companies have to be accountable first. Child safety needs to be designed in from the start, not bolted on as a PR exercise when the backlash arrives. That means diverse teams, developmental expertise in the room during product design, age-appropriate defaults, and transparency about data. As we’ve already seen with social media, voluntary commitments aren't sufficient. 

Governments need to set binding standards with meaningful enforcement. There are positive steps on this across the UK, US and Europe but the issue is speed and enforcement. It’s almost impossible for the pace of regulatory development to keep up with the pace of AI deployment. However, we’re also seeing new organisations pop up like Common Sense Media’s Youth AI Safety Institute, which will have a role to play in doing research, setting standards and benchmarks and evaluating AI.

Schools have a critical role and AI education shouldn’t just be about how to use the tools but about how to think about them and the role they play in our lives and society. That means curriculum, teacher training, and giving educators the confidence and the tools to have these conversations without feeling like they're always one step behind.

And parents need access to honest, accessible, non-alarmist information to make informed choices, which is currently hard to find. Much of what exists is either too technical, too sensationalist or produced by parties with a commercial interest in the answer.

This often goes unsaid, but children themselves need a voice in this. The generation most affected is the one least consulted - that has to change.

Subscribe to the Kids First AI Substack or follow them on Instagram to find out more.