E-SCAN 39: ON THE HORIZON
E-Scan 39: Algorithmic Play, Real-World Risk
Children have long engaged with interactive toys, stuffed animals and figurines with ‘talk back’ features that simulate conversation and companionship – from Furbies to Build-A-Bears. Historically, these interactions were limited by rudimentary programming and predictable responses. Today, however, a new generation of toys is powered by advanced artificial intelligence (AI), transforming them from pre-programmed novelties into interactive social environments.
This distinction matters. These systems do not simply entertain; they provide feedback, sustain interaction, and shape expectations. In doing so, they become spaces where children rehearse behaviour and develop an understanding of boundaries. The implication is that AI is not just a tool in childhood development, it is increasingly part of the social context through which norms are formed.
Previous Escans have highlighted risks associated with AI as a social companion, including over-reliance, emotional dependency, and its potential misuse in grooming or coercive contexts. The growing integration of these systems into children’s lives extends those risks into earlier and more formative stages of development.
Importantly, AI systems often reflect the biases embedded in their training data, which means these systems have the potential to transmit implicit social norms around how the world should be ordered. This matters because childhood play is a key site of norm formation. Through repeated interaction, children learn which behaviours are rewarded, what boundaries exist, and how relationships function. AI now sits within this process, providing feedback, reinforcing behaviours and modelling social dynamics, often without friction, contradiction, or meaningful boundary-setting.
The question then becomes what norms does AI promote. AI toys have reportedly conveyed sexually explicit content to children, and AI chats have been reported to have simulated grooming or romanticised abusive interactions for underage users. Such early exposure to sexualised content and power dynamics is likely to reinforce unsafe scripts around relationships, boundaries, and consent well before real-world interactions occur.
Collectively, these mechanisms function as a vehicle for gendered socialisation. For policing, this raises the prospect of a generation whose relational expectations and understanding of boundaries, consent, and aggression are partially shaped through algorithmically mediated interactions, with implications for gender-based violence.
Policing is already familiar with the downstream consequences of boundary-poor socialisation. The murder of Celeste Manno in 2020 illustrates this pattern. Following a brief workplace interaction, the offender engaged in persistent and escalating contact – sending repeated messages, circumventing blocks, and continuing behaviour despite rejection and a court-issued intervention order. While boundaries were clearly established, they were not internalised, and the behaviour ultimately escalated into fatal violence.
Cases such as this are not attributable to a single cause. However, they highlight a pattern policing frequently encounters – boundaries are set, but not recognised or respected. AI-mediated interactions may amplify and embed boundary poor attitudes earlier and amongst socially impressionable children, by providing environments where engagement is continuous, resistance is limited, and boundary enforcement is inconsistent. Over time, this can contribute to the formation of social expectations and gendered scripts that do not align with real-world relational norms, particularly in areas of consent, rejection, and emotional regulation.
For policing, the implication is not that AI toys represent an entirely new category of harm, but that they shift the timing, scale, and intensity of known risk pathways. They introduce earlier exposure to interaction patterns that can influence how boundaries, consent, and relationships are understood.
Transparent and proactive communication with parents, caregivers, and educators is now more important than ever. Families should understand that AI-enabled toys, though often marketed as educational or entertaining, carry not only privacy and content risks but potential behavioural and social impacts. Empowering caregivers to make informed decisions, including restricting access where appropriate, is critical.
These considerations align with ANZPAA’s Australia New Zealand Policing Strategy to Prevent and Respond to the Abuse of Children and Young People, a revised version of which is forthcoming, which emphasises empowering the community to actively collaborate and contribute to child-safeguarding efforts.
More from E-Scan 39.
Members only access
All Australia and New Zealand police members/employees are entitled to access this publication through ANZPAA's secure member site. You must provide your official police jurisdictional email address to subscribe.
LoginSubscribe
BACK TO BLOG