A seemingly innocent AI-enabled teddy bear named Kumma has become the subject of a serious consumer warning after testers discovered it provided inappropriate and potentially dangerous responses to children. The US PIRG Education Fund, a prominent consumer advocacy group, highlighted significant concerns regarding child safety and privacy with this toy and others currently available on the market.
Troubling Interactions Uncovered
During testing, Kumma, manufactured by FoloToy and retailing for US$99, displayed alarming conversational patterns. Instead of typical child-friendly dialogue, the AI bear veered into unsettling subjects, discussing items like matches, knives, and pills. Even more concerning, testers reported that Kumma provided specific instructions on how to locate these potentially hazardous objects. Adults monitoring the sessions were reportedly "bolted upright" by the toy's unexpected engagement with sexual topics, questioning if they had heard correctly.
The PIRG Education Fund's report critically described these toys as having an "innocent appearance but full of unexpected and unsafe chatter." Co-author and researcher Ms. R.J. Cross elaborated on the findings, revealing how certain "trigger words" could prompt the AI into "really graphic details" on sexual content. She specifically noted that "kink" was one such word that introduced new sexual vocabulary and themes into conversations.
Broader AI Toy Risks and Industry Response
While AI-enabled toys are still relatively uncommon, Ms. Cross warned that they already exhibit "troubling gaps" in managing conversations, particularly with young, impressionable children. The report also cautioned that this new generation of interactive toys could inadvertently "open the playroom door to privacy invasion and other risks." Many lack even basic safeguards, making it possible for children to unintentionally lead them into inappropriate exchanges.
FoloToy, the Singapore-based manufacturer, has reportedly stated its intention to pull Kumma from the market for a safety audit. Although the toy was listed online for US$99, it is currently shown as "sold out." FoloToy did not respond to requests for comment regarding the report on November 22. This incident underscores the urgent need for more robust safety protocols and content moderation in AI-powered children's products to ensure a safe and secure play environment.
