Roughly a year into the COVID-19 pandemic, Carolyn, a mother of six, finally capitulated to her kids’ begging. Her children wanted to join Roblox, a platform that allows users to create and host online multiplayer games. Roblox became a vital way for kids to connect virtually during the pandemic, and despite her concerns about her kids, ages 12 and under, interacting with others online, Carolyn and her husband decided to let them play. Two years later, a slew of content moderation changes at Roblox has made it harder for her to control what her young children see. She made the hard decision to stop allowing her kids to play.
“Alright, I’m done. DONE. I deleted my kids’ Roblox accounts, and recommend you guys do too. I just spent six hours playing the games meant for 5-year-olds and it was freaking awful. Something is very wrong with Roblox Corp,” she said in a tweet thread.
In the following tweets, Carolyn described additional troubling content she encountered while playing on the platform, including a game called Public Bathroom Simulator that mimicked a public restroom environment. Carolyn told Polygon she expected some potty humor, but what she found was much worse. In the game, her character slipped on a puddle and got stuck on the ground. Afterward, other players in the game appeared to take turns doing role-play that simulated an assault of her character, an incident that has happened to others on Roblox.
Anyone can make a game on Roblox. The platform almost works like YouTube — except instead of uploading videos, any user can design and publish a game, or as Roblox calls it, an “experience” for others to play. This gives young users vast opportunities to develop and play in any virtual worlds and games they imagine. According to a Roblox spokesperson, the platform had 58.8 million daily active users in the third quarter of 2022, 46% of whom identify as being younger than 13 years old.
This presents a content moderation problem. Users can choose between roughly 9 million experiences, according to Roblox’s website. This immense volume of experiences makes it more challenging to moderate, despite the company’s active role in monitoring the platform.
Roblox filters potentially inappropriate user-made skins and chat, as well as personal information. The company says that it employs a team of “thousands” to monitor the platform for inappropriate content and that its trust and safety team typically takes “minutes” to address any behavior that violates their terms of use and community standards. Still, games simulating slavery and fascism or ones that simulate mass shootings have cropped up on the platform.
For most of 2022, individual Roblox games did not have an “age appropriateness” designation or rating, as the platform’s intended audience was simply younger children. Parents could restrict their kids’ account so that kids could only play on a set list of games curated by Roblox, along with other safety features, like disabling chat.
However, in September, Roblox launched “Experience Guidelines,” which include age ratings for games, seemingly in order to cater to its growing number of users over the age of 13 on the platform. Games now have three possible age group labels — all ages, 9 and over, and 13 and over — and age recommendations are “grounded in child development research” and “industry standards,” according to a Roblox Developers Conference blog. Parents now have the option to limit access to certain games based on age recommendations.
It’s up to the everyday people making games on Roblox to make sure their content has an age-appropriateness rating. To do so, developers fill out an Experience Questionnaire — specifying whether violence is depicted in-game and its frequency, level of intensity, and presence of blood — that will generate the recommended age group for their game. As of January, games whose developers have not filled out an Experience Questionnaire won’t be recommended to young users. (The dev forum doesn’t specify whether young users can still find these games independently.)
Polygon spoke to a Roblox developer who specializes in creating family-friendly games and who wished to remain anonymous due to their relationship with Roblox. They believe that Roblox is taking strong steps to inform parents, but that it’s ultimately on parents to monitor their kids’ activity.
“Roblox is making great changes and efforts to inform parents about the content found in particular games with age guidelines, but it’s still up to parents to monitor their younger children and teach them online safety guidelines and rules to follow,” they said.
Carolyn and her husband took an active role in their children’s digital lives and regularly played Roblox games with their kids. She and her husband grew up playing games like RuneScape and Ragnarok Online, so the two were aware of the potential dangers of playing with others online.
“We had a pretty good idea of the things that can go on, and the things that can be published with just text and voice chat. And then, more recently, the user-manipulative games — like games where you can place things and change things about your character — I think that kind of opened up a new thing for parents to be worried about.”
These Experience Guidelines only add to existing moderation challenges on the platform. Though the content of certain games may be deemed age-appropriate by developers, kids can still interact with others on the platform through role-playing. Role-play games with imaginary elements and character interactions are among the most popular on the platform. This opens up opportunities for inappropriate interactions, either via chat or with role-play elements. So while a game might not contain blood or any of the age-restricted content marked as appropriate by the survey, a user could still go in the game and simulate an assault.
Roblox terms of use and community standards are relatively strict content rules when compared to other platforms and social media sites, like YouTube. For example, Roblox prohibits depictions of romance and depictions of political parties, in addition to explicitly banning content about extremist groups. To enforce these rules, the company also moderates through a reporting system, which allows users or a parent to report a specific user or a game that violates Roblox’s terms.
Image: Roblox
Polygon reached out to Roblox and received the following comment:
“To determine the suitability of content for each age category, we examined global industry standards and consulted child development experts to guide our policies. The experiences highlighted in [Carolyn’s] Twitter thread are marked as ‘All Ages’ because the content is generally suitable for all ages and may contain infrequent mild violence and/or light unrealistic blood,” a Roblox spokesperson told Polygon via email.
The spokesperson also said that while the new system has expanded the pool of experiences, the company has upgraded parental controls so parents can customize access to experiences based on age recommendations.
While the portion of Roblox’s older players is growing, a large number of users below age 13 remain. It’s true that Roblox takes a stronger stance in content moderation compared to other content platforms, but even then, changes like these may expose its younger users to more age-inappropriate content on the platform. After tweeting, other parents and family members said they deleted the game, and others shared their own horror stories.
So for now, caught between massive changes at a company and the whims of her own kids, Carolyn has stuck to the decision to delete their accounts. Luckily for her, it seemed like it went all right.
“I was expecting a huge, like, meltdown. But they were actually really cool with it. They’re like, All right, you know, if it becomes safe, we’ll play again. And I was like, Yeah. Wow. All right. Cool.”