Business Roblox Introduces AI System That Rewrites Users’ Chat Messages in Real Time - The message that leaves your keyboard is not the message that arrives.

  • ⚙️ Performance issue identified and being addressed.
  • Want to keep track of this thread?
    Accounts can bookmark posts, watch threads for updates, and jump back to where you stopped reading.
    Create account
1773012621720.png
Roblox has started rewriting its users’ chat messages in real time using AI, altering what people actually typed into something the platform considers more appropriate.

The feature, rolling out now, goes further than the existing filter that replaces flagged words with “#” symbols. Under the new system, banned language gets silently reworded into what Roblox calls “more respectful language that remains closer to the user’s original intent.”

The platform’s example: type “Hurry TF up!” and the message your recipient sees reads “Hurry up!” Roblox says everyone in the chat is notified when this happens, though the person who typed the original message has no way to stop the substitution before it goes out.

The definition of “banned language” extends beyond profanity. It covers “misspellings, special characters, or other methods to evade detection of profanity,” meaning the AI is also tasked with catching deliberate workarounds and rewriting those too.

Roblox is simultaneously expanding its text filtering system to “detect more variations of language that break its Community Standards,” so the net is getting wider at the same time, and the consequences of being caught in it are changing.

What Roblox has built is a system that goes beyond blocking speech. It replaces it. The message that leaves your keyboard is not the message that arrives. The recipient reads words you didn’t choose, attributed to you, with a notification that your original phrasing was deemed unacceptable. The platform decides what you said.

For now, the feature applies to “in-experience” chats between age-verified users in similar age brackets, and to conversations with “Trusted Connections,” a feature for users 13 and older who’ve completed an age check and connected with people they know. Roblox started requiring age verification for chat features last month. Once verified, users can talk with players in adjacent age groups: the 9–12 bracket can chat with the 13–15 bracket, and so on.

Minecraft filters profanity too, but its approach is more honest about what it’s doing. Flagged words get replaced with symbols, or the message gets blocked. The words you typed don’t reappear as different words under your name. Roblox’s system does something categorically different: it puts words in your mouth.

A system that rewrites what users say in real time, without consent, trained on definitions of “acceptable language” written by a private company, and expanding to catch more variations of speech the platform dislikes, is a significant piece of infrastructure. It starts with profanity. The architecture works the same way for anything else the platform decides to flag.
-----
Article
Archive
 
"yes officer that's why the message I typed to the 11 year old saying 'how's it going kid' became 'send me pics of ur butt'. Really I'm the victim here."
 
I no longer even have hope a single person will stop using it. The conditioning is complete. I'm just waiting for discord to do the same, messenger and snapchat to follow.
 
Why is this written like limiting communication between people in a literal baby game is some dark, dystopian outcome? Fucking of course there should be strict-ass limits to how kids can communicate with strangers online. Whoever wrote this needs to have their hard drive checked.
 
Why is this written like limiting communication between people in a literal baby game is some dark, dystopian outcome? Fucking of course there should be strict-ass limits to how kids can communicate with strangers online. Whoever wrote this needs to have their hard drive checked.
They are rewriting what people are saying, and this is only the starting point. Slowly more platforms will adopt this, will not notify the recipients the content was changed, and it will show the original version to the sender -- creating the illusion of no manipulation.
 
Why is this written like limiting communication between people in a literal baby game is some dark, dystopian outcome? Fucking of course there should be strict-ass limits to how kids can communicate with strangers online. Whoever wrote this needs to have their hard drive checked.

You have to be a complete retard to think Roblox gives the slightest shit about child safety when they have spent years ignoring issues and putting effort into shutting down people pointing it out. You also have to be naive to the extreme to think this won't be abused.
 
They are rewriting what people are saying, and this is only the starting point. Slowly more platforms will adopt this, will not notify the recipients the content was changed, and it will show the original version to the sender -- creating the illusion of no manipulation.
You have to be a complete retard to think Roblox gives the slightest shit about child safety when they have spent years ignoring issues and putting effort into shutting down people pointing it out. You also have to be naive to the extreme to think this won't be abused.
Ok. I don’t really care about the hypotheticals. Anyone upset that it’s harder to talk to kids in the baby game deserves the rope.
 
It would probably better if multiplayer games that are available to children and early teenagers had no chat function at all.
Emotes are enough. Pokemon does it like this.
 
It would probably better if multiplayer games that are available to children and early teenagers had no chat function at all.
Emotes are enough. Pokemon does it like this.
ToonTown Online did really well for its time before it caved to popular demand. It used to be way back when that you could only communicate using a system of drop-down menus, which were robust and useful for interacting with other players about things relating to the actual game, but if you knew somebody in real life, you could exchange lengthy "true friend codes" which let you chat with them specifically. Eventually they made a public chat available for all players with a toggle so the effort was quashed, but it proved communication could be done if developers gave a shit about safety. I think Roblox had something akin to this too for a while before they just gave up.
1773019148440.jpeg
 
Why is this written like limiting communication between people in a literal baby game is some dark, dystopian outcome? Fucking of course there should be strict-ass limits to how kids can communicate with strangers online. Whoever wrote this needs to have their hard drive checked.
Roblox, the company famous for caring a lot about child safety and being 100% honest about their intentions all the time.
 
I'm sure this won't stop people and people will find a way to fly under the radar to insult each other, as we've always done on the internet.

It would probably be better if multiplayer games with chat functions weren't available to children*
It'd be even better if the entirety of the internet wasn't available for children.
 
ToonTown Online did really well for its time before it caved to popular demand. It used to be way back when that you could only communicate using a system of drop-down menus, which were robust and useful for interacting with other players about things relating to the actual game, but if you knew somebody in real life, you could exchange lengthy "true friend codes" which let you chat with them specifically. Eventually they made a public chat available for all players with a toggle so the effort was quashed, but it proved communication could be done if developers gave a shit about safety. I think Roblox had something akin to this too for a while before they just gave up.
I don't remember Roblox having that "true friend" system, but it absolutely did have a safe chat option for users 12 and under that was virtually identical to the ToonTown approach save for it being more general and a few of the drop-down category options being different (most notably categories for text emoticons and work-safe leet/meme phrases). They removed the latter years ago and have poorly tried to establish a "trusted connections" system very recently. They could go back to what worked at any time and elect to do anything but.
1773022448715.png
 
Maybe it'll legalize/normalize everyone's ages absolving the company of legal and moral quagmires "Officer our logs don't show your purported 41/m and 14/m chatting, they both show 18 to us!"
 
Back
Top Bottom