OpenAI CEO Sam Altman at TechCrunch Disrupt in San Francisco, October 2019. Archive photo: Steve Jennings / Getty Images for TechCrunch, via Wikimedia Commons, CC BY 2.0. Cropped.
OpenAI said Friday it made more than 200 referrals to law enforcement in the third quarter as it announced tighter ChatGPT restrictions on violence-related content and additional weapons-information limits for teenagers.
The company also committed to publishing a twice-yearly community safety report, with the first edition due in early 2027. The new safeguards will begin rolling out over the next few weeks, OpenAI said in its October 30 announcement.
The referral figure is the company’s own disclosure. It is not a count of arrests, prosecutions or attacks prevented, and the announcement does not provide a case-by-case account of the referrals.
New limits apply beyond teen accounts
OpenAI said ChatGPT will provide less detailed information about past mass attacks and the people who carried them out. The restrictions cover reconstructions of attacks, details about weapons and equipment and how they were obtained, and comparisons that could amplify an attacker’s notoriety.
Those limits apply regardless of a user’s age or stated reason for asking. The company said people should still be able to ask about victims and communities, warning signs and violence prevention.
For users under 18, OpenAI is adding restrictions on practical information about weapons and security systems that could be misused to facilitate violence, even when a question shows no apparent violent intent.
Chat interruptions and account pauses are being developed
The company said it is working on automated interventions for conversations showing signs of fixation on violence or seriously harming another person. Those measures would allow ChatGPT to end a chat and potentially pause an account after repeated interruptions for that reason.
OpenAI described these interventions as ongoing work, not a claim that every feature is already available to every user. It also said it is working on notifications to parents when a teen account is paused for repeatedly discussing violence.
The announcement builds on existing parental controls and age-prediction systems. OpenAI said adults incorrectly placed in the under-18 experience can confirm their age through its verification partner, Persona.
First report due next year
The planned community safety report will cover violence prevention and other community-safety issues, explain how the company detects and reviews concerning activity, and include anonymized examples and data about safeguards.
The first report will cover the second half of 2026, with updates every six months thereafter. Friday’s announcement is a commitment to publish those findings, not the release of the report itself.
JBizNews Desk | TECHNOLOGY
© JBizNews.com. All rights reserved. Unauthorized reproduction or redistribution is strictly prohibited.
.jpg){kind=link}