AI firms such as Anthropic and OpenAI regularly share usage statistics for their chatbots, but they only disclose the portions that align with their preferred narratives, leaving no external way to verify the full picture, according to researchers.
To address this gap, a group led by Stanford PhD candidate Anka Reuel launched the AI Observatory, a public platform that pulls together consent‑based conversations from seven existing datasets covering models like Claude and Gemini. By applying the same filtering rules used in Anthropic’s Economic Index, the team found that roughly 48 % of the collected exchanges would be excluded as non‑work‑related. Those omitted chats contained markedly higher shares of health‑relationship content (44 % vs 31 %), adult or illicit material (8 % vs 2 %), harassment or hate speech (28 % vs 6 %), and sexual content (17 % vs 2 %).
Experts note that having a unified, cross‑model view—rather than isolated company reports—helps researchers see the full spectrum of how generative AI is actually employed, from professional assistance to personal companionship and, in some cases, harmful uses.
Why this matters
Source fact: The AI Observatory shows that nearly half of AI interactions are filtered out by company‑centric metrics, revealing substantial non‑work usage. Inference: This suggests that policy decisions based solely on corporate‑released usage data may underestimate risks related to mental health, harassment, and illicit content, underscoring the need for independent, transparent monitoring of AI use.