We talk a lot about how AI can be used to defend against cyberattacks — but attackers are just as interested in using AI platforms against you. A new threat campaign discovered this month proves the point in a clever and unsettling way, highlighting the growing risks of fake ChatGPT organizations.
What’s Happening
Cybersecurity researchers at Push Security recently discovered that threat actors are creating fake ChatGPT organizations — called “tenants” — and inviting real employees to join them. The twist: the invitation emails come directly from OpenAI’s own legitimate notification system, meaning they pass every email authentication check your security tools run. To a trained eye and to automated filters alike, they look completely real.
The campaign, dubbed “Poisoned Tenant” by Push Security, was uncovered when multiple employees received invitations to join an OpenAI organization named “Push Security Inc.” — their own employer. The tenant had actually been created by an attacker using Gmail addresses. This tactic demonstrates how fake ChatGPT organizations can be weaponized to bypass traditional defenses.
How Fake ChatGPT Organizations Work
This isn’t your typical spray-and-pray phishing campaign. The attackers targeted specific employees using their work email addresses, suggesting they had researched the organization before launching the campaign. Once inside the fraudulent tenant, researchers found the impersonated organization contained an attacker-controlled account posted as the company’s CEO, and invited employees had been assigned Owner privileges — full administrative access to the workspace.
To add legitimacy, a Visa credit card had already been attached to the organization’s billing account, so premium ChatGPT features were available and no payment warnings would raise suspicion. These details make fake ChatGPT organizations particularly convincing and dangerous.
What the Attackers Want
The goal appears straightforward: get employees to start using this fake ChatGPT workspace as if it were their real company’s platform. Once they do, anything typed into those chats — source code, internal documents, customer data, security research, strategic plans — flows directly to the attacker.
As Push Security put it, on an AI platform, what people type into prompts can be extraordinarily sensitive. And that’s exactly the point. With the rise of AI tools in the workplace, fake ChatGPT organizations represent a new vector for data exfiltration that exploits user trust rather than technical vulnerabilities.
Why This Is So Dangerous
Traditional phishing depends on getting you to click a bad link or open a malicious attachment. This attack is different: unlike normal phishing campaigns, these invitations to fake ChatGPT organizations originate from the platform’s own infrastructure, making them more likely to bypass email security controls. There’s no malware, no spoofed sender, no sketchy URL. Just a legitimate-looking invite to collaborate on a platform your employees may already trust and use daily.
This reflects a broader trend in modern attacks — abusing legitimate SaaS platform features rather than breaking through security tools. When the attacker’s weapon is the platform itself, your perimeter defenses have nothing to catch. The sophistication of these fake ChatGPT organizations attacks shows how quickly cybercriminals are adapting to the AI boom, turning everyday productivity tools into potential liabilities for businesses of all sizes.
What You Should Do
This threat is especially relevant for small and mid-size businesses that may not have formal policies around AI tool usage. Here are concrete steps to protect your organization from fake ChatGPT organizations and similar threats:
- Train employees to verify unexpected invitations. Any invite to join a company ChatGPT workspace — even if it looks real — should be confirmed through a separate internal channel before accepting.
- Establish an official AI policy. Employees should know which AI platforms your company officially uses and how they’re provisioned. If there’s no official ChatGPT tenant, that fact alone should raise a red flag.
- Limit what goes into AI chats. Even in legitimate workspaces, staff should understand that AI platforms are not the place for sensitive internal documents, credentials, or customer data. Implementing clear guidelines here can significantly reduce risks associated with fake ChatGPT organizations.
- Monitor SaaS organization memberships. Know what cloud platforms your employees belong to. Unexpected organization memberships are a signal worth investigating.
- Consider a managed security partner. Threats like this one don’t trigger traditional security alerts. A managed security provider can help monitor for anomalous SaaS activity and establish policies that keep pace with how attackers are evolving. This proactive approach is essential as threats involving fake ChatGPT organizations become more common.
The Bottom Line
Attackers are smart enough to know they don’t have to hack your systems if they can trick your people. AI platforms are the new frontier for social engineering — and the entry point is often a single, convincing email. The best defense is a workforce that knows what to look for and security protocols that account for the tools your team uses every day. Staying informed about tactics like fake ChatGPT organizations is crucial for maintaining strong cybersecurity posture.
Black Belt Secure helps organizations protect their people, data, and operations from the full spectrum of modern threats. Contact us to learn how we can help your team stay a step ahead.