sortiwa
Facepalm: Training new and improved AI models requires vast amounts of data, and bots are constantly scanning the internet in search of valuable information to feed the AI systems. However, this largely unregulated approach can pose serious security risks, particularly when dealing with highly sensitive data.
Popular chatbot services like Copilot and ChatGPT could theoretically be exploited to access GitHub repositories that their owners have set to private. According to Israeli security firm Lasso, this vulnerability is very real and affects tens of thousands of organizations, developers, and major technology companies.
Lasso researchers discovered the issue when they found content from their own GitHub repository accessible through Microsoft’s Copilot. Company co-founder Ophir Dror revealed that the repository had been mistakenly made public for a short period, during which Bing indexed and cached the data. Even after the repository was switched back to private, Copilot was still able to access and generate responses based on its content.
“If I was to browse the web, I wouldn’t see this data. But anyone in the world could ask Copilot the right question and get this data,” Dror explained.
After experiencing the breach firsthand, Lasso conducted a deeper investigation. The company found that over 20,000 GitHub repositories that had been set to private in 2024 were still accessible through Copilot.
Lasso reported that over 16,000 organizations were affected by this AI-generated security breach. The issue also impacted major technology companies, including IBM, Google, PayPal, Tencent, Microsoft, and Amazon Web Services. While Amazon denied being affected, Lasso was reportedly pressured by AWS’s legal team to remove any mention of the company from its findings.
Private GitHub repositories that remained accessible through Copilot contained highly sensitive data. Cybercriminals and other threat actors could potentially manipulate the chatbot into revealing confidential information, including intellectual property, corporate data, access keys, and security tokens. Lasso alerted the organizations that were “severely” impacted by the breach, advising them to rotate or revoke any compromised security credentials.
The Israeli security team notified Microsoft about the breach in November 2024, but Redmond classified it as a “low-severity” issue. Microsoft described the caching problem as “acceptable behavior,” though Bing removed cached search results related to the affected data in December 2024. However, Lasso warned that even after the cache was disabled, Copilot still retains the data within its AI model. The company has now published its research findings.
Source link