Before you throw your company code and password to AI, do you know who will eventually see that prompt? Industrial security company Dragos announced an actual test on November 19: From July to November, they sent about 2 million "bait prompts" to a number of commonly used AI services. Each one hid a unique URL and cloud key, and then saw where these secrets were used.
According to Dragosresearch report, the results found three types of problems: at least three AI service providers were hacked, sellers in the AI resale market peeked at customer prompt words, and some service providers ignored the "zero data retention" setting, captured the prompt words, and even flowed them back to China. Dragos refers to the companies involved in the report by pseudonyms and does not name them publicly.
How to set up bait: Each prompt word hides an exclusive key
Dragos uses a "canary token": a set of secrets that only exist in a certain place. As long as someone uses it, it means that the place has been leaked. Each prompt comes with a real AWS key (connected to an empty account, the entire process is recorded), a domain name, and a URL that has never appeared elsewhere.
Dragos said the decoys planted in July started triggering within days and are still ringing today. They also built a tool similar to "route tracing" to find out which intermediaries the request actually passed through. They traced about 2,000 paths, hundreds of which passed through unidentifiable systems, and a few sent data to companies that were not on public paths.
3 service providers were hacked and hundreds of keys were used
The most serious thing is that three service providers hosting AI models were hacked. Dragos noted that the decoys were processed alongside requests from real customers, and that customer data was almost certainly copied. The earliest intrusions date back to mid-July, and none of the three companies knew they were hacked until Dragos notified them.
The largest case was that all requests sent in a few periods at the end of October were stolen. Hundreds of AWS keys fell into the hands of attackers. Some requests were even rejected by the service provider before the model was executed, which means that the attacker copied the request at the service provider's entrance. Some of the other two stolen requests were marked with zero data retention, and Dragos said privacy settings don't help when hackers are already inside.
After the attacker obtained the key, he tested whether it could be used within a few hours, then searched for cloud storage and keys, and then tried to activate AWS's AI model service and create his own account to maintain access. In total, these decoy keys were used to call the AWS API tens of thousands of times. Dragos believes that an attacker has invaded two service providers at the same time, and has also used an AI-driven penetration testing tool ARTEX since November 1.
Resale Market Sellers: Open a record and wait for prompts to come to your door
The second type of problem doesn’t even use loopholes. Some AI routing services are open markets, and whoever offers the lowest price takes the order. Dragos said that as long as sellers mark the lowest price on popular models and open the request history, they can see the customer's prompt words.
Most of the people they caught were individuals trying to log in one account at a time from their home network. In one example, Dragos sent two prompt words through the same seller. About a day and a half later, someone used home broadband to open two secret URLs and began to guess the login account; requests from other sellers for the same model were not touched. Another example is a Mac in France. I manually used the password in the prompt word to try to log in to the server with the highest permissions. Dragos also discovered that a routing service privately forwarded some traffic to this marketplace.
The Chinese laboratory captured zero photos and retained the prompt words, and the link appeared in the Feishu group
In the third type of problem, no one stole the key, but the secret URL in the prompt word was repeatedly crawled. The most obvious, Dragos said, was a Chinese AI lab: More than half of the secret URLs sent to its models from July to October were crawled by its system, including about two-thirds of requests marked "no data collection" or zero data retention, which usually appeared two or three days after being sent.
Not only will you fall into this trap if you directly use the services of this laboratory. Dragos pointed out that several U.S. hosting providers resold the lab's models, and prompt words sent to these U.S. service providers also triggered the same crawler, indicating that the prompt words flowed back to the laboratory. "Customers who specifically choose American service providers and want to keep their data in the United States are not getting what they paid for."
The most bizarre one was a warning message sent to an American service provider with a zero data retention mark. 24 days later, the secret link in it was posted in a Feishu (Lark) group, and within two minutes it was opened by three computers and three mobile phones in Beijing and Shanghai. Large American service providers also have problems: one is still crawling that URL more than two months after a single search request, and another is crawling back several days after each request.
Dragos Tip: Don’t let the lowest price determine where the prompt goes.
Dragos said it has notified relevant industries and is currently cooperating with about a dozen service providers in the investigation; the service provider that was hacked in July has not yet been publicly disclosed, and one of the hacked companies at the end of October has issued a preliminary security announcement. The routing service that forwards traffic to the marketplace says it has stopped, but the stolen keys and URLs are still being accessed.
Dragos’ suggestions for users include: build your own models if you can afford it; otherwise, use first-line model manufacturers directly, sign a contract and read a privacy policy; choose your own upstream service provider when using routing services and avoid the resale market; turn on the function of automatically filtering credentials. Dragos reminds that privacy settings only restrict the first service provider to take over, and as long as the AI agent can read it, even the password will be sent.
this article Actual test of 2 million decoy prompt words: AI service provider was hacked, Chinese laboratory still caught zero data first appeared in Chain News ABMedia。




