AI chatbots recommend fake retail websites to shoppers

The risks of AI shopping

Shoppers are increasingly turning to AI chatbots to help them research products, compare prices, and find places to buy items. However, a new study suggests this convenience comes with a significant security risk. The research found that AI chatbots recommended 'risky' websites in as many as 15 percent of conversations involving shopping queries.

The study, conducted by the IP protection firm Podqi and shared with The Verge, tested three major AI engines: Gemini, Perplexity, and ChatGPT. Researchers used a series of prompts involving 151 different brands, ranging from large fashion labels like Lululemon to smaller homeware companies. The goal was to see how often these tools would direct users to untrustworthy sources.

Risky websites were defined by the researchers as several different types of threats. These included fake websites that impersonate real retailers, sites selling counterfeit goods, and old brand domains that have been reregistered by third parties. The study also identified dead sites that are no longer active but could be purchased and reused by bad actors.

How the chatbots performed

The results of the study showed a wide variance in accuracy across the different AI models. Perplexity performed the worst among the group, displaying risky sites in 15 percent of its answers. In contrast, Google’s Gemini was the most reliable of the three, linking to or mentioning a risky site in only 4 percent of the responses tested.

ChatGPT fell into the middle of the pack, surfacing risky sites in 8 percent of its answers. While it was less accurate than Gemini, it showed a specific behavior regarding safety. The study found that ChatGPT offered a warning about a risky site in 24 percent of the instances where it surfaced one.

The researchers noted that specific types of questions were more likely to trigger these errors. For example, AI engines were more likely to display risky sites when asked about regional brand websites. In some cases, the AI even fabricated fake regional sites for brands that do not actually have any official regional presence.

Vulnerability of small brands

The study highlighted a significant disparity in how AI treats large versus small companies. Podqi found that queries related to larger, well-known companies had risky sites in the answers about half as often as queries for smaller brands. This suggests that smaller retailers may be more susceptible to being overshadowed by impersonators in AI results.

This trend is particularly concerning for the upcoming holiday shopping season. During this time, consumers often search for specific items and may be more likely to visit sites they are not already familiar with. If an AI directs a user to a fake site during a high-volume shopping period, the potential for fraud increases significantly.

For brands and retailers, these errors create a major headache for maintaining brand integrity. When an AI cites a fake site as a legitimate source, it can lead to lost sales and damaged reputations. It also places a heavy burden on the consumer to distinguish between a real storefront and a deceptive one generated by a chatbot.

Company responses and safeguards

The AI companies involved have responded to the findings with different perspectives on their current safeguards. Google stated that they are aware Gemini can make mistakes and are working to improve the models. They currently display links to sources alongside responses to help users verify the information provided by the AI.

OpenAI expressed a goal of connecting people with reliable places to shop and stated they are continuing to improve safeguards. They aim to make deceptive sites less likely to appear in shopping results and expressed appreciation for the researchers sharing these examples. They view the data as a way to identify areas where the experience needs improvement.

Perplexity, despite performing the worst in the study, defended its accuracy on other benchmarks. A spokesperson for the company noted that while they do not claim to be 100 percent accurate, they are focused on research quality. The study serves as a reminder that while these tools are powerful, they are not infallible and require human oversight.