chatgptresults.hu / Articles / Analysis
ChatGPT vs Claude vs Perplexity: How Brand Answers Differ
A comparative study of 20 Hungarian B2B entities reveals that high-salience brands face higher fabrication rates and that platform behavior varies drastically by model architecture.
How the platforms were compared
The study examined 20 Hungarian B2B entities split into two groups. Ten were high-salience Tier 1 brands including OTP Bank, MOL Group, and Richter Gedeon. The other ten were low-salience Tier 3 firms such as Contlog Kft. and Codespring Kft. Researchers ran 280 specific queries across two major platforms to test response patterns.
Each entity received seven distinct query templates covering topics like GDPR compliance, ESG capabilities, and general brand facts. The total dataset included 393 unique source URLs generated by the models. This volume allowed for a granular analysis of how different models handle known versus obscure corporate entities.
The methodology focused on verifying whether every generated URL actually existed and supported the claim made. Human coding validated the accuracy of the citations against the original source material. This process distinguished between raw mentions and verified, source-faithful references.
Results showed a clear divergence in how the models approached the task. One platform aggressively generated citations for almost every query. The other platform frequently declined to offer specific links for the smaller companies. These behavioral differences define the current landscape of AI visibility.
Many sources, more fabrication
The model identified as Claude generated 360 of the 393 total sources found in the study. It produced an average of 2.57 citations per run. However, this high volume came with a significant cost. The fabrication rate for this platform reached 44.72 percent across all entities tested.
In contrast, the OpenAI model generated only 33 sources with an average of 0.24 per run. Its fabrication rate was substantially lower at 15.15 percent. This disparity highlights a trade-off between citation volume and factual reliability. More sources do not equate to better accuracy.
The data revealed a paradox where high-salience brands faced higher error rates. Tier 1 entities saw a fabrication rate of 46.22 percent compared to 36.90 percent for Tier 3 firms. Familiarity appears to create a surface for plausible but incorrect completions by the model.
Regulatory queries triggered the highest error rates overall. Questions regarding GDPR compliance resulted in a 52.94 percent fabrication rate. This suggests that models struggle most when asked to verify legal or compliance claims for specific organizations.
- Claude generated 10.9 times more sources than GPT-4o
- Tier 1 brands had a 9.32 percentage point higher fabrication rate than Tier 3
- GDPR-related queries triggered the highest hallucination frequency at 52.94 percent
Refusal as a behaviour
GPT-4o displayed a distinct pattern when handling low-salience entities. It frequently responded with statements indicating an inability to provide specific URLs. This behavior acts as a filter that protects accuracy but reduces visibility for smaller firms.
This refusal mechanism creates an engagement rate paradox. The model engages fully with well-known brands but withdraws from lesser-known ones. Users receive accurate answers for major corporations but find gaps in information for niche B2B players.
The refusal is not random. It correlates with the lack of structured data and knowledge graph presence for the specific entity. Without sufficient digital infrastructure, the model defaults to a safe non-answer rather than risking a fabrication.
This behavior impacts how brands are perceived in search results. A refusal can be interpreted as a lack of relevance or authority by the user. It effectively silences smaller entities in AI-mediated conversations.
The grounded engine
Grounded retrieval systems operate differently from pure parametric models. These engines prioritize verified sources and often decline to answer when reliable data is missing. This approach minimizes the risk of generating false claims about companies.
Studies indicate that grounded engines fabricate far less than their counterparts. They rely on external databases and live search results rather than internal weights alone. This structure forces the model to cite existing documents or admit ignorance.
However, this safety comes with limitations. Grounded systems may still fail to retrieve information for entities with poor online footprints. The lack of a primary source leads to a refusal similar to the behavior seen in GPT-4o.
The difference lies in the intent. Grounded refusals stem from a lack of retrieved evidence. Parametric refusals often stem from confidence thresholds in the model's internal knowledge. Both result in a missing answer for the user.
Choosing what to check where
Brands must select verification tools based on their specific risk profile. High-salience companies face the greatest threat from confident fabrication. They need tools that detect false attributions and incorrect citations.
Low-salience firms face the opposite challenge. Their primary risk is invisibility or refusal. They require strategies to improve their digital infrastructure so models can retrieve their data.
A multi-platform audit is essential for a complete picture. No single model provides a definitive truth. Comparing outputs across different architectures reveals the full scope of potential errors.
Regular monitoring is necessary because model behaviors change. Updates to training data or retrieval mechanisms alter how entities are represented. Continuous testing ensures that brand narratives remain accurate across all AI channels.
Key takeaways
- Claude generates significantly more citations but also produces nearly three times the fabrication rate of GPT-4o.
- High-salience brands experience a higher rate of false attribution than lesser-known firms due to the Brand Hallucination Paradox.
- Grounded retrieval engines minimize fabrication but may refuse to answer for entities lacking strong digital infrastructure.
- Regulatory and compliance queries trigger the highest rates of hallucination across all tested platforms.
- Verification requires checking multiple models as no single system offers universal accuracy for all entity types.
Related questions
Why do AI models fabricate more for famous brands?
Familiarity creates a stronger surface for plausible completions. Models associate high-salience entities with a vast amount of training data, making them likely targets for confident but incorrect attributions. This phenomenon is known as the Brand Hallucination Paradox.
Does GPT-4o refuse to answer for small companies?
Yes, GPT-4o frequently declines to provide specific URLs for low-salience entities. This behavior stems from a lack of retrievable evidence in its knowledge base. It prioritizes accuracy over engagement, resulting in a refusal rather than a potential hallucination.
Which AI assistant is safest for compliance queries?
Grounded retrieval systems generally pose the lowest risk of fabrication for compliance topics. They rely on verified external sources rather than internal memory. However, they may still refuse to answer if no authoritative source exists for the specific entity.
AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.
