2 min read
Chinese AI startup DeepSeek’s chatbot has been evaluated by NewsGuard, a trustworthiness rating service, and found to have achieved only 17% accuracy in delivering news and information. This performance placed it tenth out of eleven in a comparison with Western competitors, including OpenAI’s ChatGPT and Google’s Gemini.
The audit discovered that the DeepSeek chatbot provided incorrect information 30% of the time and gave unclear, useless responses 53% of the time for news-related questions, resulting in an 83% failure rate. The failure rate among Western competitors averaged 62%, however DeepSeek’s performance fell short of this criterion, with an 83% failure rate.
Such an evaluation calls DeepSeek’s AI system into question, despite the fact that the business touted its technology as being equivalent to or superior to Microsoft-backed OpenAI while being less expensive to develop. According to performance criteria, the chatbot lags below its Western competitors in terms of providing factual news information.
The initial deployment of DeepSeek’s chatbot resulted in its position as the App Store’s top downloaded program, leading American technology stock prices to fall by $1 trillion and raising concerns about the United States’ supremacy in AI technology.
NewsGuard employed a series of 300 queries, which DeepSeek incorporated, similar to how it did for Western news outlets, which used 30 questions based on ten bogus claims circulating on the internet. The claims investigated included the assassination of UnitedHealthcare executive Brian Thompson last month, as well as the Azerbaijan Airlines flight 8243 plane disaster incident.
The audit also revealed that DeepSeek reiterated the Chinese government’s stance on the subject in three of the ten queries, without being prompted to provide any information regarding China. DeepSeek responded with Beijing’s stance on the Azerbaijan Airlines accident in response to inquiries that were not related to China.
D.A. Davidson analyst Gil Luria commented, “The importance of the DeepSeek breakthrough is not in answering Chinese news-related questions accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models.”
Like other AI models, DeepSeek was most vulnerable to repeating false claims when responding to prompts used by people seeking to use AI models to create and spread false claims, NewsGuard added.
