New Delhi, Apr 6 (PTI) Asking ChatGPT a health-related question that included evidence was seen to confuse the AI-powered bot and affect its ability to produce accurate answers, according to new research.

Scientists were "not sure" why this happens, but they hypothesised that including the evidence in the question "adds too much noise", thereby lowering the chatbot's accuracy.

Also Read | BJP Foundation Day 2024 Date: Know the History and Significance of BJP Sthapna Diwas That Celebrates the Formation of the Bharatiya Janata Party.

They said that as large language models (LLMs) like ChatGPT explode in popularity, there is potential risk to the growing number of people using online tools for key health information. LLMs are trained on massive amounts of textual data and hence are capable of producing content in the natural language.

The researchers from the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and The University of Queensland (UQ), Australia, investigated a hypothetical scenario of an average person asking ChatGPT if 'X' treatment has a positive effect on condition 'Y'. They looked at two question formats - either just a question, or a question biased with supporting or contrary evidence.

Also Read | AMC Raising Day 2024: Army Medical Corps Celebrates 260th Raising Day as They Live Up To The Corps Motto.

The team presented 100 questions, which ranged from 'Can zinc help treat the common cold?' to 'Will drinking vinegar dissolve a stuck fish bone?'. ChatGPT's response was compared to the known correct response, or 'ground truth' that is based on existing medical knowledge.

The results revealed that while the chatbot produced answers with 80 per cent accuracy when asked in a question-only format, its accuracy fell to 63 per cent when given a prompt biased with evidence. Prompts are phrases or instructions given to a chatbot in natural language to trigger a response.

"We're not sure why this happens. But given this occurs whether the evidence given is correct or not, perhaps the evidence adds too much noise, thus lowering accuracy," said Bevan Koopman, CSIRO Principal Research Scientist and Associate Professor at UQ.

The team said continued research on using LLMs to answer people's health-related questions is needed as people increasingly search information online through tools such as ChatGPT.

"The widespread popularity of using LLMs online for answers on people's health is why we need continued research to inform the public about risks and to help them optimise the accuracy of their answers," said Koopman.

"While LLMs have the potential to greatly improve the way people access information, we need more research to understand where they are effective and where they are not," said Koopman.

The peer-reviewed study was presented at Empirical Methods in Natural Language Processing (EMNLP) in December 2023. EMNLP is a natural language processing conference.

(The above story is verified and authored by Press Trust of India (PTI) staff. PTI, India’s premier news agency, employs more than 400 journalists and 500 stringers to cover almost every district and small town in India.. The views appearing in the above post do not reflect the opinions of LatestLY)