Automated online “symptom checkers” that seem to offer patients a quick opportunity for self-diagnosis provide the right diagnosis in only about one-third of cases, a new analysis reveals.
The study team found that online checkers—which are typically free services offered by medical schools, insurance companies, and even government entities—are a more reliable and effective means to get a handle on symptoms than using web search engines such as Google.
The investigation also found that online medical checkers are about as accurate as primary care physician phone services that offer patients advice on whether or not a condition requires urgent care.
“The goal with these symptom checkers is to try and streamline the process by which people search the Internet for information on health problems,” explained study lead author Hannah Semigran, a research assistant in the department of health care policy at Harvard Medical School in Boston.
“And we found that they are a better alternative to previous attempts to conduct random searches. Symptom checkers are definitely a more organized and constructive way to go about that,” she added.
“We found that they are pretty good at effectively directing people with an (emergency) situation to seek some kind of appropriate care, and to do so quickly,” Semigran said. “But these tools are only a helpful piece of the information puzzle. And users should know that they definitely do not provide the final word on their diagnosis.”
Semigran and her colleagues reported their research online July 9 in the BMJ. Funding was provided by the U.S. National Institute of Allergy and Infectious Diseases.
To assess the pros and cons of symptom checkers, the study team made a list of symptoms from 45 medical scenarios typically presented to medical students for teaching purposes.
In 2014, those symptoms were input into 23 different English-language online symptom checkers. All were free, available to the public, and variously based in the United States, the United Kingdom, the Netherlands and Poland.
Some sites had multiple-choice symptom lists, while others allowed for users to enter their symptoms manually. These automated systems then generated a ballpark sense of what the user’s problem could be, and whether or not the person needed immediate in-person care.
Taken together, the online checkers accurately assessed symptoms on the first attempt in roughly one-third of cases. More than half the time, a correct diagnosis was listed among three top options. And that success rate rose to 58 percent among lists offering 20 options.
What’s more, the checkers were judged to be accurate 57 percent of the time when giving advice as to how to handle the symptoms and where to seek care; that figure jumped to 80 percent when faced with critical or urgent situations. The researchers pointed out that performance varied across the symptom checkers.
Prior research has suggested that random Internet searches only help patients get good advice 64 percent of the time when struggling to handle an urgent concern. Other studies have found triage phone lines to be similarly effective, providing in the range of 61 to 69 percent accuracy when diagnosing a range of conditions (compared with an in-person diagnosis rendered by a physician).
The study authors also found checkers to be relatively conservative when making judgment calls. At times that meant advising users to seek unwarranted medical care. “And sometimes the list of diagnoses options offered can be huge, which can be very confusing for users,” said Semigran.
She also said that incorrect information wasn’t uncommon. And she cautioned that not all symptom checkers are equally trustworthy.
“They definitely varied in terms of having qualified clinical content,” Semigran said. “Some clearly noted an association with a qualified panel of physicians, while others didn’t. So the safest approach is to figure out which entity actually owns the site you’re using. Is it a university? Is there a knowledgeable team behind it?”
Dr. David Hanauer, an associate professor in the department of pediatrics at the University of Michigan in Ann Arbor, suggested that symptom checkers pose a “very complicated” dilemma.
“A lot of physicians are hopeful about finding ways we can democratize the information that we have,” he said. “But the challenge is how we can really be sure that people can make the best decisions with the limited kind of information these services can offer.”
Hanauer explained that “sometimes patients can’t even describe the symptoms that they have. And it’s not always a ‘yes’ or ‘no’ answer. It’s how severe are the symptoms? Which started first, and which have gone away? A savvy clinician will know how to handle all of that, since so much of medicine is based on gut and instinct. A computer may not,” he noted.
“So I don’t want to sound like I’m putting people down for using these services,” Hanauer said. “But it’s a very tricky question.”