12. How to evaluate a scientific claim and expertise of an expert?

As a representative of the Finnish FaktaBaari, I had a privilege to be involved in a deeply engaging project coordinated by Stanford University. The final report “Science Education in the Age of Misinformation” (1) was published in spring 2022. An international team of experts examined how science education should respond to the challenges posed by the misuse of scientific information and evidence. The report also considers how to verify scientific claims made on social media and how to assess the competence of the person making the claim as an expert in the field.

It is important to be aware that all types of content circulate online. In addition to correct and useful information, there is also a great deal of incorrect information (misinformation, i.e. incorrect information spread in good faith or by mistake) and falsified information (disinformation, i.e. incorrect or inaccurate information deliberately spread). The dissemination of incorrect or falsified information is often harmful to both the individual and the community. It is therefore useful to identify who is behind the information and to verify the information from multiple sources to understand the perspective and possible bias of the source.

Every now and then we must assess the credibility of scientific news we find on social media. For example, is there scientific evidence of the benefits of using masks? Can we stop climate change? Is nuclear energy safe and is it a sustainable option? Modern science is so highly specialised that no one person can master all fields and all subjects. We are therefore dependent on experts and must evaluate whose expertise we can rely on – especially if the expert opinions are somewhat contradictory.

In the past two years we have all come across distorted claims about the Covid-19 pandemic, which fact-checkers around the world have had to correct. More than 17,000 Covid-19 claims (2) have been verified by the joint efforts of fact-checkers.  Some of these claims are based on what appear to be scientific studies and expert opinion. It is therefore important to reflect on how to take a healthy critical view of scientific claims and how to identify a true expert.

Disinformation is often dressed up as a reliable pseudo-scientific claim. Products may be marketed with misleading or non-existent references to various studies. Articles of questionable scientific quality are circulated on social media.

How to evaluate the expertise of experts?

When we choose a lawyer, plumber, dentist or architect, we look for evidence and references of the person’s previous professional skills and qualifications. But how do you assess the expertise and authority of a scientist – whether they are a well-known and respected expert in their field, and what evidence of their expertise is there?

 Being a scientist requires years of education and often a PhD. Even a doctorate covers only a narrow field of knowledge. Expertise can also be acquired through scientific professional training or practical work experience.

“Just being a practicing scientist, however, is not enough. The individual must be a practicing scientist in the relevant field. Being a Nobel prize winner in one field, does not make you an expert in other fields. Yet, individuals may easily lump all scientists together as undifferentiated ‘authorities.’ A specialist in radiology is not somebody you would ask for advice on viruses. Being a scientist in one field of science does not make you an expert in all fields of science. A theoretical cosmologist knows no more about ecology than any other competent outsider” (Osborne et al. 2022).

In recent weeks, various experts have appeared on social media commenting on the Russian invasion of Ukraine. It has often been easy to deduce from their statements which side they represent. In times of conflict, it is therefore necessary to take a more cautious and prudent approach than usual to various news reports and expert opinions. It is important to find out who is representing what, what evidence the information is based on and what the real expertise of the person making the statement is on the issue in question.  

How to evaluate a scientific claim?

Scientific information must go through a number of processes to ensure that it is reliable. Openness, critical debate and peer review drive research forward.  Science is self-correcting. Interpretations of research data are modified and refined as new knowledge emerges. Research builds on knowledge built up over decades, if not centuries.

Scientific knowledge is our best current understanding of things. It is not anyone’s opinion or personal experience, but the result of a systematic process. It can change as new research findings and understanding develops. That’s why researched science is worth more than opinions!

Figure: A schematic overview of the approach we think needs to be taken to evaluating scientific claims on the internet (Osborne et al, 2022).

When faced with a science-based claim, it is worth finding out whether the person/organisation making the claim has a conflict of interest. Are there economic, religious or political interests at stake? If so, it may be a form of paid advertising and the results should be treated with suspicion. For example, the tobacco industry and fossil fuel companies have used experts on their payrolls to disseminate information that benefits them.

If there is no conflict of interest, the following questions should be asked:

  • Does the individual/organization have relevant expertise?
  • What is the standing of the author within the scientific community?
  • Do they have a record of integrity?
  • Does the author have the appropriate credentials or other relevant experience?
  • Is there a strong scientific consensus among experts? If not, what do the majority of scientists think?
  • How certain of the claims is the scientific community?
  • Has the finding been vetted by similar experts and to what degree?

It is also worth pausing to consider the potential benefits and risks involved. For example, during the coronary period, we have had to make personal choices about following expert advice – for example, about taking COVID-19 vaccines, wearing masks, adhering to the length of quarantine periods and the reliability of home tests.

Where to find reliable information?

 To obtain an answer in the English-speaking world Wikipedia is a good place to begin. The websites of major scientific institutions, such as National Academies of Science (www.nap.edu) (4), and long-established news media are also reliable sources of information.

Fact-checkers in different countries have interesting fact-checking websites where you can learn how fact-checkers check the accuracy of various claims and the authenticity and originality of images and videos, for example. EDMO’s fact-checking community (5) has an updated list of reliable European fact-checking organisations.

It is also worth checking out the report “Science Education in the Age of Misinformation (6) for a more in-depth look at the topic.

References:

(1) Osborne, J., Pimentel, D., Alberts, B., Allchin, D., Barzilai, S., Bergstrom, C., Coffey, J., Donovan, B., Kivinen, K., Kozyreva. A., & Wineburg, S. (2022). Science Education in an Age of Misinformation. Stanford University, Stanford, CA https://sciedandmisinfo.stanford.edu/

(2) CoronaVirusFacts Alliance, Poynter,  https://www.poynter.org/coronavirusfactsalliance/

(3) Osborne, J., Pimentel, D., Alberts, B., Allchin, D., Barzilai, S., Bergstrom, C., Coffey, J., Donovan, B., Kivinen, K., Kozyreva. A., & Wineburg, S. (2022). Science Education in an Age of Misinformation. Stanford University, Stanford, CA https://sciedandmisinfo.stanford.edu/

(4) Miten tutkittu tieto syntyy? https://www.aka.fi/tietysti/mutuaihmeellisempaa/#d82904a1 

(5) National Academies https://nap.nationalacademies.org/ 

(6) Osborne, J., Pimentel, D., Alberts, B., Allchin, D., Barzilai, S., Bergstrom, C., Coffey, J., Donovan, B., Kivinen, K., Kozyreva. A., & Wineburg, S. (2022). Science Education in an Age of Misinformation. Stanford University, Stanford, CA https://sciedandmisinfo.stanford.edu/

Kari Kivinen,  PhD, is an Education outreach expert at EUIPO.   He has over 30 years of experience in international education. Since 2017 he has led the pedagogical development work at Faktabaari EDU digital information literacy service building on fact-checking methodology and co-authored and piloted the learning materials with fellow teachers around Finland and abroad. He is a member of the Commission expert group on tackling disinformation and promoting digital education. The author works for EUIPO and agency of the EU but the views expressed are purely personal and cannot be taken as being official statements of either the EU or the EUIPO. 

Artwork: Lumi Pönkä

Download the Digital Information Literacy Guide (PDF).

info@faktabaari.fi