Author: Pan Wenjing (Associate Professor, School of Journalism, Chinese University).
*: Young Journalists, Issue 23, 2023.
Compared with simple refutation, fact-checking journalism brings a higher perception of the quality of correction methods, and also brings a higher willingness to socialize. The effectiveness of fact-checking journalism is not affected by the pre-existing conspiracy theory mentality of audiences, nor does it reflect demographic differences.
Research origins. The development of Internet technology, on the one hand, has brought convenience to information acquisition, but at the same time, it has also led to uneven information quality, and even accelerated the spread of false information and fake news, and the spread speed and scope of fake news on social ** platforms even exceed that of real news [1]. Professional journalism, whether as a response to rebuilding the audience's trust in professional journalism, or as an innovative attempt to face the convergence trend, has carried out various forms and types of fact-checking practices. Globally, the practice of fact-checking has become an important step for journalism to address the challenges of digitalization in recent years[2].
Fact-checking, as a journalistic genre, can simply be defined as the evaluation of the validity of claims or claims made by public figures and institutions in a systematic manner [3]. Fact-checking first emerged in the U.S. press to check the public statements of politicians and elites, and has since been widely used on a variety of social, economic, scientific, and health-related issues due to the transparency and openness of its verification process [4]. Fact-checking, as a new type of journalistic practice, has been embraced by the American journalism industry, and fact-checking agencies have won the Pulitzer Prize for journalism [5].
In China, fact-checking may have been in its infancy for a long time, and rumor-busting is a more common form of correcting disinformation than fact-checking practices carried out by professional institutions. Internet information rumor refutation is mainly carried out by commercial institutions and entities, such as the China Internet Joint Rumor Refutation Platform established by the Cyberspace Administration of China, Science Popularization China - Scientific Rumor Refutation, and the fact-checking practice carried out by Internet business platforms, such as Tencent's "Truth". In addition, professional ** has also tried to carry out fact-checking practices, such as The Paper's global real-time verification platform "Mingcha". In addition to the diversity of fact-checking subjects, the methods of fact-checking also present a variety of characteristics. Common verification methods include expert refutation of rumors, background information verification completed by professional reporters, narrative logic verification, and automated verification.
On top of these booms, fact-checking, as an ex post facto corrective for the spread of disinformation, has been ineffective. Some studies have found that fact-checking is effective in reducing beliefs and attitudes related to disinformation [6], while others have not found a significant corrective effect [7], and some studies have even found a backfire effect, in which fact-checking reinforces people's beliefs about attitudes toward disinformation [8]. In addition, the conspiracy theory mentality, a cognitive factor at the individual level, can also affect the effectiveness of fact-checking [9]. Widely disseminated disinformation often presents conspiracy theories that are exaggerated or exaggerated, and foreign studies of vaccine-related disinformation have shown that individuals who are skeptical about the efficacy of vaccines are more distrustful of ** and other social organizations [10].
Research the question. Existing studies have focused more on disinformation-related belief or attitude changes when measuring the effectiveness of fact-checking. It is true that attitudes and beliefs are very important subjects of study, but from the perspective of improving the quality of fact-checking and promoting fact-checking news, more attention should be paid to the following two performance measures: first, the quality of the audience's perceived way of correcting, and second, the audience's willingness to socialize.
Comparisons are required to measure the effectiveness of fact-checking journalism. Among the existing studies, some adopt a pre-event and post-event experimental design, that is, read "false information + corrective information", and then compare the audience's attitudes towards false information before and after correction; There are also studies that use an "experimental group + control group" design, but the control group only reads false information [11]. When fact-checking news has not yet appeared, the more commonly used correction method is to refute rumors or simply refute them, mainly through **, enterprises, institutions or authorities** to issue a notice that false information is false information, so as to achieve the effect of refuting rumors. Fact-checking journalism is generally done by professional fact-checkers, who cross-check and verify sources, content, channels, and other aspects through a pre-established fact-checking process, adjudicating disputed facts in accordance with scientific methods and established procedures, giving verification conclusions, and judging the authenticity of claims [12]. Based on these two common forms of correction, this study raises the first research question:
RQ1: For the audience, which simple refutation or fact-checking leads to a higher perception of the quality of the correction method and a stronger willingness to do so?
When measuring the effectiveness of fact-checking journalism, the cognitive characteristics of the audience, such as the audience's existing views and attitudes, cognitive needs, media literacy, etc. [13][14]. Among them, the individual's existing conspiracy theory mentality also needs to be taken into account. Conspiracy theories refer to the assumption that actors with power are conspiring to influence major social and political events [15]. Disinformation on social media platforms often uses conspiracy theories to explain important events, and news audiences who are exposed to conspiracy theories-style disinformation will develop a conspiracy theory mentality and reduce trust in **, institutions, and ** [16]. Based on the impact of conspiracy theory mentality on disinformation acceptance and social trust, this study proposes a second research question:
RQ2: For audiences, how does the individual-level conspiracy theory mentality affect their perception of the quality of fact-checked journalism and their willingness to do so?
Methods & Processes.
a) Experimental stimulus material.
In order to answer the above two research questions, this study uses 2 (corrective methods: fact-checking vs. .Simple refutation) 2 (Conspiracy Theory Mentality: High vs. Conspiracy Theory.)Low) 2 (Topic: Vaccine vs. VaccineIn order to increase the generalizability of the research conclusions, this study used two common disinformation topics.
The selection of false information on the two topics refers to the widely circulated false information on fact-checking platforms [17][18], and the pre-test of a small sample (n= 108) is used to ensure that there is no difference in the credibility of the false information on the two topics.
The correction is done by rewriting the text. In the case of a simple refutation, the corrective information only points out that the false information is wrong; In the context of fact-checking, corrective information briefly describes the process of fact-checking, verifies false information, and finally points out that false information is false. In order to ensure that the corrective action was successful, the participants were asked to answer the method of the correction they had just read after reading the corrective information, and the answers that were incorrect or "uncertain" were excluded.
Conspiracy theory mentality, after measuring the participants' existing conspiracy theory mentality, was divided into two groups, high and low, according to the median. Measurements were done primarily by the Conspiracy Theories Mindset Scale [19]. The scale has a total of 9 items, which are completed on a seven-level Likert scale with good reliability (Cronbach's = 0.).87）。
2) Experimental process.
The experiment was completed through a questionnaire system. After reading the informed consent, participants were randomly assigned to different experimental groups. Before the experiment began, participants were first asked to answer about their pre-existing attitudes about the topic of disinformation (sunscreen vaccines) and then browsed three pages: page 1 presented false information, page 2 presented irrelevant information, and page 3 presented corrective information (simple refutation by fact checking). After the tour was completed, participants proceeded to answer questions about perceived quality of correction style, willingness to correct information**, conspiracy theory mentality, and demographic variables including gender, age, education level, income level. Finally, a total of 519 valid samples were received.
c) Measurements. Existing attitudes. The participants' pre-existing attitudes related to the topic of disinformation were measured by three items on a seven-point Likert scale, including whether the vaccine was useful, beneficial, and helpful, which was less reliable but acceptable in exploratory studies (Cronbach's = 0.).69）。
Perceive corrective quality. The quality of the corrected information subjectively perceived by the participants was measured by 5 items on a seven-point Likert scale, including whether the checks mentioned in the corrected messages read were of high quality, useful, valid, convincing, and good, with good reliability (Cronbach's = 0.).95）。
*Will. The willingness of the experimental participants was measured by 4 items scored on a seven-point Likert scale, including two dimensions: public and private (one-to-one) on social platforms, and the reliability of the 4 questions was good (cronbach's = 0.).94）。
Findings: In order to answer the research question, a multivariate analysis of covariance was used, which showed that the topic did not directly affect the perception of corrected quality, f(1,510)=019，p= 0.66；It also does not affect the willingness of **, f(1,510)=1.17，p= 0.98；Existing attitudes were used as covariates to significantly affect the perception of corrected quality, f(1,510)=1359，p< 0.001；It also significantly affects ** willingness, f(1,510)=1251，p< 0.001。Demographic variables, including gender, age, income level, and level of education, do not affect the outcome variables. In the final analysis model, the correction style and conspiracy theory mentality were regarded as independent variables, the perceived correction quality and ** willingness were regarded as dependent variables, and the existing attitude was used as covariates, and SPSS26 was used for analysis.
The first research question is about the effect of correction style on perceived quality and willingness to correct. The results showed that participants who read fact-checked news had a perceived quality of correction (m = 5.).55，sd= 0.85), which is higher than reading a simple refutation (m= 3..)88，sd= 1.50），f（1，514）=252.88，p< 0.001；The willingness of participants who read fact-checked news (m=4..)84，sd= 1.40), also higher than reading a simple refutation (m= 3.).85，sd= 1.61），f（1，514）=58.55，p< 0.001。
The second research question is about the impact of conspiracy theory mentality on the effectiveness of fact-checking. The results showed that the conspiracy theory mentality did not affect the participants' perception of the quality of correction with different correction methods, f(1,514)=013，p= 0.72；It also does not affect the participant's willingness to correct the different ways, f(1,514)=0.46，p= 0.50。
Discussion. In foreign countries, although fact-checking aims to provide an open, impartial, and transparent verification process, its role as an impartial judge is often polarized because the object of verification is often the words and deeds of politicians [20]. In China, fact-checking news is generally targeted at fake news or disinformation related to health, science, wellness, and food safety, and while such topics are more likely to draw more definitive conclusions than political topics, how effective is fact-checking?
The results show that fact-checking journalism can lead to a higher perception of corrective quality and a stronger willingness to correct than simple refutation. This result can be understood on two levels. First, the audience's assessment of the quality of fact-checked journalism is significantly higher than that of simple refutation, a conclusion that affirms the quality of fact-checked journalism. In China, simple refutation of rumors by officials, institutions, or platforms, while more common, is characterized by the identification of the results rather than the process of detailed verification[21]. The results of this study show that the audience is more likely to recognize the attempt of fact-checking journalism to emphasize the verification process than the usual simple debunking of rumors, which also affirms the innovation of fact-checking as a new news genre. For professional journalists, in the future practice of fact-checking, it is necessary to strengthen the respect for facts and the importance of evidence and logic, and continue to provide high-quality fact-checking news to the audience. Second, the audience's willingness to read fact-checked news is significantly higher than that of a simple refutation. More means that a wider audience is able to access fact-checked news, and considering that the channels for disinformation and fact-checked news are mainly social**, then this result also has implications for the dissemination and promotion of fact-checked news.
In terms of individual differences in correction effect, some foreign meta-analyses have concluded that fact-checking can positively influence the political beliefs and attitudes of the audience, but this influence will be weakened by the existing attitudes, knowledge, and opinions of individuals [22]. Specifically, some studies have suggested that a conspiracy theory mindset can lead to an individual's preference for conspiracy theories as an explanation for important social events and phenomena, and it can also deepen personal distrust of **, institutions, and officials**. However, the results of this study show that the audience's existing conspiracy theory mentality does not affect their evaluation of the quality of fact-checking and their willingness to follow-up. This result also illustrates the pervasiveness of fact-checking journalism as a means of correcting disinformation. Whether it is in correcting the perception of quality or in terms of willingness, the effect of fact-checking can break through individual differences and can be generalized to the widest possible news audience.
This paper is the interim result of the National Social Science ** General Project "Comparative Research on the Governance Model of Social Platform Disinformation" (Grant No.: 21BXW065).
References: 1] Vosoughi S, Roy D, Aral S the spread of true and false news online[j].science,2018, 359(6380):1146-1151.
2][21] Yan Wenjie, Liu Yusi, Zhou Ruiming. Fact-checking: Innovative Practices for Professional News Producers: Implications of a Critical Experiment [J].News Reporter, 2023(02): 46-59
3][22] walter n, cohen j, holbert r l, et al. fact-checking:a meta-analysis of what works and for whom[j]. political communication, 2020,37(3): 350-375.
4][5]gr**es l, nyhan b, reifler j. understanding innovations in journalistic practice: a field experiment examining motivations for fact-checking[j].journal of communication, 2016, 66(1): 102-138.
6]fridkin k, kenney p j, wintersieck a. liar, liar, pants on fire: how fact-checking influences citizens’ reactions to negative advertising[j].political communication, 2015, 32(1): 127-151.
7][9]jolley d, douglas k m. the effects of anti-vaccine conspiracy theories on vaccination intentions[j]. plos one, 2014, 9(2): e89177.
8]nyhan b, reifler j. when corrections fail: the persistence of political misperceptions[j]. political beh**ior, 2010, 32(2): 303-330.
10]larson h j, jarrett c, eckersberger e, et al. understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007-2012[j]. vaccine, 2014, 32(19): 2150-2159.
11]walter n, tukachinsky r. a meta-analytic examination of the continued influence of misinformation in the face of correction: how powerful is it, why does it happen, and how to stop it?[j]. communication research, 2020, 47(2): 155-177.
12]gr**es l. deciding what's true: the rise of political fact-checking in american journalism[m]. new york: columbia university press,2016.
13]pennycook g, epstein z, mosleh m, et al. shifting attention to accuracy can reduce misinformation online[j]. nature, 2021, 592(7855): 590-595.
14] Ma Deyong. "Matching Effect": The Psychological and Ideological Roots of Political Rumors[J].Political Science Research,2018(05):54-66+126
15]douglas k m, uscinski j e, sutton r m, et al. understanding conspiracy theories[j]. political psychology, 2019, 40(s1): 3-35.
16] Liu Yusi, Yan Wenjie, Zhou Ruiming. Making fact-checking more popular? The Multiple Consequences of Using Mass Reporting and Pro-People Discourse to Correct Conspiracy Theories[J].Journal of Global Media,2023,10(03):170-190
17]health feedback. vaccines do not cause neurological damage, aluminum levels in vaccines are safe[eb/ol]. 2020-03-24)[2023-07-01].
18]health feedback. current evidence doesn’t indicate that sunscreen ingredients increase the risk of cancer; it shows that unprotected sun exposure does[eb/ol].(2022-04-09)[2023-07-01].
19]imhoff r, bruder m. speaking (un-) truth to power: conspiracy mentality as a generalised political attitude[j]. european journal of personality, 2014,28(1): 25-43.
20]marietta m, barker d c, bowser t. fact-checking polarized politics: does the fact-check industry provide consistent guidance on disputed realities?[c]//the forum. de gruyter, 2015, 13(4): 577-596.
23]zhang j, featherstone j d, calabrese c, et al. effects of fact-checking social media vaccine misinformation on attitudes toward vaccines[j]. preventive medicine,2021,145: 106408.
This article refers to the citation format:
Pan Wenjing. Why fact-checking is "out of the circle"——The enlightenment of a ** experimental study[J].Young Journalists, 2023(23):16-18