When impact factor matters less

Average number of citations is not the most important indicator for supporting medical guidelines in Germany, study suggests

Editorial staff

What is the best way to determine the most appropriate studies for guiding medical practice? For the Association of the Scientific Medical Societies in Germany (AWMF), which establishes the country’s diagnosis, prevention, and treatment guidelines, it is not the impact factor of the journals that published them, but how utilizable the research results are.

In a study published in the journal Scientometrics, researchers from the University of Göttingen in Germany showed that the two journals with the most articles cited in 70 medical guidelines from the years 2017 and 2018 have a much lower impact factor than so-called high-impact journals.

A journal’s impact factor is calculated by dividing the total number of citations received by all the articles it has published in the last two years by the total number of articles. For example, if a journal published 100 articles and they were cited 100 times, its impact factor would be 1.

According to the study, less than 10% of the citations in the guidelines referred to scientific articles published in the previous two years and more than half were published at least eight years ago. The result suggests that the two-year cycle used to calculate whether a study remains relevant and impactful does not apply to clinical practice.

The second most cited journal in the guidelines, for example, was ranked 40^th in terms of impact factor. The 100 journals most cited in the guidelines accounted for 57.95% of the citations in the documents, written to guide medical practice in Germany.

Titles ranked below 11^th in number of citations in the guidelines each represented less than 1% of citations, while those below 100 each accounted for an average of 0.192% of the total citations in the guidelines.

“When looking at the guidelines as a whole, there was only a weak correlation between guideline citation numbers for particular journals and their impact factors […]. References in the guidelines maintain usefulness regardless of their impact factor,” wrote the authors.

As an example, they cited the Cochrane Database of Systematic Reviews, which has a moderate impact factor of 6,754 but was cited 811 times, the second highest. At the other end of the spectrum, the CA: A Cancer Journal for Clinicians has a high impact factor (244,585) but is cited just 18 times in the guidelines, not even making the top 100 most-cited journals.

“This suggests that authors of the guidelines scan through hundreds of different sources, independent of impact factor, to meet their needs,” state the researchers. The authors suggest that being cited in medical guidelines should itself be a factor when evaluating the importance of a scientific article and studies that can be used in medical practice should be encouraged.

* This article may be republished online under the CC-BY-NC-ND Creative Commons license.
The text must not be edited and the author(s) and source (Science Arena) must be credited.