Impact factors of forensic science and toxicology journals: what do the numbers really mean?

https://doi.org/10.1016/S0379-0738(03)00042-2Get rights and content

Abstract

This article presents review and opinion about the use and abuse of journal impact factors for judging the importance and prestige of scientific journals in the field of forensic science and toxicology. The application of impact factors for evaluating the published work of individual scientists is also discussed. The impact factor of a particular journal is calculated by dividing the number of current year citations to a journal’s articles that were published in the previous 2 years by the total number of citable items (articles and reviews) published in the same 2-year period. Journal impact factors differ from discipline to discipline and range from 0 for a journal whose articles are not cited in the previous 2 years to 46 for a journal where the average recent article is cited 46 times per year. The impact factor reflects the citation rate of the average article in a journal and not a specific article. Many parameters influence the citation rate of a particular journal’s articles and, therefore, its impact factor. These include the visibility and size of the circulation of the journal including availability of electronic formats and options for on-line search and retrieval. Other things to consider are editorial standards especially rapid and effective peer-reviewing and a short time lag between acceptance and appearance in print. The number of self-citations and citation density (the ratio of references to articles) and also the inclusion of many review articles containing hundreds of references to recently published articles will boost the impact factor. Judging the importance of a scientist’s work based on the average or median impact factor of the journals used to publish articles is not recommended. Instead an article-by-article citation count should be done, but this involves much more time and effort. Moreover, some weighting factor is necessary to allow for the number of co-authors on each article and the relative positioning of the individual names should also be considered. Authors should submit their research results and manuscripts to journals that are easily available and are read by their peers (the most interested audience) and pay less attention to journal impact factors. To assess the true usefulness of a person’s contributions to forensic science and toxicology one needs to look beyond impact factor and citation counts. For example, one might consider whether the articles contained new ideas or innovations that proved useful in routine forensic casework or are widely relied upon in courts of law as proof source.

Introduction

Spreading information to colleagues working in other laboratories or countries and sharing new knowledge by timely publication of research findings or providing details of new method development is fundamental for the advancement and practice of science [1]. Some have said that science does not exist until it is published [2]. The first scientific journal appeared in 1665 (The Philosophical Transactions) and the refereeing of manuscript began in 1752, when the Royal Society of London took over responsibility for this publication [3], [4]. Thousands of scientific journals are now available and many of these have wide circulation and a solid reputation for scholarly excellence. Indeed, a large proportion of manuscripts submitted to these premier journals are rejected because of shortage of space despite receiving favorable peer-review reports from the referees [5], [6]. By contrast, other periodicals have relatively small circulation numbers and some have gained a dubious reputation and seemingly publish every article submitted to them. Those persistent enough will eventually get their work published.

Besides the prestigious multidisciplinary weekly journals (Science and Nature), where only a select few manage to get their articles published, there are hundreds of specialized journals covering all branches of the pure, applied and biomedical sciences as well as the social sciences. Some journals are controlled, edited and published by scientific or professional societies and the membership is encouraged to submit their work for peer-review evaluation and possibly publication [1], [2]. With such a wide selection of journals available for submitting a manuscript, does it really matter where a paper eventually gets published?

The prestige and standing of a particular scientific journal depends on many factors including the reputation of the editor and the editorial standards, the speed of handling manuscripts, the timeliness of publication, the size of the circulation, the potential for on-line search and retrieval, and not least the rigor of the peer-review process [6]. More recently, the journal impact factor (IF) has emerged as an alternative index of quality and prestige and IFs are being used increasingly for ranking and evaluating journals and also as surrogates for judging academic performance and the quality and importance of an individual researcher’s publications [7], [8], [9], [10], [11], [12]. As impact factors are numbers reported with three decimal digits this gives them a mark of quantitative importance and prestige and they are being increasingly used by funding agencies and university search committees to evaluate individual scientists or entire departments [13], [14]. Getting articles published in journals with high impact factor is a major goal for many scientists and the way a journal’s IF fluctuates is closely monitored by publishers, editors, administrators and also by those who submit articles for publication [15]. When evaluating the productivity and scientific output of a university department, when allocating funding for research or when judging candidates for academic promotion, journal impact factors are being increasingly scrutinized [14], [15], [16], [17].

Detailed lists of journal citations and impact factors are produced and made available by the Institute for Scientific Information (ISI), which has its head offices in Philadelphia, PA. A listing of all the impact factors is included in Journal Citation Reports (JCR) and, since 1997, this product has been searchable on-line and this was the source of the data used to prepare the present review.

The impact factor of a scientific journal in a particular year is the ratio of the number of current year citations to articles published in the journal in the two preceding years divided by the number of citable items (articles and reviews) published in the same 2 years [18], [19]. For example, the 2001 impact factor for Forensic Science International (FSI) is the number of citations received by this journal in 2001 to articles published in FSI in 2000 and 1999 divided by the number of articles published in these same 2 years. Impact factors are, therefore, derived from a breakdown of the reference lists attached to the end of each article. This entails counting the number of times a particular target journal’s articles are cited as a function of the year these articles appeared in print.

One important underlying assumption of citation analysis is that citing a particular author’s work establishes a scholarly link or influence on one’s own research work [18], [19]. Journal impact factors range from 0 for the least cited journal to over 40 for some of the most highly cited journals. The median impact factor for all science journals in the ISI database for the year 2001 was about 0.80. In a relatively small discipline such as forensic and legal medicine, the impact factors of the journals are generally lower compared with broader subject categories, such as life sciences, clinical medicine and much lower than hot research topics like genetics and molecular biology [18], [19].

The aim of this article is to review and comment upon the impact factors for certain forensic science, legal medicine and toxicology journals. Many of the variables that influence the calculation of journal impact factors are discussed and some problems and pitfalls are highlighted when impact factors are used to judge the importance of the work of individual scientists.

Section snippets

Trends in impact factors of forensic science and toxicology journals

Table 1 shows the year-by-year changes (1997–2001) in journal impact factors for selected forensic science, legal medicine and toxicology journals. On the whole, impact factors of these specialist forensic science and toxicology journals changed very little over this time period [20]. The toxicology journals seemed to achieve higher impact factors than the forensic and legal medicine journals. Compared with other disciplines covered by ISI, the impact factors shown in Table 1 are relatively low

Subject categories

The Journal Citation Reports is subdivided into different subject categories. The forensic science journals are found within the subcategory “medicine, legal” and make-up a small group of only eight journals (median IF 0.96, in 2001). Moreover, some of the titles included in the list are seemingly not of any direct interest or relevance to mainstream forensic scientists, e.g. Regulatory Toxicology Pharmacology and Expert Opinion Therapeutic Patents. However, one journal that should have been

Citing and cited journal pages

The starting point for compiling journal citation reports and calculating impact factors is the reference lists or bibliographies that appear at the end of each published article in the thousands of journals covered by ISI. For example, reference lists from every article and review paper published in Forensic Science International in 2001 are scrutinized and used to find the total number of references cited. The average citation density is derived by dividing the number of references by the

Adjusting for self-citations

Research workers who publish regularly often cite their own previously published work and also articles by their colleagues who are active in similar branches of research. This is only natural considering scientists tend to build or expand on their earlier research work covering a particular area of interest. An article by one or more authors containing a bibliographic reference to earlier articles by one or more of the same authors is called a self-citation. Such self-citations usually

Long-term versus short-term impact

Some of the critics of journal impact factors maintain that a major flaw is the way that ISI makes the calculations, namely by only considering the citations to articles published in the two previous years. This method is thought to be unfair to slower moving disciplines such as Forensic Science and Toxicology as well as others. However, it is fairly easy to calculate 5- or even 10-year impact factors if considered necessary. The 5-year IF is derived by looking at all current year citations

Citation density and inclusion of review articles

Other things that are considered to influence journal impact factor and complicate comparisons between journals are the citation density and the number of review articles published each year. Citation density is the ratio of total number of references in all the articles published in a particular journal divided by the number of articles. Not many journals set limits on the number of reference items (citations) an author is permitted to include in the manuscript submitted for publication.

Concluding remarks

For some people journal impact factors have become the Holy Grail of scientific publishing [25], [26], whereas, others consider them highly contentious with many artifacts including strong national biases [27], [28], [29], [30]. This criticism is especially justified when IFs are used to evaluate and compare the productivity and importance of the work of individual scientists [31], [32], [33], [34]. There is a low correlation between the impact factor of a journal where an article is published

References (55)

  • D.A Kronick

    Peer-review in 18th century scientific journalism

    JAMA

    (1990)
  • Instructions for authors, Lancet 360...
  • A.S Relman et al.

    How good is peer-review

    New Engl. J. Med.

    (1989)
  • E Garfield

    Journal impact factors—a brief review

    CMAJ

    (1999)
  • K.S Joseph

    CMAJ’s impact factor: room for recalculation

    CMAJ

    (1999)
  • D Schoonbaert et al.

    Citation analysis for measuring the value of scientific publications: quality assessment tool or comedy of errors?

    Trop. Med. Int. Health

    (1996)
  • E Garfield

    Which medical journals have the greatest impact?

    Ann. Int. Med.

    (1986)
  • A.W Jones

    The impact of alcohol and alcoholism among substance abuse journals

    Alcohol Alcohol.

    (1999)
  • E Garfield

    Interview with Eugene Garfield, chairman emeritus of the Institute for Scientific Information (ISI)

    Cortex

    (2001)
  • A.W Jones

    Evaluating the work of forensic scientists by citation analysis

    J. Forensic Sci.

    (1995)
  • G Williams

    Misleading, unscientific, and unjust: the United Kingdom’s research assessment exercise

    Br. Med. J.

    (1998)
  • A.S Wierzbicki et al.

    Total research productivity in a pathology discipline

    J. Clin. Pathol.

    (2002)
  • D Adams

    The counting house

    Nature

    (2002)
  • S.E Gisvold

    Citation analysis and journal impact factors—is the tail wagging the dog?

    Acta Anaesthesiol. Scand.

    (1999)
  • E Garfield

    Dispelling a few common myths about journal impact factors

    Scientist

    (1997)
  • E Garfield

    How can impact factor be improved?

    Br. Med. J.

    (1996)
  • A.W Jones

    JAT’s impact factor—room for improvement

    J. Anal. Toxicol.

    (2002)
  • Cited by (63)

    • Publication trends in forensic science research: Friction ridge discipline

      2020, Forensic Science International: Synergy
      Citation Excerpt :

      Through the use of several metrics, bibliometric reviews reveal the type and impact of research being conducted and they can help inform a strategic direction in a discipline [18]. Alan Jones, forensic toxicologist and author was one of the first to apply bibliometrics to forensic science [19]. One of the metrics calculated is impact factor.

    • Publishing Trends in Toxicology

      2014, Encyclopedia of Toxicology: Third Edition
    • Publishing trends in toxicology

      2023, Encyclopedia of Toxicology, Fourth Edition: Volume 1-9
    View all citing articles on Scopus
    View full text