SAJSM EDITORIAL.indd 62 SAJSM vol. 25 No. 3 2013 EDITORIAL Evaluating the quality of science in exercise and sports medicine With the increasing ease with which information can be obtained from the internet, it is becoming more important to be able to discern ‘good quality’ information from ‘not-so-good’ information. This even applies to scientists who are accustomed to the process of peer review. Information on the internet often breaks from convention where scientists aspire to have their research papers published in high- impact journals. The practice of striving to publish in high-impact journals is based on the assumption that high-quality research papers are associated with high impact factor (IF) journals. However, this assumption is not always fulfilled. This is because the IF of a journal was originally developed by librarians to rank journals; it was not designed as a marker of the quality of the research papers published in the journal. The IF is merely a calculation of the ratio between the number of citations of papers and the number of papers published in a defined period.[1] For example, the numerator of the ratio is the number of current-year citations (e.g. citations made in 2012) and the denominator is the total number of published items in that journal in the previous two years (i.e. 2010 and 2011). An example follows: In 2012, all the papers in a particular journal were cited 185 times from papers published in 2010 (n=70 papers) and 2011 (n=75 papers); therefore, the IF for 2012 is 185 ÷145 = 1.28. The IF has been used to rate the quality of the work of scientists who have to compete for grant funding, or who have to justify reasons for promotion within their academic institutions. However, there is growing concern about the use of the IF. Firstly, it is misguided to assume that all papers published in journals with high IFs are good studies. Some studies might have slipped through the peer review process of the journal, and only in hindsight, may have been exposed as having arrived at the wrong conclusions. Secondly, the calculation of the IF of the journal also depends on the popularity of the field of research. It stands to reason that journals representing areas of research that attract fewer papers are going to have lower IFs than journals representing areas of research with a global interest, such as HIV or tuberculosis. As a consequence of the unhappiness about this situation in the scientific community, a group of editors and publishers of scholarly journals met at the end of 2012 in San Francisco. The group developed a set of recommendations referred to as the San Francisco Declaration on Research Assessment.[2] This document acknowledges that the outputs from scientific research are varied and go beyond just research papers in journals. Other factors and outputs such as developing databases and software, refinement of technology, intellectual property and training of young scientists should all be considered when the output of a scientist is evaluated. The group recommended that journal-based metrics such as the IF should be discontinued, and that research should be assessed on its own merits, rather than on the basis of the journal in which researchers publish. In the field of exercise science and sports medicine, a scientist’s contribution should be evaluated on how service delivery and treatments have been affected as a result of experimentation, whether the original work was conducted in the laboratory or in an applied setting. Outputs can also be evaluated by the impact of the research on policy or their contribution to principles of best practice. Sometimes the work of an applied scientist is not published in the formal sense, but undergoes sufficient peer review before the principles are utilised in the management of athletes or players. In this setting, the environment is so competitive and demanding that if the recommendations of the scientist do not have any discernible effect, then they will not be utilised by the practitioners or support staff. This is just another check and balance that evaluates the original work done by scientists in the field of exercise science and sports medicine. Clearly, this is far more important than the IF of the journal in which the paper is published. The publication of this edition of the South African Journal of Sports Medicine coincides with the 15th Biennial Congress of the South African Sports Medicine Association (SASMA), which will be held at the Wild Coast Sun from 24 to 27 October 2013. The organising committee, headed by Dr Glen Hagemann, SASMA President, has done a magnificent job in compiling a high- quality programme. Mike Lambert Editor-in-Chief 1. Eston R. The impact factor: A misleading and flawed measure of research quality. J Sport Sci 2005;23(1):1. [http://dx.doi.org/10.1080/02640410400014208] 2. San Francisco Declaration on Research Assessment. http://am.ascb.org/dora/ (accessed 23 September 2013). S Afr J SM 2013;25(3):62. DOI:10.7196/SAJSM.509