You are here: Service / VHB-Jourqual / VHB-JOURQUAL 3 / Accompanying documents / Tips on the responsible use of VHB-JOURQUAL3
Wednesday, October 18, 2017
DE
Links

 

80th Annual Meeting of the VHB

May 23-25, 2017

Magdeburg, Germany

Tips on the responsible use of VHB-JOURQUAL3

The journal rating VHB-JOURQUAL, which, following calls by many association members, has been published by the German Academic Association for Business Research (VHB) for many years, enables the assessment of the reputation of business research journals, especially those not included in other ratings or rankings. This is designed to improve the possibilities of evaluating the quality of journals. VHB-JOURQUAL therefore increases transparency and improves the orientation.  

 

This was also an opportunity for a substantial proportion of the members of our association, our colleagues Thorsten Hennig-Thurau and Henrik Sattler, and the VHB Board to ask for a fresh census – VHB-JOURQUAL3 – to be conducted based on a methodology that had been honed in dialogue with the association members. In order to render this dialogue as transparent and comprehensible as possible, questions and arguments were addressed in various Questions and Answers documents, which were (and still are) available online before and after VHB-JOURQUAL3 was realised.

 

In the current rating, VHB-JOURQUAL3, a journal’s academic quality is defined as the degree to which the journal in question advances business research as an academic discipline. It involves a subjective overall assessment that is naturally composed of different information and experiences for everyone polled (e.g. individual experiences as an author, articles read, experiences of review processes, discussions with colleagues etc.). Pooled from all these polls, it expresses the perception of a journal’s academic quality in the German-speaking academic community. The individual rating is extrapolated from the median of the individual assessments. VHB-JOURQUAL3 therefore provides information on a summarised evaluation by VHB members.

 

Against this backdrop, my fellow members of the VHB Board and I would like to reiterate the importance of a responsible and thoughtful use of such ratings.  

 

The rating is primarily intended as an orientation aid for individual paper submissions. It is particularly important for budding scientists to know what reputation is associated with a publication in a certain journal from the academic community’s point of view. Caution is called for when it is used as a benchmark for a performance evaluation, especially if individual researchers are being assessed. 

 

Any evaluation of research achievements that is limited to a simple addition of the sum of papers weighted according to rating values is problematic. It is important to understand why this is the case. The most important problems include: 

 

  • Scattering: in the long run, a journal’s reputation is formed from the quality of the papers published in it. However, this certainly does not mean to say that all the papers it features are equally good. Instead, their quality is subject to major fluctuations: there are ground-breaking papers that have appeared in journals with a lesser reputation and weak papers without any impact in leading journals. On average, it is the other way round, of course. A journal’s reputation is an (error-prone) indicator of the paper’s quality – no more, no less.   
  • Incompleteness: the rating exclusively refers to publications in journals. Another aspect of business research is published in other media, such as monographs, or is reflected in the development of prototypes. As these research achievements are not recorded in VHB-JOURQUAL, they are not evaluated.
  • Opportunism: the problem whenever achievement assessments are carried out based on a few key figures is that this is not anticipated by those evaluated. In such a situation, it is rational to only optimise the key figures. If this is only linked correlatively to the performance goal actually intended, it can result in substantial undesirable developments. The scientific search for relevant and innovative findings is complex, the evaluations of research achievements – as important as they are – always temporary and prone to error. Researchers who only concentrate on publication successes instead of the knowledge goal might be tempted to achieve one at the expense of the other. This can result in salami publications, deals and any other form of legal and illegal exploitation of opportunistic elbowroom.  

 

Therefore, VHB-JOURQUAL3 is by no means a substitute for the differentiated picture of publication achievements that every colleague should gain. In order to be able to assess the originality and relevance of the issue, the soundness and argumentative stringency of the contribution to theory, and the methodical rigour in the approach to and implementation of modelling and/or empiricism, one actually needs to read a paper. This is the way it is and has always been.    

 

Finally, it should be pointed out that academic research is only ever one sub-area of the extensive duties of university lecturers. Other achievements by scientists, such as teaching, services in self-management and for the researcher community are also important, of course. For this reason, the VHB Board has deliberately and repeatedly spoken out against personal rankings based on journal ratings. This stance remains unchanged and the relevant statements can be accessed here.

 

Moreover, you will find extensive thoughts and considerations on using ratings in VHB’s tips on the ethical and practical vocational orientation of its members, which are available here. In this context, we would especially like to highlight the tips on good practice when using journal and personal rankings. We also invite you to contribute new thoughts and arguments, but also questions on using ratings and rankings, to the discussion on how these and other tips on good practice can be refined. As an association member, there is a forum at your disposal, where you can submit your comments or questions once you have logged on in the Member’s Area.

  

For the VHB Board

Prof. Dr Barbara E. Weißenberger

Prof. Dr Birgitta Wolff

 

For the JOURQUAL Advisory Board

Prof. Dr Harald Dyckhoff

Prof. Dr Nikolaus Franke

Prof. Dr Georg Schreyögg