Research evaluation

Hjørland (1996) argues for a middle position in research evaluation. This middle position tries to avoid two extremes:

• One the one hand the bureaucratic and positivistic approach to evaluation, which assumes that research can and must be measured like any other line of production in society, preferably by quantitative methods.

• On the other hand the view that important research should not be governed at all and that it cannot be measured. In particular this last view holds that science cannot be measured by counting publications or citations, but must be evaluated by peers, if at all.

In arguing for a middle position, Hjørland puts forward seven theses: 

1. Any evaluation of research carries with it some implicit or explicit understandings of science and of knowledge as well as views on the goals of research (and vice versa: Any view of science have implications for, how evaluations should be carried out).

2. Science has a paradoxical problem: On the one side a big demand for new knowledge, and on the other side an overload of publications.

3. Methods for evaluation of research are based on basic assumptions on theory of science, overtly or covertly. Most methods are based on positivist norms. The debate between positivism and its alternatives is essential for understanding the nature of science and for its evaluation. 

4. Bibliometric indicators may throw light on trends in research, but they cannot tell if these trends represent fruitful developments or just populist currents.

5. Criteria for evaluation of research are domain specific.

6. In order to interpret statistical indicators of research it is essential to involve theory history and sociology of science.

7. Library and information science is a potentially important participant in research evaluation because in much of our theoretical and practical work (i.e., classification, document selection) we are concerned with criteria for quality, and information scientists have for a long time been engaged in bibliometric analysis.

Literature:

Alston, J. M.; Norton, G. W. & Pardey, P. G. (1995). Science Under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting. Ithaca, NY: Cornell University Press.

Garfield, E. (1989). Evaluation research - do bibliometric indicators provide the best measures - Introduction to to a review of bibliometrics and other science indicators and their role in research evaluation (Reprinted). Current Contents, 14, APR, 3-10.Available at:  http://www.garfield.library.upenn.edu/essays/v12p093y1989.pdf

Geuna, A. & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41(4), 277-304. 

Hjørland, B. (1996). Forskningsevaluering i videnskabsteoretisk belysning. Biblioteksarbejde, 17, #48, 35-57. (In Danish) Click for full-text;

Seglen, P. O. (1996). Bruk av siteringer og tidsskriftimpaktfaktor til forskningsevaluering. Biblioteksarbejde, 17, #48, 27-34. (In Norwegian)  Click for full-text

See also: Knowledge Production, quality of;

Birger Hjørland

Last edited: 29-03-2006

Home