CNI-ANNOUNCE@cni.org Mailing List Archive

SURF report: Users, Narcissism and Control: Tracking the I

Our friends at the SURF foundation in the Netherlands have just announced a new report that I think will be of interest to the CNI community. I've reproduced the announcement below. The report is at:

http://www.surffoundation.nl/nl/publicaties/Documents/Users%20narcissism%20and%20control.pdf

Clifford Lynch

Director, CNI

-------------------------

Individual researchers benefit from online impact assessment

Online measuring of impact not yet suited for research assessment exercises

Individual researchers are very interested in evidence of the impact of their publications. Research institutes and independent organisations assessing research have a special interest when comparing groups and organisations for research assessment. Thanks to the possibilities of web based publishing it is now possible to gauge the impact of some publications under certain conditions. New information filters and tools are helping researchers to assess their own progress and to find responses of others to their publications.

The report Users, Narcissism and Control, which was funded by SURF, offers a comprehensive overview of the current tracking tools of online publications. The report shows to what extent it is possible to follow in real-time how research results are being downloaded, read, cited, and applied.

Stricter protocols required

The fact that researchers can use these tools does not necessarily mean that this technology is also a legitimate source of information for research assessment. For this application, they need to adhere to a far stricter protocol of data quality and indicator reliability and validity (for example; what does a download imply on the use of the research results). Most new tools do not (yet) comply with these stricter quality criteria required for them to be used in research assessments.

Explosion of tracking tools

The authors of the report 'Users, narcissism and control' Paul Wouters and Rodrigo Costas (CWTS, Leiden) explore the explosion of tracking tools that has accompanied the surge of web based information instruments. A total of 16 quite different tools have been assessed. The authors also highlight the potential risks and disadvantages of the new tracking tools. Just some of the tools discussed are: Google Scholar, Microsoft Academic Research, Total-Impact, PlosONE altmetrics, en F1000.

Frank van Harmelen, professor of Knowledge Representation and Reasoning at the VU University Amsterdam writes on the report: "New web-based metrics for scientific impact will make it possible to observe the developments in science in near real-time. I'm very happy to see that such web-based measures of scientific impact are now being considered by leading scholars involved in the science of measuring and analysing science (Scientometrics)."

Research assessment exercises

The report also recommends to start a concerted research programme investigating the dynamics, properties, and potential use of new web based metrics. This programme should relate these new measures to the already established indicators for scientific and scholarly impact. It can provide insight in how these developing metrics could be applied in research assessment exercises.

Kind regards,

Keith Russell

SURFfoundation, SURFnet en SURFdiensten maken deel uit van SURF, de samenwerkingsorganisatie voor ICT in het hoger onderwijs en onderzoek.