Article,
Final Report on the Investigation into the Feasibility of Developing and Implementing Journal Usage Factors
(April 2007)
Abstract
The objective of this project was to obtain an initial assessment of the feasibility of developing and implementing journal Usage Factors (UFs). This was done by conducting a survey in two Phases. Phase 1 was series of in-depth telephone interviews with a total of 29 authors/editors, librarians and publishers. Phase 2 was a web-based survey in which almost 1400 authors and 155 librarians participated. The feedback obtained has helped determine not only whether UF is a meaningful concept with the potential to provide additional insights into the value and quality of online journals, but also how it might be implemented. The results obtained have also provided useful pointers for the topics to be explored further, if it is decided to take this project further. The apparent eagerness of senior executives to take part in the interviews and the large number of responses to the web survey indicate the high level of interest in journal quality measures in general and the Usage Factor concept in particular.
Based on these results it appears that it would not only be feasible to develop a meaningful journal Usage Factor, but that there is broad support for its implementation. Detailed conclusions and recommendations are provided in Section 4 of this report. Principal among these are:
• the COUNTER usage statistics are not yet seen as a solid enough foundation on which to build a new global measure such as Usage Factor, but confidence in them is growing and they are seen as the only viable basis for UF
• the majority of publishers are supportive of the UF concept, appear to be willing, in principle to participate in the calculation and publication of UFs, and are prepared to see their journals ranked according to UF
• there is a diversity of opinion on the way in which UF should be calculated, in particular on how to define the following terms: ‘total usage’, ‘specified usage period’, and ‘total number of articles published online’. Tests with real usage data will be required to refine the definitions for these terms.
• there is not a significant difference between authors in different areas of academic research on the validity of journal Impact Factors as a measure of quality
• the great majority of authors in all fields of academic research would welcome a new, usage-based measure of the value of journals
• UF, were it available, would be a highly ranked factor by librarians, not only in the evaluation of journals for potential purchase, but also in the evaluation of journals for retention or cancellation
• publishers are, on the whole, unwilling to provide their usage data to a third party for consolidation and for calculation of UF. The majority appear to be willing to calculate UFs for their own journals and to have this process audited. This is generally perceived as a natural extension of the work already being done for COUNTER. While it may have implications for systems, these are not seen as being problematic.
• COUNTER is on the whole trusted by librarians and publishers and is seen as having a role in the development and maintenance of UFs, possibly in partnership with another industry organization. Any organization filling this role must be trusted by both librarians and publishers and include representatives of publishers and librarians.
• there are several structural problems with online usage data that would have to be addressed for UFs to be credible. Notable among these is the perception that online usage data is much more easily manipulated than is citation data.
• should UKSG wish to take this project further there is a strong likelihood that other agencies would be interested in contributing financial support
