The panel was titled “New Possibilities in Evaluation Metrics: Authors + Altmetrics = ?”. In organizing the panel, I tried to arrange a cross-section of speakers who had experience dealing with both author identifiers and altmetrics. All three of our panelists are frequent speakers on both topics independently of one another, and I thought it would be interesting to hear their thoughts on the relationship between these trends.
In his role as Head of Academic Outreach at Mendeley, our first speaker William Gunn (http://orcid.org/0000-0002-3555-2054) is a major proponent of Mendeley readership statistics as a key new metric. Mendeley is also a major scholarly platform for user-generated researcher profiles. Mendeley has been at the center of these trends since the beginning. For example, one of the earliest experiments in altmetrics at the author-level was Dario Tarobelli‘s ReaderMeter, which was based on Mendeley data. William kicked-off by outlining three key benefits of altmetrics: “Get better data on researcher engagement with research; Get it faster; Serve all the stakeholders in research”
He continued to explore each of these in turn. William used the example of Mendeley to explain how altmetrics are the outputs of researcher (and public) engagement with research. A user storing a document in their Mendeley collection to read also results in a measure of engagement with that document. The Web has meant that not only are many more forms of communication and interaction with scholarly content possible than ever before, but that they are now also measurable. We can thus capture the broadest impact of a researchers traditional and non-traditional works. William spent some time discussing the example of the new PubMed Commons as one place he sees new forms of discussion beginning to flourish. William also touched on the value of altmetrics to improve discovery platforms. William’s talk proved an excellent introduction to the topic and laid a good foundation for the discussion.
Kristi’s talk was titled “Author identifiers & research impact: A role for libraries”. She began by discussing the growing importance of research assessment and posed the question, “How do we measure what matters?” She proceeded to outline the Becker Model and some of the meaningful things that should be measured but aren’t yet. She then spoke about how researcher networking tools and research information systems like SciVal Experts and VIVO are creating the foundation to begin tracking these new forms of impact. She followed this up with the example of VIVO and finished with a description of VIVO’s efforts to integrate with ORCID. Kristi’s primary focus was on practical advice for libraries.
Martin’s presentation focused on the big picture and a look toward the future. He began by pointing out that altmetrics have two goals, both to improve discovery and also assessment. He discussed the importance of linking authors with key institutions and funders as well as with their many forms of outputs, such as articles, datasets, and software. He touched on ORCID, ISNI/Ringgold, Fundref, Datacite and many more identifiers and platforms that are working together to enable this. One of the highlights of the panel was a table Martin introduced to argue that different metrics are used for different use cases.
Following the presentations we had time for some questions. One topic discussed was how to track the impact of research data. The ODIN (Orcid and Datacite Interoperability Network) project was brought up as a great example of ORCID being used in a way that will enable altmetrics. The topic of using altmetrics for improved discovery was also discussed. Each of the panelists had a chance to share their vision for what things might look like 10 years on. Martin closed with a very positive outlook. He believes that we will see significant transformations in discovery and assessment over the next five years as altmetrics are enabled by the uptake of author identifiers.
Following the panel, there was a Scopus 10th Anniversary reception at the booth to wrap things up.
At the 2013 SSP (Society of Scholarly Publishers) Annual conference I was on a panel about research assessment and metrics. As part of my presentation I shared some findings from a large survey conducted by Elsevier’s Research and Academic Relations group (methodology in the slides).
One finding the Twitter back-channel picked up on was the surprising statistic that, from this random sample, only 1% of academics were familiar with altmetrics. I followed this up with a more optimistic statistic showing that both researchers under 35 and also researchers from developing nations were more likely to view different types of potential altmetrics as useful. For this section of the talk, my primary point was that we need to focus on raising awareness among this demographic if altmetrics are to gain legitimacy in the researcher community.