Using Altmetrics: Experiences From the Front Line
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Roelandse, Martijn
Naish, Paul
Wright, Stuart
Gorraiz, Juan
- What can a journal do with altmetrics? - Using Altmetrics to Encourage and Track the Dissemination of a Potentially High Impact Paper - Monitoring the broad impact of the journal publication output on a country level: A Case Study for Austria
 Using other tricks and experiences from online. Here one speaker who is at online, he's not here because he has a gas leak. It's a slightly different content but we have a long talk here. Or we have a long flash. Good. The first speaker is Paul and he will talk about what can you do with online with other tricks. I'm Paul Nish, I'm from Tainan, France. I'm a partnership working in the R8 Toro team. I've got a lot of experience about what we've been doing with our metrics. We've looked at a lot of metrics today, the research behind it, but this is really about telling you a story about what we've done. Some papers have had high metrics and how we've analyzed that and how we've responded to that. So, on to the first slide. This is actually not our logo. It's our old logo from many years ago when we launched back in 1700. You can see the new one when we rebranded it much, much more recently. I think one of the interesting things here is the motto, the Larry Flamat, which means to found the flames. Nothing to do with burning, hopefully. It's really about finding the flames of science and research and that's kind of the ethos we have there. I think only one of our journals has this logo on and you'll see that later. We're kind of using our metrics in that regard here and that's how I'm talking about it anyway. So, I'm going to show you three case studies and these are little stories about journals and papers, individual papers, which have high metrics. We've monitored those metrics. We noticed them and we've decided to do something with them. Each of them we've done something different and they're quite nice. The first one is one of our flagship journal, Journal of Science, which is a journal in history of science. You can see the logo down there. It's a very old, very esteemed journal. I've caught a new share of history of science, very broad interest. Obviously, we introduced our metrics a few years ago and we monitor both as a company but also on an individual journal level metric. We get weekly updates and occasionally things pop up for whatever reason. So, a few years ago, this paper from 1962 by Sidney Ross, which is a story of a word, which is really about where the word scientist came from, the origination of that. It started very quickly, very young, unusually, just getting a spine from the old metrics. We interrogated them as a title. And if I got often, it's based on Twitter. So, in this case, we looked at tweets, we looked at all the other metrics, but in this case it was Twitter. There was a trigger tweet. In this case, it was from Rebecca Higgett, who's a well-known historian of science. After that tweet, there was a lot of online interest and there were a few responses there and it went up quite a way. And it's still getting tweets this day. But what do we do with that? What can you do with a lot of tweets to a seminar that's called the one-time-go? We decided to make it free because there was so much interest in this and it was the sort of paper that people were going to a lot online as a reference source. Also, we did a social media campaign based on that, but that's kind of an initial reaction. It's the sort of thing you can do very quickly as a publisher. We also sat down with the editors and said, okay, not just this paper, but a lot of other papers in this journal have a lot of interest, why their interest, not just within the history of science, but broadly into the scientific community and beyond. So we sort of thought, how can we capitalise on this to boost the journal and to actually communicate with the science more broadly? We came up with the term, the classic paper term with that. So what we did was initiate a series of follow-up articles, which are really within this journal, an occasional series of papers. This is written by the editor, just looking at the impact of that particular article. This is the first of the series. So it's the story of, the story of, which is getting a bit confusing. And obviously that had its own, a polymetric pattern, as I've put a few weeks ago. And it gave us the opportunity to market the journal further, both in terms of this paper, the original paper, but other papers around it. So it got a lot of attention and also communicating a really interesting paper about where the word scientist comes from. Now the outcomes here, this is by the start of the small paper, it's in the thousands. The next paper I'm going to talk about is going to be in the tens of thousands. The third paper is going to be in the hundreds of thousands. Sound a little bit like I'm talking about immigration titles, but not. So moving on, the case study two, again, this is another paper which surprised us a little bit, but also didn't surprise us. This is from a journal of experimental direct-class artificial intelligence. Most of the research here is very, very technical. It's talking about swarm analysis and various AI things. But one paper popped up a few years ago, again by the author, in the field. And to sort of cut a long story short, this is about chess robots and the AI uprising, which is clearly going to happen soon. And again, we monitored a lot of metrics and it did really, really well. And almost from day one, there was a lot of comment here. A really interesting comment you can see here, people are talking about robots taking over the world. Will there be a robot uprising, Skynet, chess, that sort of thing. So yeah, a lot of interesting conversation going on on Twitter and we got high on metrics. So what do we do? We decided that the type of research was very communicable in a visual format. So obviously, we did a comic. So this is a recent endeavor as a company. Very occasionally, very interesting piece of research, we turned into what we call a cartoon abstract or comic abstract. It's the one page, very simple, effectively a graphic model. And we combined that with such things as a press release, a screen, supplemental material allocated to the general page. And obviously a social media campaign. So we wrapped that up and it's usually boosted the research. It's very interesting, but it communicated beyond the original scientific community way beyond that. It massively boosted it on social media. As an aside, again, our cartoon abstracts, they actually won the war, strangely enough. And these are some of the sort of things that we've done. So they're the one about open plan offices, making it ill. So the global zombie attack, which is actually a pure math paper, couldn't come a reality. Physics there, I don't know what happened there, but it's very interesting. Maps, TV commercial, all that sort of thing. But it's something again, this is something we do, just as one way of communicating quite interesting research in a visual format. It broadens the research way beyond the actual scientific community. So it's a very interesting site. But in terms of outcomes, how did this do? It really well. So from the initial Twitter spurch, so to speak, we've got a lot of sites, a lot of views. I think over 20,000 views since we did this campaign. So it's really boosted not just that paper, but that journal beyond its community. Even policy. You can go and actually look at those. In terms also, I've got one more way. I'm going to look at a third case that here. Again, this is a sort of a line of thing. This is from a very big journal, The American Statistician, a very prestigious journal. And recently they issued a statement on p-values. And we knew a little bit about this coming up, but it's really, for those who aren't statisticians, it's an important statistical model which is often misused and is very controversial within statistics. Because statistics is such an important subject beyond statistical community. We know about this. We know its impact. So we've got a lot of marketing head-bit. Obviously we will monitoring our metrics quite closely. After it published, there was a lot of interest on Twitter. But because it was such a seminal important statement, we monitored more news and blogs. And these drove the old metrics a little more. And yeah, it was, I think the thing about monitoring this paper, we knew it could be big. But the old metrics reaffirmed what we thought. So we knew there could be things that we do afterwards. And we did not do that important. We would probably not do as much activity afterwards. So what do we do? First up, like I said, it's got hundreds of thousands of reads and a lot of interest. And it's been cited in many, many journals across the lines. As we went and interviewed the executive director of the American Statistical Association about why that statement was made in the first place, the statement itself and its subsequent impact. And we also did a video with Ron. There he is. Again, talking about the same sort of thing. And Q&A, a Twitter Q&A, a follow-up. And this is really to talk about, to communicate via social media where a lot of interest for that paper was occurring. The outcomes of this were massive. And a lot of this would have happened anyway. But I think the activity would have boosted it. It's got hundreds of thousands of views. The amount of citations was huge. And you can see it's had wide impact. The difficulty here was actually communicating to the editors of the journal that in a year or two their impact factors were going way down because you've not done a statement on the history year or two. So that's a third example. I'd like to sort of bring that all together and summarize what we do. Those are just snapshots, really. But I think using art metrics, it's a tool in a toolbox. We use it as part of a wider journals communication strategy where monitoring is really, really important. We monitor widely, pretty much every paper. We obviously, very published agency, positive and negatives, and you can feel different, you can feel both of those. But we also monitor, in terms of editorial work on a day-to-day basis, looking at different journals, different papers and seeing what's happening. I think that's really not just looking at the broad picture, but the individual picture and what you can do as an editor to boost your journal or to react to what's going on in the community. So it's a tool both for impact but for general interest as well, because I think most scientists also want other research to be read widely beyond the actual community. And I think the different examples I've shown show that there are different responses you can do based on the content and the community. So it's not one-size-fits-all, let's do a press release because that did well on Twitter. I don't think it's that, I think what we do with this is the most appropriate and will have the most impact and will fit well with that journal and that community. But as you're seeing, and these are just some examples, I've taken not a random, but there are lots more of these. It can really boost impact and research. I think that's good for everybody. Thank you. For our next talk, we will go to Igor and I will talk about Open I'd love to explore on the essential impact and policy outreach. Okay. My name is Juan Boca. I come from the University of Vienna. I talk about the University Biometrics Department. Also called Publication Strategies is a part of the library. And that is which services we are offering. So we are offering as well services for researchers as for the administration, as for the library, I even went for a standard task. Why for researchers, because I think if we are able to evaluate scientists, we should be also able to tell the scientists how to do it better. We think that bibliometrics is very important for almost young scientists in order to optimize the application strategies. So we are offering these services. We are also offering the bibliometric reports for individual evaluations, faculty evaluations, but one very important part of our services at the University are the monitoring services. We are also trying to offer monitoring to try to follow how, for example, the affiliations policies are moving or the open access policies moving at all institutions. And today I think we should face the new challenges of the new metrics. We are offering also the possibility of monitoring the institutional web impact, rather say the attention in web, and to see how it's also changing for the institution or as I have chosen for this presentation for the country for us. If we want to use something and in order to measure the impact, we should also know how big is the level of adoption or acceptance in the scholar community. And that is very, very different not only according to the different disciplines, but also according to the users. If young scientists need to promote, but then your professors say, what I have to promote, what I do, you tell me to do it, I don't need to do it at all. In order to find these things, we have two possibilities to carry out service. The service are taken by scientists because they have no time to talk with each other, so they have no time for service. And normally people participating in service are always engaged people and giving a very bad idea of what happened. And the other possibility is just to follow what happened and to see, okay, there are more tweeters or there are increasing or what happened at different levels. Good. For this little presentation, we were using, first of all, there are a lot of restrictions in this analysis. The first one is that we are only analyzing journal articles and journal articles in the context of science. So, you know, really, I don't need to say too much, but there are a lot of restrictions and we wanted just to see how does it work if we can do something or not. And which disciplines are moving or not, which disciplines will be important for our country or for our institution or not. And at the end, always because we're geometricians of correlations, we are making some correlations, even if we are not so interesting. But that's okay with correlations. Okay. Good. What we are using on the one side is that we are using the web of science or the complete collection. And on the other side, we are using the mix. Actually, I love this tool because they offer you the possibility not only to measure social media, but you can have as well usage data. So, I think that even as you will see, some disciplines match more interesting as altmetrics. I say like this because for me, as a librarian, usage metrics is nothing not the same as altmetrics. It's much older and it's not a new thing, I believe, today. Good. And samples, we were using the publications of the years 2014, 15, 16, 17, we are doing at the moment. I know you saw some trends, but not more. And the big first restriction is if you want to be traced, you want the good to be well traced and that publication is not moving. You need a good identifier. So, the DOIs are opening you the door to be traced in the altmetrics. And so, we were also seeing how the number of publications with DOI are increasing in the databases for the institution and in which I ask if it happens or not. I will return to this later. What's up with that? We were taking all the publications in the web of science with the DOI and putting them in the plan mix. It's very easy to do it and we find already when we are using the DOI, of course, very high accuracy, data accuracy, as it was almost 1990% in this case. And what we were doing because if we were using the classification of web of science, we will have more than 240 categories and you will not see anything at all. So, we were reducing to six categories, life sciences, physical science, engineering, technology, health science, social science and arts and humanities. When the first results, what you get in plan mix are a lot of measures, a lot of different measures, but at the end, if you see there are only very few significant. At the end, you have only citations, you have downloads and views, you have twitters, you have captures in MNAA and you have not too much more. We will see on the way mentions, we were expecting more, but that's coming because the media are not really well represented and are more of national character. And the second thing is that the distributions as well as how things in the citations are very skewed, so almost there are always few modifications getting a lot of things, but the other ones have not at all. What we will find is, one problem that we have is the window, the measuring window was the end of 2017, so of course all the publications of 2014 had much more time to be cited, to be downloaded or to be something. So it is a decrease, very clear for the number of citations and usage. And very important is what we were measuring. As you see, we were seeing how many items we have, then we were measuring the degree of availability or coverage, is how many of the publications had at least one scoring one of the tours. The intensity is just the number total of scores or signals and then the density is just the number, the intensity divided by the number of items with at least one scoring. Good to the specter decrease, the opposite was for mentions on social media, even if the window was decreasing, they were increasing. And the publications with social media increased in Austria from 28% to the degree of availability, the percentage of publications was increasing almost in 10 points. And if we are following this in 2017, we are already by 40 to 43%. Even the twitch intensity was increasing, the intensity is not so expected, but it was also increasing in the last day. The percentage of publications we mentioned was increasing but much more slowly. And the highest coverage was provided always by the number of readers in the middle age, which is almost very well known because it has been reported. And what is very, very important, again, is the usage data, very, very important and not many, they are not traced or not considered in a lot of analysis. And now we are showing the results for the six categories we were using, these are the results and to win time I will try to make a summary of the most interesting results. The first one was the number of readers in all the areas. Very interesting was the number of readers in other humanities, but not for usage. Of course, if you are working in Web of Science, the number of publications of the other humanities is lower, but this should be very, has a very big impact because they are only the things in English and in some language that should attract more scores or more tweets or more other things. So very important, of course, the citations were not playing in any role as expected, but the users data were as high as in other disciplines. If we were analyzing the behavior of the hard sciences, engineering, technology, health science, life science and physical science was very similar everywhere, as well as social media, whereas expected health and life science and the context for the high-nest cover us. And after we are getting the publications of the social sciences and after that we will see data is the physical science. The present as well is for the social science, it's supposed to be the one for the ones for the hard sciences and the ones for other humanities, but the extent by the social media. The log over us was for the physical science and very important, it was a relatively insignificant also in the arts and humanities. And density research were very good correlation with the cover us or the degree of availability and the tweets intensity has been increasing, as I've already said. Then we were performing some correlations and we got very quick. First we were working with Web of Science and with Plumbing, so it was a very easy way to make correlations of citations between SCOPUs and Web of Science. The problem that we have is that the citations window was very short, so it is very easier to get the high correlations. The correlation with cross-ray was also high, but the number of citations was not as high as in SCOPUs sometimes. Then we were making the correlations between Web of Science and Plumbing, considering the usage data. Web of Science called it usage data, actually it is just a number of clicks of abstracts of publications. And the correlation with the data that we have used in SCOPUs was very slow and something it is hinting at the fact that the user groups are quite different. The user groups use it in both data sources and have different interests. Then correlation between different type of measures we have made for readers in Mendeley, Citations, Tweeters, and Usage views and PDS. It was very slow, only CACCHUS was correlating with citations a little bit but not much. And then at the end we tried also to make correlations with the number of authors and affiliations of the publications and it was also not existing. The game for CACCHUS was the highest correlation. Generic conclusions, okay, monitoring practices are very important tasks and I think it is a new task for libraries or for universities in order to see how the things are moving. And these little exercises were showing us that social media and particularly Twitter usage is getting broader in Austria at least. And Plamix was a very good instrument in order to make these exercises with only some problems because in Plamix you cannot fix the measuring window so you will get all the things until today. For example, if you want to say that you can control the publications years of the publications but not years of the scores are falling to the publications and this will be very important in order to see how it is moving. And also we were making some analysis if this data is stable or not. And at the end yes correlations should be taken with a pinch of salt because there are different of salt essence. But that is a very important thing when we are using citations in order to evaluate or to reward some people you need to understand that we need some time and that is dependent on the half life of the discipline and sometimes you need to learn. But the other ones are moving quite quicker as usage data or Twitter are asked at the beginning until they are the same things just the opposite of salt essence training. Well we were also not only making the monitoring we were also making a survey and we were focusing on the humanities. We made this in the University of Vienna and the University of Nevada. We were only asking professors. We were not asking too much and we were asking if they were interested in following these things or not. And you see 50% of the people almost always yes and 50% say not at all I have no interest. We can't go ahead with the aestheticism or something like this but it was the result when you were asking. What I have done before was just the monitoring of journal publications but what happens for example in the humanities is we are looking at the books and we make analysis with our book collection in ESCO. And you see that for books the most important indicators are not only usage data but also very important. We have catalog holdings and the reviews. We have reviews of good reads, the number of reviews in Amazon or in some different tools and these are very relevant for books. And just at the end because this question was today also coming but we are making bibliometric profiles and there are a lot of people saying we are not using bibliometrics but we are using unmetrics but it's a completely nonsense because exactly bibliometrics to the finish you need to use any kind of quantitative indicators. And unmetrics is a part of bibliometrics of course. It's a new part, it's a new metric but it's not a new discipline different than bibliometrics. And we are making a bibliometric profile for a worldwide person. We are using the number of publications, the disability, the impact, the operation, the reference analysis and the focus path. Even if it is not included, it's another problem, we are going to, we are half calling, we are attention and you can find the most interesting things, what happened and this can be sometimes a help. Okay? Just to finish, if we are asking us if we can use unmetrics for academic evaluation, the Raspberry's computer says no. Thank you very much for your relation and happy to answer your questions. Thank you.
 Using other tricks and experiences from online.  Here one speaker who is at online, he's not here because he has a gas leak.  It's a slightly different content but we have a long talk here.  Or we have a long flash. Good.  The first speaker is Paul and he will talk about what can you do with online with other tricks.  I'm Paul Nish, I'm from Tainan, France. I'm a partnership working in the R8 Toro team.  I've got a lot of experience about what we've been doing with our metrics.  We've looked at a lot of metrics today, the research behind it, but this is really about telling you a story about what we've done.  Some papers have had high metrics and how we've analyzed that and how we've responded to that.  So, on to the first slide. This is actually not our logo.  It's our old logo from many years ago when we launched back in 1700.  You can see the new one when we rebranded it much, much more recently.  I think one of the interesting things here is the motto, the Larry Flamat, which means to found the flames.  Nothing to do with burning, hopefully.  It's really about finding the flames of science and research and that's kind of the ethos we have there.  I think only one of our journals has this logo on and you'll see that later.  We're kind of using our metrics in that regard here and that's how I'm talking about it anyway.  So, I'm going to show you three case studies and these are little stories about journals and papers, individual papers, which have high metrics.  We've monitored those metrics.  We noticed them and we've decided to do something with them.  Each of them we've done something different and they're quite nice.  The first one is one of our flagship journal, Journal of Science, which is a journal in history of science.  You can see the logo down there.  It's a very old, very esteemed journal.  I've caught a new share of history of science, very broad interest.  Obviously, we introduced our metrics a few years ago and we monitor both as a company but also on an individual journal level metric.  We get weekly updates and occasionally things pop up for whatever reason.  So, a few years ago, this paper from 1962 by Sidney Ross, which is a story of a word, which is really about where the word scientist came from, the origination of that.  It started very quickly, very young, unusually, just getting a spine from the old metrics.  We interrogated them as a title.  And if I got often, it's based on Twitter.  So, in this case, we looked at tweets, we looked at all the other metrics, but in this case it was Twitter.  There was a trigger tweet.  In this case, it was from Rebecca Higgett, who's a well-known historian of science.  After that tweet, there was a lot of online interest and there were a few responses there and it went up quite a way.  And it's still getting tweets this day.  But what do we do with that?  What can you do with a lot of tweets to a seminar that's called the one-time-go?  We decided to make it free because there was so much interest in this and it was the sort of paper that people were going to a lot online as a reference source.  Also, we did a social media campaign based on that, but that's kind of an initial reaction.  It's the sort of thing you can do very quickly as a publisher.  We also sat down with the editors and said, okay, not just this paper, but a lot of other papers in this journal  have a lot of interest, why their interest, not just within the history of science, but broadly into the scientific community and beyond.  So we sort of thought, how can we capitalise on this to boost the journal and to actually communicate with the science more broadly?  We came up with the term, the classic paper term with that.  So what we did was initiate a series of follow-up articles, which are really within this journal, an occasional series of papers.  This is written by the editor, just looking at the impact of that particular article.  This is the first of the series. So it's the story of, the story of, which is getting a bit confusing.  And obviously that had its own, a polymetric pattern, as I've put a few weeks ago.  And it gave us the opportunity to market the journal further, both in terms of this paper, the original paper, but other papers around it.  So it got a lot of attention and also communicating a really interesting paper about where the word scientist comes from.  Now the outcomes here, this is by the start of the small paper, it's in the thousands.  The next paper I'm going to talk about is going to be in the tens of thousands.  The third paper is going to be in the hundreds of thousands.  Sound a little bit like I'm talking about immigration titles, but not.  So moving on, the case study two, again, this is another paper which surprised us a little bit, but also didn't surprise us.  This is from a journal of experimental direct-class artificial intelligence.  Most of the research here is very, very technical.  It's talking about swarm analysis and various AI things.  But one paper popped up a few years ago, again by the author, in the field.  And to sort of cut a long story short, this is about chess robots and the AI uprising, which is clearly going to happen soon.  And again, we monitored a lot of metrics and it did really, really well.  And almost from day one, there was a lot of comment here.  A really interesting comment you can see here, people are talking about robots taking over the world.  Will there be a robot uprising, Skynet, chess, that sort of thing.  So yeah, a lot of interesting conversation going on on Twitter and we got high on metrics.  So what do we do?  We decided that the type of research was very communicable in a visual format.  So obviously, we did a comic.  So this is a recent endeavor as a company.  Very occasionally, very interesting piece of research, we turned into what we call a cartoon abstract or comic abstract.  It's the one page, very simple, effectively a graphic model.  And we combined that with such things as a press release, a screen, supplemental material allocated to the general page.  And obviously a social media campaign.  So we wrapped that up and it's usually boosted the research.  It's very interesting, but it communicated beyond the original scientific community way beyond that.  It massively boosted it on social media.  As an aside, again, our cartoon abstracts, they actually won the war, strangely enough.  And these are some of the sort of things that we've done.  So they're the one about open plan offices, making it ill.  So the global zombie attack, which is actually a pure math paper, couldn't come a reality.  Physics there, I don't know what happened there, but it's very interesting.  Maps, TV commercial, all that sort of thing.  But it's something again, this is something we do, just as one way of communicating quite interesting research in a visual format.  It broadens the research way beyond the actual scientific community.  So it's a very interesting site.  But in terms of outcomes, how did this do?  It really well.  So from the initial Twitter spurch, so to speak, we've got a lot of sites, a lot of views.  I think over 20,000 views since we did this campaign.  So it's really boosted not just that paper, but that journal beyond its community.  Even policy.  You can go and actually look at those.  In terms also, I've got one more way.  I'm going to look at a third case that here.  Again, this is a sort of a line of thing.  This is from a very big journal, The American Statistician, a very prestigious journal.  And recently they issued a statement on p-values.  And we knew a little bit about this coming up, but it's really, for those who aren't statisticians,  it's an important statistical model which is often misused and is very controversial within statistics.  Because statistics is such an important subject beyond statistical community.  We know about this.  We know its impact.  So we've got a lot of marketing head-bit.  Obviously we will monitoring our metrics quite closely.  After it published, there was a lot of interest on Twitter.  But because it was such a seminal important statement, we monitored more news and blogs.  And these drove the old metrics a little more.  And yeah, it was, I think the thing about monitoring this paper, we knew it could be big.  But the old metrics reaffirmed what we thought.  So we knew there could be things that we do afterwards.  And we did not do that important. We would probably not do as much activity afterwards.  So what do we do?  First up, like I said, it's got hundreds of thousands of reads and a lot of interest.  And it's been cited in many, many journals across the lines.  As we went and interviewed the executive director of the American Statistical Association  about why that statement was made in the first place, the statement itself and its subsequent impact.  And we also did a video with Ron. There he is.  Again, talking about the same sort of thing.  And Q&A, a Twitter Q&A, a follow-up.  And this is really to talk about, to communicate via social media where a lot of interest  for that paper was occurring.  The outcomes of this were massive. And a lot of this would have happened anyway.  But I think the activity would have boosted it. It's got hundreds of thousands of views.  The amount of citations was huge. And you can see it's had wide impact.  The difficulty here was actually communicating to the editors of the journal that in a year or two  their impact factors were going way down because you've not done a statement on the history year or two.  So that's a third example. I'd like to sort of bring that all together and summarize what we do.  Those are just snapshots, really.  But I think using art metrics, it's a tool in a toolbox.  We use it as part of a wider journals communication strategy where monitoring is really, really important.  We monitor widely, pretty much every paper.  We obviously, very published agency, positive and negatives, and you can feel different, you can feel both of those.  But we also monitor, in terms of editorial work on a day-to-day basis, looking at different journals, different papers  and seeing what's happening.  I think that's really not just looking at the broad picture, but the individual picture  and what you can do as an editor to boost your journal or to react to what's going on in the community.  So it's a tool both for impact but for general interest as well, because I think most scientists also want other research to be read widely  beyond the actual community.  And I think the different examples I've shown show that there are different responses you can do based on the content and the community.  So it's not one-size-fits-all, let's do a press release because that did well on Twitter.  I don't think it's that, I think what we do with this is the most appropriate and will have the most impact and will fit well with that journal and that community.  But as you're seeing, and these are just some examples, I've taken not a random, but there are lots more of these.  It can really boost impact and research.  I think that's good for everybody.  Thank you.  For our next talk, we will go to Igor and I will talk about Open I'd love to explore on the essential impact and policy outreach.  Okay.  My name is Juan Boca.  I come from the University of Vienna.  I talk about the University Biometrics Department.  Also called Publication Strategies is a part of the library.  And that is which services we are offering.  So we are offering as well services for researchers as for the administration, as for the library, I even went for a standard task.  Why for researchers, because I think if we are able to evaluate scientists, we should be also able to tell the scientists how to do it better.  We think that bibliometrics is very important for almost young scientists in order to optimize the application strategies.  So we are offering these services.  We are also offering the bibliometric reports for individual evaluations, faculty evaluations, but one very important part of our services at the University are the monitoring services.  We are also trying to offer monitoring to try to follow how, for example, the affiliations policies are moving or the open access policies moving at all institutions.  And today I think we should face the new challenges of the new metrics.  We are offering also the possibility of monitoring the institutional web impact, rather say the attention in web, and to see how it's also changing for the institution or as I have chosen for this presentation for the country for us.  If we want to use something and in order to measure the impact, we should also know how big is the level of adoption or acceptance in the scholar community.  And that is very, very different not only according to the different disciplines, but also according to the users. If young scientists need to promote, but then your professors say, what I have to promote, what I do, you tell me to do it, I don't need to do it at all.  In order to find these things, we have two possibilities to carry out service.  The service are taken by scientists because they have no time to talk with each other, so they have no time for service.  And normally people participating in service are always engaged people and giving a very bad idea of what happened.  And the other possibility is just to follow what happened and to see, okay, there are more tweeters or there are increasing or what happened at different levels.  Good. For this little presentation, we were using, first of all, there are a lot of restrictions in this analysis. The first one is that we are only analyzing journal articles and journal articles in the context of science.  So, you know, really, I don't need to say too much, but there are a lot of restrictions and we wanted just to see how does it work if we can do something or not.  And which disciplines are moving or not, which disciplines will be important for our country or for our institution or not.  And at the end, always because we're geometricians of correlations, we are making some correlations, even if we are not so interesting.  But that's okay with correlations.  Okay. Good. What we are using on the one side is that we are using the web of science or the complete collection.  And on the other side, we are using the mix. Actually, I love this tool because they offer you the possibility not only to measure social media, but you can have as well usage data.  So, I think that even as you will see, some disciplines match more interesting as altmetrics.  I say like this because for me, as a librarian, usage metrics is nothing not the same as altmetrics.  It's much older and it's not a new thing, I believe, today.  Good. And samples, we were using the publications of the years 2014, 15, 16, 17, we are doing at the moment.  I know you saw some trends, but not more.  And the big first restriction is if you want to be traced, you want the good to be well traced and that publication is not moving.  You need a good identifier.  So, the DOIs are opening you the door to be traced in the altmetrics.  And so, we were also seeing how the number of publications with DOI are increasing in the databases for the institution and in which I ask if it happens or not.  I will return to this later.  What's up with that? We were taking all the publications in the web of science with the DOI and putting them in the plan mix.  It's very easy to do it and we find already when we are using the DOI, of course, very high accuracy, data accuracy, as it was almost 1990% in this case.  And what we were doing because if we were using the classification of web of science, we will have more than 240 categories and you will not see anything at all.  So, we were reducing to six categories, life sciences, physical science, engineering, technology, health science, social science and arts and humanities.  When the first results, what you get in plan mix are a lot of measures, a lot of different measures, but at the end, if you see there are only very few significant.  At the end, you have only citations, you have downloads and views, you have twitters, you have captures in MNAA and you have not too much more.  We will see on the way mentions, we were expecting more, but that's coming because the media are not really well represented and are more of national character.  And the second thing is that the distributions as well as how things in the citations are very skewed, so almost there are always few modifications getting a lot of things, but the other ones have not at all.  What we will find is, one problem that we have is the window, the measuring window was the end of 2017, so of course all the publications of 2014 had much more time to be cited, to be downloaded or to be something.  So it is a decrease, very clear for the number of citations and usage.  And very important is what we were measuring.  As you see, we were seeing how many items we have, then we were measuring the degree of availability or coverage, is how many of the publications had at least one scoring one of the tours.  The intensity is just the number total of scores or signals and then the density is just the number, the intensity divided by the number of items with at least one scoring.  Good to the specter decrease, the opposite was for mentions on social media, even if the window was decreasing, they were increasing.  And the publications with social media increased in Austria from 28% to the degree of availability, the percentage of publications was increasing almost in 10 points.  And if we are following this in 2017, we are already by 40 to 43%.  Even the twitch intensity was increasing, the intensity is not so expected, but it was also increasing in the last day.  The percentage of publications we mentioned was increasing but much more slowly.  And the highest coverage was provided always by the number of readers in the middle age, which is almost very well known because it has been reported.  And what is very, very important, again, is the usage data, very, very important and not many, they are not traced or not considered in a lot of analysis.  And now we are showing the results for the six categories we were using, these are the results and to win time I will try to make a summary of the most interesting results.  The first one was the number of readers in all the areas.  Very interesting was the number of readers in other humanities, but not for usage.  Of course, if you are working in Web of Science, the number of publications of the other humanities is lower, but this should be very, has a very big impact because they are only the things in English and in some language that should attract more scores  or more tweets or more other things.  So very important, of course, the citations were not playing in any role as expected, but the users data were as high as in other disciplines.  If we were analyzing the behavior of the hard sciences, engineering, technology, health science, life science and physical science was very similar everywhere, as well as social media, whereas expected health and life science and the context for the high-nest cover us.  And after we are getting the publications of the social sciences and after that we will see data is the physical science.  The present as well is for the social science, it's supposed to be the one for the ones for the hard sciences and the ones for other humanities, but the extent by the social media.  The log over us was for the physical science and very important, it was a relatively insignificant also in the arts and humanities.  And density research were very good correlation with the cover us or the degree of availability and the tweets intensity has been increasing, as I've already said.  Then we were performing some correlations and we got very quick. First we were working with Web of Science and with Plumbing, so it was a very easy way to make correlations of citations between SCOPUs and Web of Science.  The problem that we have is that the citations window was very short, so it is very easier to get the high correlations. The correlation with cross-ray was also high, but the number of citations was not as high as in SCOPUs sometimes.  Then we were making the correlations between Web of Science and Plumbing, considering the usage data. Web of Science called it usage data, actually it is just a number of clicks of abstracts of publications.  And the correlation with the data that we have used in SCOPUs was very slow and something it is hinting at the fact that the user groups are quite different.  The user groups use it in both data sources and have different interests. Then correlation between different type of measures we have made for readers in Mendeley, Citations, Tweeters, and Usage views and PDS.  It was very slow, only CACCHUS was correlating with citations a little bit but not much.  And then at the end we tried also to make correlations with the number of authors and affiliations of the publications and it was also not existing.  The game for CACCHUS was the highest correlation.  Generic conclusions, okay, monitoring practices are very important tasks and I think it is a new task for libraries or for universities in order to see how the things are moving.  And these little exercises were showing us that social media and particularly Twitter usage is getting broader in Austria at least.  And Plamix was a very good instrument in order to make these exercises with only some problems because in Plamix you cannot fix the measuring window so you will get all the things until today.  For example, if you want to say that you can control the publications years of the publications but not years of the scores are falling to the publications and this will be very important in order to see how it is moving.  And also we were making some analysis if this data is stable or not. And at the end yes correlations should be taken with a pinch of salt because there are different of salt essence.  But that is a very important thing when we are using citations in order to evaluate or to reward some people you need to understand that we need some time and that is dependent on the half life of the discipline and sometimes you need to learn.  But the other ones are moving quite quicker as usage data or Twitter are asked at the beginning until they are the same things just the opposite of salt essence training.  Well we were also not only making the monitoring we were also making a survey and we were focusing on the humanities. We made this in the University of Vienna and the University of Nevada. We were only asking professors.  We were not asking too much and we were asking if they were interested in following these things or not. And you see 50% of the people almost always yes and 50% say not at all I have no interest.  We can't go ahead with the aestheticism or something like this but it was the result when you were asking. What I have done before was just the monitoring of journal publications but what happens for example in the humanities is we are looking at the books and we make analysis with our book collection in ESCO.  And you see that for books the most important indicators are not only usage data but also very important. We have catalog holdings and the reviews.  We have reviews of good reads, the number of reviews in Amazon or in some different tools and these are very relevant for books.  And just at the end because this question was today also coming but we are making bibliometric profiles and there are a lot of people saying we are not using bibliometrics but we are using unmetrics but it's a completely nonsense because exactly bibliometrics to the finish you need to use any kind of quantitative indicators.  And unmetrics is a part of bibliometrics of course. It's a new part, it's a new metric but it's not a new discipline different than bibliometrics.  And we are making a bibliometric profile for a worldwide person. We are using the number of publications, the disability, the impact, the operation, the reference analysis and the focus path.  Even if it is not included, it's another problem, we are going to, we are half calling, we are attention and you can find the most interesting things, what happened and this can be sometimes a help.  Okay? Just to finish, if we are asking us if we can use unmetrics for academic evaluation, the Raspberry's computer says no.  Thank you very much for your relation and happy to answer your questions.  Thank you.
27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_0_155.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_156_203.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_204_443.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_444_491.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_492_803.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_804_1019.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_1020_1091.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_1092_1259.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_1260_1943.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_1944_3179.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_3180_3719.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_3720_4199.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_4200_9191.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_9192_9299.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_9300_9707.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_9708_10079.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_10080_10259.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_10260_12455.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_12456_13511.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_13512_13595.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_13596_14087.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_14088_14207.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_14208_14255.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_14256_14303.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_14304_14399.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_14400_15407.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_15408_17135.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_17136_17615.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_17616_18287.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_18288_20855.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_20856_20903.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_20904_21011.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_21012_21107.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_21108_21203.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_21204_21263.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_21264_24107.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_24108_47435.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_47436_47891.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_47892_48683.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_48684_49067.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_49068_49511.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_49512_50435.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_50436_50831.png 27_09_18_5AM_Altmetrics_Conference_Day_3_pt3_EDIT1_50832_50988.png