Boosting Books in Search Results at the University of Minnesota Press
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Hamilton, Emily
Mitchell, Alec
Emily Hamilton, Alec Mitchell and Sally Kleinfeldt discuss the importance of search on the University of Minnesota Press website
 Hello and welcome to our World Fund Day session. I'm Sally Kleinmore from just Carta with the US based company specializing in open source web technologies including one of course. And with me is Emily Hamilton who is the assistant director for book publishing University of Minnesota Press and Alec Mitchell one of our developers here at Carta also a former phone release manager. And the three of us have been working together on the University of Minnesota Press's clone website since 2011 and we're going to share a little bit of that history with you with a particular focus on site search. So Emily take it away. Tell us a little bit about the press. Sure. I'm happy to. So the University of Minnesota Press is a scholarly publisher. It was established in 1925. So we've been publishing books for a very long time. Primarily we do books in the humanities and social sciences also art and architecture. Currently we publish about 130 books a year. But we have more than 3000 titles in print. So we use our clone site to market all of those books and sell them as well. And many other things that I'm sure you'll see soon. The University of Minnesota Press is a part of the university but it is a standalone publisher. So we publish topics ranging from very, very specialized sociology to children's books that have to do with the natural world in Minnesota where we're located. So it's a wide range of content. And I think that about covers it. Great. Great. And can you say just a little bit about the importance of search to your website? Sort of what do you want people to find when they search for things on your website? And while you're talking about that, I'm going to go to sort of a demo of search on the site. So there's the site and you can see the URL up at the top here everyone. Go ahead. That's your homepage. And of course when people come to any part of our website, we really want them to be able to search and browse the things that they're interested in. And so we've got the search site button global search right there. And we want, for example, if someone hears about an author that we publish, we want them to see all of the books that they've published with us right away. Perhaps it's obvious, but we want them to be able to get to know those books and buy them, hopefully. So I'm just searching on Casanova, which Emily has prompted me as I guess a popular author on your site. And although Casanova is not shown in the titles of any of these, these are all books by this author shown in the live search here. And I'm going to hit return and go to the search results for that page so you can see more about that. Another thing about the search results here is that, so if someone's a fan of Mary Casanova, the books are organized by, with the most recent coming up. So that might be the one that they've heard about now because that's the one that's just published. But they can then scroll down and see all of the books that she's published with us. Right. And if they're interested in other things, the search results page, we customize this so they're also able to see, for example, events that she might be taking place, taking part in. News articles. Yes. So we do, we use all of the functionality on the site to include media hits, you know, to recruit for events. Of course, right now they're virtual primarily, but we use it the same way. And here you can see the e-commerce link and a lot of overview and a lot of details, author biography, any reviews. So there's a lot of information that is published about this book as well as related books. Yeah. Yeah. The related publications is one of the, one of the great things about the search too. We can connect a lot of things that people might not find based on one search. So, you know, if they're fans of Mary Casanova, we have a whole lot of other books that they might be interested in as well. Yeah. Great. Why don't I ask, turn the conversation over you Alec, and maybe you can say a little bit about how this works, how it happens that the books come up first when you do a search for something like an author's name. Sure. So, typically in a Plone text search, by default, it would use something called the ZC text index, which is the standard Zcatalog text index, which does some basic kind of relevancy based sorting. And that's effectively your only option. In newer versions of Plone, you can have some kind of secondary sorting if there are relevancy matches that are similar, but you're pretty limited in terms of how you can sort things. So, what we do with the University Minoist Soda Press is we use a solar search backend, which is a Lusine based search engine using an add-on called ALM.solarIndex to power the search index for the site. And we use that for searchable text and also for title. And then, secondarily, we index the publication dates of the books and also the type, the portal types. And using solar, we can modify the queries that come in to tell it to boost certain aspects of the search results so that we can ensure not only that we're doing a relevancy based search on the searchable text, but also, and the title and the author and getting some particular boost so we make authors more important than general descriptions and titles more important than general descriptions and things like that, so that we have some kind of boosting of the importance of each of the different fields and then more importantly of the recency of the date of publication of these things. So we, for example, will push sort of all books above all events or give a boost to books that tends to push them above events and then give a boost to author names to push them above something where the person, the name Casanova, was in the blurb about the book or something. Put the back down. You're right. And so that gives us a sort of wide range of ability to tune results. And over the years, we've done a few different kind of changes to the tuning to get where we are now, where we have a set of search results that really promotes books, gets them up by date, but makes sure that the results are really relevant in terms of what people are doing. And it's, you know, a Lucine, what's called an E-Dismax query for the Lucine heads out there that we use. We have a fixed one that sort of boosts all of these particular things and gets them working in the way you need to, which can be pretty difficult for internal searches, but it's worked really well and it works with the faceted searches and with the live search and the regular search. So we're very happy with that. Thanks for mentioning the faceted searches. We'll look there and just real quickly, some of you may not have heard of solar, but you might have heard of elastic search, which is apparently, I guess, elastic searches like the most used search engine on the web these days. And both solar and elastic search rely on the underlying Lucine search library. Solar is a little older. They basically are useful for two different kinds of use cases. We've been using solar for quite a while and quite successfully here and there are integrations for both options, solar and elastic search available in the clone. Anyway, I'm going to go back to the website and look a bit at this faceted search, which you can find by clicking on this explore page when we were developing the site. I found myself just spending untold minutes playing with this explore functionality. So it sort of pulls you into the amazing collections of books that University of Minnesota Press offers. You can just go there and you'll actually maybe, Alec, you can explain how these things are being boosted now without any kind of search at all and what this technology is using. Question for Alec, that was. Sorry, I was muted. You're muted, problem. So, yeah, by default, we have the faceted search just due because these are all books. The faceted search is limited on the explore page, just a book. There's some other faceted searches for other content on the site. But this one is just books. And by default, when there's no query, we sort by publication date. And then once you put a query in, we rely a lot more on that same solar relevancy searches when you're doing a text query. I believe when you do a normal facet query using one of these sort of checkbox facets or the tag facet, then it's still sorting by default by date. But as soon as you enter searchable text query, you'll end up with the kind of solar-based relevancy search that also does some date boosting to ensure that recent content is really being promoted. As you can see, you can combine a subject with another primary subject to narrow things down, and then there are also finer-grained topics over here that you can select also. And there's one that's available there. And there's a tag cloud there to show what's happening with these books. Yeah, Emily, any thoughts on this explore search? I know it's not been perhaps as important to you guys on site as some of your kind of direct search for various reasons. I don't know. Any thoughts on the faceted search? Yeah. You know, the ideas behind the faceted search are really to sort of help people do a really sort of mix and match kind of search based on what they're interested in. And because we publish in so many different areas and in so many different subjects, people can combine a lot of different interdisciplinary interest areas in order to find books that are perhaps not all the same major discipline, but they share some characteristics that they might be interested in. So it's kind of a more browsable, I would say, feature, kind of like what you were saying, Sally, where you can just sort of click around and find things that you're interested in rather than sort of knowing what you're looking for. But we envision it as something where people can kind of enter into what we do and see books that cover a lot of different areas, which is really inherent in what we do because we do so much interdisciplinary and cross-disciplinary publishing. Yeah. Yeah. Your selection of books is fascinating. All right. Great. Well, we hope we've given you a little taste of some of the kinds of things you can do integrating a real feature-rich search engine like Solar or Elastic Search with PLON to make it easier for people to find possibly more specialized content on your website. And thank you very much to Emily and Alec for joining us and telling us all about it. Thanks, Sally. Thank you.
 Hello and welcome to our World Fund Day session.  I'm Sally Kleinmore from just Carta with the US based company specializing in open source  web technologies including one of course.  And with me is Emily Hamilton who is the assistant director for book publishing University  of Minnesota Press and Alec Mitchell one of our developers here at Carta also a former  phone release manager.  And the three of us have been working together on the University of Minnesota Press's clone  website since 2011 and we're going to share a little bit of that history with you with  a particular focus on site search.  So Emily take it away.  Tell us a little bit about the press.  Sure.  I'm happy to.  So the University of Minnesota Press is a scholarly publisher.  It was established in 1925.  So we've been publishing books for a very long time.  Primarily we do books in the humanities and social sciences also art and architecture.  Currently we publish about 130 books a year.  But we have more than 3000 titles in print.  So we use our clone site to market all of those books and sell them as well.  And many other things that I'm sure you'll see soon.  The University of Minnesota Press is a part of the university but it is a standalone publisher.  So we publish topics ranging from very, very specialized sociology to children's books  that have to do with the natural world in Minnesota where we're located.  So it's a wide range of content.  And I think that about covers it.  Great.  Great.  And can you say just a little bit about the importance of search to your website?  Sort of what do you want people to find when they search for things on your website?  And while you're talking about that, I'm going to go to sort of a demo of search on the site.  So there's the site and you can see the URL up at the top here everyone.  Go ahead.  That's your homepage.  And of course when people come to any part of our website, we really want them to be  able to search and browse the things that they're interested in.  And so we've got the search site button global search right there.  And we want, for example, if someone hears about an author that we publish, we want them  to see all of the books that they've published with us right away.  Perhaps it's obvious, but we want them to be able to get to know those books and buy  them, hopefully.  So I'm just searching on Casanova, which Emily has prompted me as I guess a popular author  on your site.  And although Casanova is not shown in the titles of any of these, these are all books  by this author shown in the live search here.  And I'm going to hit return and go to the search results for that page so you can see  more about that.  Another thing about the search results here is that, so if someone's a fan of Mary Casanova,  the books are organized by, with the most recent coming up.  So that might be the one that they've heard about now because that's the one that's just  published.  But they can then scroll down and see all of the books that she's published with us.  Right.  And if they're interested in other things, the search results page, we customize this  so they're also able to see, for example, events that she might be taking place, taking  part in.  News articles.  Yes.  So we do, we use all of the functionality on the site to include media hits, you know,  to recruit for events.  Of course, right now they're virtual primarily, but we use it the same way.  And here you can see the e-commerce link and a lot of overview and a lot of details, author  biography, any reviews.  So there's a lot of information that is published about this book as well as related books.  Yeah.  Yeah.  The related publications is one of the, one of the great things about the search too.  We can connect a lot of things that people might not find based on one search.  So, you know, if they're fans of Mary Casanova, we have a whole lot of other books that they  might be interested in as well.  Yeah.  Great.  Why don't I ask, turn the conversation over you Alec, and maybe you can say a little bit  about how this works, how it happens that the books come up first when you do a search  for something like an author's name.  Sure.  So, typically in a Plone text search, by default, it would use something called the ZC text  index, which is the standard Zcatalog text index, which does some basic kind of relevancy  based sorting.  And that's effectively your only option.  In newer versions of Plone, you can have some kind of secondary sorting if there are relevancy  matches that are similar, but you're pretty limited in terms of how you can sort things.  So, what we do with the University Minoist Soda Press is we use a solar search backend,  which is a Lusine based search engine using an add-on called ALM.solarIndex to power  the search index for the site.  And we use that for searchable text and also for title.  And then, secondarily, we index the publication dates of the books and also the type, the  portal types.  And using solar, we can modify the queries that come in to tell it to boost certain aspects  of the search results so that we can ensure not only that we're doing a relevancy based  search on the searchable text, but also, and the title and the author and getting some  particular boost so we make authors more important than general descriptions and titles more  important than general descriptions and things like that, so that we have some kind of boosting  of the importance of each of the different fields and then more importantly of the recency  of the date of publication of these things.  So we, for example, will push sort of all books above all events or give a boost to  books that tends to push them above events and then give a boost to author names to push  them above something where the person, the name Casanova, was in the blurb about the  book or something.  Put the back down.  You're right.  And so that gives us a sort of wide range of ability to tune results.  And over the years, we've done a few different kind of changes to the tuning to get where  we are now, where we have a set of search results that really promotes books, gets them  up by date, but makes sure that the results are really relevant in terms of what people  are doing.  And it's, you know, a Lucine, what's called an E-Dismax query for the Lucine heads out  there that we use.  We have a fixed one that sort of boosts all of these particular things and gets them working  in the way you need to, which can be pretty difficult for internal searches, but it's worked  really well and it works with the faceted searches and with the live search and the regular  search.  So we're very happy with that.  Thanks for mentioning the faceted searches.  We'll look there and just real quickly, some of you may not have heard of solar, but you  might have heard of elastic search, which is apparently, I guess, elastic searches like  the most used search engine on the web these days.  And both solar and elastic search rely on the underlying Lucine search library.  Solar is a little older.  They basically are useful for two different kinds of use cases.  We've been using solar for quite a while and quite successfully here and there are integrations  for both options, solar and elastic search available in the clone.  Anyway, I'm going to go back to the website and look a bit at this faceted search, which  you can find by clicking on this explore page when we were developing the site.  I found myself just spending untold minutes playing with this explore functionality.  So it sort of pulls you into the amazing collections of books that University of Minnesota Press  offers.  You can just go there and you'll actually maybe, Alec, you can explain how these things  are being boosted now without any kind of search at all and what this technology is  using.  Question for Alec, that was.  Sorry, I was muted.  You're muted, problem.  So, yeah, by default, we have the faceted search just due because these are all books.  The faceted search is limited on the explore page, just a book.  There's some other faceted searches for other content on the site.  But this one is just books.  And by default, when there's no query, we sort by publication date.  And then once you put a query in, we rely a lot more on that same solar relevancy searches  when you're doing a text query.  I believe when you do a normal facet query using one of these sort of checkbox facets  or the tag facet, then it's still sorting by default by date.  But as soon as you enter searchable text query, you'll end up with the kind of solar-based  relevancy search that also does some date boosting to ensure that recent content is  really being promoted.  As you can see, you can combine a subject with another primary subject to narrow things  down, and then there are also finer-grained topics over here that you can select also.  And there's one that's available there.  And there's a tag cloud there to show what's happening with these books.  Yeah, Emily, any thoughts on this explore search?  I know it's not been perhaps as important to you guys on site as some of your kind of  direct search for various reasons.  I don't know.  Any thoughts on the faceted search?  Yeah.  You know, the ideas behind the faceted search are really to sort of help people do a really  sort of mix and match kind of search based on what they're interested in.  And because we publish in so many different areas and in so many different subjects, people  can combine a lot of different interdisciplinary interest areas in order to find books that  are perhaps not all the same major discipline, but they share some characteristics that they  might be interested in.  So it's kind of a more browsable, I would say, feature, kind of like what you were saying,  Sally, where you can just sort of click around and find things that you're interested in  rather than sort of knowing what you're looking for.  But we envision it as something where people can kind of enter into what we do and see  books that cover a lot of different areas, which is really inherent in what we do because  we do so much interdisciplinary and cross-disciplinary publishing.  Yeah.  Yeah.  Your selection of books is fascinating.  All right.  Great.  Well, we hope we've given you a little taste of some of the kinds of things you can do  integrating a real feature-rich search engine like Solar or Elastic Search with PLON to make  it easier for people to find possibly more specialized content on your website.  And thank you very much to Emily and Alec for joining us and telling us all about it.  Thanks, Sally.  Thank you.
Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_0_5435.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_5436_6011.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_6012_7235.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_7236_7631.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_7632_13727.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_13728_13847.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_13848_15539.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_15540_16559.png Boosting_Books_in_Search_Results_at_the_University_Nuq4mz_CuM4_16560_20076.png