<commit_before>= Gem: google-site-search

== Description

This gem was created to aid in the querying and parsing of the {Google Site Search}[http://www.google.com/sitesearch] api. 

In the simplest use case it will query your google site search for a term and supply you with an object containing the results. However I've built this gem with the intention that you will want to explicitly handle _what_ and _how_ the specific results are stored. 

== Installation

Add the following to your projects Gemfile.

    gem 'google-site-search', :git => "<EMAIL>:dvallance/google-site-search.git"

Require the code if necessary (_note:_ some frameworks like rails are set to auto-require gems for you by default)

    require 'google-site-search'

== Usage

The simpliest way to use the gem is by providing just a *search* *query* *term* and your *search* *engine* *unique* *id* code (_e.g._ looks like this +00255077836266642015+:+u-scht7a-8i+ and is located in your google site search control panel) 

    # just assign the query to an object
    search = GoogleSiteSearch.query(GoogleSiteSearch::UrlBuilder.new("microsoft", "00255077836266642015:u-scht7a-8i")
  
    # object has search attributes like
    puts search.next_results_url
    puts search.previous_results_url
    puts search.xml
    puts search.spelling
    puts search.spelling_url

    # object has an array of each specific result that contains title, description and its link by default
    search.results.each do |result|
      puts result.title
      puts result.description
      puts result.link
    end
	
The _query_ method expects a valid url so if you wanted to supply your own you can! However I have created a builder class to help with proper url creation and to help do some of the work for you.

== Multiple Search

Since google only allows a max of 20 returned results I have added a method that will capture up to *n* number of results.

    # the array will be up to 5 search objects if the query actually has that many results.
    # has soon as a search doesn't have a next_results_url the method stops.
    array_of_search_results = GoogleSiteSearch.query_multiple(5, *YOUR_URL*)  

== Blocks (and a caching example)

Both the query and query_multiple methods can take a block which executes for each Search object found.

    # here is a rails example using the block 
    url = GoogleSiteSearch::UrlBuilder.new("microsoft", "00255077836266642015:u-scht7a-8i")
    @search = Rails.cache.fetch(url) do |search|
      # I can do something with the search objects.
      # possibly custom analytics for our searchs?
      search.url # we can access the object
    end
      

== Advanced Usage 

An important requirement for this gem was to be able to use {structured data}[https://developers.google.com/custom-search/docs/structured_data] for:
* querying the search api itself (_i.e._ filtering and sorting )
* displaying specific information in views (_i.e._ display a specific field like'author', or 'product_type')

Therefore I allow the developer to supply his own "*Results*" class to the query and allow them to parse each result xml element explicitly.

The default Result class is as follows:

    class Result
      attr_reader :title, :link, :description
        def initialize(node)
        @title = node.find_first("T").content
        @link = node.find_first("UE").content
        @description = node.find_first("S").content
      end
    end

As you can see it is very simple. Your class simply needs an initialize method that will recieve an xml node, which it can then do with as it pleases. After it is initialized it is added to the _search.results_ array as shown previously.

See
* {libxml-ruby}[http://libxml.rubyforge.org/rdoc/] for help on xml parsing.
* {Googles Site Search XML API reference}[https://developers.google.com/custom-search/docs/xml_results?hl=en] for the <R> tag which encapsulates the details of an individual search result.

== Pagination

The google search api does the work of pagination for us, supplying the next and previous urls. The urls are relative paths and contain the search engine id parameter. Since this is a security concern I strip out the search engine id when I store them in the Search.next_result_url and Search.previous_result_url methods. This makes them safe to put in links on views and it is why you must supply the search engine id again on the paginate call; so full url can be rebuilt for the query call.

    search2 = GoogleSiteSearch.query(GoogleSiteSearch.paginate(search1.next_results_url, "00255077836266642015:u-scht7a-8i"))

== Pagination Simple Example
<commit_msg>Updating the readme to reflect the new caching_key method.<commit_after>= Gem: google-site-search

== Description

This gem was created to aid in the querying and parsing of the {Google Site Search}[http://www.google.com/sitesearch] api. 

In the simplest use case it will query your google site search for a term and supply you with an object containing the results. However I've built this gem with the intention that you will want to explicitly handle _what_ and _how_ the specific results are stored. 

== Installation

Add the following to your projects Gemfile.

    gem 'google-site-search', :git => "<EMAIL>:dvallance/google-site-search.git"

Require the code if necessary (_note:_ some frameworks like rails are set to auto-require gems for you by default)

    require 'google-site-search'

== Usage

The simpliest way to use the gem is by providing just a *search* *query* *term* and your *search* *engine* *unique* *id* code (_e.g._ looks like this +00255077836266642015+:+u-scht7a-8i+ and is located in your google site search control panel) 

    # just assign the query to an object
    search = GoogleSiteSearch.query(GoogleSiteSearch::UrlBuilder.new("microsoft", "00255077836266642015:u-scht7a-8i")
  
    # object has search attributes like
    puts search.next_results_url
    puts search.previous_results_url
    puts search.xml
    puts search.spelling
    puts search.spelling_url

    # object has an array of each specific result that contains title, description and its link by default
    search.results.each do |result|
      puts result.title
      puts result.description
      puts result.link
    end
	
The _query_ method expects a valid url so if you wanted to supply your own you can! However I have created a builder class to help with proper url creation and to help do some of the work for you.

== Multiple Search

Since google only allows a max of 20 returned results I have added a method that will capture up to *n* number of results.

    # the array will be up to 5 search objects if the query actually has that many results.
    # has soon as a search doesn't have a next_results_url the method stops.
    array_of_search_results = GoogleSiteSearch.query_multiple(5, *YOUR_URL*)  

== Caching

To aid in caching I added a a *caching_key* method which sorts the query parameters, removes some unique parameters that google adds (i.e. an _ie_ parameter which has something to do with related searchs), and compresses the result. It should be a unique representation of a search url, and used to cache results. 

== Blocks (and a caching example)

Both the query and query_multiple methods can take a block which executes for each Search object found.

    # here is a rails example using caching and supplying a block 
    url = GoogleSiteSearch::UrlBuilder.new("microsoft", "00255077836266642015:u-scht7a-8i")

    @search = Rails.cache.fetch(GoogleSiteSearch::caching_key(url.to_s), :expires => 1.day) do |search|
      GoogleSiteSearch.query(url, SearchResult) do |search|
        # I can do something with the search objects.
        # possibly some custom analytics for our searchs?
        search.url # we can access the object
      end
    end


== Advanced Usage 

An important requirement for this gem was to be able to use {structured data}[https://developers.google.com/custom-search/docs/structured_data] for:
* querying the search api itself (_i.e._ filtering and sorting )
* displaying specific information in views (_i.e._ display a specific field like'author', or 'product_type')

Therefore I allow the developer to supply his own "*Results*" class to the query and allow them to parse each result xml element explicitly.

The default Result class is as follows:

    class Result
      attr_reader :title, :link, :description
        def initialize(node)
        @title = node.find_first("T").content
        @link = node.find_first("UE").content
        @description = node.find_first("S").content
      end
    end

As you can see it is very simple. Your class simply needs an initialize method that will recieve an xml node, which it can then do with as it pleases. After it is initialized it is added to the _search.results_ array as shown previously.

See
* {libxml-ruby}[http://libxml.rubyforge.org/rdoc/] for help on xml parsing.
* {Googles Site Search XML API reference}[https://developers.google.com/custom-search/docs/xml_results?hl=en] for the <R> tag which encapsulates the details of an individual search result.

== Pagination

The google search api does the work of pagination for us, supplying the next and previous urls. The urls are relative paths and contain the search engine id parameter. Since this is a security concern I strip out the search engine id when I store them in the Search.next_results_url and Search.previous_results_url methods. This makes them safe to put in links on views and it is why you must supply the search engine id again on the paginate call; so the full url can be rebuilt for the query call.

    search2 = GoogleSiteSearch.query(GoogleSiteSearch.paginate(search1.next_results_url, "00255077836266642015:u-scht7a-8i"))

== Pagination Simple Example

{'fraction_non_alphanumeric': 0.0633558144287758, 'fraction_numerical': 0.019313304721030045, 'mean_word_length': 3.8861707438841737, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 10, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 14}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1138295', 'n_tokens_mistral': 2788, 'n_tokens_neox': 2588, 'n_words': 1270}