<commit_before># gorange

gorange is a small Go library for interacting with range servers.

### Usage

Documentation is available via
[![GoDoc](https://godoc.org/github.com/karrick/gorange?status.svg)](https://godoc.org/github.com/karrick/gorange).

### Description

gorange provides a `Querier` interface, and a few data structures that implement this interface and
allows querying range services on remote hosts.

Querying range is a simple HTTP GET call, and Go already provides a steller http library. So why
wrap it? Well, either you write your own wrapper or use one someone else has written, it's all the
same to me. But I had to write the wrapper, so I figured I'd at least provide my implementation as a
reference piece for others doing the same.

In any eveny, this library

1. guarantees HTTP connections can be re-used by always reading all body bytes if the Get succeeded.
1. detects and parses the RangeException header, so you don't have to.
1. converts response body to slice of strings.

There are four possible error types this library returns:

1. Raw error that the underlying Get method returned.
1. ErrStatusNotOK is returned when the response status code is not OK.
1. ErrRangeException is returned when the response headers includes 'RangeException' header.
1. ErrParseException is returned by Client.Query when an error occurs while parsing the GET
response.

### Supported Use Cases

Both the `Client` and `CachingClient` data types implement the `Querier` interface. In fact,
`CachingClient` is implemented as a simple `Client` with a TTL cache, and the `NewCachingClient`
function merely wraps the provided `Client`. For a majority of use-cases, you don't need to worry
about any of this. I recommend ignoring `Client` and `CachingClient` and just think about calling
the `NewQuerier` function and getting back some opaque data structure instance that exposes the
`Query` method. See the _Simple_ code example below.

##### Simple

The easiest way to use gorange is to use a `Configurator` instance to create an object that
implements the `Querier` interface, and use that to query range.

```Go
    var querier gorange.Querier

    func init() {
    	// create a range querier; could list additional servers or include other options as well
    	var err error
        querier, err = gorange.NewQuerier(&gorange.Configurator{Servers: []string{"range.example.com"}})
    	if err != nil {
    		fmt.Fprintf(os.Stderr, "%s\n", err)
    		os.Exit(1)
    	}
    }
    
    func main() {
    	// use the range querier
    	lines, err := querier.Query("%someQuery")
    	if err != nil {
    		fmt.Fprintf(os.Stderr, "%s", err)
    		os.Exit(1)
    	}
    	for _, line := range lines {
    		fmt.Println(line)
    	}
    }
```

##### Customized

As described above, the `NewQuerier` function allows creating objects that implement `Querier` and
supports most use-cases: optional round-robin query of multiple servers, optional retry of specific
errors, and optional TTL memoization of query responses. The only requirement is to specify one or
more servers to query. Leaving any other config option at its zero value creates a viable Querier
without those optional features.

See the `examples/customized/main.go` for complete example of this code, including constants and
functions not shown here.

```Go
    func main() {
    	servers := []string{"range1.example.com", "range2.example.com", "range3.example.com"}
    
    	config := &gorange.Configurator{
    		Addr2Getter:   addr2Getter,
    		RetryCallback: retryCallback,
    		RetryCount:    len(servers),
    		Servers:       servers,
    		TTL:           time.Minute,
    	}
    <commit_msg>readme snippets include gopkg import statement<commit_after># gorange

gorange is a small Go library for interacting with range servers.

### Usage

Documentation is available via
[![GoDoc](https://godoc.org/github.com/karrick/gorange?status.svg)](https://godoc.org/github.com/karrick/gorange).

### Description

gorange provides a `Querier` interface, and a few data structures that implement this interface and
allows querying range services on remote hosts.

Querying range is a simple HTTP GET call, and Go already provides a steller http library. So why
wrap it? Well, either you write your own wrapper or use one someone else has written, it's all the
same to me. But I had to write the wrapper, so I figured I'd at least provide my implementation as a
reference piece for others doing the same.

In any eveny, this library

1. guarantees HTTP connections can be re-used by always reading all body bytes if the Get succeeded.
1. detects and parses the RangeException header, so you don't have to.
1. converts response body to slice of strings.

There are four possible error types this library returns:

1. Raw error that the underlying Get method returned.
1. ErrStatusNotOK is returned when the response status code is not OK.
1. ErrRangeException is returned when the response headers includes 'RangeException' header.
1. ErrParseException is returned by Client.Query when an error occurs while parsing the GET
response.

### Supported Use Cases

Both the `Client` and `CachingClient` data types implement the `Querier` interface. In fact,
`CachingClient` is implemented as a simple `Client` with a TTL cache, and the `NewCachingClient`
function merely wraps the provided `Client`. For a majority of use-cases, you don't need to worry
about any of this. I recommend ignoring `Client` and `CachingClient` and just think about calling
the `NewQuerier` function and getting back some opaque data structure instance that exposes the
`Query` method. See the _Simple_ code example below.

##### Simple

The easiest way to use gorange is to use a `Configurator` instance to create an object that
implements the `Querier` interface, and use that to query range.

```Go
    package main
    
    import (
        gorange "gopkg.in/karrick/gorange.v2"
    )

    var querier gorange.Querier

    func init() {
    	// create a range querier; could list additional servers or include other options as well
    	var err error
        querier, err = gorange.NewQuerier(&gorange.Configurator{Servers: []string{"range.example.com"}})
    	if err != nil {
    		fmt.Fprintf(os.Stderr, "%s\n", err)
    		os.Exit(1)
    	}
    }
    
    func main() {
    	// use the range querier
    	lines, err := querier.Query("%someQuery")
    	if err != nil {
    		fmt.Fprintf(os.Stderr, "%s", err)
    		os.Exit(1)
    	}
    	for _, line := range lines {
    		fmt.Println(line)
    	}
    }
```

##### Customized

As described above, the `NewQuerier` function allows creating objects that implement `Querier` and
supports most use-cases: optional round-robin query of multiple servers, optional retry of specific
errors, and optional TTL memoization of query responses. The only requirement is to specify one or
more servers to query. Leaving any other config option at its zero value creates a viable Querier
without those optional features.

See the `examples/customized/main.go` for complete example of this code, including constants and
functions not shown here.

```Go
    package main
    
    import (
        gorange "gopkg.in/karrick/gorange.v2"
    )

    func main() {
    	servers := []string{"range1.example.com", "range2.example.com", "range3.example.com"}
    
    	config := &gorange.Configurator{
    		Addr2Getter:   addr2Getter,
    		RetryCallback: retryCallback,
    		RetryCount:    len(servers),
    		Servers:       servers,
    		TTL:           time.Minute,
    	}
    
{'fraction_non_alphanumeric': 0.0760164216660045, 'fraction_numerical': 0.003972983710766786, 'mean_word_length': 3.5548854041013267, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3123220', 'n_tokens_mistral': 2179, 'n_tokens_neox': 2125, 'n_words': 955}