Spaces:
Sleeping
Sleeping
hiett
commited on
Commit
•
d976565
1
Parent(s):
48432f0
Rename new readme
Browse files- README-UPDATED.md +0 -145
- README.md +116 -81
README-UPDATED.md
DELETED
@@ -1,145 +0,0 @@
|
|
1 |
-
# Serverless Redis HTTP (SRH)
|
2 |
-
|
3 |
-
A Redis proxy and connection pooler that uses HTTP rather than the Redis binary protocol.\
|
4 |
-
The aim of this project is to be entirely compatible with Upstash, and work with any Upstash supported Redis version.
|
5 |
-
|
6 |
-
Use cases for SRH:
|
7 |
-
- For usage in your CI pipelines, creating Upstash databases is tedious, or you have lots of parallel runs.
|
8 |
-
- See [Using in GitHub Actions](#in-github-actions) on how to quickly get SRH setup for this context.
|
9 |
-
- For usage inside of Kubernetes, or any network whereby the Redis server is not exposed to the internet.
|
10 |
-
- See [Using in Docker Compose](#via-docker-compose) for the various setup options directly using the Docker Container.
|
11 |
-
- For local development environments, where you have a local Redis server running, or require offline access.
|
12 |
-
- See [Using the Docker Command](#via-docker-command), or [Using Docker Compose](#via-docker-compose).
|
13 |
-
|
14 |
-
## Differences between Upstash and Redis to note
|
15 |
-
SRH tests are ran nightly against the `@upstash/redis` JavaScript package. However, there are some minor differences between Upstash's implementation of Redis and the official Redis code.
|
16 |
-
|
17 |
-
- The `UNLINK` command will not throw an error when 0 keys are given to it. In Redis, and as such SRH, an error will be thrown.
|
18 |
-
- In the `ZRANGE` command, in Upstash you are not required to provide `BYSCORE` or `BYLEX` in order to use the `LIMIT` argument. With Redis/SRH, this will throw an error if not provided.
|
19 |
-
- The Upstash implementation of `RedisJSON` contains a number of subtle differences in what is returned in responses. For this reason, **it is not advisable to use SRH with Redis Stack if you are testing your Upstash implementation that uses JSON commands**. If you don't use any JSON commands, then all is good :)
|
20 |
-
- **SRH does not implement commands via paths, or accepting the token via a query param**. Only the body method is implemented, which the `@upstash/redis` SDK uses.
|
21 |
-
|
22 |
-
### Similarities to note:
|
23 |
-
|
24 |
-
Pipelines and Transaction endpoints are also implemented, also using the body data only. You can read more about the RestAPI here: [Upstash Docs on the Rest API](https://docs.upstash.com/redis/features/restapi)
|
25 |
-
|
26 |
-
Response encoding is also fully implemented. This is enabled by default by the `@upstash/redis` SDK. You can read more about that here: [Upstash Docs on Hashed Responses](https://docs.upstash.com/redis/sdks/javascriptsdk/troubleshooting#hashed-response)
|
27 |
-
|
28 |
-
## How to use with the `@upstash/redis` SDK
|
29 |
-
Simply set the REST URL and token to where the SRH instance is running. For example:
|
30 |
-
```ts
|
31 |
-
import {Redis} from '@upstash/redis';
|
32 |
-
|
33 |
-
export const redis = new Redis({
|
34 |
-
url: "http://localhost:8079",
|
35 |
-
token: "example_token",
|
36 |
-
});
|
37 |
-
```
|
38 |
-
|
39 |
-
# Setting up SRH
|
40 |
-
## Via Docker command
|
41 |
-
If you have a locally running Redis server, you can simply start an SRH container that connects to it.
|
42 |
-
In this example, SRH will be running on port `8080`.
|
43 |
-
|
44 |
-
```bash
|
45 |
-
docker run \
|
46 |
-
-it -d -p 8080:80 --name srh \
|
47 |
-
-e SRH_MODE=env \
|
48 |
-
-e SRH_TOKEN=your_token_here \
|
49 |
-
-e SRH_CONNECTION_STRING="redis://your_server_here:6379" \
|
50 |
-
hiett/serverless-redis-http:latest
|
51 |
-
```
|
52 |
-
|
53 |
-
## Via Docker Compose
|
54 |
-
If you wish to run in Kubernetes, this should contain all the basics would need to set that up. However, be sure to read the Configuration Options, because you can create a setup whereby multiple Redis servers are proxied.
|
55 |
-
```yml
|
56 |
-
version: '3'
|
57 |
-
services:
|
58 |
-
redis:
|
59 |
-
image: redis
|
60 |
-
ports:
|
61 |
-
- '6379:6379'
|
62 |
-
serverless-redis-http:
|
63 |
-
ports:
|
64 |
-
- '8079:80'
|
65 |
-
image: hiett/serverless-redis-http:latest
|
66 |
-
environment:
|
67 |
-
SRH_MODE: env
|
68 |
-
SRH_TOKEN: example_token
|
69 |
-
SRH_CONNECTION_STRING: 'redis://redis:6379' # Using `redis` hostname since they're in the same Docker network.
|
70 |
-
```
|
71 |
-
|
72 |
-
## In GitHub Actions
|
73 |
-
|
74 |
-
SRH works nicely in GitHub Actions because you can run it as a container in a job's services. Simply start a Redis server, and then
|
75 |
-
SRH alongside it. You don't need to worry about a race condition of the Redis instance not being ready, because SRH doesn't create a Redis connection until the first command comes in.
|
76 |
-
|
77 |
-
```yml
|
78 |
-
name: Test @upstash/redis compatability
|
79 |
-
on:
|
80 |
-
push:
|
81 |
-
workflow_dispatch:
|
82 |
-
|
83 |
-
env:
|
84 |
-
SRH_TOKEN: example_token
|
85 |
-
|
86 |
-
jobs:
|
87 |
-
container-job:
|
88 |
-
runs-on: ubuntu-latest
|
89 |
-
container: denoland/deno
|
90 |
-
services:
|
91 |
-
redis:
|
92 |
-
image: redis/redis-stack-server:6.2.6-v6 # 6.2 is the Upstash compatible Redis version
|
93 |
-
srh:
|
94 |
-
image: hiett/serverless-redis-http:latest
|
95 |
-
env:
|
96 |
-
SRH_MODE: env # We are using env mode because we are only connecting to one server.
|
97 |
-
SRH_TOKEN: ${{ env.SRH_TOKEN }}
|
98 |
-
SRH_CONNECTION_STRING: redis://redis:6379
|
99 |
-
|
100 |
-
steps:
|
101 |
-
# You can place your normal testing steps here. In this example, we are running SRH against the upstash/upstash-redis test suite.
|
102 |
-
- name: Checkout code
|
103 |
-
uses: actions/checkout@v3
|
104 |
-
with:
|
105 |
-
repository: upstash/upstash-redis
|
106 |
-
|
107 |
-
- name: Run @upstash/redis Test Suite
|
108 |
-
run: deno test -A ./pkg
|
109 |
-
env:
|
110 |
-
UPSTASH_REDIS_REST_URL: http://srh:80
|
111 |
-
UPSTASH_REDIS_REST_TOKEN: ${{ env.SRH_TOKEN }}
|
112 |
-
```
|
113 |
-
|
114 |
-
# Configuration Options
|
115 |
-
|
116 |
-
SRH works with multiple Redis servers, and can pool however many connections you wish it to. It will shut down un-used pools after 15 minutes of inactivity. Upon the next command, it will re-build the pool.
|
117 |
-
|
118 |
-
## Connecting to multiple Redis servers at the same time
|
119 |
-
|
120 |
-
The examples above use environment variables in order to tell SRH which Redis server to connect to. However, you can also use a configuration JSON file, which lets you create as many connections as you wish. The token provided in each request will decide which pool is used.
|
121 |
-
|
122 |
-
Create a JSON file, in this example called `tokens.json`:
|
123 |
-
```json
|
124 |
-
{
|
125 |
-
"example_token": {
|
126 |
-
"srh_id": "some_unique_identifier",
|
127 |
-
"connection_string": "redis://localhost:6379",
|
128 |
-
"max_connections": 3
|
129 |
-
}
|
130 |
-
}
|
131 |
-
```
|
132 |
-
You can provide as many entries to the base object as you wish, and configure the number of max connections per pool. The `srh_id` is used internally to keep track of instances. It can be anything you want.
|
133 |
-
|
134 |
-
Once you have created this, mount it to the docker container to the `/app/srh-config/tokens.json` file. Here is an example docker command:
|
135 |
-
|
136 |
-
`docker run -it -d -p 8079:80 --name srh --mount type=bind,source=$(pwd)/tokens.json,target=/app/srh-config/tokens.json hiett/serverless-redis-http:latest`
|
137 |
-
|
138 |
-
## Environment Variables
|
139 |
-
|
140 |
-
| Name | Default Value | Notes |
|
141 |
-
| ---- | ------------- | ----- |
|
142 |
-
| SRH_MODE | `file` | Can be `env` or `file`. If `file`, see [Connecting to multiple Redis servers](#connecting-to-multiple-redis-servers-at-the-same-time). If set to `env`, you are required to provide the following environment variables: |
|
143 |
-
| SRH_TOKEN | `<required if SRH_MODE = env>` | Set the token that the Rest API will require |
|
144 |
-
| SRH_CONNECTION_STRING | `<required if SRH_MODE = env>` | Sets the connection string to the Redis server. |
|
145 |
-
| SRH_MAX_CONNECTIONS | `3` | Only used if `SRH_MODE=env`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
@@ -1,25 +1,33 @@
|
|
1 |
-
#
|
2 |
|
3 |
-
|
|
|
4 |
|
5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
|
|
23 |
import {Redis} from '@upstash/redis';
|
24 |
|
25 |
export const redis = new Redis({
|
@@ -27,84 +35,111 @@ export const redis = new Redis({
|
|
27 |
token: "example_token",
|
28 |
});
|
29 |
```
|
30 |
-
---
|
31 |
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
-
|
40 |
-
-
|
41 |
-
-
|
42 |
-
-
|
43 |
-
-
|
44 |
-
|
45 |
|
46 |
-
##
|
47 |
-
|
48 |
-
|
49 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
50 |
|
51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
52 |
|
53 |
-
|
54 |
-
You have to options to run this:
|
55 |
-
- Via docker: `docker pull hiett/serverless-redis-http:latest` [Docker Hub link](https://hub.docker.com/r/hiett/serverless-redis-http)
|
56 |
-
- Via elixir: `(clone this repo)` -> `mix deps.get` -> `iex -S mix`
|
57 |
|
58 |
-
|
59 |
-
An example of a run command is the following:
|
60 |
|
61 |
-
|
62 |
|
63 |
-
|
64 |
|
65 |
-
|
66 |
-
Create a file: `srh-config/tokens.json`
|
67 |
```json
|
68 |
{
|
69 |
"example_token": {
|
70 |
"srh_id": "some_unique_identifier",
|
71 |
"connection_string": "redis://localhost:6379",
|
72 |
"max_connections": 3
|
73 |
-
}
|
74 |
}
|
75 |
```
|
|
|
76 |
|
77 |
-
|
78 |
-
```
|
79 |
-
SRH_MODE="env"
|
80 |
-
SRH_TOKEN="example_token"
|
81 |
-
SRH_CONNECTION_STRING="redis://redis:6379"
|
82 |
-
```
|
83 |
-
If you use this strategy, it's important that you set `SRH_MODE=env`. This is because SRH by default uses the file strategy for backwards-compatability.
|
84 |
|
85 |
-
|
86 |
-
|
87 |
-
|
88 |
-
"connection_string": "redis://redis:6379"
|
89 |
-
```
|
90 |
-
docker-compose.yaml
|
91 |
-
```yaml
|
92 |
-
version: '3'
|
93 |
-
services:
|
94 |
-
redis:
|
95 |
-
image: redis
|
96 |
-
ports:
|
97 |
-
- '6379:6379'
|
98 |
-
serverless-redis-http:
|
99 |
-
ports:
|
100 |
-
- '8079:80'
|
101 |
-
image: hiett/serverless-redis-http:latest
|
102 |
-
volumes:
|
103 |
-
- ./path/to/tokens.json:/app/srh-config/tokens.json
|
104 |
-
```
|
105 |
|
106 |
-
Notes
|
107 |
-
|
108 |
-
|
109 |
-
|
110 |
-
|
|
|
|
1 |
+
# Serverless Redis HTTP (SRH)
|
2 |
|
3 |
+
A Redis proxy and connection pooler that uses HTTP rather than the Redis binary protocol.\
|
4 |
+
The aim of this project is to be entirely compatible with Upstash, and work with any Upstash supported Redis version.
|
5 |
|
6 |
+
Use cases for SRH:
|
7 |
+
- For usage in your CI pipelines, creating Upstash databases is tedious, or you have lots of parallel runs.
|
8 |
+
- See [Using in GitHub Actions](#in-github-actions) on how to quickly get SRH setup for this context.
|
9 |
+
- For usage inside of Kubernetes, or any network whereby the Redis server is not exposed to the internet.
|
10 |
+
- See [Using in Docker Compose](#via-docker-compose) for the various setup options directly using the Docker Container.
|
11 |
+
- For local development environments, where you have a local Redis server running, or require offline access.
|
12 |
+
- See [Using the Docker Command](#via-docker-command), or [Using Docker Compose](#via-docker-compose).
|
13 |
|
14 |
+
## Differences between Upstash and Redis to note
|
15 |
+
SRH tests are ran nightly against the `@upstash/redis` JavaScript package. However, there are some minor differences between Upstash's implementation of Redis and the official Redis code.
|
16 |
+
|
17 |
+
- The `UNLINK` command will not throw an error when 0 keys are given to it. In Redis, and as such SRH, an error will be thrown.
|
18 |
+
- In the `ZRANGE` command, in Upstash you are not required to provide `BYSCORE` or `BYLEX` in order to use the `LIMIT` argument. With Redis/SRH, this will throw an error if not provided.
|
19 |
+
- The Upstash implementation of `RedisJSON` contains a number of subtle differences in what is returned in responses. For this reason, **it is not advisable to use SRH with Redis Stack if you are testing your Upstash implementation that uses JSON commands**. If you don't use any JSON commands, then all is good :)
|
20 |
+
- **SRH does not implement commands via paths, or accepting the token via a query param**. Only the body method is implemented, which the `@upstash/redis` SDK uses.
|
21 |
+
|
22 |
+
### Similarities to note:
|
23 |
+
|
24 |
+
Pipelines and Transaction endpoints are also implemented, also using the body data only. You can read more about the RestAPI here: [Upstash Docs on the Rest API](https://docs.upstash.com/redis/features/restapi)
|
25 |
+
|
26 |
+
Response encoding is also fully implemented. This is enabled by default by the `@upstash/redis` SDK. You can read more about that here: [Upstash Docs on Hashed Responses](https://docs.upstash.com/redis/sdks/javascriptsdk/troubleshooting#hashed-response)
|
27 |
+
|
28 |
+
## How to use with the `@upstash/redis` SDK
|
29 |
+
Simply set the REST URL and token to where the SRH instance is running. For example:
|
30 |
+
```ts
|
31 |
import {Redis} from '@upstash/redis';
|
32 |
|
33 |
export const redis = new Redis({
|
|
|
35 |
token: "example_token",
|
36 |
});
|
37 |
```
|
|
|
38 |
|
39 |
+
# Setting up SRH
|
40 |
+
## Via Docker command
|
41 |
+
If you have a locally running Redis server, you can simply start an SRH container that connects to it.
|
42 |
+
In this example, SRH will be running on port `8080`.
|
43 |
+
|
44 |
+
```bash
|
45 |
+
docker run \
|
46 |
+
-it -d -p 8080:80 --name srh \
|
47 |
+
-e SRH_MODE=env \
|
48 |
+
-e SRH_TOKEN=your_token_here \
|
49 |
+
-e SRH_CONNECTION_STRING="redis://your_server_here:6379" \
|
50 |
+
hiett/serverless-redis-http:latest
|
51 |
+
```
|
52 |
|
53 |
+
## Via Docker Compose
|
54 |
+
If you wish to run in Kubernetes, this should contain all the basics would need to set that up. However, be sure to read the Configuration Options, because you can create a setup whereby multiple Redis servers are proxied.
|
55 |
+
```yml
|
56 |
+
version: '3'
|
57 |
+
services:
|
58 |
+
redis:
|
59 |
+
image: redis
|
60 |
+
ports:
|
61 |
+
- '6379:6379'
|
62 |
+
serverless-redis-http:
|
63 |
+
ports:
|
64 |
+
- '8079:80'
|
65 |
+
image: hiett/serverless-redis-http:latest
|
66 |
+
environment:
|
67 |
+
SRH_MODE: env
|
68 |
+
SRH_TOKEN: example_token
|
69 |
+
SRH_CONNECTION_STRING: 'redis://redis:6379' # Using `redis` hostname since they're in the same Docker network.
|
70 |
+
```
|
71 |
|
72 |
+
## In GitHub Actions
|
73 |
+
|
74 |
+
SRH works nicely in GitHub Actions because you can run it as a container in a job's services. Simply start a Redis server, and then
|
75 |
+
SRH alongside it. You don't need to worry about a race condition of the Redis instance not being ready, because SRH doesn't create a Redis connection until the first command comes in.
|
76 |
+
|
77 |
+
```yml
|
78 |
+
name: Test @upstash/redis compatability
|
79 |
+
on:
|
80 |
+
push:
|
81 |
+
workflow_dispatch:
|
82 |
+
|
83 |
+
env:
|
84 |
+
SRH_TOKEN: example_token
|
85 |
+
|
86 |
+
jobs:
|
87 |
+
container-job:
|
88 |
+
runs-on: ubuntu-latest
|
89 |
+
container: denoland/deno
|
90 |
+
services:
|
91 |
+
redis:
|
92 |
+
image: redis/redis-stack-server:6.2.6-v6 # 6.2 is the Upstash compatible Redis version
|
93 |
+
srh:
|
94 |
+
image: hiett/serverless-redis-http:latest
|
95 |
+
env:
|
96 |
+
SRH_MODE: env # We are using env mode because we are only connecting to one server.
|
97 |
+
SRH_TOKEN: ${{ env.SRH_TOKEN }}
|
98 |
+
SRH_CONNECTION_STRING: redis://redis:6379
|
99 |
+
|
100 |
+
steps:
|
101 |
+
# You can place your normal testing steps here. In this example, we are running SRH against the upstash/upstash-redis test suite.
|
102 |
+
- name: Checkout code
|
103 |
+
uses: actions/checkout@v3
|
104 |
+
with:
|
105 |
+
repository: upstash/upstash-redis
|
106 |
+
|
107 |
+
- name: Run @upstash/redis Test Suite
|
108 |
+
run: deno test -A ./pkg
|
109 |
+
env:
|
110 |
+
UPSTASH_REDIS_REST_URL: http://srh:80
|
111 |
+
UPSTASH_REDIS_REST_TOKEN: ${{ env.SRH_TOKEN }}
|
112 |
+
```
|
113 |
|
114 |
+
# Configuration Options
|
|
|
|
|
|
|
115 |
|
116 |
+
SRH works with multiple Redis servers, and can pool however many connections you wish it to. It will shut down un-used pools after 15 minutes of inactivity. Upon the next command, it will re-build the pool.
|
|
|
117 |
|
118 |
+
## Connecting to multiple Redis servers at the same time
|
119 |
|
120 |
+
The examples above use environment variables in order to tell SRH which Redis server to connect to. However, you can also use a configuration JSON file, which lets you create as many connections as you wish. The token provided in each request will decide which pool is used.
|
121 |
|
122 |
+
Create a JSON file, in this example called `tokens.json`:
|
|
|
123 |
```json
|
124 |
{
|
125 |
"example_token": {
|
126 |
"srh_id": "some_unique_identifier",
|
127 |
"connection_string": "redis://localhost:6379",
|
128 |
"max_connections": 3
|
129 |
+
}
|
130 |
}
|
131 |
```
|
132 |
+
You can provide as many entries to the base object as you wish, and configure the number of max connections per pool. The `srh_id` is used internally to keep track of instances. It can be anything you want.
|
133 |
|
134 |
+
Once you have created this, mount it to the docker container to the `/app/srh-config/tokens.json` file. Here is an example docker command:
|
|
|
|
|
|
|
|
|
|
|
|
|
135 |
|
136 |
+
`docker run -it -d -p 8079:80 --name srh --mount type=bind,source=$(pwd)/tokens.json,target=/app/srh-config/tokens.json hiett/serverless-redis-http:latest`
|
137 |
+
|
138 |
+
## Environment Variables
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
139 |
|
140 |
+
| Name | Default Value | Notes |
|
141 |
+
| ---- | ------------- | ----- |
|
142 |
+
| SRH_MODE | `file` | Can be `env` or `file`. If `file`, see [Connecting to multiple Redis servers](#connecting-to-multiple-redis-servers-at-the-same-time). If set to `env`, you are required to provide the following environment variables: |
|
143 |
+
| SRH_TOKEN | `<required if SRH_MODE = env>` | Set the token that the Rest API will require |
|
144 |
+
| SRH_CONNECTION_STRING | `<required if SRH_MODE = env>` | Sets the connection string to the Redis server. |
|
145 |
+
| SRH_MAX_CONNECTIONS | `3` | Only used if `SRH_MODE=env`.
|