<commit_before># Using TensorFlow Serving via Docker

One of the easiest ways to get started using TensorFlow Serving is via
[Docker](http://www.docker.com/).

## Installing Docker

General installation instructions are
[on the Docker site](https://docs.docker.com/install/), but we give some quick
links here:

*   [Docker for macOS](https://docs.docker.com/docker-for-mac/install/)
*   [Docker for Windows](https://docs.docker.com/docker-for-windows/install/)
    for Windows 10 Pro or later
*   [Docker Toolbox](https://docs.docker.com/toolbox/) for much older versions
    of macOS, or versions of Windows before Windows 10 Pro

## Serving with Docker

### Pulling a serving image

Once you have Docker installed, you can pull the latest TensorFlow Serving
docker image by running:

```shell
docker pull tensorflow/serving
```

This will pull down an minimal Docker image with TensorFlow Serving installed.

See the Docker Hub
[tensorflow/serving repo](http://hub.docker.com/r/tensorflow/serving/tags/) for
other versions of images you can pull.

### Serving example

Once you have pulled the serving image, you can try serving an example model.

We will use a toy model called `Half Plus Two`, which generates `0.5 * x + 2`
for the values of `x` we provide for prediction.

To get this model, first clone the TensorFlow Serving repo.

```shell
mkdir -p /tmp/tfserving
cd /tmp/tfserving
git clone --depth=1 https://github.com/tensorflow/serving
```

Next, run the TensorFlow Serving container pointing it to this model and opening
the REST API port (8501):

```shell
docker run -p 8501:8501 \
--mount type=bind,\
source=/tmp/tfserving/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu,\
target=/models/half_plus_two \
-e MODEL_NAME=half_plus_two -t tensorflow/serving &
```

This will run the docker container and launch the TensorFlow Serving Model
Server, bind the REST API port 8501, and map our desired model from our host to
where models are expected in the container. We also pass the name of the model
as an environment variable, which will be important when we query the model.

To query the model using the predict API, you can run

```shell
curl -d '{"instances": [1.0, 2.0, 5.0]}' -X POST http://localhost:8501/v1/models/half_plus_two:predict
```

NOTE: Older versions of Windows and other systems without curl can download it
[here](https://curl.haxx.se/download.html).

This should return a set of values:

```json
{ "predictions": [2.5, 3.0, 4.5] }
```

More information on using the RESTful API can be found [here](api_rest.md).

## Serving with Docker using your GPU<commit_msg>Add a note re gRPC port to the Docker documentation<commit_after># Using TensorFlow Serving via Docker

One of the easiest ways to get started using TensorFlow Serving is via
[Docker](http://www.docker.com/).

## Installing Docker

General installation instructions are
[on the Docker site](https://docs.docker.com/install/), but we give some quick
links here:

*   [Docker for macOS](https://docs.docker.com/docker-for-mac/install/)
*   [Docker for Windows](https://docs.docker.com/docker-for-windows/install/)
    for Windows 10 Pro or later
*   [Docker Toolbox](https://docs.docker.com/toolbox/) for much older versions
    of macOS, or versions of Windows before Windows 10 Pro

## Serving with Docker

### Pulling a serving image

Once you have Docker installed, you can pull the latest TensorFlow Serving
docker image by running:

```shell
docker pull tensorflow/serving
```

This will pull down an minimal Docker image with TensorFlow Serving installed.

See the Docker Hub
[tensorflow/serving repo](http://hub.docker.com/r/tensorflow/serving/tags/) for
other versions of images you can pull.

### Serving example

Once you have pulled the serving image, you can try serving an example model.

We will use a toy model called `Half Plus Two`, which generates `0.5 * x + 2`
for the values of `x` we provide for prediction.

To get this model, first clone the TensorFlow Serving repo.

```shell
mkdir -p /tmp/tfserving
cd /tmp/tfserving
git clone --depth=1 https://github.com/tensorflow/serving
```

Next, run the TensorFlow Serving container pointing it to this model and opening
the REST API port (8501):

```shell
docker run -p 8501:8501 \
--mount type=bind,\
source=/tmp/tfserving/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu,\
target=/models/half_plus_two \
-e MODEL_NAME=half_plus_two -t tensorflow/serving &
```

This will run the docker container and launch the TensorFlow Serving Model
Server, bind the REST API port 8501, and map our desired model from our host to
where models are expected in the container. We also pass the name of the model
as an environment variable, which will be important when we query the model.

Note that if you wanted to use the gRPC API, you should instead bind to port
8500.

To query the model using the predict API, you can run

```shell
curl -d '{"instances": [1.0, 2.0, 5.0]}' -X POST http://localhost:8501/v1/models/half_plus_two:predict
```

NOTE: Older versions of Windows and other systems without curl can download it
[here](https://curl.haxx.se/download.html).

This should return a set of values:

```json
{ "predictions": [2.5, 3.0, 4.5] }
```

More information on using the RESTful API can be found [here](api_rest.md).

## Serving with Docker using your GPU
{'fraction_non_alphanumeric': 0.09532640949554896, 'fraction_numerical': 0.015949554896142432, 'mean_word_length': 4.2820763956905, 'pattern_counts': {'":': 4, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 12, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '346483', 'n_tokens_mistral': 1714, 'n_tokens_neox': 1564, 'n_words': 683}