metadata

license: apache-2.0

Template for Concrete ML

Concrete ML is Zama's open-source privacy-preserving ML package, based on fully homomorphic encryption (FHE). We refer the reader to fhe.org or Zama's websites for more information on FHE.

This directory is used:

by ML practicioners, to create Concrete ML FHE-friendly models, and make them available to HF users
by companies, institutions or people to deploy those models over HF inference endpoints
by developers, to use these entry points to make applications on privacy-preserving ML

Creating models and making them available on HF

This is quite easy. Fork this template (maybe use this experimental tool https://huggingface.co/spaces/huggingface-projects/repo_duplicator for that), and then:

install everything with: pip install -r requirements.txt
edit creating_models.py, and fill the part between "# BEGIN: insert your ML task here" and "# END: insert your ML task here"
run the python file: python creating_models.py

At the end, if the script is successful, you'll have your compiled model ready in compiled_model. Now you can commit and push your repository (with in particular compiled_model, handler.py, play_with_endpoint.py and requirements.txt, but you can include the other files as well).

We recommend you to tag your Concrete ML compiled repository with Concrete ML FHE friendly tag, such that people can find them easily.

Deploying a compiled model on HF inference endpoint

If you find an Concrete ML FHE friendly repository that you would like to deploy, it is very easy.

click on 'Deploy' button in HF interface
chose "Inference endpoints"
chose the right model repository
(the rest of the options are classical to HF end points; we refer you to their documentation for more information) and then click on 'Create endpoint'

And now, your model should be deployed, after few secunds of installation.

Using HF entry points on privacy-preserving models

Now, this is the final step: using the entry point. You should:

if your inference endpoint is private, set an environment variable HF_TOKEN with your HF token
edit play_with_endpoint.py
replace API_URL by your entry point URL
replace the part between "# BEGIN: replace this part with your privacy-preserving application" and "# END: replace this part with your privacy-preserving application" with your application

Finally, you'll be able to launch your application with python play_with_endpoint.py.