Add support for Hugging Face Persistent Storage

Add documentation on how to enable Hugging Face Persistent Storage
for database and task storage.

Dockerfile

@@ -25,13 +25,44 @@ FROM heartexlabs/label-studio:hf-latest
 
 ################################################################################
 #
-# How to Enable Configuration Persistence
-# ---------------------------------------
+# How to Enable Persistent Storage for Label Studio in Hugging Face Spaces
+# ------------------------------------------------------------------------
+#
 # By default this space stores all project configuration and data annotations
-# in local storage with Sqlite. If the space is reset, all configuration and
+# in local storage with sqlite. If the space is reset, all configuration and
 # annotation data in the space will be lost. You can enable configuration
-# persistence by connecting an external Postgres database to your space,
-# guaranteeing that all project and annotation settings are preserved.
+# persistence through one of two methods:
+#
+# 1) Enabling Hugging Face Persistent Storage for saving project and annotation
+#    settings, as well as local task storage.
+# 2) Connecting an external Postgres database for saving project and annotation
+#    settings, and cloud by connecting cloud storage for tasks.
+#
+################################################################################
+
+################################################################################
+#
+# How to Enable Hugging Face Persistent Storage for Label Studio
+# --------------------------------------------------------------
+#
+# In the Hugging Face Label Studio Space settings, select the appropriate
+# Persistent Storage tier. Note that Persistent Storage is a paid add-on.
+# By default, persistent storage is mounted to /data. In your Space settings,
+# set the following variables:
+#
+# LABEL_STUDIO_BASE_DATA_DIR=/data
+# ENV STORAGE_PERSISTENCE=1
+#
+# Your space will restart. NOTE: if you have existing settings and data,
+# they will be lost in this first restart. Data and setting will only be
+# preserved on subsequent restarts of the space.
+#
+################################################################################
+
+################################################################################
+#
+# How to Enable Configuration Persistence with Postgres
+# -----------------------------------------------------
 #
 # Set the following secret variables to match your own hosted instance of
 # Postgres. We strongly recommend setting these as secrets to prevent leaking
@@ -46,16 +77,14 @@ FROM heartexlabs/label-studio:hf-latest
 # ENV POSTGRE_PORT=<db_port>
 # ENV POSTGRE_HOST=<db_host>
 #
-# Uncomment the following line to remove the warning about ephemeral storage
+# Uncomment the following line or set the following Space variable to remove
+# the warning about ephemeral storage
 #
 # ENV STORAGE_PERSISTENCE=1
 #
 # Note that you will need to connect cloud storage to host data items that you
 # want to annotate, as local storage will not be preserved across a space reset.
 #
-################################################################################
-
-################################################################################
 #
 # How to Enable Cloud Storage
 # ---------------------------
@@ -74,25 +103,24 @@ FROM heartexlabs/label-studio:hf-latest
 # STORAGE_AWS_BUCKET_NAME="<YOUR_BUCKET_NAME>"
 # STORAGE_AWS_REGION_NAME="<YOUR_BUCKET_REGION>"
 # STORAGE_AWS_FOLDER=""
-# 
+#
 # Google Cloud Storage
 # ====================
-# 
+#
 # STORAGE_TYPE=gcs
 # STORAGE_GCS_BUCKET_NAME="<YOUR_BUCKET_NAME>"
 # STORAGE_GCS_PROJECT_ID="<YOUR_PROJECT_ID>"
 # STORAGE_GCS_FOLDER=""
 # GOOGLE_APPLICATION_CREDENTIALS="/opt/heartex/secrets/key.json"
-# 
+#
 # Azure Blob Storage
 # ==================
-# 
+#
 # STORAGE_TYPE=azure
 # STORAGE_AZURE_ACCOUNT_NAME="<YOUR_STORAGE_ACCOUNT>"
 # STORAGE_AZURE_ACCOUNT_KEY="<YOUR_STORAGE_KEY>"
 # STORAGE_AZURE_CONTAINER_NAME="<YOUR_CONTAINER_NAME>"
 # STORAGE_AZURE_FOLDER=""
-# 
 #
 ################################################################################
 

README.md

@@ -37,7 +37,7 @@ credentials.
 
 **By default, these spaces permit anyone to create a new login
 account, allowing them to view and modify project configuration, data sets, and
-annotations. Without any modifications, treat this space like a demo environment.** 
+annotations. Without any modifications, treat this space like a demo environment.**
 
 ## Creating a Labeling Project
 
@@ -58,7 +58,7 @@ resources including tutorials and documentation.
 - 🤗 [Tutorial: Using Label Studio with Hugging Face Datasets Hub](https://danielvanstrien.xyz/huggingface/huggingface-datasets/annotation/full%20stack%20deep%20learning%20notes/2022/09/07/label-studio-annotations-hub.html)
 - 💡 [Label Studio Docs](https://hubs.ly/Q01CN9Yq0)
 
-  
+ 
 ![Gif of Label Studio annotating different types of data](https://raw.githubusercontent.com/heartexlabs/label-studio/master/images/annotation_examples.gif)
 
 ### Making your Label Studio Hugging Face Space production-ready
@@ -68,8 +68,8 @@ will full access to all projects and data. This is great for trying out
 Label Studio and collaborating on projects, but you may want to restrict
 access to your space to only authorized users. Add the following environment
 variable to your spaces Dockerfile to disable public account creation for
-this space. 
- 
+this space.
+
     ENV LABEL_STUDIO_DISABLE_SIGNUP_WITHOUT_LINK=true
 
 Set secrets in your space to create an inital user, and log in with your
@@ -80,13 +80,34 @@ globally visible on a public space.
     LABEL_STUDIO_PASSWORD
 
 You will need to provide new users with an invitation link to join the space,
-which can be found in the Organizations interface of Label Studio
+which can be found in the Organizations interface of Label Studio.
 
 By default this space stores all project configuration and data annotations
 in local storage with Sqlite. If the space is reset, all configuration and
 annotation data in the space will be lost. You can enable configuration
-persistence by connecting an external Postgres database to your space,
-guaranteeing that all project and annotation settings are preserved.
+persistence in one of two ways:
+
+1. Enabling Persistent Storage in your Space settings and configuring Label
+   Studio to write its database and task storage there.
+
+2. Connecting an external Postgres database and cloud storage to your space,
+   guaranteeing that all project and annotation settings are preserved.
+
+### Enabling Hugging Face Persistent Storage
+
+In the Hugging Face Label Studio Space settings, select the appropriate
+Persistent Storage tier. Note that Persistent Storage is a paid add-on.
+By default, persistent storage is mounted to /data. In your Space settings,
+set the following variables:
+
+    LABEL_STUDIO_BASE_DATA_DIR=/data
+    ENV STORAGE_PERSISTENCE=1
+
+Your space will restart. NOTE: if you have existing settings and data,
+they will be lost in this first restart. Data and setting will only be
+preserved on subsequent restarts of the space.
+
+### Enabling Postgres Database and Cloud Storage
 
 Set the following secret variables to match your own hosted instance of
 Postgres. We strongly recommend setting these as secrets to prevent leaking
@@ -142,6 +163,6 @@ Azure Blob Storage
     STORAGE_AZURE_FOLDER=""
 
 
-## Questions? Concerns? Want to get involved? 
+## Questions? Concerns? Want to get involved?
 
 Email the community team at [[email protected]](mailto:[email protected])