Add new SentenceTransformer model

README.md

@@ -7,729 +7,472 @@ tags:
 - sentence-similarity
 - feature-extraction
 - generated_from_trainer
-- dataset_size:3284
+- dataset_size:36
 - loss:MatryoshkaLoss
 - loss:MultipleNegativesRankingLoss
 base_model: Snowflake/snowflake-arctic-embed-m-v1.5
 widget:
-- source_sentence: How do I register and activate a stack with a new orchestrator
-    using ZenML?
+- source_sentence: Where can I find older versions of ZenML documentation?
   sentences:
-  - "hestrator registered and part of our active stack:zenml orchestrator register\
-    \ <ORCHESTRATOR_NAME> \\\n    --flavor=airflow \\\n    --local=True  # set this\
-    \ to `False` if using a remote Airflow deployment\n\n# Register and activate a\
-    \ stack with the new orchestrator\nzenml stack register <STACK_NAME> -o <ORCHESTRATOR_NAME>\
-    \ ... --set\n\nDue to dependency conflicts, we need to install the Python packages\
-    \ to start a local Airflow server in a separate Python environment.\n\n# Create\
-    \ a fresh virtual environment in which we install the Airflow server dependencies\n\
-    python -m venv airflow_server_environment\nsource airflow_server_environment/bin/activate\n\
-    \n# Install the Airflow server dependencies\npip install \"apache-airflow==2.4.0\"\
-    \ \"apache-airflow-providers-docker<3.8.0\" \"pydantic~=2.7.1\"\n\nBefore starting\
-    \ the local Airflow server, we can set a few environment variables to configure\
-    \ it:\n\nAIRFLOW_HOME: This variable defines the location where the Airflow server\
-    \ stores its database and configuration files. The default value is ~/airflow.\n\
-    \nAIRFLOW__CORE__DAGS_FOLDER: This variable defines the location where the Airflow\
-    \ server looks for DAG files. The default value is <AIRFLOW_HOME>/dags.\n\nAIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL:\
-    \ This variable controls how often the Airflow scheduler checks for new or updated\
-    \ DAGs. By default, the scheduler will check for new DAGs every 30 seconds. This\
-    \ variable can be used to increase or decrease the frequency of the checks, depending\
-    \ on the specific needs of your pipeline.\n\nWhen running this on MacOS, you might\
-    \ need to set the no_proxy environment variable to prevent crashes due to a bug\
-    \ in Airflow (see this page for more information):\n\nexport no_proxy=*\n\nWe\
-    \ can now start the local Airflow server by running the following command:\n\n\
-    # Switch to the Python environment that has Airflow installed before running this\
-    \ command\nairflow standalone"
-  - "ta stores you want to migrate, then upgrade ZenML.Decide the ZenML deployment\
-    \ model that you want to follow for your projects. See the ZenML deployment documentation\
-    \ for available deployment scenarios. If you decide on using a local or remote\
-    \ ZenML server to manage your pipelines, make sure that you first connect your\
-    \ client to it by running zenml connect.\n\nUse the zenml pipeline runs migrate\
-    \ CLI command to migrate your old pipeline runs:\n\nIf you want to migrate from\
-    \ a local SQLite metadata store, you only need to pass the path to the metadata\
-    \ store to the command, e.g.:\n\nzenml pipeline runs migrate PATH/TO/LOCAL/STORE/metadata.db\n\
-    \nIf you would like to migrate any other store, you will need to set --database_type=mysql\
-    \ and provide the MySQL host, username, and password in addition to the database,\
-    \ e.g.:\n\nzenml pipeline runs migrate DATABASE_NAME \\\n  --database_type=mysql\
-    \ \\\n  --mysql_host=URL/TO/MYSQL \\\n  --mysql_username=MYSQL_USERNAME \\\n \
-    \ --mysql_password=MYSQL_PASSWORD\n\n\U0001F4BE The New Way (CLI Command Cheat\
-    \ Sheet)\n\nDeploy the server\n\nzenml deploy --aws (maybe don’t do this :) since\
-    \ it spins up infrastructure on AWS…)\n\nSpin up a local ZenML Server\n\nzenml\
-    \ up\n\nConnect to a pre-existing server\n\nzenml connect (pass in URL / etc,\
-    \ or zenml connect --config + yaml file)\n\nList your deployed server details\n\
-    \nzenml status\n\nThe ZenML Dashboard is now available\n\nThe new ZenML Dashboard\
-    \ is now bundled into the ZenML Python package and can be launched directly from\
-    \ Python. The source code lives in the ZenML Dashboard repository.\n\nTo launch\
-    \ it locally, simply run zenml up on your machine and follow the instructions:\n\
-    \n$ zenml up\nDeploying a local ZenML server with name 'local'.\nConnecting ZenML\
-    \ to the 'local' local ZenML server (http://127.0.0.1:8237).\nUpdated the global\
-    \ store configuration.\nConnected ZenML to the 'local' local ZenML server (http://127.0.0.1:8237).\n\
-    The local ZenML dashboard is available at 'http://127.0.0.1:8237'. You can\nconnect\
-    \ to it using the 'default' username and an empty password."
-  - '🐍Configure Python environments
-
-
-    Navigating multiple development environments.
-
-
-    PreviousHyperAI Service ConnectorNextHandling dependencies
-
-
-    Last updated 21 days ago'
-- source_sentence: How do you build a simple machine learning pipeline using ZenML
-    decorators in the code?
-  sentences:
-  - 'Develop a custom data validator
-
-
-    How to develop a custom data validator
-
-
-    Before diving into the specifics of this component type, it is beneficial to familiarize
-    yourself with our general guide to writing custom component flavors in ZenML.
-    This guide provides an essential understanding of ZenML''s component flavor concepts.
-
-
-    Base abstraction in progress!
-
-
-    We are actively working on the base abstraction for the Data Validators, which
-    will be available soon. As a result, their extension is not recommended at the
-    moment. When you are selecting a data validator for your stack, you can use one
-    of the existing flavors.
-
-
-    If you need to implement your own Data Validator flavor, you can still do so,
-    but keep in mind that you may have to refactor it when the base abstraction is
-    updated.
-
-
-    ZenML comes equipped with Data Validator implementations that integrate a variety
-    of data logging and validation libraries, frameworks and platforms. However, if
-    you need to use a different library or service as a backend for your ZenML Data
-    Validator, you can extend ZenML to provide your own custom Data Validator implementation.
-
-
-    Build your own custom data validator
-
-
-    If you want to implement your own custom Data Validator, you can follow the following
-    steps:
-
-
-    Create a class which inherits from the BaseDataValidator class and override one
-    or more of the abstract methods, depending on the capabilities of the underlying
-    library/service that you want to integrate.
-
-
-    If you need any configuration, you can create a class which inherits from the
-    BaseDataValidatorConfig class.
-
-
-    Bring both of these classes together by inheriting from the BaseDataValidatorFlavor.
-
-
-    (Optional) You should also provide some standard steps that others can easily
-    insert into their pipelines for instant access to data validation features.
-
-
-    Once you are done with the implementation, you can register it through the CLI.
-    Please ensure you point to the flavor class via dot notation:'
-  - " This is us if you want to put faces to the names!However, in order to improve\
-    \ ZenML and understand how it is being used, we need to use analytics to have\
-    \ an overview of how it is used 'in the wild'. This not only helps us find bugs\
-    \ but also helps us prioritize features and commands that might be useful in future\
-    \ releases. If we did not have this information, all we really get is pip download\
-    \ statistics and chatting with people directly, which while being valuable, is\
-    \ not enough to seriously better the tool as a whole.\n\nHow does ZenML collect\
-    \ these statistics?\n\nWe use Segment as the data aggregation library for all\
-    \ our analytics. However, before any events get sent to Segment, they first go\
-    \ through a central ZenML analytics server. This added layer allows us to put\
-    \ various countermeasures to incidents such as getting spammed with events and\
-    \ enables us to have a more optimized tracking process.\n\nThe client code is\
-    \ entirely visible and can be seen in the analytics module of our main repository.\n\
-    \nIf I share my email, will you spam me?\n\nNo, we won't. Our sole purpose of\
-    \ contacting you will be to ask for feedback (e.g. in the shape of a user interview).\
-    \ These interviews help the core team understand usage better and prioritize feature\
-    \ requests. If you have any concerns about data privacy and the usage of personal\
-    \ information, please contact us, and we will try to alleviate any concerns as\
-    \ soon as possible.\n\nVersion mismatch (downgrading)\n\nIf you've recently downgraded\
-    \ your ZenML version to an earlier release or installed a newer version on a different\
-    \ environment on the same machine, you might encounter an error message when running\
-    \ ZenML that says:\n\n`The ZenML global configuration version (%s) is higher than\
-    \ the version of ZenML \ncurrently being used (%s).`\n\nWe generally recommend\
-    \ using the latest ZenML version. However, there might be cases where you need\
-    \ to match the global configuration version with the version of ZenML installed\
-    \ in the current environment. To do this, run the following command:\n\nzenml\
-    \ downgrade"
-  - "⛓️Build a pipeline\n\nBuilding pipelines is as simple as adding the `@step` and\
-    \ `@pipeline` decorators to your code.\n\n@step  # Just add this decorator\ndef\
-    \ load_data() -> dict:\n    training_data = [[1, 2], [3, 4], [5, 6]]\n    labels\
-    \ = [0, 1, 0]\n    return {'features': training_data, 'labels': labels}\n\n@step\n\
-    def train_model(data: dict) -> None:\n    total_features = sum(map(sum, data['features']))\n\
-    \    total_labels = sum(data['labels'])\n\n# Train some model here\n\nprint(f\"\
-    Trained model using {len(data['features'])} data points. \"\n          f\"Feature\
-    \ sum is {total_features}, label sum is {total_labels}\")\n\n@pipeline  # This\
-    \ function combines steps together \ndef simple_ml_pipeline():\n    dataset =\
-    \ load_data()\n    train_model(dataset)\n\nYou can now run this pipeline by simply\
-    \ calling the function:\n\nsimple_ml_pipeline()\n\nWhen this pipeline is executed,\
-    \ the run of the pipeline gets logged to the ZenML dashboard where you can now\
-    \ go to look at its DAG and all the associated metadata. To access the dashboard\
-    \ you need to have a ZenML server either running locally or remotely. See our\
-    \ documentation on this here.\n\nCheck below for more advanced ways to build and\
-    \ interact with your pipeline.\n\nConfigure pipeline/step parameters\n\nName and\
-    \ annotate step outputs\n\nControl caching behavior\n\nRun pipeline from a pipeline\n\
-    \nControl the execution order of steps\n\nCustomize the step invocation ids\n\n\
-    Name your pipeline runs\n\nUse failure/success hooks\n\nHyperparameter tuning\n\
-    \nAttach metadata to steps\n\nFetch metadata within steps\n\nFetch metadata during\
-    \ pipeline composition\n\nEnable or disable logs storing\n\nSpecial Metadata Types\n\
-    \nAccess secrets in a step\n\nPreviousBest practicesNextUse pipeline/step parameters\n\
-    \nLast updated 1 month ago"
-- source_sentence: How can I integrate Large Language Models (LLMs) into my MLOps
-    workflows using ZenML?
-  sentences:
-  - '🦜LLMOps guide
-
-
-    Leverage the power of LLMs in your MLOps workflows with ZenML.
-
-
-    Welcome to the ZenML LLMOps Guide, where we dive into the exciting world of Large
-    Language Models (LLMs) and how to integrate them seamlessly into your MLOps pipelines
-    using ZenML. This guide is designed for ML practitioners and MLOps engineers looking
-    to harness the potential of LLMs while maintaining the robustness and scalability
-    of their workflows.
-
-
-    In this guide, we''ll explore various aspects of working with LLMs in ZenML, including:
-
-
-    RAG with ZenML
-
-
-    RAG in 85 lines of code
-
-
-    Understanding Retrieval-Augmented Generation (RAG)
-
-
-    Data ingestion and preprocessing
-
-
-    Embeddings generation
-
-
-    Storing embeddings in a vector database
-
-
-    Basic RAG inference pipeline
-
-
-    Evaluation and metrics
-
-
-    Evaluation in 65 lines of code
-
-
-    Retrieval evaluation
-
-
-    Generation evaluation
-
-
-    Evaluation in practice
-
-
-    Reranking for better retrieval
-
-
-    Understanding reranking
-
-
-    Implementing reranking in ZenML
-
-
-    Evaluating reranking performance
-
-
-    Improve retrieval by finetuning embeddings
-
-
-    Synthetic data generation
-
-
-    Finetuning embeddings with Sentence Transformers
-
-
-    Evaluating finetuned embeddings
-
-
-    Finetuning LLMs with ZenML
-
-
-    To follow along with the examples and tutorials in this guide, ensure you have
-    a Python environment set up with ZenML installed. Familiarity with the concepts
-    covered in the Starter Guide and Production Guide is recommended.
-
-
-    We''ll showcase a specific application over the course of this LLM guide, showing
-    how you can work from a simple RAG pipeline to a more complex setup that involves
-    finetuning embeddings, reranking retrieved documents, and even finetuning the
-    LLM itself. We''ll do this all for a use case relevant to ZenML: a question answering
-    system that can provide answers to common questions about ZenML. This will help
-    you understand how to apply the concepts covered in this guide to your own projects.'
-  - ' data with tags
-
-
-    Get arbitrary artifacts in a stepHandle custom data types
-
-
-    Load artifacts into memory
-
-
-    Datasets in ZenML
-
-
-    Manage big data
-
-
-    Skipping materialization
-
-
-    Passing artifacts between pipelines
-
-
-    Register Existing Data as a ZenML Artifact
-
-
-    📊Visualizing artifacts
-
-
-    Default visualizations
-
-
-    Creating custom visualizations
-
-
-    Displaying visualizations in the dashboard
-
-
-    Disabling visualizations
-
-
-    🪆Use the Model Control Plane
+  - '🧙Installation
 
 
-    Registering a Model
+    Installing ZenML and getting started.
 
 
-    Deleting a Model
+    ZenML is a Python package that can be installed directly via pip:
 
 
-    Associate a pipeline with a Model
+    pip install zenml
 
 
-    Connecting artifacts via a Model
+    Note that ZenML currently supports Python 3.8, 3.9, 3.10, and 3.11. Please make
+    sure that you are using a supported Python version.
 
 
-    Controlling Model versions
+    Install with the dashboard
 
 
-    Load a Model in code
+    ZenML comes bundled with a web dashboard that lives inside a sister repository.
+    In order to get access to the dashboard locally, you need to launch the ZenML
+    Server and Dashboard locally. For this, you need to install the optional dependencies
+    for the ZenML Server:
 
 
-    Promote a Model
-
-
-    Linking model binaries/data to a Model
-
-
-    Load artifacts from Model
-
-
-    📈Track metrics and metadata
-
-
-    Attach metadata to a model
-
-
-    Attach metadata to an artifact
-
-
-    Attach metadata to steps
-
-
-    Group metadata
-
-
-    Special Metadata Types
-
-
-    Fetch metadata within steps
-
-
-    Fetch metadata during pipeline composition
-
-
-    👨‍🎤Popular integrations
-
-
-    Run on AWS
-
-
-    Run on GCP
-
-
-    Run on Azure
-
-
-    Kubeflow
-
-
-    Kubernetes
-
-
-    MLflow
-
-
-    Skypilot
-
-
-    🔌Connect services (AWS, GCP, Azure, K8s etc)
-
-
-    Service Connectors guide
-
-
-    Security best practices
-
-
-    Docker Service Connector
-
-
-    Kubernetes Service Connector
-
-
-    AWS Service Connector
-
-
-    GCP Service Connector
-
-
-    Azure Service Connector
-
-
-    HyperAI Service Connector
-
-
-    🐍Configure Python environments
-
-
-    Handling dependencies
-
-
-    Configure the server environment
-
-
-    🔌Connect to a server
-
-
-    Connect in with your User (interactive)
-
-
-    Connect with a Service Account
-
-
-    🔐Interact with secrets
-
-
-    🐞Debug and solve issues
-
-
-    🤝Contribute to ZenML
-
-
-    Implement a custom integration
-
-
-    Stack Components
-
-
-    📜Overview
-
-
-    🔋Orchestrators
-
-
-    Local Orchestrator
-
-
-    Local Docker Orchestrator
+    pip install "zenml[server]"
 
 
-    Kubeflow Orchestrator
+    We highly encourage you to install ZenML in a virtual environment. At ZenML, We
+    like to use virtualenvwrapper or pyenv-virtualenv to manage our Python virtual
+    environments.
 
 
-    Kubernetes Orchestrator
+    Installing onto MacOS with Apple Silicon (M1, M2)
 
 
-    Google Cloud VertexAI Orchestrator
+    A change in how forking works on Macs running on Apple Silicon means that you
+    should set the following environment variable which will ensure that your connections
+    to the server remain unbroken:
 
 
-    AWS Sagemaker Orchestrator
+    export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
 
 
-    AzureML Orchestrator
+    You can read more about this here. This environment variable is needed if you
+    are working with a local server on your Mac, but if you''re just using ZenML as
+    a client / CLI and connecting to a deployed server then you don''t need to set
+    it.
 
 
-    Databricks Orchestrator
+    Nightly builds
 
 
-    Tekton Orchestrator
+    ZenML also publishes nightly builds under the zenml-nightly package name. These
+    are built from the latest develop branch (to which work ready for release is published)
+    and are not guaranteed to be stable. To install the nightly build, run:
 
 
-    Airflow Orchestrator
+    pip install zenml-nightly
 
 
-    Skypilot VM Orchestrator
+    Verifying installations
 
 
-    HyperAI Orchestrator
+    Once the installation is completed, you can check whether the installation was
+    successful either through Bash:
 
 
-    Lightning AI Orchestrator
+    zenml version
 
 
-    Develop a custom orchestrator
+    or through Python:
 
 
-    🏪Artifact Stores
+    import zenml
 
 
-    Local Artifact Store
+    print(zenml.__version__)
 
 
-    Amazon Simple Cloud Storage (S3)
+    If you would like to learn more about the current release, please visit our PyPi
+    package page.
 
 
-    Google Cloud Storage (GCS)'
-  - 'Troubleshoot the deployed server
+    Running with Docker'
+  - 'Finetuning LLMs with ZenML
 
 
-    Troubleshooting tips for your ZenML deployment
+    Finetune LLMs for specific tasks or to improve performance and cost.
 
 
-    In this document, we will go over some common issues that you might face when
-    deploying ZenML and how to solve them.
+    PreviousEvaluating finetuned embeddingsNextSet up a project repository
 
 
-    Viewing logs
+    Last updated 6 months ago'
+  - 'ZenML - Bridging the gap between ML & Ops
 
 
-    Analyzing logs is a great way to debug issues. Depending on whether you have a
-    Kubernetes (using Helm or zenml deploy) or a Docker deployment, you can view the
-    logs in different ways.
+    Legacy Docs
 
 
-    If you are using Kubernetes, you can view the logs of the ZenML server using the
-    following method:
+    Bleeding EdgeLegacy Docs0.67.0
 
 
-    Check all pods that are running your ZenML deployment.
+    🧙‍♂️Find older version our docs
 
 
-    kubectl -n <KUBERNETES_NAMESPACE> get pods
+    Powered by GitBook'
+- source_sentence: How can you visualize the profiles generated by whylogs in ZenML?
+  sentences:
+  - 'Whylogs
 
 
-    If you see that the pods aren''t running, you can use the command below to get
-    the logs for all pods at once.
+    How to collect and visualize statistics to track changes in your pipelines'' data
+    with whylogs/WhyLabs profiling.
 
 
-    kubectl -n <KUBERNETES_NAMESPACE> logs -l app.kubernetes.io/name=zenml
+    The whylogs/WhyLabs Data Validator flavor provided with the ZenML integration
+    uses whylogs and WhyLabs to generate and track data profiles, highly accurate
+    descriptive representations of your data. The profiles can be used to implement
+    automated corrective actions in your pipelines, or to render interactive representations
+    for further visual interpretation, evaluation and documentation.
 
 
-    Note that the error can either be from the zenml-db-init container that connects
-    to the MySQL database or from the zenml container that runs the server code. If
-    the get pods command shows that the pod is failing in the Init state then use
-    zenml-db-init as the container name, otherwise use zenml.
+    When would you want to use it?
 
 
-    kubectl -n <KUBERNETES_NAMESPACE> logs -l app.kubernetes.io/name=zenml -c <CONTAINER_NAME>
+    Whylogs is an open-source library that analyzes your data and creates statistical
+    summaries called whylogs profiles. Whylogs profiles can be processed in your pipelines
+    and visualized locally or uploaded to the WhyLabs platform, where more in depth
+    analysis can be carried out. Even though whylogs also supports other data types,
+    the ZenML whylogs integration currently only works with tabular data in pandas.DataFrame
+    format.
 
 
-    You can also use the --tail flag to limit the number of lines to show or the --follow
-    flag to follow the logs in real-time.
+    You should use the whylogs/WhyLabs Data Validator when you need the following
+    data validation features that are possible with whylogs and WhyLabs:
 
 
-    If you are using Docker, you can view the logs of the ZenML server using the following
-    method:
+    Data Quality: validate data quality in model inputs or in a data pipeline
 
 
-    If you used the zenml up --docker CLI command to deploy the Docker ZenML server,
-    you can check the logs with the command:
+    Data Drift: detect data drift in model input features
 
 
-    zenml logs -f
+    Model Drift: Detect training-serving skew, concept drift, and model performance
+    degradation
 
 
-    If you used the docker run command to manually deploy the Docker ZenML server,
-    you can check the logs with the command:
+    You should consider one of the other Data Validator flavors if you need a different
+    set of data validation features.
 
 
-    docker logs zenml -f
+    How do you deploy it?
 
 
-    If you used the docker compose command to manually deploy the Docker ZenML server,
-    you can check the logs with the command:
+    The whylogs Data Validator flavor is included in the whylogs ZenML integration,
+    you need to install it on your local machine to be able to register a whylogs
+    Data Validator and add it to your stack:
 
 
-    docker compose -p zenml logs -f
+    zenml integration install whylogs -y
 
 
-    Fixing database connection problems'
-- source_sentence: How can you disable artifact visualization in ZenML?
+    If you don''t need to connect to the WhyLabs platform to upload and store the
+    generated whylogs data profiles, the Data Validator stack component does not require
+    any configuration parameters. Adding it to a stack is as simple as running e.g.:'
+  - "Control caching behavior\n\nBy default steps in ZenML pipelines are cached whenever\
+    \ code and parameters stay unchanged.\n\n@step(enable_cache=True) # set cache\
+    \ behavior at step level\ndef load_data(parameter: int) -> dict:\n    ...\n\n\
+    @step(enable_cache=False) # settings at step level override pipeline level\ndef\
+    \ train_model(data: dict) -> None:\n    ...\n\n@pipeline(enable_cache=True) #\
+    \ set cache behavior at step level\ndef simple_ml_pipeline(parameter: int):\n\
+    \    ...\n\nCaching only happens when code and parameters stay the same.\n\nLike\
+    \ many other step and pipeline settings, you can also change this afterward:\n\
+    \n# Same as passing it in the step decorator\nmy_step.configure(enable_cache=...)\n\
+    \n# Same as passing it in the pipeline decorator\nmy_pipeline.configure(enable_cache=...)\n\
+    \nFind out here how to configure this in a YAML file\n\nPreviousStep output typing\
+    \ and annotationNextSchedule a pipeline\n\nLast updated 4 months ago"
+  - "ogsDataValidatorSettings,\n)\nfrom zenml import step@step(\n    settings={\n\
+    \        \"data_validator\": WhylogsDataValidatorSettings(\n            enable_whylabs=True,\
+    \ dataset_id=\"model-1\"\n        )\n    }\n)\ndef data_loader() -> Tuple[\n \
+    \   Annotated[pd.DataFrame, \"data\"],\n    Annotated[DatasetProfileView, \"profile\"\
+    ]\n]:\n    \"\"\"Load the diabetes dataset.\"\"\"\n    X, y = datasets.load_diabetes(return_X_y=True,\
+    \ as_frame=True)\n\n# merge X and y together\n    df = pd.merge(X, y, left_index=True,\
+    \ right_index=True)\n\nprofile = why.log(pandas=df).profile().view()\n    return\
+    \ df, profile\n\nHow do you use it?\n\nWhylogs's profiling functions take in a\
+    \ pandas.DataFrame dataset generate a DatasetProfileView object containing all\
+    \ the relevant information extracted from the dataset.\n\nThere are three ways\
+    \ you can use whylogs in your ZenML pipelines that allow different levels of flexibility:\n\
+    \ninstantiate, configure and insert the standard WhylogsProfilerStep shipped with\
+    \ ZenML into your pipelines. This is the easiest way and the recommended approach,\
+    \ but can only be customized through the supported step configuration parameters.\n\
+    \ncall the data validation methods provided by the whylogs Data Validator in your\
+    \ custom step implementation. This method allows for more flexibility concerning\
+    \ what can happen in the pipeline step, but you are still limited to the functionality\
+    \ implemented in the Data Validator.\n\nuse the whylogs library directly in your\
+    \ custom step implementation. This gives you complete freedom in how you are using\
+    \ whylogs's features.\n\nYou can visualize whylogs profiles in Jupyter notebooks\
+    \ or view them directly in the ZenML dashboard.\n\nThe whylogs standard step"
+- source_sentence: How can I build my own custom experiment tracker flavor in ZenML?
   sentences:
-  - 'Secret management
-
-
-    Configuring the secrets store.
-
-
-    PreviousCustom secret storesNextZenML Pro
-
+  - "e details of the deployment process from the user.It needs to act as a ZenML\
+    \ BaseService registry, where every BaseService instance is used as an internal\
+    \ representation of a remote model server (see the find_model_server abstract\
+    \ method). To achieve this, it must be able to re-create the configuration of\
+    \ a BaseService from information that is persisted externally, alongside, or even\
+    \ as part of the remote model server configuration itself. For example, for model\
+    \ servers that are implemented as Kubernetes resources, the BaseService instances\
+    \ can be serialized and saved as Kubernetes resource annotations. This allows\
+    \ the model deployer to keep track of all externally running model servers and\
+    \ to re-create their corresponding BaseService instance representations at any\
+    \ given time. The model deployer also defines methods that implement basic life-cycle\
+    \ management on remote model servers outside the coverage of a pipeline (see stop_model_server\
+    \ , start_model_server and delete_model_server).\n\nPutting all these considerations\
+    \ together, we end up with the following interface:\n\nfrom abc import ABC, abstractmethod\n\
+    from typing import Dict, List, Optional, Type\nfrom uuid import UUID\n\nfrom zenml.enums\
+    \ import StackComponentType\nfrom zenml.services import BaseService, ServiceConfig\n\
+    from zenml.stack import StackComponent, StackComponentConfig, Flavor\n\nDEFAULT_DEPLOYMENT_START_STOP_TIMEOUT\
+    \ = 300\n\nclass BaseModelDeployerConfig(StackComponentConfig):\n    \"\"\"Base\
+    \ class for all ZenML model deployer configurations.\"\"\"\n\nclass BaseModelDeployer(StackComponent,\
+    \ ABC):\n    \"\"\"Base class for all ZenML model deployers.\"\"\"\n\n@abstractmethod\n\
+    \    def perform_deploy_model(\n        self,\n        id: UUID,\n        config:\
+    \ ServiceConfig,\n        timeout: int = DEFAULT_DEPLOYMENT_START_STOP_TIMEOUT,\n\
+    \    ) -> BaseService:\n        \"\"\"Abstract method to deploy a model.\"\"\""
+  - 'Develop a custom experiment tracker
+
+
+    Learning how to develop a custom experiment tracker.
 
-    Last updated 21 days ago'
-  - ' visit our PyPi package page.
-
-
-    Running with Dockerzenml is also available as a Docker image hosted publicly on
-    DockerHub. Use the following command to get started in a bash environment with
-    zenml available:
-
-
-    docker run -it zenmldocker/zenml /bin/bash
 
+    Before diving into the specifics of this component type, it is beneficial to familiarize
+    yourself with our general guide to writing custom component flavors in ZenML.
+    This guide provides an essential understanding of ZenML''s component flavor concepts.
 
-    If you would like to run the ZenML server with Docker:
 
+    Base abstraction in progress!
 
-    docker run -it -d -p 8080:8080 zenmldocker/zenml-server
 
+    We are actively working on the base abstraction for the Experiment Tracker, which
+    will be available soon. As a result, their extension is not recommended at the
+    moment. When you are selecting an Experiment Tracker for your stack, you can use
+    one of the existing flavors.
 
-    Deploying the server
 
+    If you need to implement your own Experiment Tracker flavor, you can still do
+    so, but keep in mind that you may have to refactor it when the base abstraction
+    is released.
 
-    Though ZenML can run entirely as a pip package on a local system, complete with
-    the dashboard. You can do this easily:
 
+    Build your own custom experiment tracker
 
-    pip install "zenml[server]"
 
-    zenml up  # opens the dashboard locally
+    If you want to create your own custom flavor for an experiment tracker, you can
+    follow the following steps:
 
 
-    However, advanced ZenML features are dependent on a centrally-deployed ZenML server
-    accessible to other MLOps stack components. You can read more about it here.
+    Create a class that inherits from the BaseExperimentTracker class and implements
+    the abstract methods.
 
 
-    For the deployment of ZenML, you have the option to either self-host it or register
-    for a free ZenML Pro account.
+    If you need any configuration, create a class that inherits from the BaseExperimentTrackerConfig
+    class and add your configuration parameters.
 
 
-    PreviousIntroductionNextCore concepts
+    Bring both the implementation and the configuration together by inheriting from
+    the BaseExperimentTrackerFlavor class.
 
 
-    Last updated 20 days ago'
-  - "Disabling visualizations\n\nDisabling visualizations.\n\nIf you would like to\
-    \ disable artifact visualization altogether, you can set enable_artifact_visualization\
-    \ at either pipeline or step level:\n\n@step(enable_artifact_visualization=False)\n\
-    def my_step():\n    ...\n\n@pipeline(enable_artifact_visualization=False)\ndef\
-    \ my_pipeline():\n    ...\n\nPreviousDisplaying visualizations in the dashboardNextUse\
-    \ the Model Control Plane\n\nLast updated 21 days ago"
-- source_sentence: How can I programmatically manage secrets using the ZenML Client
-    API, and what are some of the methods available for tasks like fetching, updating,
-    and deleting secrets?
+    Once you are done with the implementation, you can register it through the CLI.
+    Please ensure you point to the flavor class via dot notation:
+
+
+    zenml experiment-tracker flavor register <path.to.MyExperimentTrackerFlavor>
+
+
+    For example, if your flavor class MyExperimentTrackerFlavor is defined in flavors/my_flavor.py,
+    you''d register it by doing:
+
+
+    zenml experiment-tracker flavor register flavors.my_flavor.MyExperimentTrackerFlavor'
+  - "res Spark to handle the resource configuration.\"\"\"def _backend_configuration(\n\
+    \            self,\n            spark_config: SparkConf,\n            step_config:\
+    \ \"StepConfiguration\",\n    ) -> None:\n        \"\"\"Configures Spark to handle\
+    \ backends like YARN, Mesos or Kubernetes.\"\"\"\n\ndef _io_configuration(\n \
+    \           self,\n            spark_config: SparkConf\n    ) -> None:\n     \
+    \   \"\"\"Configures Spark to handle different input/output sources.\"\"\"\n\n\
+    def _additional_configuration(\n            self,\n            spark_config: SparkConf\n\
+    \    ) -> None:\n        \"\"\"Appends the user-defined configuration parameters.\"\
+    \"\"\n\ndef _launch_spark_job(\n            self,\n            spark_config: SparkConf,\n\
+    \            entrypoint_command: List[str]\n    ) -> None:\n        \"\"\"Generates\
+    \ and executes a spark-submit command.\"\"\"\n\ndef launch(\n            self,\n\
+    \            info: \"StepRunInfo\",\n            entrypoint_command: List[str],\n\
+    \    ) -> None:\n        \"\"\"Launches the step on Spark.\"\"\"\n\nUnder the\
+    \ base configuration, you will see the main configuration parameters:\n\nmaster\
+    \ is the master URL for the cluster where Spark will run. You might see different\
+    \ schemes for this URL with varying cluster managers such as Mesos, YARN, or Kubernetes.\n\
+    \ndeploy_mode can either be 'cluster' (default) or 'client' and it decides where\
+    \ the driver node of the application will run.\n\nsubmit_args is the JSON string\
+    \ of a dictionary, which will be used to define additional parameters if required\
+    \ ( Spark has a wide variety of parameters, thus including them all in a single\
+    \ class was deemed unnecessary.).\n\nIn addition to this configuration, the launch\
+    \ method of the step operator gets additional configuration parameters from the\
+    \ DockerSettings and ResourceSettings. As a result, the overall configuration\
+    \ happens in 4 base methods:\n\n_resource_configuration translates the ZenML ResourceSettings\
+    \ object to Spark's own resource configuration.\n\n_backend_configuration is responsible\
+    \ for cluster-manager-specific configuration."
+- source_sentence: What are the steps to configure RBAC for Spark in Kubernetes and
+    register the stack component using ZenML?
   sentences:
-  - "tack:\n\nzenml stack register-secrets [<STACK_NAME>]The ZenML client API offers\
-    \ a programmatic interface to create, e.g.:\n\nfrom zenml.client import Client\n\
-    \nclient = Client()\nclient.create_secret(\n    name=\"my_secret\",\n    values={\n\
-    \        \"username\": \"admin\",\n        \"password\": \"abc123\"\n    }\n)\n\
-    \nOther Client methods used for secrets management include get_secret to fetch\
-    \ a secret by name or id, update_secret to update an existing secret, list_secrets\
-    \ to query the secrets store using a variety of filtering and sorting criteria,\
-    \ and delete_secret to delete a secret. The full Client API reference is available\
-    \ here.\n\nSet scope for secrets\n\nZenML secrets can be scoped to a user. This\
-    \ allows you to create secrets that are only accessible to one user.\n\nBy default,\
-    \ all created secrets are scoped to the active user. To create a secret and scope\
-    \ it to your active user instead, you can pass the --scope argument to the CLI\
-    \ command:\n\nzenml secret create <SECRET_NAME> \\\n    --scope user \\\n    --<KEY_1>=<VALUE_1>\
-    \ \\\n    --<KEY_2>=<VALUE_2>\n\nScopes also act as individual namespaces. When\
-    \ you are referencing a secret by name in your pipelines and stacks, ZenML will\
-    \ look for a secret with that name scoped to the active user.\n\nAccessing registered\
-    \ secrets\n\nReference secrets in stack component attributes and settings\n\n\
-    Some of the components in your stack require you to configure them with sensitive\
-    \ information like passwords or tokens, so they can connect to the underlying\
-    \ infrastructure. Secret references allow you to configure these components in\
-    \ a secure way by not specifying the value directly but instead referencing a\
-    \ secret by providing the secret name and key. Referencing a secret for the value\
-    \ of any string attribute of your stack components, simply specify the attribute\
-    \ using the following syntax: {{<SECRET_NAME>.<SECRET_KEY>}}\n\nFor example:\n\
-    \n# Register a secret called `mlflow_secret` with key-value pairs for the\n# username\
-    \ and password to authenticate with the MLflow tracking server"
-  - 'y to the active stack
-
-    zenml stack update -c <NAME>Additionally, we''ll need to log in to the container
-    registry so Docker can pull and push images. This will require your DockerHub
-    account name and either your password or preferably a personal access token.
-
-
-    docker login
-
-
-    For more information and a full list of configurable attributes of the dockerhub
-    container registry, check out the SDK Docs .
-
-
-    PreviousDefault Container RegistryNextAmazon Elastic Container Registry (ECR)
-
+  - " build to finish. More information: Build Timeout.We can register the image builder\
+    \ and use it in our active stack:\n\nzenml image-builder register <IMAGE_BUILDER_NAME>\
+    \ \\\n    --flavor=gcp \\\n    --cloud_builder_image=<BUILDER_IMAGE_NAME> \\\n\
+    \    --network=<DOCKER_NETWORK> \\\n    --build_timeout=<BUILD_TIMEOUT_IN_SECONDS>\n\
+    \n# Register and activate a stack with the new image builder\nzenml stack register\
+    \ <STACK_NAME> -i <IMAGE_BUILDER_NAME> ... --set\n\nYou also need to set up authentication\
+    \ required to access the Cloud Build GCP services.\n\nAuthentication Methods\n\
+    \nIntegrating and using a GCP Image Builder in your pipelines is not possible\
+    \ without employing some form of authentication. If you're looking for a quick\
+    \ way to get started locally, you can use the Local Authentication method. However,\
+    \ the recommended way to authenticate to the GCP cloud platform is through a GCP\
+    \ Service Connector. This is particularly useful if you are configuring ZenML\
+    \ stacks that combine the GCP Image Builder with other remote stack components\
+    \ also running in GCP.\n\nThis method uses the implicit GCP authentication available\
+    \ in the environment where the ZenML code is running. On your local machine, this\
+    \ is the quickest way to configure a GCP Image Builder. You don't need to supply\
+    \ credentials explicitly when you register the GCP Image Builder, as it leverages\
+    \ the local credentials and configuration that the Google Cloud CLI stores on\
+    \ your local machine. However, you will need to install and set up the Google\
+    \ Cloud CLI on your machine as a prerequisite, as covered in the Google Cloud\
+    \ documentation , before you register the GCP Image Builder.\n\nStacks using the\
+    \ GCP Image Builder set up with local authentication are not portable across environments.\
+    \ To make ZenML pipelines fully portable, it is recommended to use a GCP Service\
+    \ Connector to authenticate your GCP Image Builder to the GCP cloud platform."
+  - ' your GCP Image Builder to the GCP cloud platform.To set up the GCP Image Builder
+    to authenticate to GCP and access the GCP Cloud Build services, it is recommended
+    to leverage the many features provided by the GCP Service Connector such as auto-configuration,
+    best security practices regarding long-lived credentials and reusing the same
+    credentials across multiple stack components.
+
+
+    If you don''t already have a GCP Service Connector configured in your ZenML deployment,
+    you can register one using the interactive CLI command. You also have the option
+    to configure a GCP Service Connector that can be used to access more than just
+    the GCP Cloud Build service:
+
+
+    zenml service-connector register --type gcp -i
+
+
+    A non-interactive CLI example that leverages the Google Cloud CLI configuration
+    on your local machine to auto-configure a GCP Service Connector for the GCP Cloud
+    Build service:
+
+
+    zenml service-connector register <CONNECTOR_NAME> --type gcp --resource-type gcp-generic
+    --resource-name <GCS_BUCKET_NAME> --auto-configure
+
+
+    Example Command Output
+
+
+    $ zenml service-connector register gcp-generic --type gcp --resource-type gcp-generic
+    --auto-configure
+
+    Successfully registered service connector `gcp-generic` with access to the following
+    resources:
+
+    ┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓
+
+    ┃ RESOURCE TYPE  │ RESOURCE NAMES ┃
+
+    ┠────────────────┼────────────────┨
+
+    ┃ 🔵 gcp-generic │ zenml-core     ┃
+
+    ┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛
+
+
+    Note: Please remember to grant the entity associated with your GCP credentials
+    permissions to access the Cloud Build API and to run Cloud Builder jobs (e.g.
+    the Cloud Build Editor IAM role). The GCP Service Connector supports many different
+    authentication methods with different levels of security and convenience. You
+    should pick the one that best fits your use case.
+
+
+    If you already have one or more GCP Service Connectors configured in your ZenML
+    deployment, you can check which of them can be used to access generic GCP resources
+    like the GCP Image Builder required for your GCP Image Builder by running e.g.:'
+  - "ngs/python/Dockerfile -u 0 build\n\nConfiguring RBACAdditionally, you may need\
+    \ to create the several resources in Kubernetes in order to give Spark access\
+    \ to edit/manage your driver executor pods.\n\nTo do so, create a file called\
+    \ rbac.yaml with the following content:\n\napiVersion: v1\nkind: Namespace\nmetadata:\n\
+    \  name: spark-namespace\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n\
+    \  name: spark-service-account\n  namespace: spark-namespace\n---\napiVersion:\
+    \ rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n  name: spark-role\n\
+    \  namespace: spark-namespace\nsubjects:\n  - kind: ServiceAccount\n    name:\
+    \ spark-service-account\n    namespace: spark-namespace\nroleRef:\n  kind: ClusterRole\n\
+    \  name: edit\n  apiGroup: rbac.authorization.k8s.io\n---\n\nAnd then execute\
+    \ the following command to create the resources:\n\naws eks --region=$REGION update-kubeconfig\
+    \ --name=$EKS_CLUSTER_NAME\n\nkubectl create -f rbac.yaml\n\nLastly, note down\
+    \ the namespace and the name of the service account since you will need them when\
+    \ registering the stack component in the next step.\n\nHow to use it\n\nTo use\
+    \ the KubernetesSparkStepOperator, you need:\n\nthe ZenML spark integration. If\
+    \ you haven't installed it already, run\n\nzenml integration install spark\n\n\
+    Docker installed and running.\n\nA remote artifact store as part of your stack.\n\
+    \nA remote container registry as part of your stack.\n\nA Kubernetes cluster deployed.\n\
+    \nWe can then register the step operator and use it in our active stack:\n\nzenml\
+    \ step-operator register spark_step_operator \\\n\t--flavor=spark-kubernetes \\\
+    \n\t--master=k8s://$EKS_API_SERVER_ENDPOINT \\\n\t--namespace=<SPARK_KUBERNETES_NAMESPACE>\
+    \ \\\n\t--service_account=<SPARK_KUBERNETES_SERVICE_ACCOUNT>\n\n# Register the\
+    \ stack\nzenml stack register spark_stack \\\n    -o default \\\n    -s spark_step_operator\
+    \ \\\n    -a spark_artifact_store \\\n    -c spark_container_registry \\\n   \
+    \ -i local_builder \\\n    --set"
+- source_sentence: Where can I find older versions of ZenML documentation?
+  sentences:
+  - 'ZenML - Bridging the gap between ML & Ops
 
-    Last updated 4 months ago'
-  - 'nect the stack component to the Service Connector:$ zenml step-operator register
-    <NAME> --flavor kubernetes
 
-    Running with active stack: ''default'' (repository)
+    Legacy Docs
 
-    Successfully registered step operator `<NAME>`.
 
+    Bleeding EdgeLegacy Docs0.67.0
 
-    $ zenml service-connector list-resources --resource-type kubernetes-cluster -e
 
-    The following ''kubernetes-cluster'' resources can be accessed by service connectors
-    that you have configured:
+    🧙‍♂️Find older version our docs
 
-    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓
 
-    ┃             CONNECTOR ID             │ CONNECTOR NAME        │ CONNECTOR TYPE
-    │ RESOURCE TYPE         │ RESOURCE NAMES      ┃
+    Powered by GitBook'
+  - 'ZenML - Bridging the gap between ML & Ops
 
-    ┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
 
-    ┃ e33c9fac-5daa-48b2-87bb-0187d3782cde │ aws-iam-multi-eu      │ 🔶 aws         │
-    🌀 kubernetes-cluster │ kubeflowmultitenant ┃
+    Legacy Docs
 
-    ┃                                      │                       │                │                       │
-    zenbox              ┃
 
-    ┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
+    Bleeding EdgeLegacy Docs0.67.0
 
-    ┃ ed528d5a-d6cb-4fc4-bc52-c3d2d01643e5 │ aws-iam-multi-us      │ 🔶 aws         │
-    🌀 kubernetes-cluster │ zenhacks-cluster    ┃
 
-    ┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
+    🧙‍♂️Find older version our docs
 
-    ┃ 1c54b32a-4889-4417-abbd-42d3ace3d03a │ gcp-sa-multi          │ 🔵 gcp         │
-    🌀 kubernetes-cluster │ zenml-test-cluster  ┃
 
-    ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛'
+    Powered by GitBook'
+  - "tainer_registry \\\n    -i local_builder \\\n    --setOnce you added the step\
+    \ operator to your active stack, you can use it to execute individual steps of\
+    \ your pipeline by specifying it in the @step decorator as follows:\n\nfrom zenml\
+    \ import step\n\n@step(step_operator=<STEP_OPERATOR_NAME>)\ndef step_on_spark(...)\
+    \ -> ...:\n    \"\"\"Some step that should run with Spark on Kubernetes.\"\"\"\
+    \n    ...\n\nAfter successfully running any step with a KubernetesSparkStepOperator,\
+    \ you should be able to see that a Spark driver pod was created in your cluster\
+    \ for each pipeline step when running kubectl get pods -n $KUBERNETES_NAMESPACE.\n\
+    \nInstead of hardcoding a step operator name, you can also use the Client to dynamically\
+    \ use the step operator of your active stack:\n\nfrom zenml.client import Client\n\
+    \nstep_operator = Client().active_stack.step_operator\n\n@step(step_operator=step_operator.name)\n\
+    def step_on_spark(...) -> ...:\n    ...\n\nAdditional configuration\n\nFor additional\
+    \ configuration of the Spark step operator, you can pass SparkStepOperatorSettings\
+    \ when defining or running your pipeline. Check out the SDK docs for a full list\
+    \ of available attributes and this docs page for more information on how to specify\
+    \ settings.\n\nPreviousKubernetesNextDevelop a Custom Step Operator\n\nLast updated\
+    \ 4 months ago"
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 metrics:
@@ -759,49 +502,49 @@ model-index:
       type: dim_384
     metrics:
     - type: cosine_accuracy@1
-      value: 0.1917808219178082
+      value: 1.0
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
-      value: 0.5095890410958904
+      value: 1.0
       name: Cosine Accuracy@3
     - type: cosine_accuracy@5
-      value: 0.6986301369863014
+      value: 1.0
       name: Cosine Accuracy@5
     - type: cosine_accuracy@10
-      value: 0.810958904109589
+      value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 0.1917808219178082
+      value: 1.0
       name: Cosine Precision@1
     - type: cosine_precision@3
-      value: 0.16986301369863013
+      value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.13972602739726026
+      value: 0.2
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.08109589041095888
+      value: 0.1
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 0.1917808219178082
+      value: 1.0
       name: Cosine Recall@1
     - type: cosine_recall@3
-      value: 0.5095890410958904
+      value: 1.0
       name: Cosine Recall@3
     - type: cosine_recall@5
-      value: 0.6986301369863014
+      value: 1.0
       name: Cosine Recall@5
     - type: cosine_recall@10
-      value: 0.810958904109589
+      value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 0.490826354124735
+      value: 1.0
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 0.38868232224396587
+      value: 1.0
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 0.3947220516402755
+      value: 1.0
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -811,49 +554,49 @@ model-index:
       type: dim_256
     metrics:
     - type: cosine_accuracy@1
-      value: 0.19726027397260273
+      value: 0.75
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
-      value: 0.5150684931506849
+      value: 1.0
       name: Cosine Accuracy@3
     - type: cosine_accuracy@5
-      value: 0.6931506849315069
+      value: 1.0
       name: Cosine Accuracy@5
     - type: cosine_accuracy@10
-      value: 0.8191780821917808
+      value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 0.19726027397260273
+      value: 0.75
       name: Cosine Precision@1
     - type: cosine_precision@3
-      value: 0.17168949771689496
+      value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.13863013698630136
+      value: 0.2
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.08191780821917807
+      value: 0.1
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 0.19726027397260273
+      value: 0.75
       name: Cosine Recall@1
     - type: cosine_recall@3
-      value: 0.5150684931506849
+      value: 1.0
       name: Cosine Recall@3
     - type: cosine_recall@5
-      value: 0.6931506849315069
+      value: 1.0
       name: Cosine Recall@5
     - type: cosine_recall@10
-      value: 0.8191780821917808
+      value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 0.4973578695114294
+      value: 0.9077324383928644
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 0.3948053924766253
+      value: 0.875
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 0.39989299235069015
+      value: 0.875
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -863,49 +606,49 @@ model-index:
       type: dim_128
     metrics:
     - type: cosine_accuracy@1
-      value: 0.19452054794520549
+      value: 0.75
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
-      value: 0.5013698630136987
+      value: 1.0
       name: Cosine Accuracy@3
     - type: cosine_accuracy@5
-      value: 0.673972602739726
+      value: 1.0
       name: Cosine Accuracy@5
     - type: cosine_accuracy@10
-      value: 0.7835616438356164
+      value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 0.19452054794520549
+      value: 0.75
       name: Cosine Precision@1
     - type: cosine_precision@3
-      value: 0.16712328767123283
+      value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.1347945205479452
+      value: 0.2
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.07835616438356163
+      value: 0.1
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 0.19452054794520549
+      value: 0.75
       name: Cosine Recall@1
     - type: cosine_recall@3
-      value: 0.5013698630136987
+      value: 1.0
       name: Cosine Recall@3
     - type: cosine_recall@5
-      value: 0.673972602739726
+      value: 1.0
       name: Cosine Recall@5
     - type: cosine_recall@10
-      value: 0.7835616438356164
+      value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 0.47814279525126957
+      value: 0.9077324383928644
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 0.38079147640791483
+      value: 0.875
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 0.38821398163789955
+      value: 0.875
       name: Cosine Map@100
   - task:
       type: information-retrieval
@@ -915,49 +658,49 @@ model-index:
       type: dim_64
     metrics:
     - type: cosine_accuracy@1
-      value: 0.1780821917808219
+      value: 0.75
       name: Cosine Accuracy@1
     - type: cosine_accuracy@3
-      value: 0.4602739726027397
+      value: 1.0
       name: Cosine Accuracy@3
     - type: cosine_accuracy@5
-      value: 0.6547945205479452
+      value: 1.0
       name: Cosine Accuracy@5
     - type: cosine_accuracy@10
-      value: 0.7753424657534247
+      value: 1.0
       name: Cosine Accuracy@10
     - type: cosine_precision@1
-      value: 0.1780821917808219
+      value: 0.75
       name: Cosine Precision@1
     - type: cosine_precision@3
-      value: 0.15342465753424658
+      value: 0.3333333333333333
       name: Cosine Precision@3
     - type: cosine_precision@5
-      value: 0.13095890410958905
+      value: 0.2
       name: Cosine Precision@5
     - type: cosine_precision@10
-      value: 0.07753424657534246
+      value: 0.1
       name: Cosine Precision@10
     - type: cosine_recall@1
-      value: 0.1780821917808219
+      value: 0.75
       name: Cosine Recall@1
     - type: cosine_recall@3
-      value: 0.4602739726027397
+      value: 1.0
       name: Cosine Recall@3
     - type: cosine_recall@5
-      value: 0.6547945205479452
+      value: 1.0
       name: Cosine Recall@5
     - type: cosine_recall@10
-      value: 0.7753424657534247
+      value: 1.0
       name: Cosine Recall@10
     - type: cosine_ndcg@10
-      value: 0.4625139710379368
+      value: 0.9077324383928644
       name: Cosine Ndcg@10
     - type: cosine_mrr@10
-      value: 0.36313002826701446
+      value: 0.875
       name: Cosine Mrr@10
     - type: cosine_map@100
-      value: 0.37046958498969434
+      value: 0.875
       name: Cosine Map@100
 ---
 
@@ -1012,9 +755,9 @@ from sentence_transformers import SentenceTransformer
 model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m-v1.5")
 # Run inference
 sentences = [
-    'How can I programmatically manage secrets using the ZenML Client API, and what are some of the methods available for tasks like fetching, updating, and deleting secrets?',
-    'tack:\n\nzenml stack register-secrets [<STACK_NAME>]The ZenML client API offers a programmatic interface to create, e.g.:\n\nfrom zenml.client import Client\n\nclient = Client()\nclient.create_secret(\n    name="my_secret",\n    values={\n        "username": "admin",\n        "password": "abc123"\n    }\n)\n\nOther Client methods used for secrets management include get_secret to fetch a secret by name or id, update_secret to update an existing secret, list_secrets to query the secrets store using a variety of filtering and sorting criteria, and delete_secret to delete a secret. The full Client API reference is available here.\n\nSet scope for secrets\n\nZenML secrets can be scoped to a user. This allows you to create secrets that are only accessible to one user.\n\nBy default, all created secrets are scoped to the active user. To create a secret and scope it to your active user instead, you can pass the --scope argument to the CLI command:\n\nzenml secret create <SECRET_NAME> \\\n    --scope user \\\n    --<KEY_1>=<VALUE_1> \\\n    --<KEY_2>=<VALUE_2>\n\nScopes also act as individual namespaces. When you are referencing a secret by name in your pipelines and stacks, ZenML will look for a secret with that name scoped to the active user.\n\nAccessing registered secrets\n\nReference secrets in stack component attributes and settings\n\nSome of the components in your stack require you to configure them with sensitive information like passwords or tokens, so they can connect to the underlying infrastructure. Secret references allow you to configure these components in a secure way by not specifying the value directly but instead referencing a secret by providing the secret name and key. Referencing a secret for the value of any string attribute of your stack components, simply specify the attribute using the following syntax: {{<SECRET_NAME>.<SECRET_KEY>}}\n\nFor example:\n\n# Register a secret called `mlflow_secret` with key-value pairs for the\n# username and password to authenticate with the MLflow tracking server',
-    "nect the stack component to the Service Connector:$ zenml step-operator register <NAME> --flavor kubernetes\nRunning with active stack: 'default' (repository)\nSuccessfully registered step operator `<NAME>`.\n\n$ zenml service-connector list-resources --resource-type kubernetes-cluster -e\nThe following 'kubernetes-cluster' resources can be accessed by service connectors that you have configured:\n┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓\n┃             CONNECTOR ID             │ CONNECTOR NAME        │ CONNECTOR TYPE │ RESOURCE TYPE         │ RESOURCE NAMES      ┃\n┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨\n┃ e33c9fac-5daa-48b2-87bb-0187d3782cde │ aws-iam-multi-eu      │ 🔶 aws         │ 🌀 kubernetes-cluster │ kubeflowmultitenant ┃\n┃                                      │                       │                │                       │ zenbox              ┃\n┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨\n┃ ed528d5a-d6cb-4fc4-bc52-c3d2d01643e5 │ aws-iam-multi-us      │ 🔶 aws         │ 🌀 kubernetes-cluster │ zenhacks-cluster    ┃\n┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨\n┃ 1c54b32a-4889-4417-abbd-42d3ace3d03a │ gcp-sa-multi          │ 🔵 gcp         │ 🌀 kubernetes-cluster │ zenml-test-cluster  ┃\n┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛",
+    'Where can I find older versions of ZenML documentation?',
+    'ZenML - Bridging the gap between ML & Ops\n\nLegacy Docs\n\nBleeding EdgeLegacy Docs0.67.0\n\n🧙\u200d♂️Find older version our docs\n\nPowered by GitBook',
+    'tainer_registry \\\n    -i local_builder \\\n    --setOnce you added the step operator to your active stack, you can use it to execute individual steps of your pipeline by specifying it in the @step decorator as follows:\n\nfrom zenml import step\n\n@step(step_operator=<STEP_OPERATOR_NAME>)\ndef step_on_spark(...) -> ...:\n    """Some step that should run with Spark on Kubernetes."""\n    ...\n\nAfter successfully running any step with a KubernetesSparkStepOperator, you should be able to see that a Spark driver pod was created in your cluster for each pipeline step when running kubectl get pods -n $KUBERNETES_NAMESPACE.\n\nInstead of hardcoding a step operator name, you can also use the Client to dynamically use the step operator of your active stack:\n\nfrom zenml.client import Client\n\nstep_operator = Client().active_stack.step_operator\n\n@step(step_operator=step_operator.name)\ndef step_on_spark(...) -> ...:\n    ...\n\nAdditional configuration\n\nFor additional configuration of the Spark step operator, you can pass SparkStepOperatorSettings when defining or running your pipeline. Check out the SDK docs for a full list of available attributes and this docs page for more information on how to specify settings.\n\nPreviousKubernetesNextDevelop a Custom Step Operator\n\nLast updated 4 months ago',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -1058,89 +801,89 @@ You can finetune this model on your own dataset.
 * Dataset: `dim_384`
 * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
 
-| Metric              | Value      |
-|:--------------------|:-----------|
-| cosine_accuracy@1   | 0.1918     |
-| cosine_accuracy@3   | 0.5096     |
-| cosine_accuracy@5   | 0.6986     |
-| cosine_accuracy@10  | 0.811      |
-| cosine_precision@1  | 0.1918     |
-| cosine_precision@3  | 0.1699     |
-| cosine_precision@5  | 0.1397     |
-| cosine_precision@10 | 0.0811     |
-| cosine_recall@1     | 0.1918     |
-| cosine_recall@3     | 0.5096     |
-| cosine_recall@5     | 0.6986     |
-| cosine_recall@10    | 0.811      |
-| cosine_ndcg@10      | 0.4908     |
-| cosine_mrr@10       | 0.3887     |
-| **cosine_map@100**  | **0.3947** |
+| Metric              | Value   |
+|:--------------------|:--------|
+| cosine_accuracy@1   | 1.0     |
+| cosine_accuracy@3   | 1.0     |
+| cosine_accuracy@5   | 1.0     |
+| cosine_accuracy@10  | 1.0     |
+| cosine_precision@1  | 1.0     |
+| cosine_precision@3  | 0.3333  |
+| cosine_precision@5  | 0.2     |
+| cosine_precision@10 | 0.1     |
+| cosine_recall@1     | 1.0     |
+| cosine_recall@3     | 1.0     |
+| cosine_recall@5     | 1.0     |
+| cosine_recall@10    | 1.0     |
+| cosine_ndcg@10      | 1.0     |
+| cosine_mrr@10       | 1.0     |
+| **cosine_map@100**  | **1.0** |
 
 #### Information Retrieval
 * Dataset: `dim_256`
 * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
 
-| Metric              | Value      |
-|:--------------------|:-----------|
-| cosine_accuracy@1   | 0.1973     |
-| cosine_accuracy@3   | 0.5151     |
-| cosine_accuracy@5   | 0.6932     |
-| cosine_accuracy@10  | 0.8192     |
-| cosine_precision@1  | 0.1973     |
-| cosine_precision@3  | 0.1717     |
-| cosine_precision@5  | 0.1386     |
-| cosine_precision@10 | 0.0819     |
-| cosine_recall@1     | 0.1973     |
-| cosine_recall@3     | 0.5151     |
-| cosine_recall@5     | 0.6932     |
-| cosine_recall@10    | 0.8192     |
-| cosine_ndcg@10      | 0.4974     |
-| cosine_mrr@10       | 0.3948     |
-| **cosine_map@100**  | **0.3999** |
+| Metric              | Value     |
+|:--------------------|:----------|
+| cosine_accuracy@1   | 0.75      |
+| cosine_accuracy@3   | 1.0       |
+| cosine_accuracy@5   | 1.0       |
+| cosine_accuracy@10  | 1.0       |
+| cosine_precision@1  | 0.75      |
+| cosine_precision@3  | 0.3333    |
+| cosine_precision@5  | 0.2       |
+| cosine_precision@10 | 0.1       |
+| cosine_recall@1     | 0.75      |
+| cosine_recall@3     | 1.0       |
+| cosine_recall@5     | 1.0       |
+| cosine_recall@10    | 1.0       |
+| cosine_ndcg@10      | 0.9077    |
+| cosine_mrr@10       | 0.875     |
+| **cosine_map@100**  | **0.875** |
 
 #### Information Retrieval
 * Dataset: `dim_128`
 * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
 
-| Metric              | Value      |
-|:--------------------|:-----------|
-| cosine_accuracy@1   | 0.1945     |
-| cosine_accuracy@3   | 0.5014     |
-| cosine_accuracy@5   | 0.674      |
-| cosine_accuracy@10  | 0.7836     |
-| cosine_precision@1  | 0.1945     |
-| cosine_precision@3  | 0.1671     |
-| cosine_precision@5  | 0.1348     |
-| cosine_precision@10 | 0.0784     |
-| cosine_recall@1     | 0.1945     |
-| cosine_recall@3     | 0.5014     |
-| cosine_recall@5     | 0.674      |
-| cosine_recall@10    | 0.7836     |
-| cosine_ndcg@10      | 0.4781     |
-| cosine_mrr@10       | 0.3808     |
-| **cosine_map@100**  | **0.3882** |
+| Metric              | Value     |
+|:--------------------|:----------|
+| cosine_accuracy@1   | 0.75      |
+| cosine_accuracy@3   | 1.0       |
+| cosine_accuracy@5   | 1.0       |
+| cosine_accuracy@10  | 1.0       |
+| cosine_precision@1  | 0.75      |
+| cosine_precision@3  | 0.3333    |
+| cosine_precision@5  | 0.2       |
+| cosine_precision@10 | 0.1       |
+| cosine_recall@1     | 0.75      |
+| cosine_recall@3     | 1.0       |
+| cosine_recall@5     | 1.0       |
+| cosine_recall@10    | 1.0       |
+| cosine_ndcg@10      | 0.9077    |
+| cosine_mrr@10       | 0.875     |
+| **cosine_map@100**  | **0.875** |
 
 #### Information Retrieval
 * Dataset: `dim_64`
 * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
 
-| Metric              | Value      |
-|:--------------------|:-----------|
-| cosine_accuracy@1   | 0.1781     |
-| cosine_accuracy@3   | 0.4603     |
-| cosine_accuracy@5   | 0.6548     |
-| cosine_accuracy@10  | 0.7753     |
-| cosine_precision@1  | 0.1781     |
-| cosine_precision@3  | 0.1534     |
-| cosine_precision@5  | 0.131      |
-| cosine_precision@10 | 0.0775     |
-| cosine_recall@1     | 0.1781     |
-| cosine_recall@3     | 0.4603     |
-| cosine_recall@5     | 0.6548     |
-| cosine_recall@10    | 0.7753     |
-| cosine_ndcg@10      | 0.4625     |
-| cosine_mrr@10       | 0.3631     |
-| **cosine_map@100**  | **0.3705** |
+| Metric              | Value     |
+|:--------------------|:----------|
+| cosine_accuracy@1   | 0.75      |
+| cosine_accuracy@3   | 1.0       |
+| cosine_accuracy@5   | 1.0       |
+| cosine_accuracy@10  | 1.0       |
+| cosine_precision@1  | 0.75      |
+| cosine_precision@3  | 0.3333    |
+| cosine_precision@5  | 0.2       |
+| cosine_precision@10 | 0.1       |
+| cosine_recall@1     | 0.75      |
+| cosine_recall@3     | 1.0       |
+| cosine_recall@5     | 1.0       |
+| cosine_recall@10    | 1.0       |
+| cosine_ndcg@10      | 0.9077    |
+| cosine_mrr@10       | 0.875     |
+| **cosine_map@100**  | **0.875** |
 
 <!--
 ## Bias, Risks and Limitations
@@ -1161,19 +904,19 @@ You can finetune this model on your own dataset.
 #### json
 
 * Dataset: json
-* Size: 3,284 training samples
+* Size: 36 training samples
 * Columns: <code>positive</code> and <code>anchor</code>
-* Approximate statistics based on the first 1000 samples:
+* Approximate statistics based on the first 36 samples:
   |         | positive                                                                           | anchor                                                                               |
   |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
   | type    | string                                                                             | string                                                                               |
-  | details | <ul><li>min: 11 tokens</li><li>mean: 22.84 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 17 tokens</li><li>mean: 322.65 tokens</li><li>max: 512 tokens</li></ul> |
+  | details | <ul><li>min: 13 tokens</li><li>mean: 22.58 tokens</li><li>max: 44 tokens</li></ul> | <ul><li>min: 32 tokens</li><li>mean: 300.72 tokens</li><li>max: 512 tokens</li></ul> |
 * Samples:
-  | positive                                                                                                                                                      | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
-  |:--------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-  | <code>How can impersonating accounts and assuming roles enhance flexibility and control in authentication methods for Service Connectors within ZenML?</code> | <code>ethods.<br><br>Impersonating accounts and assuming rolesThese types of authentication methods require more work to set up because multiple permission-bearing accounts and roles need to be provisioned in advance depending on the target audience. On the other hand, they also provide the most flexibility and control. Despite their operational cost, if you are a platform engineer and have the infrastructure know-how necessary to understand and set up the authentication resources, this is for you.<br><br>These authentication methods deliver another way of configuring long-lived credentials in your Service Connectors without exposing them to clients. They are especially useful as an alternative to cloud provider Service Connectors authentication methods that do not support automatically downscoping the permissions of issued temporary tokens.<br><br>The processes of account impersonation and role assumption are very similar and can be summarized as follows:<br><br>you configure a Service Connector with long-lived credentials associated with a primary user account or primary service account (preferable). As a best practice, it is common to attach a reduced set of permissions or even no permissions to these credentials other than those that allow the account impersonation or role assumption operation. This makes it more difficult to do any damage if the primary credentials are accidentally leaked.<br><br>in addition to the primary account and its long-lived credentials, you also need to provision one or more secondary access entities in the cloud platform bearing the effective permissions that will be needed to access the target resource(s):<br><br>one or more IAM roles (to be assumed)<br><br>one or more service accounts (to be impersonated)<br><br>the Service Connector configuration also needs to contain the name of a target IAM role to be assumed or a service account to be impersonated.</code>                   |
-  | <code>How can I specify the kubeflow_hostname and provide authentication details for multi-tenant Kubeflow deployments when using ZenML?</code>               | <code>strator_flavor import KubeflowOrchestratorSettingskubeflow_settings = KubeflowOrchestratorSettings(<br>   client_args={},  <br>   user_namespace="my_namespace",<br>   pod_settings={<br>       "affinity": {...},<br>       "tolerations": [...]<br>   }<br>)<br><br>@pipeline(<br>   settings={<br>       "orchestrator": kubeflow_settings<br>   }<br>)<br><br>This allows specifying client arguments, user namespace, pod affinity/tolerations, and more.<br><br>Multi-Tenancy Deployments<br><br>For multi-tenant Kubeflow deployments, specify the kubeflow_hostname ending in /pipeline when registering the orchestrator:<br><br>zenml orchestrator register <NAME> \<br>   --flavor=kubeflow \<br>   --kubeflow_hostname=<KUBEFLOW_HOSTNAME> # e.g. https://mykubeflow.example.com/pipeline<br><br>And provide the namespace, username and password in the orchestrator settings:<br><br>kubeflow_settings = KubeflowOrchestratorSettings(<br>   client_username="admin",<br>   client_password="abc123", <br>   user_namespace="namespace_name"<br>)<br><br>@pipeline(<br>   settings={<br>       "orchestrator": kubeflow_settings<br>   }<br>)<br><br>For more advanced options and details, refer to the full Kubeflow Orchestrator documentation.<br><br>PreviousRun on AzureNextKubernetes<br><br>Last updated 21 days ago</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
-  | <code>How can I contribute to the ZenML project?</code>                                                                                                       | <code>ding ZenML to learn more!<br><br>Do you support Windows?ZenML officially supports Windows if you're using WSL. Much of ZenML will also work on Windows outside a WSL environment, but we don't officially support it and some features don't work (notably anything that requires spinning up a server process).<br><br>Do you support Macs running on Apple Silicon?<br><br>Yes, ZenML does support Macs running on Apple Silicon. You just need to make sure that you set the following environment variable:<br><br>export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES<br><br>This is a known issue with how forking works on Macs running on Apple Silicon and it will enable you to use ZenML and the server. This environment variable is needed if you are working with a local server on your Mac, but if you're just using ZenML as a client / CLI and connecting to a deployed server then you don't need to set it.<br><br>How can I make ZenML work with my custom tool? How can I extend or build on ZenML?<br><br>This depends on the tool and its respective MLOps category. We have a full guide on this over here!<br><br>How can I contribute?<br><br>We develop ZenML together with our community! To get involved, the best way to get started is to select any issue from the good-first-issue label. If you would like to contribute, please review our Contributing Guide for all relevant details.<br><br>How can I speak with the community?<br><br>The first point of the call should be our Slack group. Ask your questions about bugs or specific use cases and someone from the core team will respond.<br><br>Which license does ZenML use?<br><br>ZenML is distributed under the terms of the Apache License Version 2.0. A complete version of the license is available in the LICENSE.md in this repository. Any contribution made to this project will be licensed under the Apache License Version 2.0.<br><br>PreviousCommunity & content<br><br>Last updated 3 months ago</code> |
+  | positive                                                                                                               | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+  |:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>How do you configure ZenML to display data visualizations in the dashboard?</code>                               | <code>📊Visualizing artifacts<br><br>Configuring ZenML to display data visualizations in the dashboard.<br><br>PreviousRegister Existing Data as a ZenML ArtifactNextDefault visualizations<br><br>Last updated 4 months ago</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+  | <code>How does the model deployer in ZenML facilitate the deployment and management of machine learning models?</code> | <code>Develop a Custom Model Deployer<br><br>Learning how to develop a custom model deployer.<br><br>Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML's component flavor concepts.<br><br>To deploy and manage your trained machine-learning models, ZenML provides a stack component called Model Deployer. This component is responsible for interacting with the deployment tool, framework, or platform.<br><br>When present in a stack, the model deployer can also act as a registry for models that are served with ZenML. You can use the model deployer to list all models that are currently deployed for online inference or filtered according to a particular pipeline run or step, or to suspend, resume or delete an external model server managed through ZenML.<br><br>Base Abstraction<br><br>In ZenML, the base abstraction of the model deployer is built on top of three major criteria:<br><br>It needs to ensure efficient deployment and management of models in accordance with the specific requirements of the serving infrastructure, by holding all the stack-related configuration attributes required to interact with the remote model serving tool, service, or platform.<br><br>It needs to implement the continuous deployment logic necessary to deploy models in a way that updates an existing model server that is already serving a previous version of the same model instead of creating a new model server for every new model version (see the deploy_model abstract method). This functionality can be consumed directly from ZenML pipeline steps, but it can also be used outside the pipeline to deploy ad-hoc models. It is also usually coupled with a standard model deployer step, implemented by each integration, that hides the details of the deployment process from the user.</code> |
+  | <code>How can I track the improvement of my RAG pipeline using evaluation and metrics?</code>                          | <code>Evaluation and metrics<br><br>Track how your RAG pipeline improves using evaluation and metrics.<br><br>PreviousBasic RAG inference pipelineNextEvaluation in 65 lines of code<br><br>Last updated 4 months ago</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
 * Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
   ```json
   {
@@ -1328,32 +1071,11 @@ You can finetune this model on your own dataset.
 </details>
 
 ### Training Logs
-| Epoch      | Step    | Training Loss | dim_384_cosine_map@100 | dim_256_cosine_map@100 | dim_128_cosine_map@100 | dim_64_cosine_map@100 |
-|:----------:|:-------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
-| 0.1949     | 10      | 1.0787        | -                      | -                      | -                      | -                     |
-| 0.3898     | 20      | 0.5131        | -                      | -                      | -                      | -                     |
-| 0.5847     | 30      | 0.2011        | -                      | -                      | -                      | -                     |
-| 0.7795     | 40      | 0.1264        | -                      | -                      | -                      | -                     |
-| 0.9744     | 50      | 0.0754        | -                      | -                      | -                      | -                     |
-| 0.9939     | 51      | -             | 0.3934                 | 0.3870                 | 0.3630                 | 0.3397                |
-| 1.1693     | 60      | 0.0581        | -                      | -                      | -                      | -                     |
-| 1.3642     | 70      | 0.0543        | -                      | -                      | -                      | -                     |
-| 1.5591     | 80      | 0.0467        | -                      | -                      | -                      | -                     |
-| 1.7540     | 90      | 0.0154        | -                      | -                      | -                      | -                     |
-| 1.9488     | 100     | 0.0407        | -                      | -                      | -                      | -                     |
-| **1.9878** | **102** | **-**         | **0.4025**             | **0.4052**             | **0.387**              | **0.3672**            |
-| 2.1437     | 110     | 0.0098        | -                      | -                      | -                      | -                     |
-| 2.3386     | 120     | 0.0229        | -                      | -                      | -                      | -                     |
-| 2.5335     | 130     | 0.0184        | -                      | -                      | -                      | -                     |
-| 2.7284     | 140     | 0.009         | -                      | -                      | -                      | -                     |
-| 2.9233     | 150     | 0.0308        | -                      | -                      | -                      | -                     |
-| 2.9817     | 153     | -             | 0.3919                 | 0.4006                 | 0.3855                 | 0.3718                |
-| 3.1181     | 160     | 0.0048        | -                      | -                      | -                      | -                     |
-| 3.3130     | 170     | 0.0079        | -                      | -                      | -                      | -                     |
-| 3.5079     | 180     | 0.0156        | -                      | -                      | -                      | -                     |
-| 3.7028     | 190     | 0.0075        | -                      | -                      | -                      | -                     |
-| 3.8977     | 200     | 0.0254        | -                      | -                      | -                      | -                     |
-| 3.9756     | 204     | -             | 0.3947                 | 0.3999                 | 0.3882                 | 0.3705                |
+| Epoch   | Step  | dim_384_cosine_map@100 | dim_256_cosine_map@100 | dim_128_cosine_map@100 | dim_64_cosine_map@100 |
+|:-------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
+| **1.0** | **1** | **0.875**              | **0.875**              | **0.875**              | **0.875**             |
+| 2.0     | 3     | 1.0                    | 0.875                  | 0.875                  | 0.875                 |
+| 3.0     | 4     | 1.0                    | 0.875                  | 0.875                  | 0.875                 |
 
 * The bold row denotes the saved checkpoint.
 
@@ -1362,8 +1084,8 @@ You can finetune this model on your own dataset.
 - Sentence Transformers: 3.2.1
 - Transformers: 4.43.1
 - PyTorch: 2.5.1+cu124
-- Accelerate: 1.0.1
-- Datasets: 3.0.2
+- Accelerate: 1.1.0
+- Datasets: 3.1.0
 - Tokenizers: 0.19.1
 
 ## Citation

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1fc1bdb3410373f27919be5fd3542e57376d56857e01f20f4226a06d38d2ac74
+oid sha256:463d27a3e88a748997316367c419bbdab4d809300a5f24b9b19ba9b2dc08ffdd
 size 435588776