<commit_before>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xgqk7eHswDpB"
      },
      "source": [
        "# Vizier Basics\n",
        "Below, we provide examples of how to:\n",
        "\n",
        "*   Define a problem statement and study configuration.\n",
        "*   Setup a local server.\n",
        "*   Setup a client to connect to the server.\n",
        "*   Perform a typical tuning loop.\n",
        "*   Use other client APIs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O5RnMytPR8Aw"
      },
      "source": [
        "## Reference imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fzYr0bPYSHfQ"
      },
      "outputs": [],
      "source": [
        "from vizier.service import clients\n",
        "from vizier.service import pyvizier as vz\n",
        "from vizier.service import vizier_service"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJ1kRiHaKOVt"
      },
      "source": [
        "## Setting up the problem statement\n",
        "Here we setup the problem statement, which contains information about the search space and the metrics to optimize."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zX2G3_pcKYdG"
      },
      "outputs": [],
      "source": [
        "problem = vz.ProblemStatement()\n",
        "problem.search_space.root.add_float_param('x', 0.0, 1.0)\n",
        "problem.search_space.root.add_float_param('y', 0.0, 1.0)\n",
        "problem.metric_information.append(\n",
        "    vz.MetricInformation(\n",
        "        name='maximize_metric', goal=vz.ObjectiveMetricGoal.MAXIMIZE))\n",
        "\n",
        "\n",
        "def evaluate(x: float, y: float) -\u003e float:\n",
        "  return x**2 - y**2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GjDXKSIPKoex"
      },
      "source": [
        "## Setting up the study configuration\n",
        "The study configuration contains additional information, such as the algorithm to use and level of noise that we think the objective will have."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NlSWnX1wKrjZ"
      },
      "outputs": [],
      "source": [
        "study_config = vz.StudyConfig.from_problem(problem)\n",
        "study_config.algorithm = vz.Algorithm.RANDOM_SEARCH"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SwaqnJECLtwa"
      },
      "source": [
        "## Setting up the server\n",
        "As an local example, the server will be run locally here, and will process all client requests. However, it can also be run remotely, to service multiple clients in a distributed manner.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1IEUw_UfLvtx"
      },
      "outputs": [],
      "source": [
        "service = vizier_service.DefaultVizierService()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e2B91UvZYIM"
      },
      "source": [
        "## Setting up the client\n",
        "Starts a client, which will connect to the server. Note that the below can be called multiple times, one on each machine, to obtain multiple clients, all working on the same study, especially for tuning jobs which require multiple machines to parallelize the workload."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X2AR4OmXX3in"
      },
      "outputs": [],
      "source": [
        "clients.environment_variables.service_endpoint = service.endpoint  # Server address.\n",
        "study_client = clients.Study.from_study_config(\n",
        "    study_config, owner='owner', study_id='example_study_id')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vh3eNsrAdaMJ"
      },
      "source": [
        "## Obtaining suggestions\n",
        "Start requesting suggestions from the server, for evaluating objectives."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BnFKc7FadkJV"
      },
      "outputs": [],
      "source": [
        "suggestions = study_client.suggest(count=5)\n",
        "for suggestion in suggestions:\n",
        "  x = suggestion.parameters['x']\n",
        "  y = suggestion.parameters['y']\n",
        "  print('Suggested Parameters (x,y):', x, y)\n",
        "  final_measurement = vz.Measurement({'maximize_metric': evaluate(x, y)})\n",
        "  suggestion.complete(final_measurement)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V_pbH38ZfMVo"
      },
      "source": [
        "## Find optimal trial\n",
        "Find the best objective so far, with corresponding suggestion value. For multiobjective cases, there may be multiple outputs of `optimal_trials()`, all corresponding to a Pareto-optimal curve."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vTKVZHVtfXJc"
      },
      "outputs": [],
      "source": [
        "for optimal_trial in study_client.optimal_trials():\n",
        "  optimal_trial = optimal_trial.materialize()\n",
        "  print(\"Optimal Trial Suggestion and Objective:\", optimal_trial.parameters,\n",
        "        optimal_trial.final_measurement)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TPJD47ipbsQi"
      },
      "source": [
        "## Other client commands\n",
        "The `study` client can also send other requests, such as the following:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vcXrVLuMb201"
      },
      "outputs": [],
      "source": [
        "study_client.get_trial(1)  # Get the first trial.\n",
        "study_client.trials()  # Get all trials so far.\n",
        "\n",
        "# Obtain only the completed trials.\n",
        "trial_filter = vz.TrialFilter(status=[vz.TrialStatus.COMPLETED])\n",
        "study_client.trials(trial_filter=trial_filter)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "last_runtime": {
        "build_target": "",
        "kind": "local"
      },
      "name": "Running Vizier.ipynb",<commit_msg>Be more concise on user docs for RTD.<commit_after>{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xgqk7eHswDpB"
      },
      "source": [
        "# Vizier Basics\n",
        "Below, we provide examples of how to:\n",
        "\n",
        "*   Define a problem statement and study configuration.\n",
        "*   Setup a local server.\n",
        "*   Setup a client to connect to the server.\n",
        "*   Perform a typical tuning loop.\n",
        "*   Use other client APIs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O5RnMytPR8Aw"
      },
      "source": [
        "## Reference imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fzYr0bPYSHfQ"
      },
      "outputs": [],
      "source": [
        "from vizier.service import clients\n",
        "from vizier.service import pyvizier as vz\n",
        "from vizier.service import vizier_service"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJ1kRiHaKOVt"
      },
      "source": [
        "## Setting up the problem statement\n",
        "Here we setup the problem statement, which contains information about the search space and the metrics to optimize."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zX2G3_pcKYdG"
      },
      "outputs": [],
      "source": [
        "problem = vz.ProblemStatement()\n",
        "problem.search_space.root.add_float_param('x', 0.0, 1.0)\n",
        "problem.search_space.root.add_float_param('y', 0.0, 1.0)\n",
        "problem.metric_information.append(\n",
        "    vz.MetricInformation(\n",
        "        name='maximize_metric', goal=vz.ObjectiveMetricGoal.MAXIMIZE))\n",
        "\n",
        "\n",
        "def evaluate(x: float, y: float) -\u003e float:\n",
        "  return x**2 - y**2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GjDXKSIPKoex"
      },
      "source": [
        "## Setting up the study configuration\n",
        "The study configuration contains additional information, such as the algorithm to use and level of noise that we think the objective will have."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NlSWnX1wKrjZ"
      },
      "outputs": [],
      "source": [
        "study_config = vz.StudyConfig.from_problem(problem)\n",
        "study_config.algorithm = vz.Algorithm.RANDOM_SEARCH"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SwaqnJECLtwa"
      },
      "source": [
        "## Setting up the server\n",
        "As a basic example, the server will be run locally, and will process all client requests. However, it can also be run remotely, to service multiple clients in a distributed manner.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1IEUw_UfLvtx"
      },
      "outputs": [],
      "source": [
        "service = vizier_service.DefaultVizierService()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e2B91UvZYIM"
      },
      "source": [
        "## Setting up the client\n",
        "Starts a `study_client`, which will connect to the server. Multiple machines can simultaneously call the code below to work on the same study, useful for parallelizing evaluation workload."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X2AR4OmXX3in"
      },
      "outputs": [],
      "source": [
        "clients.environment_variables.service_endpoint = service.endpoint  # Server address.\n",
        "study_client = clients.Study.from_study_config(\n",
        "    study_config, owner='owner', study_id='example_study_id')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vh3eNsrAdaMJ"
      },
      "source": [
        "## Obtaining suggestions\n",
        "Start requesting suggestions from the server, for evaluating objectives."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BnFKc7FadkJV"
      },
      "outputs": [],
      "source": [
        "suggestions = study_client.suggest(count=5)\n",
        "for suggestion in suggestions:\n",
        "  x = suggestion.parameters['x']\n",
        "  y = suggestion.parameters['y']\n",
        "  print('Suggested Parameters (x,y):', x, y)\n",
        "  final_measurement = vz.Measurement({'maximize_metric': evaluate(x, y)})\n",
        "  suggestion.complete(final_measurement)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V_pbH38ZfMVo"
      },
      "source": [
        "## Find optimal trial\n",
        "Find the best objective so far, with corresponding suggestion value. For multiobjective cases, there may be multiple outputs of `optimal_trials()`, all corresponding to a Pareto-optimal curve."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vTKVZHVtfXJc"
      },
      "outputs": [],
      "source": [
        "for optimal_trial in study_client.optimal_trials():\n",
        "  optimal_trial = optimal_trial.materialize()\n",
        "  print(\"Optimal Trial Suggestion and Objective:\", optimal_trial.parameters,\n",
        "        optimal_trial.final_measurement)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TPJD47ipbsQi"
      },
      "source": [
        "## Other client commands\n",
        "The `study_client` can also send other requests, such as the following:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vcXrVLuMb201"
      },
      "outputs": [],
      "source": [
        "study_client.get_trial(1)  # Get the first trial.\n",
        "study_client.trials()  # Get all trials so far.\n",
        "\n",
        "# Obtain only the completed trials.\n",
        "trial_filter = vz.TrialFilter(status=[vz.TrialStatus.COMPLETED])\n",
        "study_client.trials(trial_filter=trial_filter)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "last_runtime": {
        "build_target": "",
        "kind": "local"
      },
      "name": "Running Vizier.ipynb",
{'fraction_non_alphanumeric': 0.15089747490112565, 'fraction_numerical': 0.0060845756008518406, 'mean_word_length': 2.0109915273643235, 'pattern_counts': {'":': 184, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3382871', 'n_tokens_mistral': 4388, 'n_tokens_neox': 4210, 'n_words': 936}