Spaces:

huggingface-projects

/

repo_duplicator

like 215

Running

cc @osanseviero with a potential TODO for @Wauplin :)

hmm I think we can't use the duplicate api for models/datasets yet? @Sylvestre

also ping @Sylvestre for a quick review as you implemented the feature 😁

hmm @coyotte508 yes i think so

It looks ok to me

Hey @julien-c , I just took over your code and added build_hf_headers(token=token) to create the request headers. Also made use of hf_raise_for_status(r) instead of handling the error manually. Both helpers are exposed under huggingface_hub.utils.

Copy-pasting the code here instead of creating a new PR to keep track of the discussion in a single place. Would be good to edit suggestions in the HF PRs but I guess that will come :)

# app.py
import gradio as gr
import requests
from huggingface_hub import whoami
from huggingface_hub.utils import build_hf_headers, hf_raise_for_status

ENDPOINT = "https://huggingface.co"
# ENDPOINT = "http://localhost:5564"

REPO_TYPES = ["model", "dataset", "space"]


def duplicate(source_repo, dst_repo, token, repo_type):
    try:
        if not repo_type in REPO_TYPES:
            raise ValueError("need to select valid repo type")
        _ = whoami(token)
        # ^ this will throw if token is invalid

        r = requests.post(
            f"{ENDPOINT}/api/{repo_type}s/{source_repo}/duplicate",
            headers=build_hf_headers(token=token),
            json={"repository": dst_repo},
        )
        hf_raise_for_status(r)

        repo_url = r.json().get("url")

        return (
            f'Find your repo <a href=\'{repo_url}\' target="_blank" style="text-decoration:underline">here</a>',
            "sp.jpg",
        )

    except Exception as e:
        return (
            f"""
        ### Error 😢😢😢
        
        {e}
        """,
            None,
        )


interface = gr.Interface(
    fn=duplicate,
    inputs=[
        gr.Textbox(placeholder="Source repository (e.g. osanseviero/src)"),
        gr.Textbox(placeholder="Destination repository (e.g. osanseviero/dst)"),
        gr.Textbox(placeholder="Write access token"),
        gr.Dropdown(choices=REPO_TYPES, value="model"),
    ],
    outputs=[
        gr.Markdown(label="output"),
        gr.Image(show_label=False),
    ],
    title="Duplicate your repo!",
    description="Duplicate a Hugging Face repository! You need to specify a write token obtained in https://hf.co/settings/tokens. This Space is a an experimental demo.",
    article="<p>Find your write token at <a href='https://huggingface.co/settings/tokens' target='_blank'>token settings</a></p>",
    allow_flagging="never",
    live=False,
)
interface.launch(enable_queue=True)

I am also in favor of exposing a modified requests module from huggingface_hub to wrap HTTP calls, add the header properly and monkeypatch the raise_for_status method. I have created an issue on huggingface_hub repo for it.

Ah for some reason i was under the impression hf_raise_for_status and build_hf_headers were not exposed from the library! Ok, it's easy then. Thanks

You cannot push directly on this PR due to permissions, correct?

You cannot push directly on this PR due to permissions, correct?

Exactly. I tried and got a 403 forbidden:

>>> from huggingface_hub import upload_file
>>> upload_file(path_or_fileobj="app.py", path_in_repo="app.py", repo_id="osanseviero/repo_duplicator", repo_type="space", revision="refs/pr/3")

(...)
huggingface_hub.utils._errors.HfHubHTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/api/spaces/osanseviero/repo_duplicator/commit/refs%2Fpr%2F3 (Request ID: 1m_npZRy8dWuXYxxIAKEl)
Forbidden: pass `create_pr=1` as a query parameter to create a Pull Request

And actually passing create_pr=1 doesn't work either. But from a workflow perspective, idk if we want to be able to create a PR against another PR.

Revision Not Found for url: https://huggingface.co/api/spaces/osanseviero/repo_duplicator/commit/refs%2Fpr%2F3?create_pr=1.
Invalid rev id: refs/heads/refs/pr/3

But from a workflow perspective, idk if we want to be able to create a PR against another PR.

not for now, probably

Ok I pushed your change (f4326699). Thanks!

@osanseviero this should be ready to merge now :)

Thanks a lot everyone!