# What does this PR do?

This PR creates the precedent of deprecating models in the library. By deprecating we indicate that we will stop maintaining such models, but there is no intention of actually removing those models and breaking support for them (they might one day move into a separate repo/on the Hub but we would still add the necessary imports to make sure backward compatibility stays). The main point is that we stop testing those models (to ease a bit the burden on our CI).

Deprecated models are moved in models/deprecated so direct import of objects from their modeling files will break (though that's easily fixed by adding the `.deprecated` in the path). They are removed from the `tests` folder and a mention is added in the doc page of the model.

The heuristic to pick the deprecated models in this PR is: models older than a year that got less than a cumulated 1,000 downloads (over all checkpoints) in the last 30 days (counting deduplicated downloads).
Deprecate models. # What does this PR do?

This PR creates the precedent of deprecating models in the library. By deprecating we indicate that we will stop maintaining such models, but there is no intention of actually removing those models and breaking support for them (they might one day move into a separate repo/on the Hub but we would still add the necessary imports to make sure backward compatibility stays). The main point is that we stop testing those models (to ease a bit the burden on our CI).

Deprecated models are moved in models/deprecated so direct import of objects from their modeling files will break (though that's easily fixed by adding the `.deprecated` in the path). They are removed from the `tests` folder and a mention is added in the doc page of the model.

The heuristic to pick the deprecated models in this PR is: models older than a year that got less than a cumulated 1,000 downloads (over all checkpoints) in the last 30 days (counting deduplicated downloads).
name: Check Tiny Models

on:
  push:
    branches:
      - check_tiny_models*
  repository_dispatch:
  schedule:
    - cron: "0 2 * * *"

env:
  TOKEN: ${{ secrets.TRANSFORMERS_HUB_BOT_HF_TOKEN }}

jobs:
  check_tiny_models:
    name: Check tiny models
    runs-on: ubuntu-latest
    steps:
      - name: Checkout transformers
        uses: actions/checkout@v3
        with:
          fetch-depth: 2

      - uses: actions/checkout@v3
      - name: Set up Python 3.8
        uses: actions/setup-python@v4
        with:
          # Semantic version range syntax or exact version of a Python version
          python-version: '3.8'
          # Optional - x64 or x86 architecture, defaults to x64
          architecture: 'x64'

      - name: Install
        run: |
          sudo apt-get -y update && sudo apt-get install -y libsndfile1-dev espeak-ng cmake
          pip install --upgrade pip
          python -m pip install -U .[sklearn,torch,testing,sentencepiece,torch-speech,vision,timm,video,tf-cpu]
          pip install tensorflow_probability
          python -m pip install -U natten

      - name: Create all tiny models (locally)
        run: |
          python utils/create_dummy_models.py tiny_local_models --all --num_workers 2

      - name: Local tiny model reports artifacts
        if: ${{ always() }}
        uses: actions/upload-artifact@v3
        with:
          name: tiny_local_model_creation_reports
          path: tiny_local_models/reports

      # GitHub-hosted runners have 2-core CPUs
      - name: Run pipeline tests against all new (local) tiny models
        run: |
          OMP_NUM_THREADS=1 TRANSFORMERS_TINY_MODEL_PATH=tiny_local_models python -m pytest --max-worker-restart=0 -n 2 --dist=loadfile -s -rA --make-reports=tests_pipelines tests/models -m is_pipeline_test -k "test_pipeline_" | tee tests_output.txt

      - name: Test suite reports artifacts
        if: ${{ always() }}
        uses: actions/upload-artifact@v3
        with:
          name: tiny_local_model_creation_reports
          path: reports/tests_pipelines

      - name: Create + Upload tiny models for new model architecture(s)
        run: | 
          python utils/update_tiny_models.py --num_workers 2

      - name: Full report
        run: cat tiny_models/reports/tiny_model_creation_report.json

      - name: Failure report
        run: cat tiny_models/reports/simple_failed_report.txt

      - name: Summary report
        run: cat tiny_models/reports/tiny_model_summary.json

      - name: New tiny model creation reports artifacts
        if: ${{ always() }}
        uses: actions/upload-artifact@v3
        with:
          name: tiny_model_creation_reports
          path: tiny_models/reports

name: Check Tiny Models

on:
  push:
    branches:
      - check_tiny_models*
  repository_dispatch:
  schedule:
    - cron: "0 2 * * *"

env:
  TOKEN: ${{ secrets.TRANSFORMERS_HUB_BOT_HF_TOKEN }}

jobs:
  check_tiny_models:
    name: Check tiny models
    runs-on: ubuntu-latest
    steps:
      - name: Checkout transformers
        uses: actions/checkout@v3
        with:
          fetch-depth: 2

      - uses: actions/checkout@v3
      - name: Set up Python 3.8
        uses: actions/setup-python@v4
        with:
          # Semantic version range syntax or exact version of a Python version
          python-version: '3.8'
          # Optional - x64 or x86 architecture, defaults to x64
          architecture: 'x64'

      - name: Install
        run: |
          sudo apt-get -y update && sudo apt-get install -y libsndfile1-dev espeak-ng cmake
          pip install --upgrade pip
          python -m pip install -U .[sklearn,torch,testing,sentencepiece,torch-speech,vision,timm,video,tf-cpu]
          pip install tensorflow_probability
          python -m pip install -U natten

      - name: Create all tiny models (locally)
        run: |
          python utils/create_dummy_models.py tiny_local_models --all --num_workers 2

      - name: Local tiny model reports artifacts
        if: ${{ always() }}
        uses: actions/upload-artifact@v3
        with:
          name: tiny_local_model_creation_reports
          path: tiny_local_models/reports

      # GitHub-hosted runners have 2-core CPUs
      - name: Run pipeline tests against all new (local) tiny models
        run: |
          OMP_NUM_THREADS=1 TRANSFORMERS_TINY_MODEL_PATH=tiny_local_models python -m pytest --max-worker-restart=0 -n 2 --dist=loadfile -s -rA --make-reports=tests_pipelines tests/models -m is_pipeline_test -k "test_pipeline_" | tee tests_output.txt

      - name: Test suite reports artifacts
        if: ${{ always() }}
        uses: actions/upload-artifact@v3
        with:
          name: tiny_local_model_creation_reports
          path: reports/tests_pipelines

      - name: Create + Upload tiny models for new model architecture(s)
        run: | 
          python utils/update_tiny_models.py --num_workers 2

      - name: Full report
        run: cat tiny_models/reports/tiny_model_creation_report.json

      - name: Failure report
        run: cat tiny_models/reports/simple_failed_report.txt

      - name: Summary report
        run: cat tiny_models/reports/tiny_model_summary.json

      - name: New tiny model creation reports artifacts
        if: ${{ always() }}
        uses: actions/upload-artifact@v3
        with:
          name: tiny_model_creation_reports
          path: tiny_models/reports
