File size: 1,872 Bytes
0acdbca db67ffb d277006 03e06d3 d277006 9f1591a 8ad8e8f d49a54e 9f1591a b90b282 9f1591a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
license: mit
---
This is the trained model for the controlnet-stablediffusion for the scene text eraser (Diff_SceneTextEraser)
We have to customize the pipeline for controlnet-stablediffusion-inpaint
Here is the training and inference code for [Diff_SceneTextEraser](https://github.com/Onkarsus13/Diff_SceneTextEraser)
For direct inference
step 1: Clone the GitHub repo to get the customized ControlNet-StableDiffusion-inpaint Pipeline Implementation
```
git clone https://github.com/Onkarsus13/Diff_SceneTextEraser
```
Step2: Go into the repository and install repository, dependency
```
cd Diff_SceneTextEraser
pip install -e ".[torch]"
pip install -e .[all,dev,notebooks]
```
Step3: Run `python test_eraser.py` OR You can run the code given below
```python
from diffusers import (
UniPCMultistepScheduler,
DDIMScheduler,
EulerAncestralDiscreteScheduler,
StableDiffusionControlNetSceneTextErasingPipeline,
)
import torch
import numpy as np
import cv2
from PIL import Image, ImageDraw
import math
import os
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model_path = "onkarsus13/controlnet_stablediffusion_scenetextEraser"
pipe = StableDiffusionControlNetSceneTextErasingPipeline.from_pretrained(model_path)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to(device)
# pipe.enable_xformers_memory_efficient_attention()
pipe.enable_model_cpu_offload()
generator = torch.Generator(device).manual_seed(1)
image = Image.open("<path to scene text image>").resize((512, 512))
mask_image = Image.open('<path to the corrospoinding mask image>').resize((512, 512))
image = pipe(
image,
mask_image,
[mask_image],
num_inference_steps=20,
generator=generator,
controlnet_conditioning_scale=1.0,
guidance_scale=1.0
).images[0]
image.save('test1.png')
``` |