Japanese Multimodal Models
Collection
Suite of multimodal models focusing on Japan/Japanese-related usage
โข
4 items
โข
Updated
โข
7
You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
By clicking "Agree", you agree to the License Agreement and acknowledge Stability AI's Privacy Policy.
Log in or Sign Up to review the conditions and access this model content.
Configuration Parsing
Warning:
In UNKNOWN_FILENAME: "diffusers._class_name" must be a string
Japanese Stable Diffusion XL
Please note: for commercial usage of this model, please see https://stability.ai/license
ๅ็จๅฉ็จใซ้ขใใๆฅๆฌ่ชใงใฎๅใๅใใใฏใ[email protected] ใพใงใ้กใ่ดใใพใใ
Model Details
Japanese Stable Diffusion XL (JSDXL) is a Japanese-specific SDXL model that is capable of inputting prompts in Japanese and generating Japanese-style images.
Usage
from diffusers import DiffusionPipeline
import torch
pipeline = DiffusionPipeline.from_pretrained(
"stabilityai/japanese-stable-diffusion-xl", trust_remote_code=True
)
pipeline.to("cuda")
# if using torch < 2.0
# pipeline.enable_xformers_memory_efficient_attention()
prompt = "ๆด็ฌใใซใฉใใซใขใผใ"
image = pipeline(prompt=prompt).images[0]
Model Details
- Developed by: Stability AI
- Model type: Diffusion-based text-to-image generative model
- Model Description: This model is a fine-tuned model based on SDXL 1.0. In order to maximize the understanding of the Japanese language and Japanese culture/expressions while preserving the versatility of the pre-trained model, we performed a PEFT training using one Japanese-specific compatible text encoder. As a PEFT method, we applied Orthogonal Fine-tuning (OFT) for better results and training stability.
- License: STABILITY AI COMMUNITY LICENSE
Uses
Direct Use
Commercial use: for commercial usage of this model, please see https://stability.ai/license
ๅ็จๅฉ็จใซ้ขใใๆฅๆฌ่ชใงใฎๅใๅใใใฏใ[email protected] ใพใงใ้กใ่ดใใพใใ
Research: possible research areas/tasks include:
- Generation of artworks and use in design and other artistic processes.
- Applications in educational or creative tools.
- Research on generative models.
- Safe deployment of models which have the potential to generate harmful content.
- Probing and understanding the limitations and biases of generative models.
Excluded uses are described below.
Out-of-Scope Use
The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
Limitations and Bias
Limitations
- The model does not achieve perfect photorealism
- The model cannot render legible text
- The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to โA red cube on top of a blue sphereโ
- Faces and people in general may not be generated properly.
- The autoencoding part of the model is lossy.
Bias
While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
How to cite
@misc{JSDXL,
url = {[https://huggingface.co/stabilityai/japanese-stable-diffusion-xl](https://huggingface.co/stabilityai/japanese-stable-diffusion-xl)},
title = {Japanese Stable Diffusion XL},
author = {Shing, Makoto and Akiba, Takuya and Chi, Jerry}
}
Contact
- For questions and comments about the model, please join Stable Community Japan.
- For future announcements / information about Stability AI models, research, and events, please follow https://twitter.com/StabilityAI_JP.
- For business and partnership inquiries, please contact [email protected]. ใใธใในใๅๆฅญใซ้ขใใใๅใๅใใใฏ[email protected]ใซใ้ฃ็ตกใใ ใใใ
- Downloads last month
- 117
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.