modeltype hybridclip iscomposition true def initself projectiondim512 kwargs super initkwargs if textconfig not in kwargs raise valueerrortextconfig can not be none if visionconfig not in kwargs raise valueerrorvisionconfig can not be none textconfig kwargs poptextconfig visionconfig kwargs popvisionconfig textmodeltype textconfig popmodeltype visionmodeltype visionconfig popmodeltype from transformers import autoconfig self textconfig autoconfig formodeltextmodeltype textconfig if visionmodeltype clip self visionconfig autoconfig formodelvisionmodeltype visionconfig visionconfig elif visionmodeltype clipvisionmodel from transformers import clipvisionconfig self visionconfig clipvisionconfigvisionconfig else self visionconfig autoconfig formodelvisionmodeltype visionconfig self projectiondim projectiondim self initializerfactor 1 0 classmethod def fromtextvisionconfigscls textconfig pretrainedconfig visionconfig pretrainedconfig kwargs r instantiate a class hybridclipconfig or a derived class from text model configuration and vision model configuration returns class hybridclipconfig an instance of a configuration object serializes this instance to a python dictionary override the default meth transformers pretrainedconfig todict returns obj dictstr any dictionary of all the attributes that make up this configuration instance class hybridclipconfig is the configuration class to store the configuration of a class hybridclipmodel it is used to instantiate hybridclipmodel model according to the specified arguments defining the text model and vision model configs configuration objects inherit from class transformers pretrainedconfig and can be used to control the model outputs read the documentation from class transformers pretrainedconfig for more information args text_config_dict obj dict dictionary of configuration options that defines text model config vision_config_dict obj dict dictionary of configuration options that defines vison model config projection_dim obj int optional defaults to 512 dimentionality of text and vision projection layers kwargs optional dictionary of keyword arguments examples from transformers import bertconfig clipconfig hybridclipconfig flaxhybridclip initializing a bert and clip configuration config_text bertconfig config_vision clipconfig config hybridclipconfig from_text_vision_configs config_text config_vision projection_dim 512 initializing a bert and clipvision model model encoderdecodermodel config config accessing the model configuration config_text model config text_config config_vision model config vision_config saving the model including its configuration model save_pretrained my model loading model and config from pretrained folder encoder_decoder_config hybridclipconfig from_pretrained my model model flaxhybridclip from_pretrained my model config encoder_decoder_config instantiate a class hybridclipconfig or a derived class from text model configuration and vision model configuration returns class hybridclipconfig an instance of a configuration object serializes this instance to a python dictionary override the default meth transformers pretrainedconfig to_dict returns obj dict str any dictionary of all the attributes that make up this configuration instance
import copy

from transformers.configuration_utils import PretrainedConfig
from transformers.utils import logging


logger = logging.get_logger(__name__)


class HybridCLIPConfig(PretrainedConfig):
    r

    model_type = "hybrid-clip"
    is_composition = True

    def __init__(self, projection_dim=512, **kwargs):
        super().__init__(**kwargs)

        if "text_config" not in kwargs:
            raise ValueError("`text_config` can not be `None`.")

        if "vision_config" not in kwargs:
            raise ValueError("`vision_config` can not be `None`.")

        text_config = kwargs.pop("text_config")
        vision_config = kwargs.pop("vision_config")

        text_model_type = text_config.pop("model_type")
        vision_model_type = vision_config.pop("model_type")

        from transformers import AutoConfig

        self.text_config = AutoConfig.for_model(text_model_type, **text_config)

        if vision_model_type == "clip":
            self.vision_config = AutoConfig.for_model(vision_model_type, **vision_config).vision_config
        elif vision_model_type == "clip_vision_model":
            from transformers import CLIPVisionConfig

            self.vision_config = CLIPVisionConfig(**vision_config)
        else:
            self.vision_config = AutoConfig.for_model(vision_model_type, **vision_config)

        self.projection_dim = projection_dim
        self.initializer_factor = 1.0

    @classmethod
    def from_text_vision_configs(cls, text_config: PretrainedConfig, vision_config: PretrainedConfig, **kwargs):
        r

        return cls(text_config=text_config.to_dict(), vision_config=vision_config.to_dict(), **kwargs)

    def to_dict(self):
        
        output = copy.deepcopy(self.__dict__)
        output["text_config"] = self.text_config.to_dict()
        output["vision_config"] = self.vision_config.to_dict()
        output["model_type"] = self.__class__.model_type
        return output