codingutf8 2022 meta platforms inc and the huggingface inc team licensed under the apache license version 2 0 the license you may not use this file except in compliance with the license you may obtain a copy of the license at http www apache orglicenseslicense2 0 unless required by applicable law or agreed to in writing software distributed under the license is distributed on an as is basis without warranties or conditions of any kind either express or implied see the license for the specific language governing permissions and limitations under the license maskformer model configuration from typing import dict optional from configurationutils import pretrainedconfig from utils import logging from auto import configmapping from detr import detrconfig from swin import swinconfig maskformerpretrainedconfigarchivemap facebookmaskformerswinbaseade https huggingface cofacebookmaskformerswinbaseadeblobmainconfig json see all maskformer models at https huggingface comodels filtermaskformer logger logging getloggername class maskformerconfigpretrainedconfig r this is the configuration class to store the configuration of a maskformermodel it is used to instantiate a maskformer model according to the specified arguments defining the model architecture instantiating a configuration with the defaults will yield a similar configuration to that of the maskformer facebookmaskformerswinbaseadehttps huggingface cofacebookmaskformerswinbaseade architecture trained on ade20k150https huggingface codatasetssceneparse150 configuration objects inherit from pretrainedconfig and can be used to control the model outputs read the documentation from pretrainedconfig for more information currently maskformer only supports the swin transformerswin as backbone args maskfeaturesize int optional defaults to 256 the masks features size this value will also be used to specify the feature pyramid network features size noobjectweight float optional defaults to 0 1 weight to apply to the null no object class useauxiliarylossbool optional defaults to false if true maskformerforinstancesegmentationoutput will contain the auxiliary losses computed using the logits from each decoder s stage backboneconfig dict optional the configuration passed to the backbone if unset the configuration corresponding to swinbasepatch4window12384 will be used decoderconfig dict optional the configuration passed to the transformer decoder model if unset the base config for detrresnet50 will be used initstd float optional defaults to 0 02 the standard deviation of the truncatednormalinitializer for initializing all weight matrices initxavierstd float optional defaults to 1 the scaling factor used for the xavier initialization gain in the hm attention map module diceweight float optional defaults to 1 0 the weight for the dice loss crossentropyweight float optional defaults to 1 0 the weight for the cross entropy loss maskweight float optional defaults to 20 0 the weight for the mask loss outputauxiliarylogits bool optional should the model output its auxiliarylogits or not raises valueerror raised if the backbone model type selected is not in swin or the decoder model type selected is not in detr examples python from transformers import maskformerconfig maskformermodel initializing a maskformer facebookmaskformerswinbaseade configuration configuration maskformerconfig initializing a model with random weights from the facebookmaskformerswinbaseade style configuration model maskformermodelconfiguration accessing the model configuration configuration model config fall back to https huggingface comicrosoftswinbasepatch4window12384in22k verify that the backbone is supported fall back to https huggingface cofacebookdetrresnet50 verify that the decoder is supported main feature dimension for the model initializer hungarian matcher loss instantiate a maskformerconfig or a derived class from a pretrained backbone model configuration and detr model configuration args backboneconfig pretrainedconfig the backbone configuration decoderconfig pretrainedconfig the transformer decoder configuration to use returns maskformerconfig an instance of a configuration object coding utf 8 2022 meta platforms inc and the huggingface inc team licensed under the apache license version 2 0 the license you may not use this file except in compliance with the license you may obtain a copy of the license at http www apache org licenses license 2 0 unless required by applicable law or agreed to in writing software distributed under the license is distributed on an as is basis without warranties or conditions of any kind either express or implied see the license for the specific language governing permissions and limitations under the license maskformer model configuration see all maskformer models at https huggingface co models filter maskformer this is the configuration class to store the configuration of a maskformermodel it is used to instantiate a maskformer model according to the specified arguments defining the model architecture instantiating a configuration with the defaults will yield a similar configuration to that of the maskformer facebook maskformer swin base ade https huggingface co facebook maskformer swin base ade architecture trained on ade20k 150 https huggingface co datasets scene_parse_150 configuration objects inherit from pretrainedconfig and can be used to control the model outputs read the documentation from pretrainedconfig for more information currently maskformer only supports the swin transformer swin as backbone args mask_feature_size int optional defaults to 256 the masks features size this value will also be used to specify the feature pyramid network features size no_object_weight float optional defaults to 0 1 weight to apply to the null no object class use_auxiliary_loss bool optional defaults to false if true maskformerforinstancesegmentationoutput will contain the auxiliary losses computed using the logits from each decoder s stage backbone_config dict optional the configuration passed to the backbone if unset the configuration corresponding to swin base patch4 window12 384 will be used decoder_config dict optional the configuration passed to the transformer decoder model if unset the base config for detr resnet 50 will be used init_std float optional defaults to 0 02 the standard deviation of the truncated_normal_initializer for initializing all weight matrices init_xavier_std float optional defaults to 1 the scaling factor used for the xavier initialization gain in the hm attention map module dice_weight float optional defaults to 1 0 the weight for the dice loss cross_entropy_weight float optional defaults to 1 0 the weight for the cross entropy loss mask_weight float optional defaults to 20 0 the weight for the mask loss output_auxiliary_logits bool optional should the model output its auxiliary_logits or not raises valueerror raised if the backbone model type selected is not in swin or the decoder model type selected is not in detr examples python from transformers import maskformerconfig maskformermodel initializing a maskformer facebook maskformer swin base ade configuration configuration maskformerconfig initializing a model with random weights from the facebook maskformer swin base ade style configuration model maskformermodel configuration accessing the model configuration configuration model config fall back to https huggingface co microsoft swin base patch4 window12 384 in22k verify that the backbone is supported fall back to https huggingface co facebook detr resnet 50 verify that the decoder is supported main feature dimension for the model initializer hungarian matcher loss instantiate a maskformerconfig or a derived class from a pre trained backbone model configuration and detr model configuration args backbone_config pretrainedconfig the backbone configuration decoder_config pretrainedconfig the transformer decoder configuration to use returns maskformerconfig an instance of a configuration object
from typing import Dict, Optional

from ...configuration_utils import PretrainedConfig
from ...utils import logging
from ..auto import CONFIG_MAPPING
from ..detr import DetrConfig
from ..swin import SwinConfig


MASKFORMER_PRETRAINED_CONFIG_ARCHIVE_MAP = {
    "facebook/maskformer-swin-base-ade": (
        "https://huggingface.co/facebook/maskformer-swin-base-ade/blob/main/config.json"
    )
    
}

logger = logging.get_logger(__name__)


class MaskFormerConfig(PretrainedConfig):
    r

    model_type = "maskformer"
    attribute_map = {"hidden_size": "mask_feature_size"}
    backbones_supported = ["resnet", "swin"]
    decoders_supported = ["detr"]

    def __init__(
        self,
        fpn_feature_size: int = 256,
        mask_feature_size: int = 256,
        no_object_weight: float = 0.1,
        use_auxiliary_loss: bool = False,
        backbone_config: Optional[Dict] = None,
        decoder_config: Optional[Dict] = None,
        init_std: float = 0.02,
        init_xavier_std: float = 1.0,
        dice_weight: float = 1.0,
        cross_entropy_weight: float = 1.0,
        mask_weight: float = 20.0,
        output_auxiliary_logits: Optional[bool] = None,
        **kwargs,
    ):
        if backbone_config is None:
            
            backbone_config = SwinConfig(
                image_size=384,
                in_channels=3,
                patch_size=4,
                embed_dim=128,
                depths=[2, 2, 18, 2],
                num_heads=[4, 8, 16, 32],
                window_size=12,
                drop_path_rate=0.3,
                out_features=["stage1", "stage2", "stage3", "stage4"],
            )

        if isinstance(backbone_config, dict):
            backbone_model_type = backbone_config.pop("model_type")
            config_class = CONFIG_MAPPING[backbone_model_type]
            backbone_config = config_class.from_dict(backbone_config)

        
        if backbone_config.model_type not in self.backbones_supported:
            logger.warning_once(
                f"Backbone {backbone_config.model_type} is not a supported model and may not be compatible with MaskFormer. "
                f"Supported model types: {','.join(self.backbones_supported)}"
            )

        if decoder_config is None:
            
            decoder_config = DetrConfig()
        else:
            
            decoder_type = (
                decoder_config.pop("model_type") if isinstance(decoder_config, dict) else decoder_config.model_type
            )
            if decoder_type not in self.decoders_supported:
                raise ValueError(
                    f"Transformer Decoder {decoder_type} not supported, please use one of"
                    f" {','.join(self.decoders_supported)}"
                )
            if isinstance(decoder_config, dict):
                config_class = CONFIG_MAPPING[decoder_type]
                decoder_config = config_class.from_dict(decoder_config)

        self.backbone_config = backbone_config
        self.decoder_config = decoder_config
        
        self.fpn_feature_size = fpn_feature_size
        self.mask_feature_size = mask_feature_size
        
        self.init_std = init_std
        self.init_xavier_std = init_xavier_std
        
        self.cross_entropy_weight = cross_entropy_weight
        self.dice_weight = dice_weight
        self.mask_weight = mask_weight
        self.use_auxiliary_loss = use_auxiliary_loss
        self.no_object_weight = no_object_weight
        self.output_auxiliary_logits = output_auxiliary_logits

        self.num_attention_heads = self.decoder_config.encoder_attention_heads
        self.num_hidden_layers = self.decoder_config.num_hidden_layers
        super().__init__(**kwargs)

    @classmethod
    def from_backbone_and_decoder_configs(
        cls, backbone_config: PretrainedConfig, decoder_config: PretrainedConfig, **kwargs
    ):
        
        return cls(
            backbone_config=backbone_config,
            decoder_config=decoder_config,
            **kwargs,
        )