I'm currently working with a dataset where each row contains 2 things:TextRelated tagsBoth are dutch.I'm trying to build a model that allows me to assign tags to new text based on what it learned from the dataset.The problem I think I'm having is that the dataset itself is very small (around 2000 messages) but the tags are also very unevenly distributed.For instance, the most used tag occurs 121 times, but there are around 40 tags that only occur 5 times in a different article.I tried to solve the data quantity problem by using dutch word-vectors and use them in an Embedding layer. This allows to give more context to the final model.The tags problem I haven't got a solution for yet. I think I should make a custom loss function that lets text with tags occurring more frequently weigh less when they are valid, and tags occurring less will score higher. But I have no idea how to get started on that. If you can help me, thanks!My model looks like this:MAX_SEQUENCE_LENGTH = 100EMBEDDING_DIM = 300embedding_layer = Embedding(num_words,                            EMBEDDING_DIM,                            weights=[embedding_matrix],                            input_length=MAX_SEQUENCE_LENGTH,                            trainable=False)sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')embedded_sequences = embedding_layer(sequence_input)x = Dropout(0.1)(embedded_sequences)x = Conv1D(128, 5, activation='relu')(x)x = MaxPooling1D(5)(x)x = Conv1D(128, 5, activation='relu')(x)x = MaxPooling1D(5)(x)x = Flatten()(x)x = Dense(128, activation='relu')(x)preds = Dense(len(labels_index), activation='sigmoid')(x)model = Model(sequence_input, preds)model.compile(loss='binary_crossentropy',              optimizer='adam',              metrics=['accuracy'])Data input is a padded sequence as input and output is a binary list of tags that are related to the input (made using MultiLabelBinarizer).
Multi-label classification of text with variable tag distribution in Keras
python;keras;multilabel classification