
first sentence: predicting the first sentence from the description as the slogan, which is simple but surprisingly competitive for document summarization [1]}. We use the sentence splitter in the Spacy library https://spacy.io/ to extract the first sentence.

first-k words: predicting the first-\(k\)  words from the description as the slogan. We choose \(k\)  that yields the highest ROUGE-1 F\(_1\)  score on the validation dataset. We add this baseline because the first sentence of the description is usually much longer than a typical slogan.

Skeleton-Based [2]}: a skeleton-based slogan generation system using genetic algorithms and various heuristic-based scoring functions. We sample a random compatible slogan skeleton from the training dataset and realise the slogan with keywords extracted from the company description. We follow [2]}'s implementation closely. However, we omit the database of frequent grammatical relations and the bigram function derived from Corpus of Contemporary American English because the resources are not available.

Encoder-Decoder [4]}: a strong and versatile GRU encoder-decoder baseline. We use identical hyper-parameters as [5]} and remove the reconstruction loss and copying mechanism to make the models directly comparable. Specifically, the model has a single hidden layer for both the bi-directional encoder and the auto-regressive decoder. We apply a dropout of 0.5 between layers. The embedding and hidden dimensions are 200 and 512 separately, and the vocabulary contains 30K most frequent words. The embedding matrix is randomly initialised and trained jointly with the model. We use Adam optimiser with a learning rate of 1e-3 and train for 10 epochs (The encoder-decoder models take more epochs to converge than the transformer models, likely because the models are randomly initialised).

Pointer-Generator [6]}: encoder-decoder model with copying mechanism to handle unknown words. Equivalent to [5]} with the reconstruction loss removed.

[5]}: a GRU encoder-decoder model for slogan generation with additional reconstruction loss to generate distinct slogans and copying mechanism to handle unknown words.

