Brief Review — GLU Variants Improve Transformer
GEGLU & SwiGLU, Better Activation Functions for Transformer —  GLU Variants Improve Transformer,
ReGLU, GEGLU & SwiGLU, by Google
2020 arXiv v1, Over 70 Citations (Sik-Ho Tsang @ Medium)
NLP, NMT, LLM, Language Model, Transformer, GLU, T5 2.1. Language Model
1991 … 2021 [Performer] [gMLP] [Roformer] [PPBERT] [DeBERTa] [DeLighT] [Transformer-LS] [R-Drop] 2022 [GPT-NeoX-20B] [InstructGPT] [GLM]
==== My Other Paper Readings Are…
Brief Review — GLU Variants Improve Transformer GEGLU & SwiGLU, Better Activation Functions for Transformer —  GLU Variants Improve Transformer,
ReGLU, GEGLU & SwiGLU, by Google
2020 arXiv v1, Over 70 Citations (Sik-Ho Tsang @ Medium)
NLP, NMT, LLM, Language Model, Transformer, GLU, T5 2.1. Language Model
1991 … 2021 [Performer] [gMLP] [Roformer] [PPBERT] [DeBERTa] [DeLighT] [Transformer-LS] [R-Drop] 2022 [GPT-NeoX-20B] [InstructGPT] [GLM]
==== My Other Paper Readings Are… nlp
Brief Review — GLU Variants Improve Transformer GEGLU SwiGLU Better Activation Functions Transformer — GLU Variants Improve Transformer ReGLU GEGLU SwiGLU Google 2020 arXiv v1 70 Citations SikHo Tsang Medium NLP NMT LLM Language Model Transformer GLU T5 21 Language Model 1991 … 2021 Performer gMLP Roformer PPBERT DeBERTa DeLighT TransformerLS RDrop 2022 GPTNeoX20B InstructGPT GLM Paper Readings Are… nlp