Iphepha elithi "Ukunaka yikho konke okudingayo" laphakamisa imodeli yeTransformer okokuqala ngqa, eyaba nomthelela omkhulu ku-LLM elandelayo. Sicela unikeze umqondo oyinhloko we-Transformer ochazwe kuleli phepha
Context:I-transformer iyimodeli yokufunda ejulile esebenzisa indlela yokunakwa komuntu siqu, ehlukanisa ukubaluleka kwengxenye ngayinye yedatha yokufaka. Isetshenziswa kakhulu emkhakheni wokucubungula ulimi lwemvelo (NLP) kanye nombono wekhompyutha (CV).  Njengezinethiwekhi ze-neural eziphindaphindwayo (ama-RNN), ama-transformer aklanyelwe ukucubungula idatha yokufaka elandelanayo, efana nolimi lwemvelo, ngezicelo eziya emisebenzini efana nokuhumusha nokufingqa umbhalo. Nokho, ngokungafani nama-RNN, ama-transformer acubungula konke okufakiwe ngesikhathi esisodwa. Isimiso sokunaka sinikeza umongo wanoma yisiphi isikhundla kulandelana okufakiwe. Ngokwesibonelo, uma idatha yokungenisa ingumusho wolimi lwemvelo, i-transformer akudingeki icubungule igama elilodwa ngesikhathi. Lokhu kuvumela ukufana okuningi kunama-RNN ngakho-ke kunciphisa izikhathi zokuqeqesha.  Ama-Transformers angeniswa ngo-2017 yithimba lakwaGoogle Brain futhi aya ngokuya eba yimodeli ekhethiwe yezinkinga ze-NLP, esikhundleni semodeli ye-RNN efana nenkumbulo yesikhathi esifushane (i-LSTM). Ukuqeqeshwa okuhambisanayo okungeziwe kuvumela ukuqeqeshwa kumasethi wedatha amakhulu. Lokhu kwaholela ekuthuthukisweni kwezinhlelo eziqeqeshwe ngaphambilini ezifana ne-BERT (Bidirectional Encoder Representations from Transformers) ne-GPT (Generative Pre-trained Transformer), ezaqeqeshwa ngamaqoqo amakhulu edatha yolimi, njenge-Wikipedia Corpus ne-Common Crawl, futhi zingahle zilungiswe kahle ngemisebenzi ethile.
Leli phepha libhalwe nguVaswani, Ashish & Shazeer, Noam & Parmar, Niki & Uszkoreit, Jakob & Jones, Llion & Gomez, Aidan & Kaiser, Lukasz & Polosukhin, Illia. futhi yashicilelwa ngo-2017. Wabeka isisekelo salolu chungechunge lwamamodeli we-GPT alandelayo kanye nochungechunge lwamamodeli weBERT. Imodeli ye-GPT isebenzisa ingxenye ye-decoder ye-Transfomer, kuyilapho imodeli ye-BERT isebenzisa ingxenye ye-encoder ye-Transfomer. Isakhiwo esiyinhloko semodeli ye-Transformer sifaka: Uhlelo Lokuziqapha, Ukufaka Ikhodi Kokuma, Ukwakhiwa Kwe-Encoder-Decoder, I-Multi-Head Attension. I-Transformer ixazulula ngempumelelo inkinga eyinkimbinkimbi ye-RNN / LSTM yangaphambilini ngokusebenzisa indlela yokunakwa. Ukusebenzisa Ukuzinaka Kuxazulula inkinga yemininingwane elinganiselwe enophawu ku-NLP. Ulwazi lokulandelana embhalweni wokufaka lugcinwa nge-Positional Encoding. Futhi i-Multi-Head Attention ivumela i-Transformer ukuba ifunde amaphethini ahlukahlukene wedatha yokuqeqesha afana nomqondo we-Channel kumodeli ye-CNN.