Uma sicabangela lezi zigaba ezimayelana namamodeli amakhulu olimi, iyini i-hallucination?
Context:Imodeli enkulu yolimi (LLM) iyimodeli yolimi equkethe inethiwekhi ye-neural enamapharamitha amaningi (imvamisa izigidigidi zezisindo noma ngaphezulu), eqeqeshwe ngamanani amakhulu wombhalo ongabhalisiwe kusetshenziswa ukufunda okuziqondisayo. Ama-LLM avele cishe ngo-2018 futhi enza kahle emisebenzini ehlukahlukene. Lokhu kuye kwashintsha ukugxila ocwaningweni lokucubungula ulimi lwemvelo kude nesibonelo sangaphambilini sokuqeqesha amamodeli aqondiswayo akhethekile emisebenzini ethile.  Izakhiwo Nakuba igama elithi imodeli enkulu yolimi lingenayo incazelo engokomthetho, livame ukubhekisela kumamodeli okufunda okujulile anenani lamapharamitha elilinganiselwa ezigidigidini noma ngaphezulu. Ama-LLM ayizinhlobo zezinhloso ezijwayelekile ezivelele emisebenzini ehlukahlukene, ngokungafani nokuqeqeshelwa umsebenzi owodwa othize (njengokuhlaziywa kwemizwa, ukuqashelwa kwezinto ezinamagama, noma ukucabanga kwezibalo). Ikhono abafeza ngalo imisebenzi, nohla lwemisebenzi abakwazi ukuyenza, kubonakala kuwumsebenzi wenani lezinsiza (idatha, ubukhulu bepharamitha, amandla okucubungula) abazinikezele kubo, ngendlela engahambisani nokuthuthuka okwengeziwe ekwakhiweni.  Nakuba ziqeqeshelwe ukwenza imisebenzi elula njengokubikezela igama elilandelayo emshweni, izinhlobo zolimi lwezinzwa eziqeqeshwe ngokwanele nezinezinombolo zamapharamitha zitholakala zithatha iningi le-syntax ne-semantics yolimi lomuntu. Ngaphezu kwalokho, amamodeli amakhulu olimi abonisa ulwazi oluningi olujwayelekile mayelana nezwe, futhi bayakwazi "ukukhumbula" inani elikhulu lamaqiniso ngesikhathi sokuqeqeshwa.  Ukubona izinto ezingekho Isihloko esiyinhloko: I-hallucination (ubuhlakani bokufakelwa) Ebuhlakani bokufakelwa ngokujwayelekile, futhi ikakhulukazi kumamodeli amakhulu olimi, "i-hallucination" iyimpendulo yokuzethemba engabonakali ifanelekile ngemininingwane yokuqeqeshwa kwemodeli.  Amakhono avelayo  Ezingxenyeni eziningi zezivivinyo zolimi lwemvelo ezibandakanya imisebenzi efana nokuphendula imibuzo, amamodeli awenzi kangcono kunethuba elingahleliwe kuze kube yilapho efinyelela esikalini esithile (kulokhu, kulinganiswa ngokuqeqeshwa kokuqeqeshwa), okuyilapho ukusebenza kwawo kukhuphuka kakhulu. Lezi yizibonelo zamakhono avelayo. Amakhono angalindelekile aye abonwa kumamodeli amakhulu olimi kodwa ayengekho kumamodeli alula (futhi ayengaklanyelwe ngokucacile kumodeli) ngokuvamile abizwa ngokuthi "amakhono avelayo". Abacwaningi baphawula ukuthi amakhono anjalo "awenakubikezelwa ngokumane nje ucabange ngokusebenza kwamamodeli amancane". Lawa makhono atholakala kunokuhlelwa noma ukuklanywa, kwezinye izimo kuphela ngemuva kokuthi i-LLM isetshenziswe esidlangalaleni. Kuye kwachazwa amakhulu amakhono avelayo. Izibonelo zihlanganisa izibalo zezinyathelo eziningi, ukuthatha izivivinyo ezisezingeni lasekolishi, ukukhomba incazelo ehlosiwe yegama, ukucela uchungechunge lokucabanga, ukucacisa i-International Phonetic Alphabet, ukucacisa izinhlamvu zegama, ukukhomba okuqukethwe okucasulayo ezigabeni zeHinglish (inhlanganisela yesiHindi nesiNgisi), nokukhiqiza okufana nesiNgisi okufana nezisho zesiKiswahili.  Izakhiwo nokuqeqeshwa Amamodeli amakhulu olimi asetshenziswe kakhulu ukwakheka kwe-transformer, okuthi, kusukela ngo-2018, kube yindlela ejwayelekile yokufunda ejulile yedatha elandelanayo (ngaphambilini, ukwakheka okuphindaphindiwe okufana ne-LSTM kwakuvame kakhulu). Ama-LLM aqeqeshwa ngendlela engaqondiswayo embhalweni ongachazwanga. Umguquli osuka kwesobunxele uye kwesokudla uqeqeshelwa ukuba akhulise amathuba anikezwe igama elilandelayo kumininingwane yokuqeqeshwa, uma kunikezwe umongo wangaphambilini. Ngenye indlela, i-LLM ingasebenzisa i-transformer enamacala amabili (njengasesibonelweni se-BERT), enikeza ukwabiwa kwamathuba phezu kwamagama anikezwe ukufinyelela kokubili kokuqala nokulandelayo. Ngaphezu komsebenzi wokubikezela igama elilandelayo noma "ukugcwalisa izikhala", ama-LLM angaqeqeshwa emisebenzini yokusiza evivinya ukuqonda kwabo ukusabalalisa kwedatha njengokuqagela komusho olandelayo (i-NSP), lapho kunikezwa khona imisho emibili futhi imodeli kufanele ibikezele ukuthi ivela yini eceleni komunye nomunye ku-corpus yokuqeqesha.  Ama-LLM okuqala ayeqeqeshelwa ama-corpora anamagama ayizigidigidi. Imodeli yokuqala ochungechungeni lwe-OpenAI's GPT yaqeqeshwa ngo-2018 ku-BookCorpus, equkethe amagama ayizigidi ezingama-985. Ngawo lowo nyaka, i-BERT yaqeqeshwa ngokuhlanganiswa kwe-BookCorpus ne-English Wikipedia, ingqikithi yamagama ayizigidi eziyizinkulungwane ezingu-3.3. Eminyakeni edlule kusukela ngaleso sikhathi, izinhlangano zokuqeqesha ama-LLM ziye zanda ngokwezinga elikhulu, zafinyelela emakhulwini ezigidi noma ezigidigidini zamathokheni.  Ama-LLM abiza kakhulu ukuba aqeqeshwe. Ucwaningo lwango-2020 lwalulinganisela izindleko zokuqeqesha imodeli yepharamitha eyizigidi eziyizinkulungwane eziyi-1.5 (ama-oda angu-1-2 amancane kunezinga lobuciko ngaleso sikhathi) ku- $ 1.6 million.  Ukuhlaziywa kuka-2020 kutholakale ukuthi ikhono lamamodeli olimi lwe-neural (njengoba kulinganiswa ngokulahlekelwa ukuqeqeshwa) lenyuke kahle ebudlelwaneni bomthetho wamandla nenani lamapharamitha, inani lemininingwane yokuqeqeshwa, kanye nokubala okusetshenziselwe ukuqeqeshwa. Lezi zixhumanisi zahlolwa ebangeni elibanzi lamagugu (kuze kufike kuma-oda ayisikhombisa wobukhulu) futhi akukho kuncipha kobudlelwano okwabonwa ekugcineni okuphezulu kwebanga (kufaka phakathi ubukhulu benethiwekhi kuze kufike kumathriliyoni wamapharamitha).  Ukusetshenziswa emisebenzini engezansi Phakathi kuka-2018 no-2020, indlela ejwayelekile yokusebenzisa i-LLM ngomsebenzi othile wokucubungula ulimi lwemvelo (i-NLP) kwakuwukulungisa imodeli ngokuqeqeshwa okwengeziwe komsebenzi othile. Kamuva kutholakale ukuthi ama-LLM anamandla kakhulu afana ne-GPT-3 angaxazulula imisebenzi ngaphandle kokuqeqeshwa okwengeziwe ngokusebenzisa amasu "okusheshisa", lapho inkinga ezolungiswa yethulwa kumodeli njengokusheshisa umbhalo, mhlawumbe ngezibonelo ezithile zemibhalo yezinkinga ezifanayo nezixazululo zazo.  Ukulungisa kahle Isihloko esiyinhloko: Ukuhlela kahle (ukufunda ngomshini) Ukulungiswa kahle kuwumkhuba wokuguqula imodeli yolimi esevele iqeqeshiwe ngokuyiqeqesha (ngendlela eqondisiwe) emsebenzini othize (isib. ukuhlaziywa kwemizwa, ukuqashelwa kwezinto ezinamagama, noma ukumakwa kwengxenye yenkulumo). Kuyindlela yokufunda ngokudlulisela. Ngokuvamile kuhilela ukwethulwa kweqoqo elisha lezilinganiso ezixhumanisa ungqimba lokugcina lwemodeli yolimi nomphumela womsebenzi olandelayo. Izisindo zokuqala zemodeli yolimi zingase "zincibilike", ukuze kufundwe kuphela ungqimba olusha lwezisindo oluzixhumanisa nomphumela ngesikhathi sokuqeqeshwa. Ngenye indlela, isisindo sokuqala singathola izibuyekezo ezincane (mhlawumbe ngezingqimba zangaphambilini ezifriziwe).  Ukukhuthaza Bheka futhi: Ubunjiniyela obusheshayo nokufunda okuncane (ukucubungula ulimi lwemvelo) Ku-prompting paradigm, eyaziwa yi-GPT-3, inkinga okufanele ixazululwe ifakwa nge-text prompt, okumele imodeli ixazulule ngokunikeza ukuqedela (ngokusebenzisa ukucabanga). "Ekuphakamiseni okusheshayo", ukuphakanyiswa kuhlanganisa inani elincane lezibonelo zamabhangqa afanayo (inkinga, isixazululo). Isibonelo, umsebenzi wokuhlaziya imizwa wokufaka ilebula emizweni yokubuyekezwa kwebhayisikobho ungacelwa kanje:  Ukubuyekeza: Le filimu ayimnandi. Isimo sengqondo: esibi  Ukubuyekeza: Le filimu iyamangalisa! Imizwa:  Uma imodeli ikhipha "okuvumayo", kusho ukuthi ixazulule umsebenzi ngendlela efanele. Ekuceleni kokudutshulwa okungu-zero, akunikezwanga izibonelo zokuxazulula. Isibonelo sesikhuthazo se-zero-shot somsebenzi ofanayo wokuhlaziywa kwemizwa kungaba "Imizwa ehambisana nokubuyekezwa kwe-movie 'Le movie iyamangalisa!' ".  Ukusebenza okuncane kwe-LLM kuboniswe ukuthi kufinyelela imiphumela yokuncintisana emisebenzini ye-NLP, kwesinye isikhathi kudlula izindlela zangaphambilini zokulungisa ubuciko. Izibonelo zemisebenzi enjalo ye-NLP ukuhumusha, ukuphendula imibuzo, imisebenzi ye-cloze, ukuhumusha amagama, nokusebenzisa igama elisha emshweni. Ukwakhiwa nokwenziwa ngcono kwalezi zikhombisi-ndlela kubizwa ngokuthi i-prompt engineering.  Ukuhlela imiyalo Ukuqondiswa kwe-Instruction kuyindlela yokulungiswa okuhle okwenzelwe ukwenza lula ukusebenzisana okusheshayo kokudubula okusheshayo okungokwemvelo nokunembile. Njengoba kunikezwe umbhalo, imodeli yolimi eqeqeshwe ngaphambilini izokhiqiza ukuqedela okufana nokusatshalaliswa kombhalo lapho iqeqeshwe khona. Imodeli yolimi engenalwazi enikezwe umyalo othi "Bhala indaba ephathelene nezingqikithi eziyinhloko zikaHamlet". ingase inikeze isiphetho esinjengokuthi "Kuzofakwa inhlawulo yokulibala engu-10% ngosuku ezicelweni ezitholwe ngemva kuka-March 17". Ekuqondiseni imiyalo, imodeli yolimi iqeqeshwa ngezibonelo eziningi zemisebenzi ehlelwe njengemiyalo yolimi lwemvelo, kanye nezimpendulo ezifanele. Amasu ahlukahlukene okulungisa imiyalo asetshenzisiwe ekusebenzeni. I-OpenAI's InstructGPT protocol ifaka ukuqondiswa okuhleliwe kusethi yedatha yamabhangqa enziwe ngumuntu (okusheshayo, impendulo), elandelwa ukufunda okuqinisiwe kusuka ekuphenduleni komuntu (i-RLHF), lapho umsebenzi womvuzo wafundwa ngokusekelwe kusethi yedatha yezintandokazi zabantu. Enye inqubo, "ukuzifundisa", ilungisa imodeli yolimi kusethi yokuqeqeshwa yezibonelo zona ngokwazo ezikhiqizwa yi-LLM (eziqalwa kusethi yokuqala encane yezibonelo ezikhiqizwe ngabantu).  https://en.wikipedia.org/wiki/Large_language_model
Ebuhlakani bokufakelwa ngokujwayelekile, futhi ikakhulukazi kumamodeli amakhulu olimi, "i-hallucination" iyimpendulo yokuzethemba engabonakali ifanelekile ngemininingwane yokuqeqeshwa kwemodeli.