Zvichipiwa ndima idzi pamusoro pemhando huru dzemitauro, chii chinonzi LLM?
Context:A guru mutauro womuenzaniso (LLM) ndiwo mutauro womuenzaniso rinoumbwa tsinga network vane parameters zhinji (kazhinji mabhiriyoni weights kana kupfuura), vakadzidziswa pamusoro yakawanda unlabeled rugwaro vachishandisa kuzvidzora rinotarisirwa kudzidza. LLM yakabuda munenge muna 2018 uye inoita zvakanaka pamabasa akasiyana siyana. Ikoku kwakatamisa ngwariro yokutsvakurudza kwokushandura mutauro wepanyama kure neparadigm yapfuura yokurovedza mienzaniso inochengetwa inochengetwa nokuda kwemabasa chaiwo.  Zvinhu Kunyange zvazvo shoko rokuti muenzaniso mukuru wemitauro risina tsanangudzo yakarurama, rinowanzoreva mienzaniso yokudzidza kwakadzama ine nhamba yemiparamita inenge mabhiriyoni kana kupfuura. LLMs ndivo general chinangwa yokutevedzera kuti zvakaisvonaka panguva zvakawanda mabasa, kusiyana nokudzidziswa rimwe basa chairo (zvakadai mafungiro ongororo, ainzi sangano kuzivikanwa, kana yemasvomhu kufunga). Unyanzvi hwavanopedza nahwo mabasa, uye nhevedzano yemabasa avanokwanisa, inoratidzika kuva basa rehuwandu hwezvinhu (data, parameter-size, computing power) yakatsaurirwa kwavari, nenzira isingavimbi nedzimwe nzira dzokufambira mberi mukugadzira.  Kunyange zvazvo vakarovedzwa pamabasa ari nyore akadai sokufanotaura shoko rinotevera mumutsara, mienzaniso yemitauro yetsinga ine kurovedzwa kwakakwana uye kuverengwa kweparameter kunowanikwa kuti inobata zvakawanda zvechimiro nemashoko omutauro womunhu. Mukuwedzera, mienzaniso mikuru yemitauro inoratidza ruzivo rwakakura rwepasi rose pamusoro penyika, uye inokwanisa "kuyeuka" huwandu hukuru hwechokwadi panguva yekudzidziswa.  Kufungidzira Nyaya huru: Kufungidzira (uchenjeri hwekunyepedzera) Muungwaru hwekunyepedzera, uye mumhando huru dzemitauro kunyanya, "kufungidzira" kupindura nechivimbo kunoratidzika kunge kusiri kururamiswa nedata rekudzidzisa remhando.  Unyanzvi huri kubuda  Pamatanho anoverengeka emitauro yemasikirwo anosanganisira mabasa akadai sekupindura mibvunzo, mienzaniso haibudirire kupfuura mukana wekusarudzika kudzamara vasvika pachiyero chakati (munyaya iyi, chakayerwa nekurovedza kuverenga), panguva iyo mashandiro avo anowedzera zvakanyanya. Iyi mienzaniso yesimba riri kubuda. Unyanzvi husingafanoonekwi hwakamboonekwa mumhando huru dzemitauro asi hwakanga husiripo mumhando dzakareruka (uye idzo dzakanga dzisina kugadzirirwa zvakajeka mumuenzaniso) dzinowanzonzi "unyanzvi hunobuda". Vanotsvakurudza vanotaura kuti mano akadaro "haagoni kufanotaurwa nokungofungidzira mashandiro anoita mienzaniso miduku". Aya mano anowanwa panzvimbo pokurongwa-mukati kana kugadzirwa, mune zvimwe zviitiko bedzi pashure pokunge LLM yaiswa pachena. Mazana emano matsva akarondedzerwa. Mienzaniso inosanganisira masvomhu ematanho akawanda, kutora bvunzo dzechikoro chepamusoro, kuziva zvinorehwa neshoko, kufungidzira kwepfungwa, kududzira International Phonetic Alphabet, kududzira tsamba dzeshoko, kuziva zvinyorwa zvinogumbura mundima dzeHinglish (musanganiswa wechiHindi neChirungu), uye kugadzira zvakafanana neChirungu zvakaenzana nezvirevo zveKiswahili.  Kugadzira uye kudzidzisa Large mutauro mienzaniso kunge kazhinji akashandisa transformer mapurani, izvo, kubva 2018, rava mureza zvakadzama kudzidza unyanzvi nokuda sequential mashoko (kare, recurrent mapurani akadai LSTM vakanga vakawanda). LLMs vanodzidziswa nenzira isingatarisirwi pamashoko asina kunyorwa. A kuruboshwe-kurudyi transformer anodzidziswa kuti maximize mukana vakapiwa rinotevera shoko kudzidziswa mashoko, akapiwa yapfuura akapoteredza. Neimwe nzira, LLM inogona kushandisa bidirectional transformer (somuenzaniso weBERT), iyo inopa mukana wekugovera pamusoro pemashoko anopiwa mukana kune zvose zviri mberi uye zvinotevera. Mukuwedzera kubasa rokufanotaura shoko rinotevera kana kuti "kuzadza nzvimbo dzisina chinhu", LLM dzinogona kudzidziswa pamabasa okuyamura anoedza kunzwisisa kwavo kugoverwa kwemashoko akadai seNext Sentence Prediction (NSP), umo vaviri vemitsara vanoratidzwa uye muenzaniso unofanira kufanotaura kana vachioneka vari pamwe chete mubhuku rekudzidzisa.  VaLLM vokutanga vakarovedzwa pamakorpora ane mashoko ane mabhiriyoni. Muenzaniso wekutanga muOpenAI's GPT akateedzana akadzidziswa muna 2018 paBookCorpus, iine mazwi emamiriyoni mazana mapfumbamwe nemakumi masere neshanu. Mugore rimwe chetero, BERT yakadzidziswa mubatanidzwa weBookCorpus neChirungu Wikipedia, inosvika mazwi mabhiriyoni 3,3. Mumakore kubvira ipapo, makorasi okudzidzisa LLM akawedzera zvikuru, kusvika kumazana emamiriyoni kana kuti matiririyoni ematani.  LLM dzinodhura kurovedza. Chidzidzo cha2020 chakafungidzira mutengo wekudzidzisa 1.5 bhiriyoni parameter modhi (1-2 mirairo yehukuru idiki pane mamiriro ehunyanzvi panguva iyoyo) pa $ 1.6 mamirioni.  Ongororo yakaitwa muna 2020 yakawana kuti kugona kwemitauro yemitauro (sezvinoyerwa nekurasikirwa nekudzidziswa) kwakawedzera zvinyoronyoro muhukama hwemutemo wesimba nenhamba yemiparamende, huwandu hwe data rekudzidzira, uye kuverenga kunoshandiswa pakudzidziswa. Hukama uhwu hwakaedzwa pamwero wakakura wemitengo (kusvika pamatanho manomwe ehukuru) uye hapana kudzikisira kwehukama kwakaonekwa kumugumo wepamusoro wepakati (kusanganisira kune hukuru hwemambure kusvika kumatirioni ematanho).  Kushandiswa kumabasa anotevera Pakati pa2018 na2020, nzira yakajairika yekushandisa LLM kune rimwe basa rakasarudzika rekugadzirisa mutauro (NLP) yaive yekugadzirisa iyo modhi nekuwedzera basa-rakasarudzika kudzidziswa. Zvakazoonekwa kuti LLM dzakasimba dzakadai seGPT-3 dzinogona kugadzirisa mabasa pasina kudzidziswa kwekuwedzera kuburikidza ne "kukurudzira" maitiro, umo dambudziko rinofanira kugadziriswa rinoratidzwa kumhando sechikumbiro chemashoko, zvichida nemimwe mienzaniso yemashoko yezvinetso zvakafanana nemhinduro dzavo.  Kugadziridza Nyaya huru: Kugadziridza (kudzidza kwemuchina) Kugadziridza kwakanaka muitiro wokugadziridza mutauro uripo wakadzidziswa kare nokudzidzisa (nenzira inotarisirwa) pane rimwe basa (semuenzaniso kuongorora mafungiro, kuzivikanwa kwezvinhu zvinonzi, kana chikamu-chekutaura tagging). Chimiro chokudzidza chinopfuudzwa. Kazhinji kunobatanidza kuunzwa kwechikwata chitsva chezviyereso zvinobatanidza rukoko rwokupedzisira rwomuenzaniso womutauro kumigumisiro yebasa rinotevera. Kurema kwokutanga kwomuenzaniso womutauro kungava "kwakasungirirwa", zvokuti bedzi rukoko rutsva rwokurema runovasonganisa nomugumisiro runodzidzwa mukati mokurovedza. Neimwe nzira, huremu hwepakutanga hunogona kugamuchira zvigadziriso zvidiki (pamwe nezvikamu zvekare zvakaomeswa nechando).  Kukurudzira Onawo: Kurumidza engineering uye Few-shot kudzidza (chisikigo mutauro kubudiswa) Muchikurudziro paradigm, yakakurumbira neGPT-3, dambudziko rinogadziriswa rinoumbwa kuburikidza nemashoko ekukurudzira, ayo muenzaniso unofanira kugadzirisa nekupa kupedzisa (kuburikidza nekugadzirisa). Mu"shoma-kupfura kukumbira", kukumbira kunosanganisira nhamba shoma yemienzaniso yakafanana (chinetso, mhinduro) vaviri. Somuenzaniso, basa rokunzvera manzwiro rokunyora manzwiro eongororo yefirimu rinogona kukumbirwa sezvinotevera:  Ongorora: Iyi firimu inosemesa. Mafungiro: akaipa  Ongorora: Iyi firimu inoshamisa! Manzwiro:  Kana muenzaniso wacho ukabudisa mashoko okuti "zvakanaka", zvinoreva kuti wapedza basa racho nemazvo. Mu zero-shot kukurudzira, hapana mienzaniso yekugadzirisa inopiwa. Muenzaniso we zero-shot inokurudzira yeiyo imwechete manzwiro ongororo basa ingave "Manzwiro ane chekuita nekuongorora bhaisikopo 'Iyi bhaisikopo inonakidza!' iri".  Kushanda kweLLM kunoratidza kuti kunoguma nemigumisiro yakanaka pamabasa eNLP, dzimwe nguva ichipfuura nzira dzekugadzirisa dzakapfuura. Mienzaniso yemabasa akadaro eNLP inosanganisira kushandura, kupindura mibvunzo, mabasa ekutsvaira, kutsanangura mashoko, uye kushandisa shoko risingazivikanwi mumutsara. Kugadzirwa nokugadziriswa kwezvikurudziro zvakadaro kunonzi uinjiniya hwechikurudziro.  Kuronga mirayiridzo Instruction tuning inzira yekugadzirisa yakagadzirirwa kuita kuti zvive nyore uye zvakarurama zero-shot inokurudzira kudyidzana. Zvichipiwa mashoko okuisa, mutauro womuenzaniso wakadzidziswa kare uchabereka kupedzisa kunowirirana nokuparadzirwa kwemashoko ayo akadzidziswa nawo. Muenzaniso womutauro usina kuchenjera unopiwa chikumbiro chokuti "Nyora rondedzero pamusoro pemisoro mikuru yaHamlet". ungapa kupedzisa kwakadai sokuti "Chirango chokunonoka che10% pazuva chichashandiswa kuzvinyorwa zvakagamuchirwa pashure paMarch 17". Mukudzora mirayiridzo, mutauro wemuenzaniso unodzidziswa pane mienzaniso yakawanda yemabasa akaumbwa semirayiridzo yemutauro wepanyama, pamwe chete nemhinduro dzakakodzera. Unyanzvi hwakasiyana-siyana hwokudzora mirayiridzo hwakashandiswa mukuita. OpenAI's InstructGPT protocol inobatanidza kutariswa kwakanaka pane dhatabhesi yevanhu-yakagadzirwa (kukurumidzira, kupindura) vaviri vaviri, ichiteverwa nekudzidzira kusimbisa kubva kumhinduro yevanhu (RLHF), umo basa remubairo rakadzidzwa zvichibva pane dhatabhesi yezvido zvevanhu. Imwe nzira, "kuzvidzidzisa", inonyatsogadzirisa mutauro wemitauro pamusangano wekudzidzisa wemienzaniso iyo inogadzirwa neLLM (inotanga kubva kune imwe duku yekutanga yemienzaniso yakagadzirwa nevanhu).  https://en.wikipedia.org/wiki/Large_language_model
A guru mutauro womuenzaniso (LLM) ndiwo mutauro womuenzaniso rinoumbwa tsinga network vane parameters zhinji (kazhinji mabhiriyoni weights kana kupfuura), vakadzidziswa pamusoro yakawanda unlabeled rugwaro vachishandisa kuzvidzora rinotarisirwa kudzidza.