Hindi, English, Kannada, Telugu, Assamese, Bengali and Malayalam
ABSTRACTIVE
Kannada, Hindi, Telugu, Malayalam, Bengali,  English and Assamese (in table, missing in text)
ABSTRACTIVE
FLOAT SELECTED: Table 1: Dataset
In this section, we describe our dataset collection process. We collected and curated around 635Hrs of audio data for 7 Indian languages, namely Kannada, Hindi, Telugu, Malayalam, Bengali, and English. We collected the data from the All India Radio news channel where an actor will be reading news for about 5-10 mins. To cover many speakers for the dataset, we crawled data from 2010 to 2019. Since the audio is very long to train any deep neural network directly, we segment the audio clips into smaller chunks using Voice activity detector. Since the audio clips will have music embedded during the news, we use Inhouse music detection model to remove the music segments from the dataset to make the dataset clean and our dataset contains 635Hrs of clean audio which is divided into 520Hrs of training data containing 165K utterances and 115Hrs of testing data containing 35K utterances. The amount of audio data for training and testing for each of the language is shown in the table bellow. FLOAT SELECTED: Table 1: Dataset
2002.01664
IN TR ODUCT ION </s> The Ġidea Ġof Ġlanguage Ġidentification Ġis Ġto Ġclassify Ġa Ġgiven Ġaudio Ġsignal Ġinto Ġa Ġparticular Ġclass Ġusing Ġa Ġclassification Ġalgorithm . ĠCommon ly Ġlanguage Ġidentification Ġtask Ġwas Ġdone Ġusing Ġi - vector Ġsystems Ġ[ 1 ]. ĠA Ġvery Ġwell Ġknown Ġapproach Ġfor Ġlanguage Ġidentification Ġproposed Ġby ĠN . ĠDa he k Ġet Ġal . Ġ[ 1 ] Ġuses Ġthe ĠGM M - UB M Ġmodel Ġto Ġobtain Ġutter ance Ġlevel Ġfeatures Ġcalled Ġi - ve ctors . ĠRecent Ġadvances Ġin Ġdeep Ġlearning Ġ[ 15 , 16 ] Ġhave Ġhelped Ġto Ġimprove Ġthe Ġlanguage Ġidentification Ġtask Ġusing Ġmany Ġdifferent Ġneural Ġnetwork Ġarchitectures Ġwhich Ġcan Ġbe Ġtrained Ġefficiently Ġusing ĠGPUs Ġfor Ġlarge Ġscale Ġdatasets . ĠThese Ġneural Ġnetworks Ġcan Ġbe Ġconfigured Ġin Ġvarious Ġways Ġto Ġobtain Ġbetter Ġaccuracy Ġfor Ġlanguage Ġidentification Ġtask . ĠEarly Ġwork Ġon Ġusing ĠDeep Ġlearning Ġfor Ġlanguage ĠIdentification Ġwas Ġpublished Ġby ĠPavel ĠMate j ka Ġet Ġal . Ġ[ 2 ], Ġwhere Ġthey Ġused Ġstacked Ġbottleneck Ġfeatures Ġextracted Ġfrom Ġdeep Ġneural Ġnetworks Ġfor Ġlanguage Ġidentification Ġtask Ġand Ġshowed Ġthat Ġthe Ġbottleneck Ġfeatures Ġlearned Ġby ĠDeep Ġneural Ġnetworks Ġare Ġbetter Ġthan Ġsimple ĠMF CC Ġor ĠPL P Ġfeatures . ĠLater Ġthe Ġwork Ġby ĠI . ĠLopez - More no Ġet Ġal . Ġ[ 3 ] Ġfrom ĠGoogle Ġshowed Ġhow Ġto Ġuse ĠDeep Ġneural Ġnetworks Ġto Ġdirectly Ġmap Ġthe Ġsequence Ġof ĠMF CC Ġframes Ġinto Ġits Ġlanguage Ġclass Ġso Ġthat Ġwe Ġcan Ġapply Ġlanguage Ġidentification Ġat Ġthe Ġframe Ġlevel . ĠSpeech Ġsignals Ġwill Ġhave Ġboth Ġspatial Ġand Ġtemporal Ġinformation , Ġbut Ġsimple ĠD NN s Ġare Ġnot Ġable Ġto Ġcapture Ġtemporal Ġinformation . ĠWork Ġdone Ġby ĠJ . ĠGonzalez - D oming uez Ġet Ġal . Ġ[ 4 ] Ġby ĠGoogle Ġdeveloped Ġan ĠL ST M Ġbased Ġlanguage Ġidentification Ġmodel Ġwhich Ġimproves Ġthe Ġaccuracy Ġover Ġthe ĠD NN Ġbased Ġmodels . ĠWork Ġdone Ġby ĠAlicia Ġet Ġal . Ġ[ 5 ] Ġused ĠCNN s Ġto Ġimprove Ġupon Ġi - vector Ġ[ 1 ] Ġand Ġother Ġpreviously Ġdeveloped Ġsystems . ĠThe Ġwork Ġdone Ġby ĠDaniel ĠGarcia - Rom ero Ġet Ġal . Ġ[ 6 ] Ġhas Ġused Ġa Ġcombination Ġof ĠAc oustic Ġmodel Ġtrained Ġfor Ġspeech Ġrecognition Ġwith ĠTime - delay Ġneural Ġnetworks Ġwhere Ġthey Ġtrain Ġthe ĠTD NN Ġmodel Ġby Ġfeeding Ġthe Ġstacked Ġbottleneck Ġfeatures Ġfrom Ġacoustic Ġmodel Ġto Ġpredict Ġthe Ġlanguage Ġlabels Ġat Ġthe Ġframe Ġlevel . ĠRecently ĠX - ve ctors Ġ[ 7 ] Ġis Ġproposed Ġfor Ġspeaker Ġidentification Ġtask Ġand Ġare Ġshown Ġto Ġoutper form Ġall Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġspeaker Ġidentification Ġalgorithms Ġand Ġare Ġalso Ġused Ġfor Ġlanguage Ġidentification Ġby ĠDavid ĠSnyder Ġet Ġal . Ġ[ 8 ]. </s> In Ġthis Ġpaper , Ġwe Ġexplore Ġmultiple Ġpool ing Ġstrategies Ġfor Ġlanguage Ġidentification Ġtask . ĠMain ly Ġwe Ġpropose ĠGhost - VL AD Ġbased Ġpool ing Ġmethod Ġfor Ġlanguage Ġidentification . ĠInspired Ġby Ġthe Ġrecent Ġwork Ġby ĠW . ĠXie Ġet Ġal . Ġ[ 9 ] Ġand ĠY . ĠZh ong Ġet Ġal . Ġ[ 10 ], Ġwe Ġuse ĠGhost - VL AD Ġto Ġimprove Ġthe Ġaccuracy Ġof Ġlanguage Ġidentification Ġtask Ġfor ĠIndian Ġlanguages . ĠWe Ġexplore Ġmultiple Ġpool ing Ġstrategies Ġincluding ĠNet VL AD Ġpool ing Ġ[ 11 ], ĠAverage Ġpool ing Ġand ĠStatistics Ġpool ing ( Ġas Ġproposed Ġin ĠX - ve ctors Ġ[ 7 ]) Ġand Ġshow Ġthat ĠGhost - VL AD Ġpool ing Ġis Ġthe Ġbest Ġpool ing Ġstrategy Ġfor Ġlanguage Ġidentification . ĠOur Ġmodel Ġob tains Ġthe Ġbest Ġaccuracy Ġof Ġ98 . 24 %, Ġand Ġit Ġoutper forms Ġall Ġthe Ġother Ġpreviously Ġproposed Ġpool ing Ġmethods . ĠWe Ġconduct Ġall Ġour Ġexperiments Ġon Ġ6 35 h rs Ġof Ġaudio Ġdata Ġfor Ġ7 ĠIndian Ġlanguages Ġcollected Ġfrom Ġ$\ text bf Ġ{ All ĠIndia ĠRadio } $ Ġnews Ġchannel . ĠThe Ġpaper Ġis Ġorganized Ġas Ġfollows . ĠIn Ġsection Ġ2 , Ġwe Ġexplain Ġthe Ġproposed Ġpool ing Ġmethod Ġfor Ġlanguage Ġidentification . ĠIn Ġsection Ġ3 , Ġwe Ġexplain Ġour Ġdataset . ĠIn Ġsection Ġ4 , Ġwe Ġdescribe Ġthe Ġexperiments , Ġand Ġin Ġsection Ġ5 , Ġwe Ġdescribe Ġthe Ġresults . </s> PO OL ING ĠSTR ATE G IES </s> In Ġany Ġlanguage Ġidentification Ġmodel , Ġwe Ġwant Ġto Ġobtain Ġutter ance Ġlevel Ġrepresentation Ġwhich Ġhas Ġvery Ġgood Ġlanguage Ġdiscrim in ative Ġfeatures . ĠThese Ġrepresentations Ġshould Ġbe Ġcompact Ġand Ġshould Ġbe Ġeasily Ġsepar able Ġby Ġa Ġlinear Ġclass ifier . ĠThe Ġidea Ġof Ġany Ġpool ing Ġstrategy Ġis Ġto Ġpool Ġthe Ġframe - level Ġrepresentations Ġinto Ġa Ġsingle Ġutter ance Ġlevel Ġrepresentation . ĠPrevious Ġworks Ġby Ġ[ 7 ] Ġhave Ġused Ġsimple Ġmean Ġand Ġstandard Ġdeviation Ġaggregation Ġto Ġpool Ġthe Ġframe - level Ġfeatures Ġfrom Ġthe Ġtop Ġlayer Ġof Ġthe Ġneural Ġnetwork Ġto Ġobtain Ġthe Ġutter ance Ġlevel Ġfeatures . ĠRecently Ġ[ 9 ] Ġused ĠV L AD Ġbased Ġpool ing Ġstrategy Ġfor Ġspeaker Ġidentification Ġwhich Ġis Ġinspired Ġfrom Ġ[ 10 ] Ġproposed Ġfor Ġface Ġrecognition . ĠThe ĠNet VL AD Ġ[ 11 ] Ġand ĠGhost - VL AD Ġ[ 10 ] Ġmethods Ġare Ġproposed Ġfor ĠPlace Ġrecognition Ġand Ġface Ġrecognition , Ġrespectively , Ġand Ġin Ġboth Ġcases , Ġthey Ġtry Ġto Ġaggregate Ġthe Ġlocal Ġdescript ors Ġinto Ġglobal Ġfeatures . ĠIn Ġour Ġcase , Ġthe Ġlocal Ġdescript ors Ġare Ġfeatures Ġextracted Ġfrom ĠRes Net Ġ[ 15 ], Ġand Ġthe Ġglobal Ġutter ance Ġlevel Ġfeature Ġis Ġobtained Ġby Ġusing ĠGhost VL AD Ġpool ing . ĠIn Ġthis Ġsection , Ġwe Ġexplain Ġdifferent Ġpool ing Ġmethods , Ġincluding ĠNet VL AD , ĠGhost - VL AD , ĠStat istic Ġpool ing , Ġand ĠAverage Ġpool ing . </s> PO OL ING ĠSTR ATE G IES Ġ: :: ĠNet VL AD Ġpool ing </s> The ĠNet VL AD Ġpool ing Ġstrategy Ġwas Ġinitially Ġdeveloped Ġfor Ġplace Ġrecognition Ġby ĠR . ĠAr and j el ovic Ġet Ġal . Ġ[ 11 ]. ĠThe ĠNet VL AD Ġis Ġan Ġextension Ġto ĠV L AD Ġ[ 18 ] Ġapproach Ġwhere Ġthey Ġwere Ġable Ġto Ġreplace Ġthe Ġhard Ġassignment Ġbased Ġclust ering Ġwith Ġsoft Ġassignment Ġbased Ġclust ering Ġso Ġthat Ġit Ġcan Ġbe Ġtrained Ġwith Ġneural Ġnetwork Ġin Ġan Ġend Ġto Ġend Ġfashion . ĠIn Ġour Ġcase , Ġwe Ġuse Ġthe ĠNet VL AD Ġlayer Ġto Ġmap ĠN Ġlocal Ġfeatures Ġof Ġdimension ĠD Ġinto Ġa Ġfixed Ġdimensional Ġvector , Ġas Ġshown Ġin ĠFigure Ġ1 Ġ( Left Ġside ). </s> The Ġmodel Ġtakes Ġspect rogram Ġas Ġan Ġinput Ġand Ġfeeds Ġinto ĠCNN Ġbased ĠRes Net Ġarchitecture . ĠThe ĠRes Net Ġis Ġused Ġto Ġmap Ġthe Ġspect rogram Ġinto Ġ3 D Ġfeature Ġmap Ġof Ġdimension ĠH x W x D . ĠWe Ġconvert Ġthis Ġ3 D Ġfeature Ġmap Ġinto Ġ2 D Ġby Ġunfolding ĠH Ġand ĠW Ġdimensions , Ġcreating Ġa ĠN x D Ġdimensional Ġfeature Ġmap , Ġwhere ĠN = H x W . ĠThe ĠNet VL AD Ġlayer Ġis Ġkept Ġon Ġtop Ġof Ġthe Ġfeature Ġextraction Ġlayer Ġof ĠRes Net , Ġas Ġshown Ġin ĠFigure Ġ1 . ĠThe ĠNet VL AD Ġnow Ġtakes ĠN Ġfeatures Ġvectors Ġof Ġdimension ĠD Ġand Ġcomp utes Ġa Ġmatrix ĠV Ġof Ġdimension ĠK x D , Ġwhere ĠK Ġis Ġthe Ġnumber Ġclusters Ġin Ġthe ĠNet VL AD Ġlayer , Ġand ĠD Ġis Ġthe Ġdimension Ġof Ġthe Ġfeature Ġvector . ĠThe Ġmatrix ĠV Ġis Ġcomputed Ġas Ġfollows . </s> Where Ġ$ w _ k $, $ b _ k $ Ġand Ġ$ c _ k $ Ġare Ġtrain able Ġparameters Ġfor Ġthe Ġcluster Ġ$ k $ Ġand ĠV ( j , k ) Ġrepresents Ġa Ġpoint Ġin Ġthe ĠV Ġmatrix Ġfor Ġ( j , k ) th Ġlocation . ĠThe Ġmatrix Ġis Ġconstructed Ġusing Ġthe Ġequation Ġ( 1 ) Ġwhere Ġthe Ġfirst Ġterm Ġcorresponds Ġto Ġthe Ġsoft Ġassignment Ġof Ġthe Ġinput Ġ$ x _ i $ Ġto Ġthe Ġcluster Ġ$ c _ k $, Ġwhereas Ġthe Ġsecond Ġterm Ġcorresponds Ġto Ġthe Ġresidual Ġterm Ġwhich Ġtells Ġhow Ġfar Ġthe Ġinput Ġdescriptor Ġ$ x _ i $ Ġis Ġfrom Ġthe Ġcluster Ġcenter Ġ$ c _ k $. </s> PO OL ING ĠSTR ATE G IES Ġ: :: ĠGhost VL AD Ġpool ing </s> Ghost VL AD Ġis Ġan Ġextension Ġof Ġthe ĠNet VL AD Ġapproach , Ġwhich Ġwe Ġdiscussed Ġin Ġthe Ġprevious Ġsection . ĠThe ĠGhost VL AD Ġmodel Ġwas Ġproposed Ġfor Ġface Ġrecognition Ġby ĠY . ĠZh ong Ġ[ 10 ]. ĠGhost VL AD Ġworks Ġexactly Ġsimilar Ġto ĠNet VL AD Ġexcept Ġit Ġadds ĠGhost Ġclusters Ġalong Ġwith Ġthe ĠNet VL AD Ġclusters . ĠSo , Ġnow Ġwe Ġwill Ġhave Ġa ĠK + G Ġnumber Ġof Ġclusters Ġinstead Ġof ĠK Ġclusters . ĠWhere ĠG Ġis Ġthe Ġnumber Ġof Ġghost Ġclusters , Ġwe Ġwant Ġto Ġadd Ġ( typically Ġ2 - 4 ). ĠThe ĠGhost Ġclusters Ġare Ġadded Ġto Ġmap Ġany Ġnoisy Ġor Ġirrelevant Ġcontent Ġinto Ġghost Ġclusters Ġand Ġare Ġnot Ġincluded Ġduring Ġthe Ġfeature Ġaggregation Ġstage , Ġas Ġshown Ġin ĠFigure Ġ1 Ġ( Right Ġside ). ĠWhich Ġmeans Ġthat Ġwe Ġcompute Ġthe Ġmatrix ĠV Ġfor Ġboth Ġnormal Ġcluster ĠK Ġand Ġghost Ġclusters ĠG , Ġbut Ġwe Ġwill Ġnot Ġinclude Ġthe Ġvectors Ġbelongs Ġto Ġghost Ġcluster Ġfrom ĠV Ġduring Ġconc aten ation Ġof Ġthe Ġfeatures . ĠDue Ġto Ġwhich , Ġduring Ġfeature Ġaggregation Ġstage Ġthe Ġcontribution Ġof Ġthe Ġnoisy Ġand Ġunwanted Ġfeatures Ġto Ġnormal ĠV L AD Ġclusters Ġare Ġassigned Ġless Ġweights Ġwhile ĠGhost Ġclusters Ġabsorb Ġmost Ġof Ġthe Ġweight . ĠWe Ġillustrate Ġthis Ġin ĠFigure Ġ1 ( Right ĠSide ), Ġwhere Ġthe Ġghost Ġclusters Ġare Ġshown Ġin Ġred Ġcolor . ĠWe Ġuse ĠGhost Ġclusters Ġwhen Ġwe Ġare Ġcomputing Ġthe ĠV Ġmatrix , Ġbut Ġthey Ġare Ġexcluded Ġduring Ġthe Ġconc aten ation Ġstage . ĠThese Ġconc aten ated Ġfeatures Ġare Ġfed Ġinto Ġthe Ġprojection Ġlayer , Ġfollowed Ġby Ġsoft max Ġto Ġpredict Ġthe Ġlanguage Ġlabel . </s> PO OL ING ĠSTR ATE G IES Ġ: :: ĠStat istic Ġand Ġaverage Ġpool ing </s> In Ġstatistic Ġpool ing , Ġwe Ġcompute Ġthe Ġfirst Ġand Ġsecond Ġorder Ġstatistics Ġof Ġthe Ġlocal Ġfeatures Ġfrom Ġthe Ġtop Ġlayer Ġof Ġthe ĠRes Net Ġmodel . ĠThe Ġ3 - D Ġfeature Ġmap Ġis Ġunfolded Ġto Ġcreate ĠN Ġfeatures Ġof ĠD Ġdimensions , Ġand Ġthen Ġwe Ġcompute Ġthe Ġmean Ġand Ġstandard Ġdeviation Ġof Ġall Ġthese ĠN Ġvectors Ġand Ġget Ġtwo ĠD Ġdimensional Ġvectors , Ġone Ġfor Ġmean Ġand Ġthe Ġother Ġfor Ġstandard Ġdeviation . ĠWe Ġthen Ġconc aten ate Ġthese Ġ2 Ġfeatures Ġand Ġfeed Ġit Ġto Ġthe Ġprojection Ġlayer Ġfor Ġpredicting Ġthe Ġlanguage Ġlabel . </s> In Ġthe ĠAverage Ġpool ing Ġlayer , Ġwe Ġcompute Ġonly Ġthe Ġfirst - order Ġstatistics Ġ( mean ) Ġof Ġthe Ġlocal Ġfeatures Ġfrom Ġthe Ġtop Ġlayer Ġof Ġthe ĠCNN Ġmodel . ĠThe Ġfeature Ġmap Ġfrom Ġthe Ġtop Ġlayer Ġof ĠCNN Ġis Ġunfolded Ġto Ġcreate ĠN Ġfeatures Ġof ĠD Ġdimensions , Ġand Ġthen Ġwe Ġcompute Ġthe Ġmean Ġof Ġall Ġthese ĠN Ġvectors Ġand Ġget ĠD Ġdimensional Ġrepresentation . ĠWe Ġthen Ġfeed Ġthis Ġfeature Ġto Ġthe Ġprojection Ġlayer Ġfollowed Ġby Ġsoft max Ġfor Ġpredicting Ġthe Ġlanguage Ġlabel . </s> D AT AS ET </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġour Ġdataset Ġcollection Ġprocess . ĠWe Ġcollected Ġand Ġcurated Ġaround Ġ6 35 H rs Ġof Ġaudio Ġdata Ġfor Ġ7 ĠIndian Ġlanguages , Ġnamely ĠK ann ada , ĠHindi , ĠTel ugu , ĠMal ay alam , ĠBeng ali , Ġand ĠEnglish . ĠWe Ġcollected Ġthe Ġdata Ġfrom Ġthe ĠAll ĠIndia ĠRadio Ġnews Ġchannel Ġwhere Ġan Ġactor Ġwill Ġbe Ġreading Ġnews Ġfor Ġabout Ġ5 - 10 Ġmins . ĠTo Ġcover Ġmany Ġspeakers Ġfor Ġthe Ġdataset , Ġwe Ġcrawled Ġdata Ġfrom Ġ2010 Ġto Ġ2019 . ĠSince Ġthe Ġaudio Ġis Ġvery Ġlong Ġto Ġtrain Ġany Ġdeep Ġneural Ġnetwork Ġdirectly , Ġwe Ġsegment Ġthe Ġaudio Ġclips Ġinto Ġsmaller Ġchunks Ġusing ĠVoice Ġactivity Ġdetector . ĠSince Ġthe Ġaudio Ġclips Ġwill Ġhave Ġmusic Ġembedded Ġduring Ġthe Ġnews , Ġwe Ġuse ĠIn house Ġmusic Ġdetection Ġmodel Ġto Ġremove Ġthe Ġmusic Ġsegments Ġfrom Ġthe Ġdataset Ġto Ġmake Ġthe Ġdataset Ġclean Ġand Ġour Ġdataset Ġcontains Ġ6 35 H rs Ġof Ġclean Ġaudio Ġwhich Ġis Ġdivided Ġinto Ġ520 H rs Ġof Ġtraining Ġdata Ġcontaining Ġ165 K Ġutter ances Ġand Ġ115 H rs Ġof Ġtesting Ġdata Ġcontaining Ġ35 K Ġutter ances . ĠThe Ġamount Ġof Ġaudio Ġdata Ġfor Ġtraining Ġand Ġtesting Ġfor Ġeach Ġof Ġthe Ġlanguage Ġis Ġshown Ġin Ġthe Ġtable Ġbell ow . </s> EX PER IM ENTS </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġthe Ġfeature Ġextraction Ġprocess Ġand Ġnetwork Ġarchitecture Ġin Ġdetail . ĠWe Ġuse Ġspectral Ġfeatures Ġof Ġ256 Ġdimensions Ġcomputed Ġusing Ġ512 Ġpoint ĠF FT Ġfor Ġevery Ġframe , Ġand Ġwe Ġadd Ġan Ġenergy Ġfeature Ġfor Ġevery Ġframe Ġgiving Ġus Ġtotal Ġ257 Ġfeatures Ġfor Ġevery Ġframe . ĠWe Ġuse Ġa Ġwindow Ġsize Ġof Ġ25 ms Ġand Ġframe Ġshift Ġof Ġ10 ms Ġduring Ġfeature Ġcomputation . ĠWe Ġcrop Ġrandom Ġ5 sec Ġaudio Ġdata Ġfrom Ġeach Ġutter ance Ġduring Ġtraining Ġwhich Ġresults Ġin Ġa Ġspect rogram Ġof Ġsize Ġ257 x 500 Ġ( features Ġx Ġnumber Ġof Ġfeatures ). ĠWe Ġuse Ġthese Ġspect rog rams Ġas Ġinput Ġto Ġour ĠCNN Ġmodel Ġduring Ġtraining . ĠDuring Ġtesting , Ġwe Ġcompute Ġthe Ġprediction Ġscore Ġirrespective Ġof Ġthe Ġaudio Ġlength . </s> For Ġthe Ġnetwork Ġarchitecture , Ġwe Ġuse ĠRes Net - 34 Ġarchitecture , Ġas Ġdescribed Ġin Ġ[ 9 ]. ĠThe Ġmodel Ġuses Ġconv olution Ġlayers Ġwith ĠRel u Ġactiv ations Ġto Ġmap Ġthe Ġspect rogram Ġof Ġsize Ġ257 x 500 Ġinput Ġinto Ġ3 D Ġfeature Ġmap Ġof Ġsize Ġ1 x 32 x 512 . ĠThis Ġfeature Ġcube Ġis Ġconverted Ġinto Ġ2 D Ġfeature Ġmap Ġof Ġdimension Ġ32 x 512 Ġand Ġfed Ġinto ĠGhost - VL AD / Net VL AD Ġlayer Ġto Ġgenerate Ġa Ġrepresentation Ġthat Ġhas Ġmore Ġlanguage Ġdiscrimination Ġcapacity . ĠWe Ġuse ĠAdam Ġoptim izer Ġwith Ġan Ġinitial Ġlearning Ġrate Ġof Ġ0 . 01 Ġand Ġa Ġfinal Ġlearning Ġrate Ġof Ġ0 . 0000 1 Ġfor Ġtraining . ĠEach Ġmodel Ġis Ġtrained Ġfor Ġ15 Ġepoch s Ġwith Ġearly Ġstopping Ġcriteria . </s> For Ġthe Ġbaseline , Ġwe Ġtrain Ġan Ġi - vector Ġmodel Ġusing ĠGM M - UB M . ĠWe Ġfit Ġa Ġsmall Ġclass ifier Ġon Ġtop Ġof Ġthe Ġgenerated Ġi - ve ctors Ġto Ġmeasure Ġthe Ġaccuracy . ĠThis Ġmodel Ġis Ġreferred Ġas Ġi - vector + s vm Ġ. ĠTo Ġcompare Ġour Ġmodel Ġwith Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġsystem , Ġwe Ġset Ġup Ġthe Ġx - vector Ġlanguage Ġidentification Ġsystem Ġ[ 8 ]. ĠThe Ġx - vector Ġmodel Ġused Ġtime - delay Ġneural Ġnetworks Ġ( TD NN ) Ġalong Ġwith Ġstatistic - pool ing . ĠWe Ġuse Ġ7 Ġlayer ĠTD NN Ġarchitecture Ġsimilar Ġto Ġ[ 8 ] Ġfor Ġtraining . ĠWe Ġrefer Ġto Ġthis Ġmodel Ġas Ġtd nn + stat - pool Ġ. ĠFinally , Ġwe Ġset Ġup Ġa ĠDeep ĠL ST M Ġbased Ġlanguage Ġidentification Ġsystem Ġsimilar Ġto Ġ[ 4 ] Ġbut Ġwith Ġlittle Ġmodification Ġwhere Ġwe Ġadd Ġstatistics Ġpool ing Ġfor Ġthe Ġlast Ġlayers Ġhidden Ġactivities Ġbefore Ġclassification . ĠWe Ġuse Ġ3 Ġlayer ĠBi - L ST M Ġwith Ġ256 Ġhidden Ġunits Ġat Ġeach Ġlayer . ĠWe Ġrefer Ġto Ġthis Ġmodel Ġas ĠL ST M + stat - pool . ĠWe Ġtrain Ġour Ġi - vector + s vm Ġand ĠTD NN + stat - pool Ġusing ĠK aldi Ġtool kit . ĠWe Ġtrain Ġour ĠNet VL AD Ġand ĠGhost VL AD Ġexperiments Ġusing ĠKer as Ġby Ġmodifying Ġthe Ġcode Ġgiven Ġby Ġ[ 9 ] Ġfor Ġlanguage Ġidentification . ĠWe Ġtrain Ġthe ĠL ST M + stat - pool Ġand Ġthe Ġremaining Ġexperiments Ġusing ĠPy tor ch Ġ[ 14 ] Ġtool kit , Ġand Ġwe Ġwill Ġopens ource Ġall Ġthe Ġcodes Ġand Ġdata Ġsoon . </s> RESULTS </s> In Ġthis Ġsection , Ġwe Ġcompare Ġthe Ġperformance Ġof Ġour Ġsystem Ġwith Ġthe Ġrecent Ġstate Ġof Ġthe Ġart Ġlanguage Ġidentification Ġapproaches . ĠWe Ġalso Ġcompare Ġdifferent Ġpool ing Ġstrategies Ġand Ġfinally , Ġcompare Ġthe Ġrobust ness Ġof Ġour Ġsystem Ġto Ġthe Ġlength Ġof Ġthe Ġinput Ġspect rogram Ġduring Ġtraining . ĠWe Ġvisualize Ġthe Ġembed d ings Ġgenerated Ġby Ġthe ĠGhost VL AD Ġmethod Ġand Ġconclude Ġthat Ġthe ĠGhost VL AD Ġembed d ings Ġshows Ġvery Ġgood Ġfeature Ġdiscrimination Ġcapabilities . </s> RESULTS Ġ: :: ĠComparison Ġwith Ġdifferent Ġapproaches </s> We Ġcompare Ġour Ġsystem Ġperformance Ġwith Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġlanguage Ġidentification Ġapproaches , Ġas Ġshown Ġin ĠTable Ġ2 . ĠThe Ġi - vector + s vm Ġsystem Ġis Ġtrained Ġusing ĠGM M - UB M Ġmodels Ġto Ġgenerate Ġi - ve ctors Ġas Ġproposed Ġin Ġ[ 1 ]. ĠOnce Ġthe Ġi - ve ctors Ġare Ġextracted , Ġwe Ġfit ĠS VM Ġclass ifier Ġto Ġclassify Ġthe Ġaudio . ĠThe ĠTD NN + stat - pool Ġsystem Ġis Ġtrained Ġwith Ġa Ġstatistics Ġpool ing Ġlayer Ġand Ġis Ġcalled Ġthe Ġx - vector Ġsystem Ġas Ġproposed Ġby ĠDavid ĠSnyder Ġet Ġal . Ġ[ 11 ] Ġand Ġis Ġcurrently Ġthe Ġstate Ġof Ġthe Ġart Ġlanguage Ġidentification Ġapproach Ġas Ġfar Ġas Ġour Ġknowledge . ĠOur Ġmethods Ġoutper form Ġthe Ġstate Ġof Ġthe Ġart Ġx - vector Ġsystem Ġby Ġabsolute Ġ1 . 88 % Ġimprovement Ġin ĠF 1 - score , Ġas Ġshown Ġin ĠTable Ġ2 . </s> RESULTS Ġ: :: ĠComparison Ġwith Ġdifferent Ġpool ing Ġtechniques </s> We Ġcompare Ġour Ġapproach Ġwith Ġdifferent Ġpool ing Ġstrategies Ġin ĠTable Ġ3 . ĠWe Ġuse ĠRes Net Ġas Ġour Ġbase Ġfeature Ġextraction Ġnetwork . ĠWe Ġkeep Ġthe Ġbase Ġnetwork Ġthe Ġsame Ġand Ġchange Ġonly Ġthe Ġpool ing Ġlayers Ġto Ġsee Ġwhich Ġpool ing Ġapproach Ġperforms Ġbetter Ġfor Ġlanguage Ġidentification Ġtask . ĠOur Ġexperiments Ġshow Ġthat ĠGhost VL AD Ġpool ing Ġoutper forms Ġall Ġthe Ġother Ġpool ing Ġmethods Ġby Ġachieving Ġ98 . 43 % ĠF 1 - Score . </s> RESULTS Ġ: :: ĠDuration Ġanalysis </s> To Ġobserve Ġthe Ġperformance Ġof Ġour Ġmethod Ġwith Ġdifferent Ġinput Ġd urations , Ġwe Ġconducted Ġan Ġexperiment Ġwhere Ġwe Ġtrain Ġour Ġmodel Ġon Ġdifferent Ġinput Ġd urations . ĠSince Ġour Ġmodel Ġuses ĠRes Net Ġas Ġthe Ġbase Ġfeature Ġextract or , Ġwe Ġneed Ġto Ġfeed Ġfixed - length Ġspect rogram . ĠWe Ġconducted Ġ4 Ġdifferent Ġexperiments Ġwhere Ġwe Ġtrained Ġthe Ġmodel Ġusing Ġ2 sec , Ġ3 sec , Ġ4 sec Ġand Ġ5 sec Ġspect rog rams Ġcontaining Ġ200 , 300 , 400 Ġand Ġ500 Ġframes Ġrespectively . ĠWe Ġobserved Ġthat Ġthe Ġmodel Ġtrained Ġwith Ġa Ġ5 sec Ġspect rogram Ġis Ġthe Ġbest Ġmodel , Ġas Ġshown Ġin ĠTable Ġ4 . </s> RESULTS Ġ: :: ĠVisual ization Ġof Ġembed d ings </s> We Ġvisualize Ġthe Ġembed d ings Ġgenerated Ġby Ġour Ġapproach Ġto Ġsee Ġthe Ġeffectiveness . ĠWe Ġextracted Ġ512 - dimensional Ġembed d ings Ġfor Ġour Ġtesting Ġdata Ġand Ġreduced Ġthe Ġdimension ality Ġusing Ġt - s ne Ġprojection . ĠThe Ġt - s ne Ġplot Ġof Ġthe Ġembed d ings Ġspace Ġis Ġshown Ġin ĠFigure Ġ3 . ĠThe Ġplot Ġshows Ġthat Ġthe Ġembed d ings Ġlearned Ġby Ġour Ġapproach Ġhas Ġvery Ġgood Ġdiscrim in ative Ġproperties </s> Conclusion </s> In Ġthis Ġwork , Ġwe Ġuse ĠGhost - VL AD Ġpool ing Ġapproach Ġthat Ġwas Ġoriginally Ġproposed Ġfor Ġface Ġrecognition Ġto Ġimprove Ġlanguage Ġidentification Ġperformance Ġfor ĠIndian Ġlanguages . ĠWe Ġcollected Ġand Ġcurated Ġ630 Ġhrs Ġaudio Ġdata Ġfrom Ġnews ĠAll ĠIndia ĠRadio Ġnews Ġchannel Ġfor Ġ7 ĠIndian Ġlanguages . ĠOur Ġexperimental Ġresults Ġshows Ġthat Ġour Ġapproach Ġoutper forms Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġmethods Ġby Ġan Ġabsolute Ġ1 . 88 % ĠF 1 - score . ĠWe Ġhave Ġalso Ġconducted Ġexperiments Ġwith Ġdifferent Ġpool ing Ġstrategies Ġproposed Ġin Ġthe Ġpast , Ġand Ġthe ĠGhost VL AD Ġpool ing Ġapproach Ġturns Ġout Ġto Ġbe Ġthe Ġbest Ġapproach Ġfor Ġaggreg ating Ġframe - level Ġfeatures Ġinto Ġa Ġsingle Ġutter ance Ġlevel Ġfeature . ĠOur Ġexperiments Ġalso Ġprove Ġthat Ġour Ġapproach Ġworks Ġmuch Ġbetter Ġeven Ġif Ġthe Ġinput Ġduring Ġtraining Ġcontains Ġsmaller Ġd urations . ĠFinally , Ġwe Ġsee Ġthat Ġthe Ġembed d ings Ġgenerated Ġby Ġour Ġmethod Ġhas Ġvery Ġgood Ġlanguage Ġdiscrim in ative Ġfeatures Ġand Ġhelps Ġto Ġimprove Ġthe Ġperformance Ġof Ġlanguage Ġidentification .
INTRODUCTION The idea of language identification is to classify a given audio signal into a particular class using a classification algorithm. Commonly language identification task was done using i-vector systems [1]. A very well known approach for language identification proposed by N. Dahek et al. [1] uses the GMM-UBM model to obtain utterance level features called i-vectors. Recent advances in deep learning [15,16] have helped to improve the language identification task using many different neural network architectures which can be trained efficiently using GPUs for large scale datasets. These neural networks can be configured in various ways to obtain better accuracy for language identification task. Early work on using Deep learning for language Identification was published by Pavel Matejka et al. [2], where they used stacked bottleneck features extracted from deep neural networks for language identification task and showed that the bottleneck features learned by Deep neural networks are better than simple MFCC or PLP features. Later the work by I. Lopez-Moreno et al. [3] from Google showed how to use Deep neural networks to directly map the sequence of MFCC frames into its language class so that we can apply language identification at the frame level. Speech signals will have both spatial and temporal information, but simple DNNs are not able to capture temporal information. Work done by J. Gonzalez-Dominguez et al. [4] by Google developed an LSTM based language identification model which improves the accuracy over the DNN based models. Work done by Alicia et al. [5] used CNNs to improve upon i-vector [1] and other previously developed systems. The work done by Daniel Garcia-Romero et al. [6] has used a combination of Acoustic model trained for speech recognition with Time-delay neural networks where they train the TDNN model by feeding the stacked bottleneck features from acoustic model to predict the language labels at the frame level. Recently X-vectors [7] is proposed for speaker identification task and are shown to outperform all the previous state of the art speaker identification algorithms and are also used for language identification by David Snyder et al. [8]. In this paper, we explore multiple pooling strategies for language identification task. Mainly we propose Ghost-VLAD based pooling method for language identification. Inspired by the recent work by W. Xie et al. [9] and Y. Zhong et al. [10], we use Ghost-VLAD to improve the accuracy of language identification task for Indian languages. We explore multiple pooling strategies including NetVLAD pooling [11], Average pooling and Statistics pooling( as proposed in X-vectors [7]) and show that Ghost-VLAD pooling is the best pooling strategy for language identification. Our model obtains the best accuracy of 98.24%, and it outperforms all the other previously proposed pooling methods. We conduct all our experiments on 635hrs of audio data for 7 Indian languages collected from $\textbf {All India Radio}$ news channel. The paper is organized as follows. In section 2, we explain the proposed pooling method for language identification. In section 3, we explain our dataset. In section 4, we describe the experiments, and in section 5, we describe the results. POOLING STRATEGIES In any language identification model, we want to obtain utterance level representation which has very good language discriminative features. These representations should be compact and should be easily separable by a linear classifier. The idea of any pooling strategy is to pool the frame-level representations into a single utterance level representation. Previous works by [7] have used simple mean and standard deviation aggregation to pool the frame-level features from the top layer of the neural network to obtain the utterance level features. Recently [9] used VLAD based pooling strategy for speaker identification which is inspired from [10] proposed for face recognition. The NetVLAD [11] and Ghost-VLAD [10] methods are proposed for Place recognition and face recognition, respectively, and in both cases, they try to aggregate the local descriptors into global features. In our case, the local descriptors are features extracted from ResNet [15], and the global utterance level feature is obtained by using GhostVLAD pooling. In this section, we explain different pooling methods, including NetVLAD, Ghost-VLAD, Statistic pooling, and Average pooling. POOLING STRATEGIES ::: NetVLAD pooling The NetVLAD pooling strategy was initially developed for place recognition by R. Arandjelovic et al. [11]. The NetVLAD is an extension to VLAD [18] approach where they were able to replace the hard assignment based clustering with soft assignment based clustering so that it can be trained with neural network in an end to end fashion. In our case, we use the NetVLAD layer to map N local features of dimension D into a fixed dimensional vector, as shown in Figure 1 (Left side). The model takes spectrogram as an input and feeds into CNN based ResNet architecture. The ResNet is used to map the spectrogram into 3D feature map of dimension HxWxD. We convert this 3D feature map into 2D by unfolding H and W dimensions, creating a NxD dimensional feature map, where N=HxW. The NetVLAD layer is kept on top of the feature extraction layer of ResNet, as shown in Figure 1. The NetVLAD now takes N features vectors of dimension D and computes a matrix V of dimension KxD, where K is the number clusters in the NetVLAD layer, and D is the dimension of the feature vector. The matrix V is computed as follows. Where $w_k$,$b_k$ and $c_k$ are trainable parameters for the cluster $k$ and V(j,k) represents a point in the V matrix for (j,k)th location. The matrix is constructed using the equation (1) where the first term corresponds to the soft assignment of the input $x_i$ to the cluster $c_k$, whereas the second term corresponds to the residual term which tells how far the input descriptor $x_i$ is from the cluster center $c_k$. POOLING STRATEGIES ::: GhostVLAD pooling GhostVLAD is an extension of the NetVLAD approach, which we discussed in the previous section. The GhostVLAD model was proposed for face recognition by Y. Zhong [10]. GhostVLAD works exactly similar to NetVLAD except it adds Ghost clusters along with the NetVLAD clusters. So, now we will have a K+G number of clusters instead of K clusters. Where G is the number of ghost clusters, we want to add (typically 2-4). The Ghost clusters are added to map any noisy or irrelevant content into ghost clusters and are not included during the feature aggregation stage, as shown in Figure 1 (Right side). Which means that we compute the matrix V for both normal cluster K and ghost clusters G, but we will not include the vectors belongs to ghost cluster from V during concatenation of the features. Due to which, during feature aggregation stage the contribution of the noisy and unwanted features to normal VLAD clusters are assigned less weights while Ghost clusters absorb most of the weight. We illustrate this in Figure 1(Right Side), where the ghost clusters are shown in red color. We use Ghost clusters when we are computing the V matrix, but they are excluded during the concatenation stage. These concatenated features are fed into the projection layer, followed by softmax to predict the language label. POOLING STRATEGIES ::: Statistic and average pooling In statistic pooling, we compute the first and second order statistics of the local features from the top layer of the ResNet model. The 3-D feature map is unfolded to create N features of D dimensions, and then we compute the mean and standard deviation of all these N vectors and get two D dimensional vectors, one for mean and the other for standard deviation. We then concatenate these 2 features and feed it to the projection layer for predicting the language label. In the Average pooling layer, we compute only the first-order statistics (mean) of the local features from the top layer of the CNN model. The feature map from the top layer of CNN is unfolded to create N features of D dimensions, and then we compute the mean of all these N vectors and get D dimensional representation. We then feed this feature to the projection layer followed by softmax for predicting the language label. DATASET In this section, we describe our dataset collection process. We collected and curated around 635Hrs of audio data for 7 Indian languages, namely Kannada, Hindi, Telugu, Malayalam, Bengali, and English. We collected the data from the All India Radio news channel where an actor will be reading news for about 5-10 mins. To cover many speakers for the dataset, we crawled data from 2010 to 2019. Since the audio is very long to train any deep neural network directly, we segment the audio clips into smaller chunks using Voice activity detector. Since the audio clips will have music embedded during the news, we use Inhouse music detection model to remove the music segments from the dataset to make the dataset clean and our dataset contains 635Hrs of clean audio which is divided into 520Hrs of training data containing 165K utterances and 115Hrs of testing data containing 35K utterances. The amount of audio data for training and testing for each of the language is shown in the table bellow. EXPERIMENTS In this section, we describe the feature extraction process and network architecture in detail. We use spectral features of 256 dimensions computed using 512 point FFT for every frame, and we add an energy feature for every frame giving us total 257 features for every frame. We use a window size of 25ms and frame shift of 10ms during feature computation. We crop random 5sec audio data from each utterance during training which results in a spectrogram of size 257x500 (features x number of features). We use these spectrograms as input to our CNN model during training. During testing, we compute the prediction score irrespective of the audio length. For the network architecture, we use ResNet-34 architecture, as described in [9]. The model uses convolution layers with Relu activations to map the spectrogram of size 257x500 input into 3D feature map of size 1x32x512. This feature cube is converted into 2D feature map of dimension 32x512 and fed into Ghost-VLAD/NetVLAD layer to generate a representation that has more language discrimination capacity. We use Adam optimizer with an initial learning rate of 0.01 and a final learning rate of 0.00001 for training. Each model is trained for 15 epochs with early stopping criteria. For the baseline, we train an i-vector model using GMM-UBM. We fit a small classifier on top of the generated i-vectors to measure the accuracy. This model is referred as i-vector+svm . To compare our model with the previous state of the art system, we set up the x-vector language identification system [8]. The x-vector model used time-delay neural networks (TDNN) along with statistic-pooling. We use 7 layer TDNN architecture similar to [8] for training. We refer to this model as tdnn+stat-pool . Finally, we set up a Deep LSTM based language identification system similar to [4] but with little modification where we add statistics pooling for the last layers hidden activities before classification. We use 3 layer Bi-LSTM with 256 hidden units at each layer. We refer to this model as LSTM+stat-pool. We train our i-vector+svm and TDNN+stat-pool using Kaldi toolkit. We train our NetVLAD and GhostVLAD experiments using Keras by modifying the code given by [9] for language identification. We train the LSTM+stat-pool and the remaining experiments using Pytorch [14] toolkit, and we will opensource all the codes and data soon. RESULTS In this section, we compare the performance of our system with the recent state of the art language identification approaches. We also compare different pooling strategies and finally, compare the robustness of our system to the length of the input spectrogram during training. We visualize the embeddings generated by the GhostVLAD method and conclude that the GhostVLAD embeddings shows very good feature discrimination capabilities. RESULTS ::: Comparison with different approaches We compare our system performance with the previous state of the art language identification approaches, as shown in Table 2. The i-vector+svm system is trained using GMM-UBM models to generate i-vectors as proposed in [1]. Once the i-vectors are extracted, we fit SVM classifier to classify the audio. The TDNN+stat-pool system is trained with a statistics pooling layer and is called the x-vector system as proposed by David Snyder et al. [11] and is currently the state of the art language identification approach as far as our knowledge. Our methods outperform the state of the art x-vector system by absolute 1.88% improvement in F1-score, as shown in Table 2. RESULTS ::: Comparison with different pooling techniques We compare our approach with different pooling strategies in Table 3. We use ResNet as our base feature extraction network. We keep the base network the same and change only the pooling layers to see which pooling approach performs better for language identification task. Our experiments show that GhostVLAD pooling outperforms all the other pooling methods by achieving 98.43% F1-Score. RESULTS ::: Duration analysis To observe the performance of our method with different input durations, we conducted an experiment where we train our model on different input durations. Since our model uses ResNet as the base feature extractor, we need to feed fixed-length spectrogram. We conducted 4 different experiments where we trained the model using 2sec, 3sec, 4sec and 5sec spectrograms containing 200,300,400 and 500 frames respectively. We observed that the model trained with a 5sec spectrogram is the best model, as shown in Table 4. RESULTS ::: Visualization of embeddings We visualize the embeddings generated by our approach to see the effectiveness. We extracted 512-dimensional embeddings for our testing data and reduced the dimensionality using t-sne projection. The t-sne plot of the embeddings space is shown in Figure 3. The plot shows that the embeddings learned by our approach has very good discriminative properties Conclusion In this work, we use Ghost-VLAD pooling approach that was originally proposed for face recognition to improve language identification performance for Indian languages. We collected and curated 630 hrs audio data from news All India Radio news channel for 7 Indian languages. Our experimental results shows that our approach outperforms the previous state of the art methods by an absolute 1.88% F1-score. We have also conducted experiments with different pooling strategies proposed in the past, and the GhostVLAD pooling approach turns out to be the best approach for aggregating frame-level features into a single utterance level feature. Our experiments also prove that our approach works much better even if the input during training contains smaller durations. Finally, we see that the embeddings generated by our method has very good language discriminative features and helps to improve the performance of language identification.
Which 7 Indian languages do they experiment with?
75df70ce7aa714ec4c6456d0c51f82a16227f2cb
Which Ġ7 ĠIndian Ġlanguages Ġdo Ġthey Ġexperiment Ġwith ?
H ind i , ĠEnglish , ĠK ann ada , ĠTel ugu , ĠAss ames e , ĠBeng ali Ġand ĠMal ay alam
<s> Which Ġ7 ĠIndian Ġlanguages Ġdo Ġthey Ġexperiment Ġwith ? </s> IN TR ODUCT ION </s> The Ġidea Ġof Ġlanguage Ġidentification Ġis Ġto Ġclassify Ġa Ġgiven Ġaudio Ġsignal Ġinto Ġa Ġparticular Ġclass Ġusing Ġa Ġclassification Ġalgorithm . ĠCommon ly Ġlanguage Ġidentification Ġtask Ġwas Ġdone Ġusing Ġi - vector Ġsystems Ġ[ 1 ]. ĠA Ġvery Ġwell Ġknown Ġapproach Ġfor Ġlanguage Ġidentification Ġproposed Ġby ĠN . ĠDa he k Ġet Ġal . Ġ[ 1 ] Ġuses Ġthe ĠGM M - UB M Ġmodel Ġto Ġobtain Ġutter ance Ġlevel Ġfeatures Ġcalled Ġi - ve ctors . ĠRecent Ġadvances Ġin Ġdeep Ġlearning Ġ[ 15 , 16 ] Ġhave Ġhelped Ġto Ġimprove Ġthe Ġlanguage Ġidentification Ġtask Ġusing Ġmany Ġdifferent Ġneural Ġnetwork Ġarchitectures Ġwhich Ġcan Ġbe Ġtrained Ġefficiently Ġusing ĠGPUs Ġfor Ġlarge Ġscale Ġdatasets . ĠThese Ġneural Ġnetworks Ġcan Ġbe Ġconfigured Ġin Ġvarious Ġways Ġto Ġobtain Ġbetter Ġaccuracy Ġfor Ġlanguage Ġidentification Ġtask . ĠEarly Ġwork Ġon Ġusing ĠDeep Ġlearning Ġfor Ġlanguage ĠIdentification Ġwas Ġpublished Ġby ĠPavel ĠMate j ka Ġet Ġal . Ġ[ 2 ], Ġwhere Ġthey Ġused Ġstacked Ġbottleneck Ġfeatures Ġextracted Ġfrom Ġdeep Ġneural Ġnetworks Ġfor Ġlanguage Ġidentification Ġtask Ġand Ġshowed Ġthat Ġthe Ġbottleneck Ġfeatures Ġlearned Ġby ĠDeep Ġneural Ġnetworks Ġare Ġbetter Ġthan Ġsimple ĠMF CC Ġor ĠPL P Ġfeatures . ĠLater Ġthe Ġwork Ġby ĠI . ĠLopez - More no Ġet Ġal . Ġ[ 3 ] Ġfrom ĠGoogle Ġshowed Ġhow Ġto Ġuse ĠDeep Ġneural Ġnetworks Ġto Ġdirectly Ġmap Ġthe Ġsequence Ġof ĠMF CC Ġframes Ġinto Ġits Ġlanguage Ġclass Ġso Ġthat Ġwe Ġcan Ġapply Ġlanguage Ġidentification Ġat Ġthe Ġframe Ġlevel . ĠSpeech Ġsignals Ġwill Ġhave Ġboth Ġspatial Ġand Ġtemporal Ġinformation , Ġbut Ġsimple ĠD NN s Ġare Ġnot Ġable Ġto Ġcapture Ġtemporal Ġinformation . ĠWork Ġdone Ġby ĠJ . ĠGonzalez - D oming uez Ġet Ġal . Ġ[ 4 ] Ġby ĠGoogle Ġdeveloped Ġan ĠL ST M Ġbased Ġlanguage Ġidentification Ġmodel Ġwhich Ġimproves Ġthe Ġaccuracy Ġover Ġthe ĠD NN Ġbased Ġmodels . ĠWork Ġdone Ġby ĠAlicia Ġet Ġal . Ġ[ 5 ] Ġused ĠCNN s Ġto Ġimprove Ġupon Ġi - vector Ġ[ 1 ] Ġand Ġother Ġpreviously Ġdeveloped Ġsystems . ĠThe Ġwork Ġdone Ġby ĠDaniel ĠGarcia - Rom ero Ġet Ġal . Ġ[ 6 ] Ġhas Ġused Ġa Ġcombination Ġof ĠAc oustic Ġmodel Ġtrained Ġfor Ġspeech Ġrecognition Ġwith ĠTime - delay Ġneural Ġnetworks Ġwhere Ġthey Ġtrain Ġthe ĠTD NN Ġmodel Ġby Ġfeeding Ġthe Ġstacked Ġbottleneck Ġfeatures Ġfrom Ġacoustic Ġmodel Ġto Ġpredict Ġthe Ġlanguage Ġlabels Ġat Ġthe Ġframe Ġlevel . ĠRecently ĠX - ve ctors Ġ[ 7 ] Ġis Ġproposed Ġfor Ġspeaker Ġidentification Ġtask Ġand Ġare Ġshown Ġto Ġoutper form Ġall Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġspeaker Ġidentification Ġalgorithms Ġand Ġare Ġalso Ġused Ġfor Ġlanguage Ġidentification Ġby ĠDavid ĠSnyder Ġet Ġal . Ġ[ 8 ]. </s> In Ġthis Ġpaper , Ġwe Ġexplore Ġmultiple Ġpool ing Ġstrategies Ġfor Ġlanguage Ġidentification Ġtask . ĠMain ly Ġwe Ġpropose ĠGhost - VL AD Ġbased Ġpool ing Ġmethod Ġfor Ġlanguage Ġidentification . ĠInspired Ġby Ġthe Ġrecent Ġwork Ġby ĠW . ĠXie Ġet Ġal . Ġ[ 9 ] Ġand ĠY . ĠZh ong Ġet Ġal . Ġ[ 10 ], Ġwe Ġuse ĠGhost - VL AD Ġto Ġimprove Ġthe Ġaccuracy Ġof Ġlanguage Ġidentification Ġtask Ġfor ĠIndian Ġlanguages . ĠWe Ġexplore Ġmultiple Ġpool ing Ġstrategies Ġincluding ĠNet VL AD Ġpool ing Ġ[ 11 ], ĠAverage Ġpool ing Ġand ĠStatistics Ġpool ing ( Ġas Ġproposed Ġin ĠX - ve ctors Ġ[ 7 ]) Ġand Ġshow Ġthat ĠGhost - VL AD Ġpool ing Ġis Ġthe Ġbest Ġpool ing Ġstrategy Ġfor Ġlanguage Ġidentification . ĠOur Ġmodel Ġob tains Ġthe Ġbest Ġaccuracy Ġof Ġ98 . 24 %, Ġand Ġit Ġoutper forms Ġall Ġthe Ġother Ġpreviously Ġproposed Ġpool ing Ġmethods . ĠWe Ġconduct Ġall Ġour Ġexperiments Ġon Ġ6 35 h rs Ġof Ġaudio Ġdata Ġfor Ġ7 ĠIndian Ġlanguages Ġcollected Ġfrom Ġ$\ text bf Ġ{ All ĠIndia ĠRadio } $ Ġnews Ġchannel . ĠThe Ġpaper Ġis Ġorganized Ġas Ġfollows . ĠIn Ġsection Ġ2 , Ġwe Ġexplain Ġthe Ġproposed Ġpool ing Ġmethod Ġfor Ġlanguage Ġidentification . ĠIn Ġsection Ġ3 , Ġwe Ġexplain Ġour Ġdataset . ĠIn Ġsection Ġ4 , Ġwe Ġdescribe Ġthe Ġexperiments , Ġand Ġin Ġsection Ġ5 , Ġwe Ġdescribe Ġthe Ġresults . </s> PO OL ING ĠSTR ATE G IES </s> In Ġany Ġlanguage Ġidentification Ġmodel , Ġwe Ġwant Ġto Ġobtain Ġutter ance Ġlevel Ġrepresentation Ġwhich Ġhas Ġvery Ġgood Ġlanguage Ġdiscrim in ative Ġfeatures . ĠThese Ġrepresentations Ġshould Ġbe Ġcompact Ġand Ġshould Ġbe Ġeasily Ġsepar able Ġby Ġa Ġlinear Ġclass ifier . ĠThe Ġidea Ġof Ġany Ġpool ing Ġstrategy Ġis Ġto Ġpool Ġthe Ġframe - level Ġrepresentations Ġinto Ġa Ġsingle Ġutter ance Ġlevel Ġrepresentation . ĠPrevious Ġworks Ġby Ġ[ 7 ] Ġhave Ġused Ġsimple Ġmean Ġand Ġstandard Ġdeviation Ġaggregation Ġto Ġpool Ġthe Ġframe - level Ġfeatures Ġfrom Ġthe Ġtop Ġlayer Ġof Ġthe Ġneural Ġnetwork Ġto Ġobtain Ġthe Ġutter ance Ġlevel Ġfeatures . ĠRecently Ġ[ 9 ] Ġused ĠV L AD Ġbased Ġpool ing Ġstrategy Ġfor Ġspeaker Ġidentification Ġwhich Ġis Ġinspired Ġfrom Ġ[ 10 ] Ġproposed Ġfor Ġface Ġrecognition . ĠThe ĠNet VL AD Ġ[ 11 ] Ġand ĠGhost - VL AD Ġ[ 10 ] Ġmethods Ġare Ġproposed Ġfor ĠPlace Ġrecognition Ġand Ġface Ġrecognition , Ġrespectively , Ġand Ġin Ġboth Ġcases , Ġthey Ġtry Ġto Ġaggregate Ġthe Ġlocal Ġdescript ors Ġinto Ġglobal Ġfeatures . ĠIn Ġour Ġcase , Ġthe Ġlocal Ġdescript ors Ġare Ġfeatures Ġextracted Ġfrom ĠRes Net Ġ[ 15 ], Ġand Ġthe Ġglobal Ġutter ance Ġlevel Ġfeature Ġis Ġobtained Ġby Ġusing ĠGhost VL AD Ġpool ing . ĠIn Ġthis Ġsection , Ġwe Ġexplain Ġdifferent Ġpool ing Ġmethods , Ġincluding ĠNet VL AD , ĠGhost - VL AD , ĠStat istic Ġpool ing , Ġand ĠAverage Ġpool ing . </s> PO OL ING ĠSTR ATE G IES Ġ: :: ĠNet VL AD Ġpool ing </s> The ĠNet VL AD Ġpool ing Ġstrategy Ġwas Ġinitially Ġdeveloped Ġfor Ġplace Ġrecognition Ġby ĠR . ĠAr and j el ovic Ġet Ġal . Ġ[ 11 ]. ĠThe ĠNet VL AD Ġis Ġan Ġextension Ġto ĠV L AD Ġ[ 18 ] Ġapproach Ġwhere Ġthey Ġwere Ġable Ġto Ġreplace Ġthe Ġhard Ġassignment Ġbased Ġclust ering Ġwith Ġsoft Ġassignment Ġbased Ġclust ering Ġso Ġthat Ġit Ġcan Ġbe Ġtrained Ġwith Ġneural Ġnetwork Ġin Ġan Ġend Ġto Ġend Ġfashion . ĠIn Ġour Ġcase , Ġwe Ġuse Ġthe ĠNet VL AD Ġlayer Ġto Ġmap ĠN Ġlocal Ġfeatures Ġof Ġdimension ĠD Ġinto Ġa Ġfixed Ġdimensional Ġvector , Ġas Ġshown Ġin ĠFigure Ġ1 Ġ( Left Ġside ). </s> The Ġmodel Ġtakes Ġspect rogram Ġas Ġan Ġinput Ġand Ġfeeds Ġinto ĠCNN Ġbased ĠRes Net Ġarchitecture . ĠThe ĠRes Net Ġis Ġused Ġto Ġmap Ġthe Ġspect rogram Ġinto Ġ3 D Ġfeature Ġmap Ġof Ġdimension ĠH x W x D . ĠWe Ġconvert Ġthis Ġ3 D Ġfeature Ġmap Ġinto Ġ2 D Ġby Ġunfolding ĠH Ġand ĠW Ġdimensions , Ġcreating Ġa ĠN x D Ġdimensional Ġfeature Ġmap , Ġwhere ĠN = H x W . ĠThe ĠNet VL AD Ġlayer Ġis Ġkept Ġon Ġtop Ġof Ġthe Ġfeature Ġextraction Ġlayer Ġof ĠRes Net , Ġas Ġshown Ġin ĠFigure Ġ1 . ĠThe ĠNet VL AD Ġnow Ġtakes ĠN Ġfeatures Ġvectors Ġof Ġdimension ĠD Ġand Ġcomp utes Ġa Ġmatrix ĠV Ġof Ġdimension ĠK x D , Ġwhere ĠK Ġis Ġthe Ġnumber Ġclusters Ġin Ġthe ĠNet VL AD Ġlayer , Ġand ĠD Ġis Ġthe Ġdimension Ġof Ġthe Ġfeature Ġvector . ĠThe Ġmatrix ĠV Ġis Ġcomputed Ġas Ġfollows . </s> Where Ġ$ w _ k $, $ b _ k $ Ġand Ġ$ c _ k $ Ġare Ġtrain able Ġparameters Ġfor Ġthe Ġcluster Ġ$ k $ Ġand ĠV ( j , k ) Ġrepresents Ġa Ġpoint Ġin Ġthe ĠV Ġmatrix Ġfor Ġ( j , k ) th Ġlocation . ĠThe Ġmatrix Ġis Ġconstructed Ġusing Ġthe Ġequation Ġ( 1 ) Ġwhere Ġthe Ġfirst Ġterm Ġcorresponds Ġto Ġthe Ġsoft Ġassignment Ġof Ġthe Ġinput Ġ$ x _ i $ Ġto Ġthe Ġcluster Ġ$ c _ k $, Ġwhereas Ġthe Ġsecond Ġterm Ġcorresponds Ġto Ġthe Ġresidual Ġterm Ġwhich Ġtells Ġhow Ġfar Ġthe Ġinput Ġdescriptor Ġ$ x _ i $ Ġis Ġfrom Ġthe Ġcluster Ġcenter Ġ$ c _ k $. </s> PO OL ING ĠSTR ATE G IES Ġ: :: ĠGhost VL AD Ġpool ing </s> Ghost VL AD Ġis Ġan Ġextension Ġof Ġthe ĠNet VL AD Ġapproach , Ġwhich Ġwe Ġdiscussed Ġin Ġthe Ġprevious Ġsection . ĠThe ĠGhost VL AD Ġmodel Ġwas Ġproposed Ġfor Ġface Ġrecognition Ġby ĠY . ĠZh ong Ġ[ 10 ]. ĠGhost VL AD Ġworks Ġexactly Ġsimilar Ġto ĠNet VL AD Ġexcept Ġit Ġadds ĠGhost Ġclusters Ġalong Ġwith Ġthe ĠNet VL AD Ġclusters . ĠSo , Ġnow Ġwe Ġwill Ġhave Ġa ĠK + G Ġnumber Ġof Ġclusters Ġinstead Ġof ĠK Ġclusters . ĠWhere ĠG Ġis Ġthe Ġnumber Ġof Ġghost Ġclusters , Ġwe Ġwant Ġto Ġadd Ġ( typically Ġ2 - 4 ). ĠThe ĠGhost Ġclusters Ġare Ġadded Ġto Ġmap Ġany Ġnoisy Ġor Ġirrelevant Ġcontent Ġinto Ġghost Ġclusters Ġand Ġare Ġnot Ġincluded Ġduring Ġthe Ġfeature Ġaggregation Ġstage , Ġas Ġshown Ġin ĠFigure Ġ1 Ġ( Right Ġside ). ĠWhich Ġmeans Ġthat Ġwe Ġcompute Ġthe Ġmatrix ĠV Ġfor Ġboth Ġnormal Ġcluster ĠK Ġand Ġghost Ġclusters ĠG , Ġbut Ġwe Ġwill Ġnot Ġinclude Ġthe Ġvectors Ġbelongs Ġto Ġghost Ġcluster Ġfrom ĠV Ġduring Ġconc aten ation Ġof Ġthe Ġfeatures . ĠDue Ġto Ġwhich , Ġduring Ġfeature Ġaggregation Ġstage Ġthe Ġcontribution Ġof Ġthe Ġnoisy Ġand Ġunwanted Ġfeatures Ġto Ġnormal ĠV L AD Ġclusters Ġare Ġassigned Ġless Ġweights Ġwhile ĠGhost Ġclusters Ġabsorb Ġmost Ġof Ġthe Ġweight . ĠWe Ġillustrate Ġthis Ġin ĠFigure Ġ1 ( Right ĠSide ), Ġwhere Ġthe Ġghost Ġclusters Ġare Ġshown Ġin Ġred Ġcolor . ĠWe Ġuse ĠGhost Ġclusters Ġwhen Ġwe Ġare Ġcomputing Ġthe ĠV Ġmatrix , Ġbut Ġthey Ġare Ġexcluded Ġduring Ġthe Ġconc aten ation Ġstage . ĠThese Ġconc aten ated Ġfeatures Ġare Ġfed Ġinto Ġthe Ġprojection Ġlayer , Ġfollowed Ġby Ġsoft max Ġto Ġpredict Ġthe Ġlanguage Ġlabel . </s> PO OL ING ĠSTR ATE G IES Ġ: :: ĠStat istic Ġand Ġaverage Ġpool ing </s> In Ġstatistic Ġpool ing , Ġwe Ġcompute Ġthe Ġfirst Ġand Ġsecond Ġorder Ġstatistics Ġof Ġthe Ġlocal Ġfeatures Ġfrom Ġthe Ġtop Ġlayer Ġof Ġthe ĠRes Net Ġmodel . ĠThe Ġ3 - D Ġfeature Ġmap Ġis Ġunfolded Ġto Ġcreate ĠN Ġfeatures Ġof ĠD Ġdimensions , Ġand Ġthen Ġwe Ġcompute Ġthe Ġmean Ġand Ġstandard Ġdeviation Ġof Ġall Ġthese ĠN Ġvectors Ġand Ġget Ġtwo ĠD Ġdimensional Ġvectors , Ġone Ġfor Ġmean Ġand Ġthe Ġother Ġfor Ġstandard Ġdeviation . ĠWe Ġthen Ġconc aten ate Ġthese Ġ2 Ġfeatures Ġand Ġfeed Ġit Ġto Ġthe Ġprojection Ġlayer Ġfor Ġpredicting Ġthe Ġlanguage Ġlabel . </s> In Ġthe ĠAverage Ġpool ing Ġlayer , Ġwe Ġcompute Ġonly Ġthe Ġfirst - order Ġstatistics Ġ( mean ) Ġof Ġthe Ġlocal Ġfeatures Ġfrom Ġthe Ġtop Ġlayer Ġof Ġthe ĠCNN Ġmodel . ĠThe Ġfeature Ġmap Ġfrom Ġthe Ġtop Ġlayer Ġof ĠCNN Ġis Ġunfolded Ġto Ġcreate ĠN Ġfeatures Ġof ĠD Ġdimensions , Ġand Ġthen Ġwe Ġcompute Ġthe Ġmean Ġof Ġall Ġthese ĠN Ġvectors Ġand Ġget ĠD Ġdimensional Ġrepresentation . ĠWe Ġthen Ġfeed Ġthis Ġfeature Ġto Ġthe Ġprojection Ġlayer Ġfollowed Ġby Ġsoft max Ġfor Ġpredicting Ġthe Ġlanguage Ġlabel . </s> D AT AS ET </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġour Ġdataset Ġcollection Ġprocess . ĠWe Ġcollected Ġand Ġcurated Ġaround Ġ6 35 H rs Ġof Ġaudio Ġdata Ġfor Ġ7 ĠIndian Ġlanguages , Ġnamely ĠK ann ada , ĠHindi , ĠTel ugu , ĠMal ay alam , ĠBeng ali , Ġand ĠEnglish . ĠWe Ġcollected Ġthe Ġdata Ġfrom Ġthe ĠAll ĠIndia ĠRadio Ġnews Ġchannel Ġwhere Ġan Ġactor Ġwill Ġbe Ġreading Ġnews Ġfor Ġabout Ġ5 - 10 Ġmins . ĠTo Ġcover Ġmany Ġspeakers Ġfor Ġthe Ġdataset , Ġwe Ġcrawled Ġdata Ġfrom Ġ2010 Ġto Ġ2019 . ĠSince Ġthe Ġaudio Ġis Ġvery Ġlong Ġto Ġtrain Ġany Ġdeep Ġneural Ġnetwork Ġdirectly , Ġwe Ġsegment Ġthe Ġaudio Ġclips Ġinto Ġsmaller Ġchunks Ġusing ĠVoice Ġactivity Ġdetector . ĠSince Ġthe Ġaudio Ġclips Ġwill Ġhave Ġmusic Ġembedded Ġduring Ġthe Ġnews , Ġwe Ġuse ĠIn house Ġmusic Ġdetection Ġmodel Ġto Ġremove Ġthe Ġmusic Ġsegments Ġfrom Ġthe Ġdataset Ġto Ġmake Ġthe Ġdataset Ġclean Ġand Ġour Ġdataset Ġcontains Ġ6 35 H rs Ġof Ġclean Ġaudio Ġwhich Ġis Ġdivided Ġinto Ġ520 H rs Ġof Ġtraining Ġdata Ġcontaining Ġ165 K Ġutter ances Ġand Ġ115 H rs Ġof Ġtesting Ġdata Ġcontaining Ġ35 K Ġutter ances . ĠThe Ġamount Ġof Ġaudio Ġdata Ġfor Ġtraining Ġand Ġtesting Ġfor Ġeach Ġof Ġthe Ġlanguage Ġis Ġshown Ġin Ġthe Ġtable Ġbell ow . </s> EX PER IM ENTS </s> In Ġthis Ġsection , Ġwe Ġdescribe Ġthe Ġfeature Ġextraction Ġprocess Ġand Ġnetwork Ġarchitecture Ġin Ġdetail . ĠWe Ġuse Ġspectral Ġfeatures Ġof Ġ256 Ġdimensions Ġcomputed Ġusing Ġ512 Ġpoint ĠF FT Ġfor Ġevery Ġframe , Ġand Ġwe Ġadd Ġan Ġenergy Ġfeature Ġfor Ġevery Ġframe Ġgiving Ġus Ġtotal Ġ257 Ġfeatures Ġfor Ġevery Ġframe . ĠWe Ġuse Ġa Ġwindow Ġsize Ġof Ġ25 ms Ġand Ġframe Ġshift Ġof Ġ10 ms Ġduring Ġfeature Ġcomputation . ĠWe Ġcrop Ġrandom Ġ5 sec Ġaudio Ġdata Ġfrom Ġeach Ġutter ance Ġduring Ġtraining Ġwhich Ġresults Ġin Ġa Ġspect rogram Ġof Ġsize Ġ257 x 500 Ġ( features Ġx Ġnumber Ġof Ġfeatures ). ĠWe Ġuse Ġthese Ġspect rog rams Ġas Ġinput Ġto Ġour ĠCNN Ġmodel Ġduring Ġtraining . ĠDuring Ġtesting , Ġwe Ġcompute Ġthe Ġprediction Ġscore Ġirrespective Ġof Ġthe Ġaudio Ġlength . </s> For Ġthe Ġnetwork Ġarchitecture , Ġwe Ġuse ĠRes Net - 34 Ġarchitecture , Ġas Ġdescribed Ġin Ġ[ 9 ]. ĠThe Ġmodel Ġuses Ġconv olution Ġlayers Ġwith ĠRel u Ġactiv ations Ġto Ġmap Ġthe Ġspect rogram Ġof Ġsize Ġ257 x 500 Ġinput Ġinto Ġ3 D Ġfeature Ġmap Ġof Ġsize Ġ1 x 32 x 512 . ĠThis Ġfeature Ġcube Ġis Ġconverted Ġinto Ġ2 D Ġfeature Ġmap Ġof Ġdimension Ġ32 x 512 Ġand Ġfed Ġinto ĠGhost - VL AD / Net VL AD Ġlayer Ġto Ġgenerate Ġa Ġrepresentation Ġthat Ġhas Ġmore Ġlanguage Ġdiscrimination Ġcapacity . ĠWe Ġuse ĠAdam Ġoptim izer Ġwith Ġan Ġinitial Ġlearning Ġrate Ġof Ġ0 . 01 Ġand Ġa Ġfinal Ġlearning Ġrate Ġof Ġ0 . 0000 1 Ġfor Ġtraining . ĠEach Ġmodel Ġis Ġtrained Ġfor Ġ15 Ġepoch s Ġwith Ġearly Ġstopping Ġcriteria . </s> For Ġthe Ġbaseline , Ġwe Ġtrain Ġan Ġi - vector Ġmodel Ġusing ĠGM M - UB M . ĠWe Ġfit Ġa Ġsmall Ġclass ifier Ġon Ġtop Ġof Ġthe Ġgenerated Ġi - ve ctors Ġto Ġmeasure Ġthe Ġaccuracy . ĠThis Ġmodel Ġis Ġreferred Ġas Ġi - vector + s vm Ġ. ĠTo Ġcompare Ġour Ġmodel Ġwith Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġsystem , Ġwe Ġset Ġup Ġthe Ġx - vector Ġlanguage Ġidentification Ġsystem Ġ[ 8 ]. ĠThe Ġx - vector Ġmodel Ġused Ġtime - delay Ġneural Ġnetworks Ġ( TD NN ) Ġalong Ġwith Ġstatistic - pool ing . ĠWe Ġuse Ġ7 Ġlayer ĠTD NN Ġarchitecture Ġsimilar Ġto Ġ[ 8 ] Ġfor Ġtraining . ĠWe Ġrefer Ġto Ġthis Ġmodel Ġas Ġtd nn + stat - pool Ġ. ĠFinally , Ġwe Ġset Ġup Ġa ĠDeep ĠL ST M Ġbased Ġlanguage Ġidentification Ġsystem Ġsimilar Ġto Ġ[ 4 ] Ġbut Ġwith Ġlittle Ġmodification Ġwhere Ġwe Ġadd Ġstatistics Ġpool ing Ġfor Ġthe Ġlast Ġlayers Ġhidden Ġactivities Ġbefore Ġclassification . ĠWe Ġuse Ġ3 Ġlayer ĠBi - L ST M Ġwith Ġ256 Ġhidden Ġunits Ġat Ġeach Ġlayer . ĠWe Ġrefer Ġto Ġthis Ġmodel Ġas ĠL ST M + stat - pool . ĠWe Ġtrain Ġour Ġi - vector + s vm Ġand ĠTD NN + stat - pool Ġusing ĠK aldi Ġtool kit . ĠWe Ġtrain Ġour ĠNet VL AD Ġand ĠGhost VL AD Ġexperiments Ġusing ĠKer as Ġby Ġmodifying Ġthe Ġcode Ġgiven Ġby Ġ[ 9 ] Ġfor Ġlanguage Ġidentification . ĠWe Ġtrain Ġthe ĠL ST M + stat - pool Ġand Ġthe Ġremaining Ġexperiments Ġusing ĠPy tor ch Ġ[ 14 ] Ġtool kit , Ġand Ġwe Ġwill Ġopens ource Ġall Ġthe Ġcodes Ġand Ġdata Ġsoon . </s> RESULTS </s> In Ġthis Ġsection , Ġwe Ġcompare Ġthe Ġperformance Ġof Ġour Ġsystem Ġwith Ġthe Ġrecent Ġstate Ġof Ġthe Ġart Ġlanguage Ġidentification Ġapproaches . ĠWe Ġalso Ġcompare Ġdifferent Ġpool ing Ġstrategies Ġand Ġfinally , Ġcompare Ġthe Ġrobust ness Ġof Ġour Ġsystem Ġto Ġthe Ġlength Ġof Ġthe Ġinput Ġspect rogram Ġduring Ġtraining . ĠWe Ġvisualize Ġthe Ġembed d ings Ġgenerated Ġby Ġthe ĠGhost VL AD Ġmethod Ġand Ġconclude Ġthat Ġthe ĠGhost VL AD Ġembed d ings Ġshows Ġvery Ġgood Ġfeature Ġdiscrimination Ġcapabilities . </s> RESULTS Ġ: :: ĠComparison Ġwith Ġdifferent Ġapproaches </s> We Ġcompare Ġour Ġsystem Ġperformance Ġwith Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġlanguage Ġidentification Ġapproaches , Ġas Ġshown Ġin ĠTable Ġ2 . ĠThe Ġi - vector + s vm Ġsystem Ġis Ġtrained Ġusing ĠGM M - UB M Ġmodels Ġto Ġgenerate Ġi - ve ctors Ġas Ġproposed Ġin Ġ[ 1 ]. ĠOnce Ġthe Ġi - ve ctors Ġare Ġextracted , Ġwe Ġfit ĠS VM Ġclass ifier Ġto Ġclassify Ġthe Ġaudio . ĠThe ĠTD NN + stat - pool Ġsystem Ġis Ġtrained Ġwith Ġa Ġstatistics Ġpool ing Ġlayer Ġand Ġis Ġcalled Ġthe Ġx - vector Ġsystem Ġas Ġproposed Ġby ĠDavid ĠSnyder Ġet Ġal . Ġ[ 11 ] Ġand Ġis Ġcurrently Ġthe Ġstate Ġof Ġthe Ġart Ġlanguage Ġidentification Ġapproach Ġas Ġfar Ġas Ġour Ġknowledge . ĠOur Ġmethods Ġoutper form Ġthe Ġstate Ġof Ġthe Ġart Ġx - vector Ġsystem Ġby Ġabsolute Ġ1 . 88 % Ġimprovement Ġin ĠF 1 - score , Ġas Ġshown Ġin ĠTable Ġ2 . </s> RESULTS Ġ: :: ĠComparison Ġwith Ġdifferent Ġpool ing Ġtechniques </s> We Ġcompare Ġour Ġapproach Ġwith Ġdifferent Ġpool ing Ġstrategies Ġin ĠTable Ġ3 . ĠWe Ġuse ĠRes Net Ġas Ġour Ġbase Ġfeature Ġextraction Ġnetwork . ĠWe Ġkeep Ġthe Ġbase Ġnetwork Ġthe Ġsame Ġand Ġchange Ġonly Ġthe Ġpool ing Ġlayers Ġto Ġsee Ġwhich Ġpool ing Ġapproach Ġperforms Ġbetter Ġfor Ġlanguage Ġidentification Ġtask . ĠOur Ġexperiments Ġshow Ġthat ĠGhost VL AD Ġpool ing Ġoutper forms Ġall Ġthe Ġother Ġpool ing Ġmethods Ġby Ġachieving Ġ98 . 43 % ĠF 1 - Score . </s> RESULTS Ġ: :: ĠDuration Ġanalysis </s> To Ġobserve Ġthe Ġperformance Ġof Ġour Ġmethod Ġwith Ġdifferent Ġinput Ġd urations , Ġwe Ġconducted Ġan Ġexperiment Ġwhere Ġwe Ġtrain Ġour Ġmodel Ġon Ġdifferent Ġinput Ġd urations . ĠSince Ġour Ġmodel Ġuses ĠRes Net Ġas Ġthe Ġbase Ġfeature Ġextract or , Ġwe Ġneed Ġto Ġfeed Ġfixed - length Ġspect rogram . ĠWe Ġconducted Ġ4 Ġdifferent Ġexperiments Ġwhere Ġwe Ġtrained Ġthe Ġmodel Ġusing Ġ2 sec , Ġ3 sec , Ġ4 sec Ġand Ġ5 sec Ġspect rog rams Ġcontaining Ġ200 , 300 , 400 Ġand Ġ500 Ġframes Ġrespectively . ĠWe Ġobserved Ġthat Ġthe Ġmodel Ġtrained Ġwith Ġa Ġ5 sec Ġspect rogram Ġis Ġthe Ġbest Ġmodel , Ġas Ġshown Ġin ĠTable Ġ4 . </s> RESULTS Ġ: :: ĠVisual ization Ġof Ġembed d ings </s> We Ġvisualize Ġthe Ġembed d ings Ġgenerated Ġby Ġour Ġapproach Ġto Ġsee Ġthe Ġeffectiveness . ĠWe Ġextracted Ġ512 - dimensional Ġembed d ings Ġfor Ġour Ġtesting Ġdata Ġand Ġreduced Ġthe Ġdimension ality Ġusing Ġt - s ne Ġprojection . ĠThe Ġt - s ne Ġplot Ġof Ġthe Ġembed d ings Ġspace Ġis Ġshown Ġin ĠFigure Ġ3 . ĠThe Ġplot Ġshows Ġthat Ġthe Ġembed d ings Ġlearned Ġby Ġour Ġapproach Ġhas Ġvery Ġgood Ġdiscrim in ative Ġproperties </s> Conclusion </s> In Ġthis Ġwork , Ġwe Ġuse ĠGhost - VL AD Ġpool ing Ġapproach Ġthat Ġwas Ġoriginally Ġproposed Ġfor Ġface Ġrecognition Ġto Ġimprove Ġlanguage Ġidentification Ġperformance Ġfor ĠIndian Ġlanguages . ĠWe Ġcollected Ġand Ġcurated Ġ630 Ġhrs Ġaudio Ġdata Ġfrom Ġnews ĠAll ĠIndia ĠRadio Ġnews Ġchannel Ġfor Ġ7 ĠIndian Ġlanguages . ĠOur Ġexperimental Ġresults Ġshows Ġthat Ġour Ġapproach Ġoutper forms Ġthe Ġprevious Ġstate Ġof Ġthe Ġart Ġmethods Ġby Ġan Ġabsolute Ġ1 . 88 % ĠF 1 - score . ĠWe Ġhave Ġalso Ġconducted Ġexperiments Ġwith Ġdifferent Ġpool ing Ġstrategies Ġproposed Ġin Ġthe Ġpast , Ġand Ġthe ĠGhost VL AD Ġpool ing Ġapproach Ġturns Ġout Ġto Ġbe Ġthe Ġbest Ġapproach Ġfor Ġaggreg ating Ġframe - level Ġfeatures Ġinto Ġa Ġsingle Ġutter ance Ġlevel Ġfeature . ĠOur Ġexperiments Ġalso Ġprove Ġthat Ġour Ġapproach Ġworks Ġmuch Ġbetter Ġeven Ġif Ġthe Ġinput Ġduring Ġtraining Ġcontains Ġsmaller Ġd urations . ĠFinally , Ġwe Ġsee Ġthat Ġthe Ġembed d ings Ġgenerated Ġby Ġour Ġmethod Ġhas Ġvery Ġgood Ġlanguage Ġdiscrim in ative Ġfeatures Ġand Ġhelps Ġto Ġimprove Ġthe Ġperformance Ġof Ġlanguage Ġidentification .