contemporary bcis are typically based on mental visual and motor imagery paradigms , which require extensive user training and good eyesight from the users  .recently alternative solutions have been proposed to make use of spatial auditory  or tactile ( somatosensory ) modalities  to enhance brain - computer interface comfort and increase the information transfer rate ( itr ) achieved by users .the concept reported in this paper further extends the previously reported by the authors in  brain somatosensory ( tactile ) channel to allow targeting of the tactile sensory domain for the operation of robotic equipment such as personal vehicles , life support systems , etc .the rationale behind the use of the tactile channel is that it is usually far less loaded than auditory or even visual channels in interfacing applications .the first report  of the successful employment of steady - state somatosensory responses to create a bci targeted a low frequency vibrotactile stimulus in the range of  hz to evoke the subjects attentional modulation , which was then used to define interfacing commands .a more recent report  proposed using a braille stimulator with  ms static push stimulus delivered to each of six fingers to evoke a somatosensory evoked potential ( sep ) response and the following p300 .the p300 response is a positive electroencephalogram event - related potential ( erp ) deflection starting at around  ms and lasting for  ms after an expected stimulus in a random series of distractors ( the so - called oddball eeg experimental paradigm )  .examples of averaged p300 response are depicted with red lines with standard errors in figures [ fig : alltopo ] , [ fig : allerp ] , [ fig : s3erp ] , and [ fig : s2erp ] .the p300 responses are commonly used in bci approaches and are considered to be the most reliable erps  with even beginner subjects .the results in  indicated that the experiments achieved information transfer rates of  bit / min on average and  bit / min for the best subject .a very recent report  additionally confirmed superiority of the tactile bci ( tbci ) in comparison with visual and auditory modalities tested with a locked in syndrome ( lis ) subject  .this paper reports improvement of our previously reported finger stimulus tbci  based on p300 responses evoked by tactile stimulus delivered via vibrotactile transducers attached to five positions on the subject s chest this time .the proposal is similar to the previously reported waist positions based tbci reported in  with a difference that we propose to use the chest area simulation which simplifies a vehicular robot operation in comparison to our previous hand- and head - stimulus based tbci solutions reported in  .the rest of the paper is organized as follows .the next section introduces the materials and methods used in the study .it also outlines the experiments conducted .the results obtained in electroencephalogram online and offline experiments with five bci subjects are then discussed . finally , conclusions are formulated and directions for future research are outlined .five volunteer male bci subjects participated in the experiments . the subjects mean age was , with a standard deviation of .all the experiments were performed at the life science center of tara , university of tsukuba , japan .the online ( real - time ) eeg tbci paradigm experiments were conducted in accordance with the _ wma declaration of helsinki - ethical principles for medical research involving human subjects_.   the tactile stimulus was delivered as sinusoidal wave generated by a portable computer with max / msp software  .the stimuli were delivered via five channel outputs of an external _ digital - to - analog _ signal converter rme fireface ucx coupled with the two acoustic yamaha p4050 power amplifiers ( four acoustic frequency channels each ) .the stimuli were delivered to the subjects chest locations via the tactile transducers hiwave hiax25c10 - 8/hs operating in the acoustic frequency spectrum of  hz , as depicted in figure [ fig : subject ] .each transducer in the experiments was set to emit a sinusoidal wave at  hz to match the transducer s resonance frequency .tactile impulses were designed to stimulate the _ pacini endings _( fast - adapting type ii afferent type tactile sensory innervation receptors ) which are the large receptive field mechanoreceptors in deeper layers of human skin  .the training instructions were presented visually by means of the _ bci2000 _ program with numbers representing robot movement directions ( see table [ tab : commands ] ) communicated via vibrotactile transducers attached to the subject s chest ( see figure [ fig : subject ] ) .eeg signals were captured with an eeg amplifier system g.usbamp by g.tec medical engineering gmbh , austria , using active electrodes .the electrodes were attached to the head locations : _ cz , pz , p3 , p4 , c3 , c4 , cp5 , cp6 , p1 , p2 , poz , c1 , c2 , fc1 , fc2 , _ and _ fcz _ , as in the extended international system  ( see the topographic plot in the top panel of figure [ fig : alltopo ] ) .the ground electrode was attached to _ fpz _ position and reference to the left earlobe respectively .no electromagnetic interference was observed from the vibrotactile transducers operating in higher frequencies comparing to the eeg frequency spectrum .details of the eeg experimental protocol are summarized in table [ tb : p4 ] .the captured eeg signals were processed online by bci2000-based application  , using a stepwise linear discriminant analysis ( swlda ) classifier  with features drawn from the  ms erp intervals . additionally in offline mode the classification accuracy was compared and improved using linear svm classifier  .the eeg recording sampling rate was set at  hz , the high pass filter at  hz , and the low pass filter at  hz . the isi was  ms , and each stimulus duration was  ms .the subjects were instructed to spell out the number sequences ( corresponding to the interactive robotic application commands shown in table [ tab : commands ] ) communicated by the transducers in each session .each was presented seven times in a single command trial .each subject performed three experimental sessions ( randomized and each ) , which were later averaged for the online swlda classifier case or treated as single trial ( only the first erp was used from a sequence of seven ) for linear svm , as discussed in a next section .the first online tbci session was a subject practice , the second was used for training the classifiers , while the last experimental session was used for testing interfacing accuracy .this section discusses the results obtained in the eeg online and offline data processing experiments , which are summarized in in table [ tb : bpmr3 ] and figures [ fig : alltopo ] , [ fig : allerp ] , [ fig : s3erp ] , and [ fig : s2erp ] .all the participating in the study tbci subjects scored well above the chance level of , reaching an itr in the range from  bit / min to  bit / min in case of the online bci experiment with swlda classifier , which may be considered to be a good outcome for online vehicular robot driving experiments .the itr was calculated as follows  : where stands for the number of bits / selection ; is the number of classes ( in this study ) ; is the classifier accuracy ( see table [ tb : bpmr3 ] ) ; and is the classification speed in selections / minute (  selections / minute for this study in case of averaging of seven responses in online swlda case or  selections / minute for the single trial based linear svm classifier ) .the maximum itr possible for the bci subjects to achieve in the settings presented were  bit / min and  bit / min , for seven averaged and single trial cases respectively .this case study demonstrated results obtained with a novel five commands and chest locations based tbci paradigm developed and used in experiments with five `` body able '' subjects .the proposed interface could be used for a real time operation of the robotic vehicle .the experiment results obtained in this study confirmed the validity of the chest tbci for interactive applications and the possibility to further improve the results with utilization of the single trial based linear svm classifier .  the eeg experiment with the paradigm has confirmed that tactile stimuli can be used to operate robotic devices with five commands and with the interfacing rate ranging from  bit / min to  bit / min for online case using swlda and  bit / min to  bit / min in the offline post processing case with linear svm classifier respectively .the results presented offer a step forward in the development of novel neurotechnology applications . due to the still not very high interfacing rate achieved in online bci case among the subjects , the current paradigm would obviously need improvements and modifications to implement also online the proposed and tested offline linear svm classifier based processing .these needs determine the major lines of study for future research .however , even in its current form , the proposed tbci can be regarded as a practical solution for lis patients ( locked into their own bodies despite often intact cognitive functioning ) , who can not use vision or auditory based interfaces due to sensory or other disabilities .we plan to continue this line of the tactile bci research in order to further optimize the signal processing and machine learning ( classification ) methods .next we will test the paradigm with the lis patients in need for bci technology .hm , tmr : performed the eeg experiments and analyzed the data .tmr : conceived the concept of the spatial tactile bci and designed the eeg experiments .sm : supported the project .hm , tmr : wrote the paper .this research was supported in part by the strategic information and communications r&d promotion program no .121803027 of the ministry of internal affairs and communication in japan , and by kakenhi , the japan society for the promotion of science , grant no .we also acknowledge the technical support of yamaha sound & it development division in hamamatsu , japan .  10  brouwer ,a.m. , van erp , j.b.f . : a tactile p300 brain - computer interface .frontiers in neuroscience * 4*(19 ) ( 2010 )  fan , r.e . , chang , k.w . ,hsieh , c.j . ,wang , x.r . ,lin , c.j .: library for large linear classification .journal of machine learning research * 9 * ( june 2008 ) 18711874  halder , s. , rea , m. , andreoni , r. , nijboer , f. , hammer , e. , kleih , s. , birbaumer , n. , kbler , a. : an auditory oddball brain computer interface for binary choices .clinical neurophysiology * 121*(4 ) ( 2010 ) 516 523  http://cycling74.com/ : max 6 ( 2012 )  johansson , r.s . ,flanagan , j.r . : coding and use of tactile signals from the fingertips in object manipulation tasks .nature reviews neuroscience * 10*(5 ) ( 2009 ) 345359  jurcak , v. , tsuzuki , d. , dan , i. : 10/20 , 10/10 , and 10/5 systems revisited : their validity as relative head - surface - based positioning systems .neuroimage * 34*(4 ) ( 2007 ) 1600 1611  kaufmann , t. , holz , e.m . , kbler , a. : comparison of tactile , auditory and visual modality for brain - computer interface use : a case study with a patient in the locked - in state .frontiers in neuroscience * 7*(129 ) ( 2013 )  krusienski , d.j . , sellers , e.w . , cabestaing , f. , bayoudh , s. , mcfarland , d.j . ,vaughan , t.m . ,wolpaw , j.r . : a comparison of classification techniques for the p300 speller .journal of neural engineering * 3*(4 ) ( 2006 ) 299  mori , h. , matsumoto , y. , struzik , z.r . , mori , k. , makino , s. , mandic , d. , rutkowski , t.m .: multi - command tactile and auditory brain computer interface based on head position stimulation . in : proceedings of the fifth international brain - computer interface meeting 2013 , asilomar conference center , pacific grove , ca usa , graz university of technology publishing house , austria ( june 3 - 7 , 2013 ) article i d : 095  mori , h. , matsumoto , y. , kryssanov , v. , cooper , e. , ogawa , h. , makino , s. , struzik , z. , rutkowski , t.m .: multi - command tactile brain computer interface : a feasibility study . in oakley ,i. , brewster , s. , eds .: haptic and audio interaction design 2013 ( haid 2013 ) .volume 7989 of lecture notes in computer science .springer verlag berlin heidelberg ( 2013 ) 5059  mori , h. , matsumoto , y. , makino , s. , kryssanov , v. , rutkowski , t.m .: vibrotactile stimulus frequency optimization for the haptic bci prototype . in : proceedings of the 6th international conference on soft computing and intelligent systems , and the 13th international symposium on advanced intelligent systems , kobe , japan ( november 20 - 24 , 2012 ) 21502153  muller - putz , g. , scherer , r. , neuper , c. , pfurtscheller , g. : steady - state somatosensory evoked potentials : suitable brain signals for brain - computer interfaces ? neural systems and rehabilitation engineering , ieee transactions on * 14*(1 ) ( march 2006 ) 3037  niedermeyer , e. , da silva , f.l . , eds .: electroencephalography : basic principles , clinical applications , and related fields . 5 edn .lippincott williams & wilkins ( 2004 )  plum , f. , posner , j.b .: the diagnosis of stupor and coma .fa davis , philadelphia , pa , usa ( 1966 )  rutkowski , t.m . , cichocki , a. , mandic , d.p . : spatial auditory paradigms for brain computer / machine interfacing . in : international workshop on the principles and applications of spatial hearing 2009 ( iwpash 2009 ) - proceedings of the international workshop , miyagi - zao royal hotel , sendai , japan ( november 11 - 13 , 2009 ) p5  schalk , g. , mellinger , j. : a practical guide to brain computer interfacing with bci2000 .springer - verlag london limited ( 2010 )  schreuder , m. , blankertz , b. , tangermann , m. : a new auditory multi - class brain - computer interface paradigm : spatial hearing as an informative cue .plos one * 5*(4 ) ( 04 2010 ) e9813  stocks , c.r.m .: - https://github.com/collinstocks/py3gui ( 2011 )  theodoridis , s. , koutroumbas , k. : pattern recognition .fourh edn .acedemic press ( 2009 )  van der waal , m. , severens , m. , geuze , j. , desain , p. : introducing the tactile speller : an erp - based brain computer interface for communication .journal of neural engineering * 9*(4 ) ( 2012 ) 045002  wolpaw , j. , wolpaw , e.w ., eds . : brain - computer interfaces : principles and practice .oxford university press ( 2012 )figure [ fig : subject ] : :  subject wearing eeg cap with active electrodes  attached to the g.usbamp amplifier .the five vibrotactile transducers  are attached to an elastic belt on the subject chest .small vehicle  robot by lego mindstroms , operated via tbci application developed by  the authors , is placed on the floor in front of the subject in the  picture .figure [ fig : alltopo ] : :  all the five subjects grand mean averaged results of the chest  stimulation eeg experiment .the left panel presents the head  topographic plot of the versus   area under the curve ( auc ) , a measure  commonly used in machine learning intra class discriminative analysis .( is usually assumed to be confirmation of  feature separability  ) .the top right panel presents the  largest difference as obtained from the data displayed in the bottom  panel .the topographic plot also depicts the electrode positions .the  fact that all the electrodes received similar auc values ( red )  supports the initial electrode placement .the second panel from the  top presents averaged sep responses to the   stimuli ( note the clear p300 response in the range of   ms ) .the third panel presents averaged sep  responses to the stimuli ( no p300 observed ) .finally , the bottom panel presents the auc of   versus responses ( again , p300 could easily be  identified ) .figure [ fig : allerp ] : :  all five subjects grand mean averaged results for each electrode  plotted separately .the red and blue lines depict targets and  non targets respectively together with standard error bars .the p300  related responses could be observed in the  ms  latencies .figure [ fig : s3erp ] : :  averaged results for each electrode separately of the  subject  for whom response was  dominating leading for the best latency used in classification .the  red and blue lines depict targets and non targets respectively  together with standard error bars .figure [ fig : s2erp ] : :  averaged results for each electrode separately of the  subject  for whom response was  significant leading for improved early latency used in classification .  observe the very short latency around where the  standard error bars do nt overlap .the red and blue lines depict  targets and non targets respectively together with standard error  bars ..conditions of the eeg experiment . [ cols="<,<",options="header " , ]    subject  & number of averaged trials  & maximum accuracy  & itr + & & &  bit / min  + & & &  bit / min  + & & &  bit / min  + & & &  bit / min  + & & &  bit / min  +  + subject  & number of averaged trials  & maximum accuracy  & itr + & & &  bit / min  + & & &  bit / min  + & & &  bit / min  + & & &  bit / min  + & & &  bit / min  +active electrodes attached to the g.usbamp amplifier .the five vibrotactile transducers are attached to an elastic belt on the subject chest .small vehicle robot by lego mindstroms , operated via tbci application developed by the authors , is placed on the floor in front of the subject in the picture . ]   versus area under the curve ( auc ) , a measure commonly used in machine learning intra class discriminative analysis .( is usually assumed to be confirmation of feature separability  ) .the top right panel presents the largest difference as obtained from the data displayed in the bottom panel .the topographic plot also depicts the electrode positions .the fact that all the electrodes received similar auc values ( red ) supports the initial electrode placement .the second panel from the top presents averaged sep responses to the stimuli ( note the clear p300 response in the range of  ms ) .the third panel presents averaged sep responses to the stimuli ( no p300 observed ) .finally , the bottom panel presents the auc of versus responses ( again , p300 could easily be identified ) . ]for whom response was significant leading for improved early latency used in classification .observe the very short latency around where the standard error bars do nt overlap .the red and blue lines depict targets and non targets respectively together with standard error bars . ]

Abstract: the presented study explores the extent to which tactile stimuli delivered to five chest positions of a healthy user can serve as a platform for a brain computer interface ( bci ) that could be used in an interactive application such as robotic vehicle operation . the five chest locations are used to evoke tactile brain potential responses , thus defining a tactile brain computer interface ( tbci ) . experimental results with five subjects performing online tbci provide a validation of the chest location tbci paradigm , while the feasibility of the concept is illuminated through information - transfer rates . additionally an offline classification improvement with a linear svm classifier is presented through the case study .   * keywords : * tactile bci , p300 , robotic vehicle interface , eeg , neurotechnology


recent advances in self - driving vehicles and advance driver assistance system ( adas ) have attracted keen attention and interest from researchers and automobile manufacturers .vision sensor plays an important role in these areas due to its faster response , lower price and power consumption compared with other popular sensors such as lidar and millimeter - wave radar .moreover , vision sensor has the ability to capture rich information from traffic scene ( such as luminance , color and texture )  , which is beneficial for object detection .vision - based object detection is one of the fundamental functions for self - driving vehicle systems and advance driver assistance systems ( adas ) , which need to detect the objects around and check whether they are dangerous to the host vehicle .however , object detection in traffic scene from image are still challenging job . on one hand ,traffic scenes are diverse and the presence of objects ( e.g. vehicles and pedestrians ) in traffic scenes are extremely flexible and unpredictable . on the one hand ,most traffic scene applications have several special requirements such as real - time , portable ( e.g. for adas device ) , low price and power consumption , which are quite different from object detection tasks in the ilsvrc  and coco  competitions .recently , a great number of researchers have been interested in learning based approaches . especially in recent years , the success of deep learning boosts the development of vision - based object detection .convolutional neural network ( cnn )  is one of the most popular forms of deep networks . by using the strategies of local receptive fields , weight sharing and spatial pooling  , cnn has made a breakthrough in computer vision and image processing .for instance , alexnet  , which is a type of cnn , has made a startling achievement in the competition of ilsvrc-2012  and demonstrated its superiority in image classification .however , with the increasing of network depth , cnn has met the bottleneck in training  . recently , he _ et al . _proposed the deep residual network  which employs shortcut connections to overcome the limitations in training a deeper cnns .  despite the fact that cnn has achieve a tremendous success in computer vision and image processing ,there still exist two main problems that have to be solved when applied to objects detection tasks .firstly , cnn has a multitude of convolutions that have to be calculated , it would be rather inefficient in object detection if using traditional dense sliding windows paradigm  .secondly , cnn has the property of shift invariance , that is , it is less sensitive to the shift of objects in input image patch. consequently , it can not achieve precise localization if it is directly applied to objects detection  .  to overcome these limitations of cnn , girshick _ et al ._ introduced the framework of region proposal based cnn ( called r - cnn )  and successfully applied it to object detection .the main idea of region proposal based cnn is performing cnns on candidate bounding - boxes ( called `` proposals '' ) which have potential to contain objects  . later , the updated version `` fast r - cnn ''  is proposed to improve the efficiency of object detection .this method combines r - cnn with sppnet  . in this way ,computation of proposals can be sharing and runtime of object detection can be reduced dramatically . however , in both r - cnn and fast r - cnn , the proposals are generated by selective search  , which is quite inefficient and thus limits the detection speed of these methods . to overcome this limitation, faster r - cnn  proposed to used region proposal networks ( rpns ) to generate proposals instead of selective search  . by this means , the stage proposal generation and cnn - based classification / regression can be performed under an unified framework and thus the detection speed can be boosted with the help of gpu.  however , region proposal based cnns are complex and not easy to optimize  .for this reason , several works attempt to achieve real - time objects detection by regarding it as a regression problem .yolo  is one of the pioneering works on deep regression networks based object detection .the approach is able to simultaneously outputs the location of bounding box , category and its confidence score for each object in the image at a extremely high frame rate .nevertheless , it does not work well in small objects detection .in the same vein , liu _et al . _ proposed ssd ( single shot multibox detector )  for real - time objects detection .it equipped the deep regression networks with several new techniques such as multi - scale feature maps and default boxes  . despite the fact that it is able to be faster and more accurate compared with yolo  ,its performance on small objects detection is still unsatisfactory .    in this paper, we introduce a practical and robust approach for real - time object detection in traffic scene .the approach requires less storage and computing resources , and thus it is feasible for traffic scene applications .the novelties of this work lie in four fold :  1 .we present a local regression strategy for accurate objects detection . as we know, current regression based solutions ( such as yolo  and ssd  ) employ global regression strategy to regress the bounding - box of each object from the whole image , which is actually a much more difficult task compared with local regression .thus , global regression strategy often needs to be supported by a complex or large - scale network and yet its performance on small objects detection is still unsatisfactory . to solve these problem, we introduce the local regression strategy that performs tiny regression networks on small image patches to detect and locate objects. as the regression networks are less sensitive to the scaling and shift of the objects in image patches , a series of sparse sliding - windows can be obtained from multi - scale image pyramid .finally , objects can be detected and located efficiently based on these sparse sliding - windows ; 2 .we introduce the generalized haar filter based deep networks where each weight larger than are constrained to the form of generalized haar filter in training phase . owing to the strong representation of haar filters , the networks are able to achieve a high performance . besides , the networks consume much less storage and computing resources compared with traditional deep networks , which make them possible to be utilized in traffic scene applications .in addition , the constraint of generalized haar filter provides a form of regularization which is able to improve the generalization ability of the deep networks ; 3 .  for object detection problems in traffic scene ,a sparse windows generation method is proposed .the method first generates a series of sparse sliding - windows in multi - scale image pyramid by setting a specific stride according to the scale and shift tolerance of our deep network . in this way , it can be ensure that each object is completely contained in at least one window .besides , in most object detection systems for traffic scene , camera is mounted on a fixed position ( e.g. mounted on the top of a vehicle windshield ) .consequently , objects ( e.g. vehicles and pedestrians ) produce the predetermined location - specific patterns in images . according to this assumption, perspective geometry is also utilized to further reduce the candidate windows ; 4 .we construct a tiny deep network that simultaneously outputs the bounding box , category and confidence score of detected object through two output channels : localization channel and classification channel .the network is efficient and consumes less resources . as our proposed method decomposes the global regression task into several easier local regression tasks , which can be handled effectively without the support of the complex or large - scale networks .thus , our proposed approach can work efficiently and effectively by combining this tiny a deep network with local regression tasks .the rest of this paper is organized as follows : in section [ rec_ghaar ] , we introduce the architecture of our generalized haar filter based deep networks and describe how to design their weights . in section [ sec_sparse_windows ], we propose the algorithm of sparse windows generation .experimental results and corresponding discussion are presented in section [ sec_res_dis ] , while conclusions and future works are given in section [ sec_conclu ] .to solve the problem of object detection , we construct a deep network that simultaneously outputs the bounding box , category and confidence score of detected object via two output channels : localization channel and classification channel ( see fig .[ fig_pipeline_net ] ) .the location channel focuses on bounding box regression and outputs a 4-dimensional vector which is used to describe a bounding box ( see fig .[ fig_locshow ] ) .the classification channel outputs the category and confidence score which are denoted by a 2-dimensional vector .  as shown in fig .[ fig_pipeline_net ] , our deep network consists of 11 convolution layers , 4 max - pooling layers and 1 softmax layer . to reduce the memory consumption ,several low - level features are shared by localization and classification channels via conv1 .as these two channels aim at different tasks , each of them has 4 independent layers to focus on different problems .our deep network is less sensitive to the scaling and shift of the input object owing to regression based localization channel .consequently , instead of generating region proposals , objects can be detected and located efficiently based on sparse sliding - window paradigm and perspective geometry .more details on this issue are introduced in section [ subsec_sliding_windows ] and related experiments are presented in section [ sec_res_dis ] .despite the fact that deep neural networks have achieved state - of - the - art performance in object detection , they consume considerable storage , computing resources and power  .consequently , they are often unsuitable for power and memory constrained devices such as vehicle - mounted embedded systems and mobile devices . to overcome this limitation, we introduce a novel convolution weights design method which is based on generalized haar filter .haar - like filters have been successfully applied in object detection owing to their strong representation and high efficiency  . instead of using a fixed number of rectangles and configuration types , generalized haar filtersare based on arbitrary configurations and number of rectangles  . in this work ,the weights of our deep network are constrained based on generalized haar filters .  unlike original generalized haar filters with arbitrary configuration and number of rectangles  , in our work , the configuration and number of rectangles are obtained in a data - driven way . for a weight of size ( ) , we constrain it to the following form : where is a multiplication factor and is the -th generalized haar filter in the haar filters space . as we know , there are types of configurations for the generalized haar filters of size . in our case , and its negative form can be regarded as a same filter .thus , the haar filter space of size contains filters .that is ,  ] and $ ] respectively represent the location ranges of the corresponding windows in and axes of the world coordinates . by solving the equation set that consists of equation ( [ eq_fxy ] ) and ( [ eq_widhei ] ) , we can find 4 boundary planes in space which jointly limit the locations and sizes of input windows : where :   space.,scaledwidth=47.0% ]  as shown in fig .[ fig_boundaryplane ] , in the space , possible input windows are distributed in the region of inverted pyramid which is enclosed by 4 boundary planes .we use to represent the set of input windows in this region , then the final sparse windows can be obtained by :   as shown in fig .[ fig_exp_pyramid ] , we illustrate the final sparse windows in each image of the image pyramid using the tme motorway dataset  , which is a challenging and widely - used dataset in vehicle detection . note that , all of the sparse windows have the same size ( in this work ) , which means that they can be directly utilized as the input of the deep network . in this way , the sparse windows are avoided to be resized and the efficiency of the approach is ensured .the experiments in this section mainly focus on vehicle detections in traffic scene , which is an important issue in a wide range of applications such as autonomous driving , autonomous navigation and advance driver assistance systems ( adas )  . in this work, the performance of our approach is evaluated on a broadly used datasets : tme motorway dataset  , which is designed for vehicle detection and localization in challenging traffic scene with various lighting conditions and complex traffic situations  .  in the following experiments ,the output bounding - boxes of our approach are refined by mean shift and non - maximum suppression , and we use an intersection - over - union ( iou ) threshold of 0.7 to determine the correctness of detection . all the experiments in this section are performed on a gtx1080 gpu .   as our deep network has two output channels : classification channel and localization channel .we respectively use the classification and localization error of training and test as the performance metrics for the following experiments .the classification error of both training and test is defined as : where is the total number of training or test samples , and respectively denotes the number of positive and negative samples which are incorrectly classified .  for the localization channel, the error of both training and test is defined as : where the vector represents the output of the localization channel , and is the ground truth location vector .[ fig_exp_tr_vs_haar ] illustrates how the errors of classification and localization changed when increasing the number of training iterations .according to this figure , our deep network is able to achieve a high performance when training is convergent . despite the fact that the deep network with generalized haar filter based weights ( hereafter called g - haar weights ) has a slightly larger training error and needs more training iterations to reach the state of convergence , it is able to produce a less test error in both classification and localization tasks when training is convergent . in other words ,the deep network with g - haar weights has a stronger generalization ability than that with traditional weights .this is owing to the regularization effect of g - haar weights . besides , the network employing g - haar weights with less ( the number of selected filters ) needs more training iterations and tend to have stronger generalization ability when training iteration is convergent .we investigate the storage and computing resources consumptions in this subsection .the experiment is also performed on tme motorway dataset  . in the experiment ,the number of selected filters is 32 , and all the weights are stored using single - precision floating - point format ( 32 bits ) . in the deep network ( see fig .[ fig_pipeline_net ] ) , the size of conv1 is , and the rest of convolution kernels have the size of . as the weights of size consume much less resources than that of size , we only evaluate the dimensions of conv1 and their effects on resources consumptions in table [ tbl_source_con ] .as shown in table [ tbl_source_con ] ( column of mem . ) , the networks using g - haar weights are able to dramatically reduce memory resources ( about times , in this work ) .this is due to the fact that only a filter index and a multiplication factor are needed to stored for each g - haar based convolution kernel , which totally consumes 5 bytes of memory space .by contrast , there are weights needed to stored for each traditional convolution kernel of , thus , of memory space is required .besides , as multipliers are major computing resource consumed by deep networks , we use the number of multiply operations required for each convolution step to measure the consumption of computing resource for the deep networks .as we know , each traditional convolution kernel of needs multiply operations for each convolution step .nevertheless , for any convolution kernel using g - haar weights , each convolution step can be transformed to the form of equation ( [ eq_convstep ] ) . in this way , only one multiply operation is required for each g - haar convolution step . accordingly , as shown in table [ tbl_source_con ] ( column of mul. . ) , the computing resource consumption of the deep networks using g - haar weights is only of that using traditional weights ( in this experiment ) .in addition , power consumption is directly influenced by storage and computing resources utilizations  .as our deep network with g - haar weights can markedly reduce storage and computing resources ( including memory accesses ) , it would have a great increase in power - efficiency .this merit is especially meaningful when the deep network implements on embedded systems or mobile devices ( such as fpga and arm ) , which are quite sensitive to power consumption .  [cols="^,^,^,^,^,^,^,^,^,^",options="header " , ]in this paper , we presents a novel network system for object detection tasks in traffic scene . in this system, we introduce a local regression strategy for accurate objects detection . compared with traditional global regression based object detection ,the local regression task is easier to handled without the support of the complex or large - scale networks . according to this fact, we handle the local regression tasks by using several tiny deep networks which simultaneously output the bounding boxes , categories and confidence scores of detected objects . in order to satisfy the storage , power and computing source requirements of the platforms for traffic scene applications ,the weights of the deep networks are constrained to the form of generalized haar filter in training phase .furthermore , to achieve the real - time performance , we introduce the strategy of sparse windows generation to reduce the runtime of our system .  in the experiments , we first evaluate the performance and generalization ability of our generalized haar filter based weights by comparison with traditional weights .then the consumptions of storage and computing resources are evaluated .finally , the effectiveness and efficiency of our approach are validated in comparison with some recently published state - of - the - art methods .experimental results demonstrate that the proposed approach is proved to be efficient , robust and source saving in challenging traffic scene .  as the proposed approach is suitable for the platforms which have strict limitations on memory , power and computing sources, our future works will focus on implementing the proposed approach ( path forward ) to fpga ( field programmable gate array ) for advance driver assistance system ( adas ) .owing to the generalized haar filter based weights , only one multiply operation is required for each convolution step .this make it possible to construct more parallel pipelines in fpga where the number of multipliers is limited .moreover , each local regression task is an independent computation .thus , all of the local regression tasks can run in parallel in fpga . in this way, the system can achieve real - time response without extra effort .this work is supported by the national natural science foundation of china ( grant no .61473303 ) .       p.f. felzenszwalb , r. b. girshick , d. mcallester , d. ramanan , object detection with discriminatively trained part - based models , ieee transactions on pattern analysis and machine intelligence ( tpami ) 32(9 ) ( 2010 ) 16271645 .r. girshick , j. donahue , t. darrell , j. malik , rich feature hierarchies for accurate object detection and semantic segmentation , in : ieee international conference on computer vision and pattern recognition ( cvpr ) , 2014 , pp .580587 .           c. caraffi , t. voj , j. trefn , j. ochman , j. matas , a system for real - time detection and tracking of vehicles from a single car - mounted camera , in : ieee intelligent transportation systems conference , 2012 , pp .

Abstract: vision - based object detection is one of the fundamental functions in numerous traffic scene applications such as self - driving vehicle systems and advance driver assistance systems ( adas ) . however , it is also a challenging task due to the diversity of traffic scene and the storage , power and computing source limitations of the platforms for traffic scene applications . this paper presents a generalized haar filter based deep network which is suitable for the object detection tasks in traffic scene . in this approach , we first decompose a object detection task into several easier local regression tasks . then , we handle the local regression tasks by using several tiny deep networks which simultaneously output the bounding boxes , categories and confidence scores of detected objects . to reduce the consumption of storage and computing resources , the weights of the deep networks are constrained to the form of generalized haar filter in training phase . additionally , we introduce the strategy of sparse windows generation to improve the efficiency of the algorithm . finally , we perform several experiments to validate the performance of our proposed approach . experimental results demonstrate that the proposed approach is both efficient and effective in traffic scene compared with the state - of - the - art .