nearly every work on image tagging to date has treated the tags that accompany an image as a bag - of - words with no inherent ordering , most especially works which use the nearest neighbor approach for tagging . some recent work , has shown that this bag - of - words assumption is actually not the reality .they show that if a user is asked to tag an image with a fixed set of tags multiple times , the order of the tags they choose tends to stay the same rather than random .more so , a recent design change on flickr  that reversed the users tag input ordering created a backlash from the community that caused flickr to revert back to the original design with the following apology : `` we thought it would be more intuitive for newest tags to appear at the top , so that when you add a tag and refresh , it s in the place you expect it .however , thanks to your points about meticulous tag ordering , we ve decided we should leave it as is .you should now see your tags in the correct order '' .these insights lead us to believe that tag ordering is inherently personal , and should be incorportated into tagging systems . hence, we propose a new model that seeks to exploit the history of a user s tag order to improve personalized image tagging .we also compare the same model which instead treats tag order as a global phenomenon , and show that indeed such a model underperforms the personalized ordering model significantly .there has been a lot of work in the area of automatic image tagging , much of which has treated tagging as more of a labelling problem , thereby implicitly imposing that tagging is a global task rather than a user - specific task .most notably the work by li et al . has tried to treat the tagging problem from the point of view of personalization by using a cross - entropy method to decide how to weight different image tagging functions for each user .there has been work seeking to tag based on object importance , but there the focus has been an explicitly categorical approach to importance by measuring properties of objects in the image ( eg , size , salience ) to estimate their relative importances . these object - property based approaches to importance typically ignore particular user preferences , and treat importance as a global phenomenon .nearest neighbor approaches are usually more common than their explicit classification counterparts in tagging because one does not have to learn how to recognize or detect specific objects in the image , which is not scalable , nor are all concepts one would like to describe an image visual . in many cases , an initial set of tags for a given query image is provided and the aim is to refine or propagate these tags through some , similarity measure ( image , user , or tag co - similarity ) . whereas their goal is to create global tags from other tags , our goal is to find the words most suitable to describe an image for specific individuals .there has also been some work that looks at the recommendation perspective , with  having an explicit goal of personalized tagging .  in this work, we employ the nearest neighbor approach to tagging because we would like to use a relatively large tagging vocabulary , which introduces scalability issues with the explicit / classification approaches , and we also do not want to restrict our vocabulary to words that describe concrete visual concepts .instead we would like to have more abstract tags because we are primarily interested in personalized tagging . finally , we also do not want to consider tag importances solely on a global scale , but per user preference .in this section we discuss our model for the tagging behavior of users and derive an objective function based on the model of tagging behavior .we then discuss a practical representation of this model and how it relates to other known problems .the main assumption , following the claims from  , is that given an image , the order that users present tags to an online tagging system , say flickr , implies an underlying preference order for those tags . for example , if a user presents the tags in that order , it would imply that for that image , the user found it more preferable ( or important ) to mention before , before , and before .our main idea is that with enough observed instances of pairs of tags mentioned together by a particular user , we can estimate these implicit biases , and in turn improve the task of automatic image tagging for new images on a personal level .our main tagging goal is to output a list of tags for an image in order of preference for the target user . to that end ,given a set of pairwise tag - preferences ( as probabilities ) for a given user and a query image , we want to find the ordering of candidate tags ( generated by some baseline tag generator ) , for that query image , that is maximal .details are provided in the supplementary appendix .   for the objective described above ,we represent the tag - preferences as probabilities .that is , is the strength of preference for tag over tag .we estimate this quantity as the number of times occurs before , divided by the number of times they occur together , regardless of order .this allows for anti - symmetry : . in this representation, implies there is no preference .also note the we only need to keep track of the strongest preference ( either , or , but not both ) .   to maximize our tagging objective, we came up with a modified version of the topological sort algorithm .since we can represent the preferences as directed edges with weighs as the probabilities , we can use topological sort  to produce an ordering that is faithful to these preferences .  to deal with cycles , which are assumed not to exist for the topological sort algorithm to work ,when we find a cycle , we delete the edge of least strength . under certain mild conditions ,this is guaranteed to still be optimal .also , since there could be multiple correct topological sort orderings , in order to break ties , we use a baseline ordering of the candidate tags ( gotten from the baseline tag generator as mentioned in section [ model : objective ] ) , as we run the modified topological sort algorithm , whenever there is an ambiguous ordering .details and proofs of claims made here are provided in the appendix .in this section , we will discuss how we go about verifying our model , from the choice of the dataset , choice of baseline and choice of evaluation metric .   for this work we chose to work with the nus - wide dataset .the nus - wide dataset is a subset of 269,648 images from flickr . for each image in the dataset , we know , via the flickr api , the corresponding user that uploaded that image , and the sequence of tags that user chose to annotate the image with . since we are particularly concerned about personalization , we only select images from this database which satisfy the following criteria : the users who uploaded the image must have at least 6 images in this dataset , similar to the setting in .this results in about 91,400 images from 5000 users .we split this dataset into a training and test partition , by randomly assigning half of each users images to the training , and half to the test set . for each imagewe only retain the tags that occurred frequently enough across the dataset , in order to make some sort of meaningful inference on the tags . in this work, we made the design choice of working with tags that occurred at least 50 times in the dataset .this results in a vocabulary of unique tags .   as we mentioned in section[ sec : related ] , we go the nearest neighbor route for the task of image tagging . to that end our image features are a 500-d bag of words based on sift descriptors , and we use the euclidean distance between feature vectors to encode the notion of closeness or similarity .   we take as our baseline the work of li et al . , which we considered the reproducable state of the art for _ personalized _ image tagging and evaluated their claims on the nus - wide dataset as well .their main idea is that for a given tag , each user has 2 weighting variables , one weighting how much to rank that tag according to its frequency independent of visual content , and the other how much to rank the frequency of the tag according to its frequency in visually similar images .we re - implemented their method using the two tagging functions described in their paper ( _ personalpreference _ function and _ _visual__ function ) . for the visual features ,we also use the 500-d bag of words based on sift descriptors for consistency .  for a given image query , we define the ordered set of tags returned by the baseline method as .since our assumption is that the order that a user tags an image important for automatic tagging , our metric should take the groundtruth user order into account .more concretely , given an ordered set of tags, , we define the relevance of each tag according to it s reciprocal rank in the ordered set : tags not in the groundtruth ordered set have zero relevance .more common metrics , such as precision , recall , and average precision , assume that all tags are equally relevant , so we do not utilize these metrics in this paper .instead we use the more appropriate _ normalized discounted cumulative gain _ ( ndcg ) , a common metric used in evaluating search engine results . for an ordered set , such that , we define the dcg with respect to the ground - truth as :    this metric is called _ discounted _ because the later we include a tag in our ranking , the less gain we get from it ( i.e. its relevance is discounted by the inverse of the log of its position in the ranking , not the groundtruth ) .note that this metric is maximal when the most relevant items are listed first .we also define for a given ranked list , , its ideal ranking , such that for , if , then in the groundtruth ( i.e. , the ideal ranking is ranked from most relevant to least ) .then the normalized dcg is defined as :   note that the ndcg is maximal ( equal to 1 ) when .we can also parameterize the dcg , and ndcg to calculate the and .that is , calculate the metrics evaluated only for the first entries of the ranked lists .let $ ] be the first k entries of , then : )}{dcg(\bar{\sigma}(t)[:k])}\ ] ]given a query image from the test set , we find the nearest neighbors ( visual similarity ) from the training set , and given some initial rank on the set of tags used to describe these visually similar images , we find optimal ranking , , as described in section [ model : objective ] , using the preference probabilities derived from the training set according to section [ model : parameter_estimation ] .we set the baseline order of candidate tags as , which is the result of our baseline ( and the current state - of - art for personalization ) given the query , from section [ exp : baseline ] .   in evaluating our approach, we made a few design decisions which are parameters in our model .the first is the minimum number of times tags and occur together , which we denote as .if any pair of tags do nt co - occur at least times , we do not have a high confidence in the order bias that we estimate from the occurrences because of noise and overfitting . the second which we denote as is the minimum strength factor between a pair of tags .that is , we ensure that , , for to be an edge to be considered by the prioritizedtoposort .the purpose of these parameters is to control for noise and overfitting to insufficient data .   in this section ,we describe the tests with which we evaluate our method . for the evaluation metrics discussed in section [ exp :metrics ] , we run evaluations under the following settings :  * number of visually similar neighbors : * minimum co - occurrence , c : * minimum strength factor , r :   we compare our performance ( personalized pairwise preference ) , to the state - of - art  , and also to global pairwise preferences .global pairwise preferences is similar to what we have proposed so far in this paper with the exception of ignoring the users , so the images are treated as though they all come from the `` average user '' .we provide evaluation for our metrics on entire ranks , and also the top tags in the rank for .we show the mean performance averaged both over each image ( included in the appendix ) , and also average across its mean performance per user since we are interested in personalization .we also calculate the number of times our approach produces better rankings ( in terms of ndcg ) , than the baseline , and the number of times a user prefers our approach on average over the baseline .we report these numbers in table [ tab : duels ] ..average ndcg percentage improvement over the baseline using pairwise personal preferences [ cols="<,^,^",options="header " , ]   from [ fig : usr_ndcg_all ] and [ fig : ndcg_k_usr ] , we notice that our algorithm which personalizes via pairwise tag preferences ( red bar / line ) always outperforms the state - of - the - art baseline ( blue bar / line ) .[ fig : usr_ndcg_all ] shows this increase in performance for all parameters ( the minimum strength factor described in section [ res : design_dec ] ) , and ( the minimum number of co - occurrences ) , that we considered .we notice that holding constant as the parameter increases , the performance of our personalization method decreases .this is likely due to a lack of enough data per user so that very few tag pairs can meet our relatively aggressive over - fitting criteria .the same trend is noticed when we hold constant and increase .we also evaluate the performance of the global ( or `` average user '' , yellow bar / line ) , pairwise preferences , and we see that enforcing pairwise orders based on a global / average notion of preference actually degrades the performance . except for , the metric for the global preferences is worse than the baseline .this might be because , users tend to use more global / popular tags in the beginning of their ranked lists , so that we are actually able to learn with enough data , certain global biases among the popular tags .beyond that , because users tag differently , enforcing a global preference is actually counterproductive .we also observe the reverse trend as we fix and vary ( and vice - versa ) to that of the personal preferences .this is because to estimate global preferences we need to observe a lot more data than for a single user , and so estimating for pairs without a lot of co - occurrences will invariably lead to over - fitting .[ fig : ndcg_k_usr ] , shows the as varies under the best choices of parameters and , for both the global and personal preferences .the best choices for the global were , , and for the personal , .we see , averaging over both users and images(included in the appendix ) , that the baseline outperforms the global ( except at ) , and the personal outperforms the baseline .these observations imply that picking lower parameters is usually better for the personal preferences , while higher values of and are necessary for the global preferences to be useful .table [ tab : avg_stats ] shows that our method on average is better than the baseline method , and this improvement goes up to about , for , as can be seen from figure [ fig : ndcg_k_usr ] .the above observations are true for both the average across all the images in our test set , and across all the users mean .this shows that the gains we get are relatively consistent for each user .this combined with the fact that the global pairwise preferences under - perform , imply that true personal pairwise preferences indeed exist , can be estimated , and can be used to improve the performance of automatic image tagging .we include more detailed plots in the supplemntary appendix .in this work we proposed a new measurement of tag preferences , and demonstrated that there is indeed a tag - order bias , that is , when a user mentions tag before tag , in a list of tags for a given image , the user is implying that he prefers , or considers to be of greater importance than .we showed that this bias can be learned from historical data using the maximum likelihood estimate based on a pair s co - occurrence , and subsequently showed that such information can be exploited to improve the performance of current state - of - art automated image tagging systems .we also defined a new tagging objective function that assumes the inherent pairwise bias between tags , and provided an algorithm which optimizes the new objective ( under some mild conditions ) , and helped verify our claims and assumptions .this leads us to conclude that although there are many visual factors that may affect what tags a user will provide for an image , it is useful to characterize instead ( or rather in conjunction ) the users tagging habits to learn what tags are of more importance to the users , whether they are visually motivated or not , and automatic tagging systems should employ this technique to improve their overall performance .we believe that there are several ways that this work could be extended or extend other works .one direction we see is working on algorithms that provide tight guarantees for solving our tagging objective , or even solves it optimally , even in the presence of dependent cycles .another direction comes from the observation that most tag pairs never occur , but it may be possible to learn a function that maps a pair of tags to a real number indicating the direction and strength of the preference. it would also be interesting to see how the assumptions made in this paper can be used to extend the works on learning for personalized ranking .we think it would also be interesting to explore the cognitive dimensions that drive tag ordering , and how these cognitive dimensions contribute to the tagging choices both independently and collectively .

Abstract: what makes a person pick certain tags over others when tagging an image ? does the order that a person presents tags for a given image follow an implicit bias that is personal ? can these biases be used to improve existing automated image tagging systems ? we show that tag ordering , which has been largely overlooked by the image tagging community , is an important cue in understanding user tagging behavior and can be used to improve auto - tagging systems . inspired by the assumption that people order their tags , we propose a new way of measuring tag preferences , and also propose a new personalized tagging objective function that explicitly considers a user s preferred tag orderings . we also provide a ( partially ) greedy algorithm that produces good solutions to our new objective and under certain conditions produces an optimal solution . we validate our method on a subset of flickr images that spans 5000 users , over 5200 tags , and over 90,000 images . our experiments show that exploiting personalized tag orders improves the average performance of state - of - art approaches both on per - image and per - user bases .   ordered tags , personalized tagging , user behavior , personalization , tag preferences


the internet has revolutionized science and society in many ways .studies of human communication networks nowadays deal with data sets of thousands people or more . gone are the days when one had to rely solely on interview surveys and observational studies . these new and larger data sets open up new possibilities one does not only get higher precision in the measurement of network structure , one can also rid the structural quantities of finite size effects that would hinder the prediction of phenomena such as large - scale information spreading .the accessibility of large data sets has also drawn the interest of statistical physicists , habitually working in the large - scale limit , to this traditionally sociological field .the sociologists themselves , naturally , contribute to the development , and make the study of communication networks a thriving interdisciplinary arena .internet has , of course , changed other aspects of society than science . in the perennial human quest to find a partner , romantic orotherwise , the internet provides a new _ modus operandi_. in this paper we study a swedish internet community , pussokram.com , intended for romantic communication among adolescents .the methods we propose do not only work for dating communities , but for internet communities in general .our definition of an internet community is a set of html - pages facilitating some type of directed messages ( i.e. not all messages are broadcasted to the whole community ) , and where each member is associated with a home page .this definition includes social networking sites but excludes , for example , news groups .the data we use , the same as in ref . , are anonymized lists of contacts triples where is the id - number of the sender , is the id - number of the receiver , and is the time ( in seconds ) since an arbitrary start time .the ids are integer numbers with no relation to the users of the real community .we do not have access to the messages themselves , nor the individual presentations on the user homepages ( or any other text or images either ) .researched data sets of electronic communication have often been constructed from email exchange , and to a lesser extent from communication within internet communities  .the advantage with internet communities is that they are a closed system all communication can be recorded . the network of emails is a much larger and more important phenomenon . but studies of email networks are plagued by statistical biases due to the openness of the system . for email communication one typically samples a set of individuals and either restricts the data to messages sent within the group ( which leaves the message set incomplete )  or one includes contacts to outer vertices , which does not include contact between outer vertices .for this reason the study of internet communities may not only be of interest _ per se _ , but is also illuminating towards the general structure of electronic communication .  the goal of this paper is to say something about what makes a user successful in the community . given our very restricted data needless to say , the most relevant information is in the text and imagery we do not have is there anything that can be said ?we propose simple measures for attractiveness and activity and observe that they are positively correlated in the contact sequences .more than that , these two quantities are more strongly correlated for the real - world data than for the null model we propose .it thus pays off , directly or ( more likely ) indirectly , to be an active community member ..[tab : sizes ] the sizes of our three data sets : the number of vertices , edges and contacts . [ cols="<,^,^,^",options="header " , ]   in the previous section we defined three vertex - specific measures for internet community contact sequences .we measured the probability distribution and the correlations between the measures .the design of the measures does not exclude that correlations may be induced by the growth of the community . to frame the correlations associated with the psychology of the community members, we need something to compare the values with , i.e. a null model .in fact , we propose two null models , one more and one less restricted . we follow the general approach of ref . and sample randomizations of the real - world data set rather than constructing parametric null models . in both these modelswe will keep the sizes , and the same as in the original contact sequences . not only that, we assume the growth of is unrelated to the psychology of the community members , we also keep the time evolution of the same as in the real data . furthermore ,as the community grows with time the communication rate in the data should also be growing . with a parametric null modelthis would be hard to implement , but we just keep the set of times from the real data and the communication rate exactly the same as in the real - world contact sequence .  in the first , less constrained , model we apply the restrictions above .a randomized contact sequence is constructed as follows :  1 .[ enu:1c ] for each vertex add one contact to or from ( with equal probability ) at time .the other vertex of the contact is chosen with uniform randomness among the vertices present in the data at this time .[ enu : rest ] draw times randomly from the set of non - first contacts .( i.e. contacts , such that and . )add contacts between vertices present in the community at these times .step [ enu:1c ] ensures that the number of active members is the same in the randomized sequence as in the real - world data .step [ enu : rest ] makes the set of times almost equal to the empirical set of times .note that , contacts between two vertices and at will give rise to two contacts in the randomized sequence ; so the set of non - first contacts used in step [ enu : rest ] will be slightly ( ) larger than .  as mentioned in the previous sectiona broad activity distribution has been observed in many data sets similar to ours .one may argue that activity is a rather independent trait , little connected to the contact dynamics . whether or not this is true, we construct a null model to test how the reality differs from such a scenario . to generate a random sequence realization for this model we loop over all contacts and replace ( the vertex the message is sent to ) by a random vertex present in the community at time .clearly this procedure conserves .it may result in a few isolated vertices ( vertices of zero out - degree who looses their incoming links ) , and a slightly altered time evolution of ( vertices whose first contact is incoming may appear later in the randomized community than in the real - world data ) . for our data these effects are small , and we assume them to be negligible .values of the linear ( pearson s ) correlation coefficient between attractiveness , activity and participance rates for the real and null model networks can be found in table [ tab : corr ] .first we note that the attractiveness and activity are more strongly correlated in the real community than in both models , and that this holds for all three networks .this means that the correlation between attractiveness and activity is not an artifact of the growth of the community ( by comparison with model 1 ) , neither is it a result of a skewed distribution of a ( hypothetical ) intrinsic activity ( by comparison with model 2 ) . unlike the attractiveness vs. activity correlation, the correlation between activity and participance rate seems to a large extent to be an effect of the growth of the network .we see that model 2 induces a very high correlation between activity and participance rate .model 2 randomizes the recipients but keeps the rest of the communication the same as in the original data .the fact that the users engage in dialogs is an explanation for the lower correlation in the real world data two users sending contacts to only each other , but frequently is active but have low participance ratio .the correlation between the participance rate and attractiveness of the real data is even more different from the null model than the attractiveness - activity correlation .this means that the vertices of highest degree in the social network generated by the communication are also the one that most frequently get incoming new contacts .another interesting observation is that the messages data have higher attractiveness correlations than the guest book and all contacts data .since the messages are not publicly visible one can assume that the real romantic communication takes place here rather than in the guest book writing .this strengthen the conclusion that active users are attractive in internet dating .we note that ( spearman type ) rank correlations are typically a little stronger than the linear correlations , but since they add little new information to the discussion we do not include them in our tables .in this paper we have introduced three measures to characterize users in online communities : attractiveness , activity and participance rate ( measuring how fast a user get to know others in the community ) .while these three measures are sensible in general internet communities , we evaluate them for an internet dating community  . how and why people end up as partners is well studied in an off - line setting  ; attractiveness in terms of body shape  , facial characteristics  and body odor  are all well - known . with internet communities playing an increasingly important role on the dating scene one need to complement the studies of in - real - life attractiveness with investigations of online romantic preferences . in this paperwe find that the attractiveness ( the number of incoming new contacts per time ) is positively correlated with activity ( the number of contacts taken per time ) , and that this correlation is significantly stronger than for our two null models .we find this heightened correlation in the real - world data even stronger in the dataset of e - mail - like messages than in the other data sets ( including messages visible to the whole community ) .since the activity , as we measure it , is rather invisible to other members , this correlation has to be a secondary effect .( writing in guest books increases ones visibility and is a , presumably small , direct effect . )it is natural to assume that the users that send many messages , and thus invest much time into their community , also spend comparatively more time embellishing their homepages .the conclusion is thus that it pays off to spend effort on one s personal presentation .another factor is that highly active users are most likely logged in more often than low - activity users .since currently logged in users are displayed , the increased visibility of frequently logged in members may boost the attractiveness of active users .we also find highly skewed , power - law - like distributions of attractiveness , activity and participance rate .the mechanisms behind electronic communication and offline behavior are presumably very different . nevertheless , similar quantities in off - line relationships , such as the number of partners per time , are also known have this feature  .research on the structure and dynamics of internet communities is still a young field .their statistical advantages ( being closed systems ) compared to e - mail exchange make us anticipate much future work with both data analysis and modeling approaches .d. i. perrett , k. j. lee , i. s. penton - voak , d. r. rowland , s. yoshikawa , d. m. burt , s. p. henzi , d. l. castles , and s. akamatsu .effects of sexual dimorphism on facial attractiveness . , 394:884887 , 1998 .j. r. tyler , d. m. wilkinson , and b. a. huberman .email as spectroscopy : automated discovery of community structure within organizations . in m.huysman , e. wenger , and v. wulf , editors , _ communities and technologies _ , pages 8196 .kluwer , deventer , 2003 .s. whittaker , l. terveen , w. hill , and l. cherny .the dynamics of mass interaction . in s.poltrock and j. grudin , editors , _ proceedings of the 1998 acm conference on computer supported cooperative work _ ,pages 257264 , new york , 1998 .association for computing machinery .

Abstract: datasets of online communication often take the form of contact sequences ordered lists contacts ( where a contact is defined as a triple of a sender , a recipient and a time ) . we propose measures of attractiveness and activity for such data sets and analyze these quantities for anonymized contact sequences from an internet dating community . for this data set the attractiveness and activity measures show broad power - law like distributions . our attractiveness and activity measures are more strongly correlated in the real - world data than in our reference model . effects that indirectly can make active users more attractive are discussed .   internet community ; network dynamics ; online dating


the wta is a computational framework in which a group of recurrent neurons cooperate and compete with each other for activation .the computational power of wta and its function in cortical processing have been studied in detail .various models and hardware implementations of wta have been proposed for both rate and spike based neural networks . in recent past , researchers have looked into the application of stdp learning rule on wta circuits . the performance of competitive spiking neurons trained with stdp has been studied for different types of input such as discrete spike volleys , periodic inputs and inputs with random intervals .  in this paper , for the first time we are proposing a winner - take - all ( wta ) network which uses neurons with nonlinear dendrites ( nnld ) and binary synapses as the basic computational units .this architecture , which we refer to as winner - take - all employing neurons with nonlinear dendrites ( wta - nnld ) , uses a novel branch - specific spike timing dependent plasticity based network rewiring ( stdp - nrw ) learning rule for its training .we have earlier presented a branch - specific stdp rule for batch learning of a supervised classifier constructed of nnlds .the primary differences of our current approach with are :  * we present an unsupervised learning rule for training a wta network . *we propose an online learning scheme where connection modifications occur after presentation of each pattern .  in this articlewe consider spike train inputs with patterns occurring at random order which is the same type presented in .the primary differences between our work and the one proposed in are :  * our wta network is composed of neurons with nonlinear dendrites instead of traditional neurons with no dendrites .* unlike the network proposed in that requires high resolution weights , the proposed network uses low - resolution non - negative integer weights and trains itself through modifying connections of inputs to dendrites .hence , change of the ` morphology ' or structure of the neuron ( in terms of connectivity pattern ) reflects the learning .this results in easier hardware implementation since a low - resolution non - negative integer weight of can be implemented by activating a shared binary synapse times through time multiplexing schemes like address event representation ( aer) . * in , though the neurons were allowed to learn and respond to subpatterns , there was no actual guideline or control parameter to set the number of subpatterns to be learned . herewe utilize the slow time constant of the inhibitory signal to select the number of subpatterns we want to divide a pattern into .  in the following section, we will present a overview of nnld , propose the wta - nnld architecture and stdp - nrw learning rule and show how the inhibitory slow time constant can be used to select subpatterns within a pattern .then we shall provide guidelines on selecting the parameters associated with wta - nnld and stdp - nrw . in section iv we will describe the classification task considered in this article which will be followed by the results .we will also present the robustness of the proposed method to variations of parameters in section v , a quality that is essential for its implementation in low - power , subthreshold neuromorphic designs that are plagued with mismatch .we will conclude the paper by discussing the implications of our work and future directions in the last section .in this section , we shall first present the working principle of a nnld .this will be followed by a description of the wta - nnld architecture and stdp - nrw learning rule .lastly , we will throw some light on the role of inhibitory time constant in balancing the specificity and sensitivity of the network .   the computational model of nnldwas first proposed by mel et al . in .they showed that such neurons have higher storage capacity than their non - dendritic counterparts .they used two such nnlds to construct a supervised classifier and demonstrated its performance in pattern memorization .recently nnld has also been employed to develop computationally powerful rate and spike based supervised classifiers .the structure of nnld is isomorphic to a feedforward spiking neural network with a single layer of hidden neurons and one output neuron .the lumped dendritic nonlinearities are equivalent to the hidden neurons interposed between the input and output layers . however, spiking neurons implement nonlinear thresholding , integration , refractory period etc .hence , it is typically a much larger circuit compared to the square law nonlinearity of a dendrite , which makes the nnld an area efficient architecture ., scaledwidth=50.0% ]  as depicted in fig .[ fig : nnld ] , a nnld consists of dendritic branches having lumped nonlinearities , with each branch containing excitatory synaptic contact points of weight 1 .if we consider a dimensional input pattern , then each synapse is driven by any one of these input dimensions where .we use the leaky integrate - and - fire ( lif ) model to generate output spikes .thus , the neuronal membrane voltage is guided by the following differential equation : where , , and are the membrane voltage , threshold voltage , input current and output spikes of the nnld respectively .let us denote the input spike train arriving at the input line as which is given by :    where is the label of the spike .then , the input current to the neuron can be calculated as : where is the nonlinear activation function of the dendritic branch characterized by , is the output current of the dendrite and is the input current to the dendritic nonlinearity . here, describes the behavior of the dendritic nonlinear function by setting a limit on the minimum number of coactive synapses required to produce a supra - linear response . denotes the post - synaptic current kernel given by : where and are the fast and slow time constants governing the rise and fall times respectively and is a normalizing constant . in this article, we consider low resolution non - negative integer weights associated with the input lines .so means the number of times the input line is connected to the dendritic branch . like our earlier work , we allow multiple connections of one input dimension to a single dendrite but restrict the number of connections per dendrite to by enforcing for each .the output of the nnld is a spike train and can be denoted as where is the spike index .we propose a spike based wta network , depicted in fig .[ fig : wta_model ] , which is composed of such nnlds .each nnld is composed of dendrites , where each dendrite chooses ( repetition allowed ) of the available input lines and connects to synapses having weight 1 .the membrane voltage , threshold voltage and input current of the nnld are denoted by , and respectively and their dynamics is governed by equation [ eq : lif ] . for the nnld , while the pre - synaptic spike - train arriving at the input line is denoted by as before , the emitted output spike train is given by .note that for any applied input pattern , is measured from its beginning .we have modelled the effect of lateral inhibition by providing each nnld with a global inhibitory current signal supplied by a single inhibitory neuron through synapses .the signal is provided by the inhibitory neuron to all the nnld whenever any one of them fires an output spike . is modeled as , when the last post - synaptic spike is produced by the nnld at .the inhibitory post - synaptic kernel , , is given by : where and are the fast and slow time constants dictating the rise and fall times of the inhibitory current respectively and sets its amplitude .since we consider binary synapses with weight or , we do not have the provision to keep real valued weights associated with them .hence , to guide the unsupervised learning , we define a correlation coefficient based fitness value for the synaptic contact point on the dendrite of the nnld of the wta network , as a substitute for its weight . in the proposed algorithm , structural plasticity or connection modifications happen in longer timescales ( at the end of patterns ) which is guided by the fitness function updated by a stdp inspired rule in shorter timescales ( at each pre- and post - synaptic spike ) .the operation of the network and learning process comprises the following steps whenever a pattern is presented :  * is initialized as . *the value of is depressed at pre - synaptic and potentiated at post - synaptic spikes according to the following rule : 1 .depression : if the pre - synaptic spike occurs at the synapse on the dendritic branch of the nnld at time , then the value of at is updated by a quantity given by : where is the post - synaptic trace of the nnld and denotes derivative of the nonlinear function .potentiation : if the nnld of the wta - nnld network fires a post - synaptic spike at time then at  i.e. for each synapse connected to the nnld is updated by given by : where is the pre - synaptic trace of the corresponding input line connected to it . + a pictorial explanation of this update rule of is shown in fig .[ fig : pre_post_pre ] .note that for a square law nonlinearity , and hence can be easily computed in hardware without requiring any extra circuitry to calculate the derivative .* during the presentation of the pattern whenever a spike is produced by any of the excitatory nnlds , the inhibitory neuron sends an inhibitory signal to all the nnlds of the wta .* after the network has been integrated over the current pattern of duration , the synaptic connections of the nnlds which have produced at least one spike are modified . * if we consider that out of nnlds have produced post - synaptic spike / spikes for the current pattern , then the connectivity of the nnld is updated by tagging the synapse ( ) having the lowest value of correlation coefficient at out of the synapses connected to it for possible replacement . * to aid the unsupervised learning process , randomly chosen sets containing of the input dimensions are forced to make silent synapses of weight 1 on the dendritic branch of  .we term these synapses as silent " since they do not contribute to the computation of - so they do not alter the classification when the same pattern set is re - applied .the value of is calculated for synapses in and the synapse having maximum in denoted by  is identified .next , the input line connected to is swapped with the input line connected to .hence , instead of the traditional method of training by changing of high - resolution synaptic weights , our learning rule modifies the connections between the inputs and dendrites based on the fitness values . *all the values are reset to zero and the above mentioned steps are repeated whenever a pattern is presented . here , we define an epoch for class classification as a set of patterns consisting of one pattern from each of the classes- in random order .we define another term as the average of the latencies of the post - synaptic spikes in the network over time period of the last epoch which is given by : where denotes averaging over one epoch .we note the value of for every epoch and the learning is considered to converge when the value of a ` convergence measure ' ( ) based on reaches saturation .we define our ` convergence measure ' in section [ params ] .  )when a post - synaptic spike occurs at the value of increases by . due to the appearance of a pre - synaptic spike at , reduces by as shown in the figure.,scaledwidth=50.0% ]   when a pattern is presented to the wta - nnld and any one of the nnlds produce an output spike , a global inhibition current is injected into all the nnlds .the slow time constant of this signal controls the output firing activity of the wta - nnld .typically , a large value of ( w.r.t to ) is set , and only one nnld produces an output spike i.e. patterns of same class are encoded by a single nnld . the post - synaptic spike latency for a pattern is defined as the time difference between the start of the pattern and the first spike produced by any one of the neurons of wta - nnld . during training of wta - nnld for this case ,different nnlds get locked onto different classes of pattern and the latency gradually decreases until the end of the training .thus , after completion of training , the unique nnlds which have learned different classes of pattern rely only on the first few spikes ( determined by the latency at the end of training ) to predict the pattern s class thereby significantly reducing the prediction time .so , the sensitivity of the network is increased. however , the problems with this approach are :  * the percentage of successful classifications can be less due to the strict requirement of different neurons firing based only on first few spikes of different patterns ( shown in section [ exp_res ] ) .* though the prediction time of a pattern s class is significantly reduced , this method neglects most part of the pattern after the first few spikes which may lead to a lot of false detections .   to respond.,scaledwidth=50.0% ]  we demonstrate the limitation mentioned in the above point by a simple example in fig .[ fig : eg_nsub1 ] .let us consider we are performing class classification and assume that after the training phase is complete , nnld responds to patterns belonging to class 1 .nnld has trained itself to provide an output spike depending on the position of the first few spikes ( red spikes in dashed box of fig .[ fig : eg_nsub1 ] ) of the pattern .it neglects the rest of the pattern while providing a prediction .however , for longer patterns there is a chance that this spike set can occur anywhere inside a random pattern ( not belonging to any class or to another class ) .the same nnld responds to such patterns by producing a post - synaptic spike .thus , we see that though trained wta - nnld is very sensitive in this case , it loses specificity . on the other hand ,if we set a moderate value of , then for a single pattern multiple nnlds are capable of producing output spikes .hence , patterns of the same class are now encoded by a sequence of successive firing of few nnlds where each nnld fires for one subpattern .let be the number of subpatterns that is set by a proper choice of .thus the original case of one nnld firing for each pattern corresponds to . in this article, for a class classification we define a successful trial as one in which ( a ) during the training phase wta - nnld learns different unique representations for patterns of different classes and ( b ) after completion of training and achieving success in ( a ) , the network produces the same representation , when presented with testing patterns corresponding to classes that it had learned during the training phase .when i.e. no pattern subdivisions are made , this unique representation is a different neuron firing for different classes of patterns . when 1 , the unique representation is a different sequence of successive nnlds firing for different classes of patterns .when , 1 , we allow the nnlds to detect subpatterns within patterns .since in this approach the wta - nnld gives weightage to the entire pattern before predicting its class , the number of false detections can be largely reduced .however , this method has a limitation of being less jitter resilient one of the many subpatterns can be easily corrupt by noisy jitters in spike ( shown in section [ exp_res ] ) and fail to produce a unique identifier during testing phase . hence , the choice of and consequently the inhibitory time constant depends on the amount of temporal jitter in the application .the following is an exhaustive list of the parameters used by wta - nnld and stdp - nrw :  1 . : duration of a pattern 2 . : dimension of the input 3 . : number of dendrites per nnld 4 . : number of synapses per dendrite 5 . : number of input dimensions in replacement set 6 . and : slow and fast time constant of excitatory current kernel 7 . : normalization constant of excitatory current kernel 8 . and : slow and fast time constant of inhibitory current kernel 9 . : normalization constant of inhibitory current kernel 10 . : threshold of dendritic nonlinearity 11 . : firing threshold voltage of nnld 12 . : number of nnlds in wta 13 . : number of classes of patterns  we will now provide some guidelines on choosing the key parameters :  [ [ total - number - of - synapses - per - nnld - s ] ] total number of synapses per nnld ( s ) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  the number of synapses allocated to each neuronal cell of wta - nnld are kept as equal to the dimension ( ) of the input patterns .this is done to ensure nnld uses the same amount of synaptic resources as the simplest neuron a perceptron .thus , if the proposed network is comprised of such neuronal cells then the total number of synaptic resources required are .  [ [ number - of - dendrites - per - nnld - m ] ] number of dendrites per nnld ( m ) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  in a measure of the pattern memorization capacity, , of the nnld ( fig.[fig : model_nl ] ) has been provided by counting all possible functions realizable as : where , and are the number of dendrites , number of synapses per dendrites and dimension of the input respectively for this neuronal cell .when a new classification problem is encountered , we first note down the value of , which in turn sets our since we have considered . since , for a fixed all possible values which can take are factors of .we calculate for these values of by equation [ eq : bn ] .the value of for which attains its maxima is set as in our experiment .as an example , we show in fig . [fig : capacity ] the variation of with when .it is evident from the curve that the capacity is maximum when and so in our simulations for classifying dimensional patterns we employ neuronal cells having 25 dendrites .  )is plotted as function of the number of dendrites ( ) for a fixed number of input dimensions ( ) and synapses ( ).,scaledwidth=40.0% ]  [ [ number - of - synapses - per - branch - k ] ] number of synapses per branch ( k ) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  after and have been set , the value of can be computed as .[ [ the - normalization - constant - i_0-slow - tau_s - and - fast - time - constant - tau_f - of - excitatory - psc - kernel ] ] the normalization constant ( ) , slow ( ) and fast time constant ( ) of excitatory psc kernel + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  the fast time constant ( ) and slow time constant ( ) have been defined in section [ sec : stdp - nrw ] . in hardware implementation of a synapse  takes a small positive value and is typically not tuned .the slow time constant , , is responsible for integration across temporally correlated spikes and the performance of the network is dependent on its value .if takes too small a value , then the post synaptic current due to individual spikes dies down rapidly and thus temporal integration of separated inputs does not take place . on the other hand large values of all spikes effectively simultaneous .so , in both extremes the extraction of temporal features from the input pattern is hampered .in we have provided a mathematical formula for calculating , the optimal value of , with respect to the inter spike interval ( isi ) of the input pattern for which optimal performance of the network is obtained .if we are considering dimensional patterns and the mean firing rate of each dimension is , then the mean isi across the entire pattern is given by .we can then set according to the formula :   in our simulations , we keep as . since the weights of all the active synapses are , we set to normalize the amplitude of the psc to be .[ [ threshold - of - nonlinearity - x_thr ] ] threshold of nonlinearity ( ) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  during the training of wta - nnld , the stdp - nrw rule preferably selects those connection topologies where correlated inputs for synaptic connections are connected to the same branch .thus , the lumped dendritic nonlinearity should give a supra - linear output only when correlated input dimensions are connected to the dendrite . to ensure this we keep the value of equal to the average input to the nonlinear function in case of random connections .we create numerous instances of dendrites having synapses and calculate the average input to the nonlinear function , , for the pattern set at hand .then we set the value of as , .[ [ v_thr - of - nnld ] ] of nnld + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  the nnld should provide a post - synaptic spike only when correlated inputs have been connected to its dendrites .we consider a nnld having dendrites and synapses and create numerous instances of random connections to these synapses .we measure the average value of the maximum membrane voltage ( ) produced when this nnld is integrated over the pattern duration for all these instances and set .[ [ the - normalization - constant - i_0inh - slow - tau_sinh - and - fast - tau_finh - time - constant - of - i_inht ] ] the normalization constant ( ) , slow ( ) and fast ( ) time constant of + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  the post - synaptic firing activity of the wta - nnld network is dependent on and . to simulate the hardware scenario we set to a small value given by . to set and , we first excite wta - nnld with epochs of patterns prior to training and calculate the average excitatory current ( ) to the nnlds as : where denotes averaging over epochs .the idea is to generate a which , if provided by at the beginning of a subpattern , decays exponentially to at the end of the subpattern i.e. after time has elapsed .this ensures that once a post - synaptic spike is generated by a nnld in a particular time window , other nnlds are unable to fire during that same time window .assuming , we can derive that the required is implemented by setting as :   note that has an inverse logarithmic relation to .[ [ convergence - measure - cm ] ] convergence measure ( ) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  the formula for calculating , applicable to both and , for detecting the convergence of learning is given by :   note that for , and so it computes the time - to - first spike for patterns averaged over an epoch .for , calculates the average time - to - first spike from the beginning of each subpattern of patterns of an epoch .we consider the learning has converged when the value of saturates .in this section , we will describe the classification task considered in this article . to show how the classification performance generalizes to multi - classwe will consider two , four and six class classification .we will be showing the performance of wta - nnld and stdp - nrw for three values of given by = 1 , 5 and 10 .the benchmark task we have selected to analyze the performance of the proposed method is the spike train classification problem . in the generalized spike train classification problem , arrays of poisson spike trains having frequency and length are present which are labeled as classes 1 to .jittered versions of these templates are created by altering the position of each spike within the templates by a random amount that is randomly drawn from a gaussian distribution with zero mean and standard deviation .the network is trained by these jittered versions of spike trains , and the task is to correctly identify a pattern s class . in this article ,unless otherwise mentioned , we have considered = 100 and poisson spike trains are present in each afferent , = 20 and = 0.5 sec and varied and .inspired by , we also consider the scenario when randomly chosen afferents do not contain any spikes , while the remaining afferents are poisson spike trains .+   in this case we have , so one nnld is capable of firing only once when a pattern is presented .considering = 0 , we have varied the number of nnlds and noted the percentage of successful trials which is depicted in fig . [n_sub_1](a ) .to make the horizontal axis invariant of the number of classes , we have taken as the horizontal axis . from the figure we can conclude that the percentage of successful trials gradually increases with an increase in and finally becomes constant after .thus , unless otherwise mentioned , we will keep when .it can be seen from fig .[ n_sub_1](a ) that the percentage of successful trials can not go beyond 92% , 88% and 82% for two , four and six class classification respectively .  earlier in section [ params ]we have mentioned that learning converges when saturates . for , is the average of the time - to - first spikes for patterns in an epoch . as an example we consider a particular trial of four - class classification and show in fig . [fig : nojit_latency_vs_epoch__classes_experi_1 ] that during training , the latencies of the four nnlds , , , and which uniquely recognize the four class of patterns gradually reduce until reaching a saturation point .moreover , in fig .[ n_sub_1](b ) we show the epochwise evolution of averaged over 50 trials for two - class , four - class and six - class classification .it is evident from the figure that the value of decreases thereby showing that the algorithm is favoring correlated inputs such that the post - synaptic spikes can occur faster and finally saturates after some epochs have passed .we denote the number of epochs taken by the algorithm for saturation of as and note its value for 50 trials . the average value of for 50 trials , , is then computed to be = 149 , 157 and 165 for two - class , four - class and six - class classification respectively .moreover , this phenomenon clearly indicates that while the wta - nnld network is being trained by stdp - nrw learning rule , unique nnlds which have locked onto the different classes of pattern , are trying to recognize the start of repeating patterns for different classes .[ n_sub_1](b ) also suggests that after the training has stopped , these unique nnlds , instead of looking at the whole pattern of duration , can now look only at the starting , and of the patterns for 2 , 4 and 6 respectively to predict its class .let us now consider the effect of jitter and we show in fig .[ n_sub_1](c ) the performance of the proposed method when the intensity of jitter is varied .next , we look into the performance of the proposed method when patterns with 50% empty afferents are considered .[ nsub_more_1](a ) depicts the results obtained by our network for this case with varying amounts of jitter .next , we consider i.e. we divide each pattern into 5 subpatterns by setting and as per equation [ tau_inh_i0_inh ] . for class classification ,the maximum number of subpatterns can be so we set in this case i.e. we keep = 10 , 20 and 30 for two , four and six class classification respectively . considering , the evolution of with epochs for two , four and six class classification averaged over 50 trials is shown in fig .[ nsub_more_1](b ) .moreover , the value of ( averaged over 50 trials ) is found out to be 210 , 221 and 230 when = 2 , 4 and 6 respectively .unlike case 1 , here we consider the response to a pattern as a unique firing sequence of few nnlds . as an example, we consider a particular trial of four class classification and look into the first and last 3 epochs during its training .it is evident from fig .[ first3_last3_epochs ] that during the first 3 epochs , wta - nnld produces arbitrary sequences of spikes .however , it can be seen that after the training of the network is complete , wta - nnld produces different firing sequences for different patterns while producing the same sequence when same patterns are encountered .wta - nnld trained by this method produces a 100% accuracy in recognizing different patterns by producing its unique firing sequence for two , four and six class classification . the performance of the network with varying intensity of jitter is provided in fig .[ n_sub_1](c ) ( spikes present in all afferents ) and fig .[ nsub_more_1](a ) ( spikes present in only half of the afferents ) which depict that the = 5 case is less jitter resilient than the = 1 case .we further increase the resolution of pattern subdivision by decreasing .we consider = 10 and following the principle of = 5 , the number of nnlds employed for are 20 , 40 and 60 for two - class , four - class and six - class classification .this approach also provides 100% accuracy in providing a unique sequence of firing whenever a particular pattern is encountered when .however , the performance of the network falls rapidly with the increase of as shown in fig . [ n_sub_1](c ) and fig . [ nsub_more_1](a ) .we also show the evolution of with the epochs in fig . [ nsub_more_1](b ) .furthermore , the number of epochs needed for convergence of in this case is much more than the previous cases as depicted in fig . [ nsub_more_1](c ) .we conclude that dividing a pattern into too many subpatterns hampers the network performance .next we delve a bit further and show the statistics of causes for the failure of the system in producing successful trials .a trial may fail if either condition ( a ) or ( b ) ( described in section [ spec_sens ] ) is not satisfied .we denote the failure of condition ( a ) as .note that for a trial , condition ( b ) can fail if a pattern is misclassified as a pattern of another class ( denoted as ) or as a random pattern ( denoted as ) .table [ fail_ana ] shows the statistics of failed trials for 1 , 5 and 10 when .note that is high for 1 since a unique nnld might lock onto multiple classes of patterns . reduces for 5 and increases again for 10 since sometimes the network fails to produce a unique 10 indices long representation for all patterns of the same class .moreover , we test our network with random patterns and note the cases where a learnt unique representation is produced for a random input pattern i.e. a false positive error occurs .the percentage of false positive errors produced for 1 , 5 and 10 when are 8% , 0% and 0% respectively .note that no false positive errors occur for and 10 since it is highly unlikely for a random pattern to make a sequence of neuron firing same as any of the learnt representation ..analysis of failure statistics [ cols="^,^,^,^,^",options="header " , ]   [ fail_ana ]in this section we analyze the stability of our algorithm to hardware nonidealities by incorporating the statistical variations of the key subcircuits .the primary subcircuits needed to implement our architecture are synapse , dendritic squaring block , neuron and calculator . while the variabilities of the synapse circuit are modeled by mismatch in the amplitude ( ) and time constant ( ) of the synaptic kernel function , the variabilities of the squaring block are captured by a multiplicative constant ( ) .we do not consider the variation of inhibitory current kernel since it is global and only a single instance is present in the architecture . in our earlier work , we proposed the circuits for implementing the synapse and squaring block of nnld and performed monte carlo analysis to find their variabilities .we presented that the of , and for the worst case scenario are 13% , 10.1% and 18% respectively .the mismatch of the lif neuron circuit proposed in was captured by variations in the firing threshold , the of which was computed to be 12.5% .lastly , the nonidealities of the calculator block , described in , are modeled as a multiplicative constant ( ) .monte carlo analysis of the calculator block revealed that its for the worst case is 18% .[ unsup_nonid ] shows the performance of the proposed method when these nonidealities are included in the model for and keeping 0.1 .the bars corresponding to , , , and denote the performance degradation when statistical variations of , , , and are included individually .the results of fig .[ unsup_nonid](a ) and fig .[ unsup_nonid](b ) depict that the performance of the proposed algorithm is most affected by and and least by . finally , to mimic the proper hardware scenario we consider the simultaneous implementations of all the nonidealities , which is marked by ( ... ) .the ( ... ) bars show that there is an 8% and 6% decrease in performance for and respectively .we have proposed a new neuro - inspired winner - take - all architecture ( wta - nnld ) and a stdp inspired dendrite specific structural plasticity based learning rule ( stdp - nrw ) for its training .motivated by recent biological evidences and models suggesting nonlinear processing properties of neuronal dendrites we employ neurons with nonlinear dendrites to construct our wta architecture .moreover , we consider binary synapses instead of high resolution synaptic weights . thus our learning rule , instead of weight updates , trains the network by modification of the connections between input and synapses .we have also provided a method by which the number of subpatterns per pattern learned by wta - nnld can be controlled .wta - nnld encodes patterns of different classes by either activity of distinct nnlds or by a distinct sequence of nnld firings . to demonstrate the performance of wta - nnld and stdp - nrw ,we have considered two , four and six class classification of 100 dimensional poisson spike trains .we can conclude from the result that the slow time constant of inhibitory signal ( ) can be properly set to obtain a tradeoff between specificity and sensitivity of the network .our immediate future work will include studying the effects of connection changes after the network gets integrated over multiple patterns .this will reduce the number of required computations . on another note, we will look into the classification of spike based mnist datasets by our method .our network can be immediately scaled to learn the digits of mnist dataset , the only requirement being additional simulation time and computational memory compared to the tasks considered in this article .furthermore , to achieve invariance to scaling and rotation during image classification , we will be constructing nnld based convolutional neural networks trained by structural plasticity .we will also implement the proposed network in hardware and apply it for real time online unsupervised classification of spatio - temporal spike trains .t. serrano and b. l. barranco , `` a modular current - mode high - precision winner - take - all circuit , '' in _ proceedings of the ieee international symposium on circuits and systems ( iscas ) _ , may 1994 , vol . 5 , pp . 557560 .a. delorme , l. perrinet , and s. j. thorpe , `` networks of integrate - and - fire neurons using rank order coding b : spike timing dependent plasticity and emergence of orientation selectivity , '' , vol .40 , pp . 539545 , 2001 .s. brink , s. nease , p. hasler , s. ramakrishnan , r. wunderlich , a. basu , and b. degnan , `` a learning - enabled neuron array ic based upon transistor channel models of biological phenomena , '' , vol .1 , pp . 7181 , feb . 2013 .s. hussain , r. gopalakrishnan , a. basu , and s. c. liu , `` morphological learning : increased memory capacity of neuromorphic systems with binary synapses exploiting aer based reconfiguration , '' in _ ieee intl .joint conference on neural networks ( ijcnn ) _ , aug .2013 , pp . 1 7 .s. roy , a. basu , and s. hussain , `` hardware efficient , neuromorphic dendritically enhanced readout for liquid state machines , '' in _ proceedings of the ieee biomedical circuits and systems ( biocas ) _ , nov 2013 , pp . 302305 .jadi , b.f .behabadi , a. poleg - polsky , j. schiller , and b.w .mel , `` an augmented two - layer model captures nonlinear analog spatial integration effects in pyramidal neuron dendrites , '' , vol .782798 , may 2014 .a. banerjee , a. bhaduri , s. roy , s. kar , and a. basu , `` a current - mode spiking neural classifier with lumped dendritic nonlinearity , '' in _ proceedings of the ieee international sympoisum on circuits and systems ( iscas ) _, 2015 , number may .s. roy , s.k .kar , and a. basu , `` architectural exploration for on - chip , online learning in spiking neural networks , '' in _14th international symposium on integrated circuits ( isic ) _ , dec 2014 , pp . 128131 .

Abstract: in this article , we propose a novel winner - take - all ( wta ) architecture employing neurons with nonlinear dendrites and an online unsupervised structural plasticity rule for training it . further , to aid hardware implementations , our network employs only binary synapses . the proposed learning rule is inspired by spike time dependent plasticity ( stdp ) but differs for each dendrite based on its activation level . it trains the wta network through formation and elimination of connections between inputs and synapses . to demonstrate the performance of the proposed network and learning rule , we employ it to solve two , four and six class classification of random poisson spike time inputs . the results indicate that by proper tuning of the inhibitory time constant of the wta , a trade - off between specificity and sensitivity of the network can be achieved . we use the inhibitory time constant to set the number of subpatterns per pattern we want to detect . we show that while the percentage of successful trials are 92% , 88% and 82% for two , four and six class classification when no pattern subdivisions are made , it increases to 100% when each pattern is subdivided into 5 or 10 subpatterns . however , the former scenario of no pattern subdivision is more jitter resilient than the later ones .