magic squares , being a recreation mathematical topic in nature , have been the subject of entertainment and interest to many mathematicians and math - enthusiasts alike for hundreds of years .normal _ magic square is an square matrix whose entries are distinctly the integers , such that each row , column , and major and minor traces sum to one constant .magic squares of order 3 and 4 ( and even higher orders ) have appeared in paintings , literature , and artifacts dated as far back as 650 bc .one of the first magic squares found in ancient chinese literature is the * lo shu square * , which is told to have been painted on the shell of a sea turtle :    one of the most notable magic squares in western civilization is the order 4 magic square in albrecht durer s engraving _melencolia i _ : the durer magic square has many fascinating properties .aside the rows and columns ( and major / minor diagonal ) summing to the magic number 34 , the sum can also be found in each 2 by 2 quadrant ,  the center 2 by 2 square , the four outer corner squares , and the four outer corner squares of each 3 by 3 subsquare .furthermore , the sum can be found clockwise from the corners ( ) and counter - clockwise ( ) .what s most fascinating about the durer square is the bottom row : 1514 was the year the painting was made , and 4 and 1 correspond to the letters d and a , the initials of albrecht durer .  today , we now know that there are exactly 7,040 fourth order normal magic squares and 2,202,441,792 fifth order normal magic squares in existence .what s most intriguing is that , like most other famous mathematicians of the time in our past , results were found without the modern - day computing and research power we have access to today .algorithms were developed to create magic squares of _ any _ size , without ever knowing how many there really are .this paper seeks to unveil that question : how to develop an algorithm that creates magic squares while systemically showing the methods it uses .for any normal magic square of order , by careful algebra one can show that which , for the case of is the number . in a 1917 publication of _ amusements in mathematics _ , henry dudeney , an english mathematician specializing in logic puzzles and mathematical games , used this important fact and found a way to classify all 7040 fourth order normal magic squares into 12 distinct configurations ( coincidentally named types i - xii ) .his classification was developed by considering the fact that the numbers from contain 8 pairs of numbers that sum to , and diagrams can be drawn for each magic square to show these pairings ( fig 1 . )    these 12 classifications are independent of orientation , as many of the diagrams are not symmetric about some axis of reflection or rotation . given this classification scheme , dudeney later showed precisely how many of the 7040 magic squares were of each type  curiously , one may note that type i - iii have an equivalent number of magic squares , as do type vii - x and type xi - xii .looking back to fig .1 , it also seems that these groups have similar structures in the configurations of half - sums . by 1948 , in a paper _ determinants of fourth order magic squares _ published in the american mathematical monthly , c.w .trigg proved that these similarities between certain dudeney types existed by calculating their determinants .trigg categorized the 12 dudeney types into 4 distinct groups ( unexcitingly named type a - d ) .indeed , type a was the set of type i - iii , type b the set of type iv - vi , type c the set of type vii - x , and type d the set of type xi - xii .furthermore , since the magic squares in these groups shared determinants , trigg showed that certain row and column operations could be performed on magic squares preserving the determinant , but reordering the configuration of half - sums within that group .for instance , by interchanging the second and third rows , then interchanging the second and third columns of a type ii magic square , the result would be a type i magic square ( both within the type a group ) .alternatively , interchanging the third and fourth rows , then interchanging the third and fourth rows of a type iii magic square results in a type i magic square as well .further research by peter staab in 2010 investigated the trigg groups to find if additional operations existed within the groups to reconfigure one dudeney type magic square to another , and if operations existed which did not affect the configuration of the square .interestingly , he noted that type vi needed to be split within the trigg type b into two distinct subsets : type vi and type vi .this was due to some of the type vi magic squares having the additional property that a broken diagonal also summed to .after this reclassification , he found the following transformation groups existed for the trigg classification ( fig 2 . ) :   : :  indentities and even bisymmetric transformations : :  semisymmetric transformations : :  broken - diagonal ( translation ) transformations : :  odd bisymmetric and rotational transformations : :  identity transformations : :  odd bisymmetric transformations : :  even bisymmetric transformations : :  rotational transformations  lastly , although tangential to the purpose of this paper , it is worth mentioning one key result from 1998 . k. pinn and c. wieczerkowski approximated that , through the use of parallel tempering monte carlo simulations , the total number of sixth order normal magic squares is .implications from these findings suggest that modern day computing power may still not be fast enough to work in the space of sixth order magic squares .given that much work has been done in categorizing the fourth order magic squares by their symmetries and pairwise similarities , the next logical step would be to investigate these symmetry groups to conclude which of the 7040 magic squares are necessary to _ generate _ the fourth order space .we make the following definitions : * def : * _ an * permutation matrix * is a square matrix whose entries are such that is in each column and each row of the matrix exactly once . _a permutation matrix acting on a matrix of similar dimension essentially reorders either the rows or columns of that matrix dependent on whether it is a left- or right - multiplication .for example , consider the following matrix multiplication : \left [ \begin{array}{ccc } 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{array } \right ] = \left [ \begin{array}{ccc } 1 & 2 & 3 \\ 7 & 8 & 9 \\ 4 & 5 & 6 \end{array } \right]\ ] ] note that the second and third rows of this matrix are swapped after being multiplied by the permutation matrix on the left .thus permutes rows and permutes columns , while permutes both rows and columns . * def : * _ the * symmetry group * of a set of magic squares is the set of all pairs of permutation matrices in such that for any magic square in the set , and is also a magic square and in the set . often times this will be called the * transformation group * of . _* def : * _ two magic squares and are * symmetric * if and only if can be written as or for a pair of permutation matrices ._ as an example , in * fig .2 * the set can be thought of as trigg s type a , and the transformation group of that set is any pair of permutation matrices from the set , or .clearly any magic square in that set will be again be a magic square in the set if any of the former transformations are applied .* def : * _ the * order * of a transformation group of ( denoted as ) is the cardinality of the set , i.e. , ._ * def : * the choice of the word generator in this case is not used in the traditional sense for algebra . a set of generators of a group of magic squares is simply the set of magic squares that are pairwise asymmetric and together generate the entire set using the transformation group of . with these definitions in place , we are ready to approach the question to be answered : * _ of the fourth order normal magic squares , how many unique generators are necessary and sufficient to generate the entire space , and what are the associated transformation groups for these generators ? _ *  as mentioned in the previous section , progress has been made towards the answer ; however , the lower bound for generators itself is unconfirmed and the categorization of transformation groups is incomplete . successfully determining a lower bound and completing the transformation groups would provide for two key results :  1 .  that there exists a set of magic squares for the n - th order that describe the entire space and 2 .  that this space can be described by a categorical enumeration of transformation groups imposed on the spacesince the topology of matrices is well - defined and inherit many nice algebraic properties , the above results would hold for the space of magic squares of any order .this suggests that such a finding would be a methodical approach towards identifying the properties of higher order magic squares with relative ease .   as mentioned previously , a normal magic square of the fourth order has the following properties :  * * each distinct * * each row , column , and trace sum to   the last two properties will be investigated first .let s consider the generic matrix from above that inherits these properties . we may notice that this produces a total of 10 linear constraints , namely  1 . 2 . 3 . 4 . 5 . 6 . 7 . 8 . 9 . 10 . .however , the solution to the system of simultaneous linear equations above finds that of the 16 variables in the matrix , only 7 are independent while the remaining 9 are necessarily defined by their dependence .that is to say , a fourth order normal magic square can be uniquely defined by only 7 entries , thus lowering the dimensionality of the problem greatly .for example , assume the entries are independent ( the choices for these being independent will be examined later ) , then                   .this does not necessarily suffice in showing that arbitrary numbers chosen for the entries will guarantee a normal magic square .it is certainly true that the constraints above will hold so that each row , column , and diagonal sum to 34 , but often times ( at least with an arbitrary arrangement ) , entries may be duplicated , and others could be negative , 0 , or larger than 16 .however , if we condition the above constraints using the first two properties from above , we yield an important restriction on the arrangement of numbers for the independent cells : they must be configured in such a way as to obey the laws that every cell must contain a distinct number from the set .determining the arrangements which do not follow this law requires an algorithmic approach , due to having no prior knowledge of which arrangements yield a normal magic square . because we are tasked with many constraints , of which require very specific parameters ( each entry being distinct ) , a backtracking algorithm that used constraint propagation seemed to be the most logical implementation , considering the analysis was focused on creating an efficient , dynamic system .furthermore , backtracking has the property of quickly withdrawing from partial solutions which are guaranteed to not resolve as a full solution for instance , when one of the constraints wo nt be met since the number necessary to be assigned has already previously been assigned .these assignments are carried out by the constraint propagation , which accounts for all properties listed in the previous section .the algorithm begins by placing arbitrary numbers along the top row into slots .the numbers in slots are considered `` free '' since they do not propagate any constraints , however , after and have been assigned , must be assigned in such a way that : is a valid number ( 1 to 16 ) that has not been propagated or previously assigned .for instance , if then the assignment would be invalid , since which is previously assigned to .    should a valid assignment occur for , then the number resulting from be removed from the list of available terms for the backtracking algorithm to test .this is necessary , as without it , deeper iterations would test this number as an assignment despite it being necessarily assigned prior to .after computing the first row , the algorithm computes the first column , having already registered the assignment for .the choice for this is not so trivial : the cells are also considered `` free '' since they will not propagate any constraints . because of this , `` free '' cells check for every valid assignment from the list of available numbers .the goal would then be to avoid `` free '' cells until absolutely necessary .assigning the first column requires only one `` free '' cell , which in turn requires much less complexity .  ]  after completing the first column , the last and final step of the backtracking algorithm is to compute the assignment for , as they are the last two remaining cells necessary to be assigned to thusly define the square unique to these 7 numbers . from above, is a free cell , so values are arbitrarily assigned from the available list .the cell requires the most effort : since this is the final cell , the number assigned here must be such that all constraints that contain are satisfied , and the dependence values are available . if either of these two assumptions fail , then can not be assigned and the backtracking algorithm withdraws from this arrangement of numbers .    once the algorithm successfully generates a normal magic square using the independent 7 numbers for the cells ( fig .5 ) , a simple supervised machine learning technique is employed to find the pairs of 17 ( dudeney s classification scheme ) in the magic square that can uniquely classify it .this is done by creating the classification scheme prior to using the diagram in fig . 1 .  ]  then , using the 7 numbers that define the magic square , the program naively iterates across the set to find the pairs .this is surprisingly efficient , since it does not have to iterate across all 16 numbers to find pairs , and the classification scheme guarantees that only two pairs are necessary to determine the dudeney type .    after each squareis generated and classified , the next step was to classify each square , as an extension , by their transformation groups .the properties necessary for this included their innate symmetries ( such as how many broken diagonals also added to ) , which trigg group they were in , and also by using previous research of the permutation sets staab had proven .finally , an arbitrary magic square was chosen from each transformation group within the trigg type , and had applied to it every set of permutation matrices defined on that magic square .as the permutations were applied , magic squares that were symmetric were deleted from the group , until the transformation group had been exhausted .if more than one magic square remained in the group , the process was repeated with another magic square from the group ( which clearly defined another subset of magic squares which it generates ) until the minimum was reached .an example of this transformation group search can be seen for the third - order case in fig .8 , 9 , and 10 .if we consider the dihedral group ( rotations , reflections , transpositions ) as the transformation group for this set , then you can see that as the candidate square rotates about , transposes , and reflects about each axis , it becomes symmetric with every other magic square in the set , thus deleting them from the group .note that since there are only 8 normal magic squares of order 3 , that the candidate magic square is indeed the sole generator of the entire group .even though there are 7040 magic squares , results found that only 95 magic squares are necessary and sufficient to generate the entire space of fourth order .what was immediately perplexing about this number was how it was distributed among the different trigg types .for the type b and type c magic squares , the resultant number of generators for the group did not evenly divide the enumeration of the magic squares classified as that type ; the expectation was that a transformation group would provide a reduction of the set by some factor , but evidence suggested that additional information was necessary .the magic squares that exist in this group , dudeney types i - iii , have the additional property that each broken diagonal also sums to . because of this property, many symmetries are introduced into this group .analysis of the results found that , from the 1152 magic squares classified as type a , only 3 generators are necessary to generate the entire group ( fig .11 ) .thus each generator from this set has a transformation group of order 384 .furthermore , we can claim that the entire type a space is uniquely defined by these 3 generators and the transformation group of order 384 .one observation worth mentioning is how similar each of the generators are ; only 8 of 16 cells differ between each generator , and yet there is no arrangement of permutations in the confines of the space to transform one to the other .analysis of the dudeney types iv - vi provided the most interesting results , primarily in the fact that so little was previously understood .it was mentioned earlier that type vi had the additional property that one of its broken diagonals also summed to .the assumption , then , would be that this group would undoubtedly have generators belonging to two separate transformation groups .    thiswas found to be not entirely true ( fig .in fact , hidden inside the group of 3968 magic squares classified as type b was a total of 46 generators , each belonging to one of 4 different transformation groups .the transformation groups were found to have orders 192 , 96 , 64 , and 32 , with generators 12 , 4 , 10 , and 20 respectively .unlike type a , type b space could not be uniquely defined by its generators and a transformation group ; however , if one were to further reduce type b into 4 pairwise disjoint subsets ( aptly named b-1 , b-2 , b-3 , and b-4 ) containing only the generators of a particular transformation group , then we can indeed uniquely define each _ subgroup _ by its generators and the transformation group .while analysis of the trigg type b group provided intriguing results , trigg type c dudeney types vii - x had perhaps the most surprising result of all four types . given that there were 1792 magic squares in type c evenly distributed among all four dudeney types , the assumption would be that the behavior of this group was much like type a.    in fact , it was found that there were 44 generators for this group , each belonging to one of 2 different transformation groups .the transformation groups were found to have orders 64 and 32 , with generators 12 and 32 , respectively ( fig .13 ) . again , because of this result , the justification that type c could be uniquely defined by a set of generators and a transformation group was invalid . by imitating the revision for type b and splitting the type c group into two disjoint subsets ( c-1 , c-2 ) categorized by separate generators and their transformation sets , we again arrive at a distinction of uniquely defining a group of magic squares based on those generators .   with the small amount of magic squares classified dudeney type xi - xii in this group ( 128 ) ,the results were expectantly underwhelming and categorically likened to type a. the exception in this case is that trigg type d magic squares lack many symmetries which would provide for a high - ordered transformation group ( fig .1 ) .it was found that only 2 generators were necessary to generate the space of type d magic squares , and each generator had a transformation group of order 64 ( fig .much like type a , without the need to reclassify the group , we can uniquely define the type d magic squares by these two generators and the corresponding transformation group of order 64 .another observation worth mentioning related to the similarity of these generators : there s practically none , and yet due to the lack of symmetry within the group , no arrangement of permutations in the confines of this space generate one from the other .the results above motivate a theoretical discussion on the potential of the methodology scaled to higher orders .firstly , it was found that very minimal information was necessary to completely describe the entire space of fourth order magic squares . from the previous sections ,it was noted that the problem in itself could be lower - dimensionalized to a basis of only 7 out of the 16 numbers that define a magic square .further , only 95 generators are sufficient to generate the 7040 magic squares for order four .this implies , that given the knowledge of trigg classification and transformation groups , less than 700 bytes of information ( and assuming each number requires one byte of storage ) is required .since we know _ some _ properties about higher order magic squares , such as transformation groups of odd order have the same behavior as transformation groups of one less order ( even ) , we can hypothesize that this scheme is adaptable at least for the fifth order case .this is further emphasized by the fact that the methods used in this analysis were efficient and easy to implement . with careful linear algebra, one can show that the fifth order magic squares can be lower - dimensionalized to a basis of only 13 out of 25 numbers that define that magic square .the implications from this suggest that a backtracking algorithm with constraint propagation may still be suitable and efficient enough with modern computing power to produce meaningful results .lastly , there is consideration to be had to appropriate a better approximation ( or more concrete bound ) on the total number of sixth order magic squares . while a previously mentioned result from 1998 showed an approximation for this number , certain applications from this analysis could perhaps be abstracted to find the theoretical transformation groups of the set , and determine from those what the generators of those sets would look like . while there is no information currently in literature suggesting any formulation of `` trigg types '' for these magic squares, one could again theorize their configurations based on small simulations of this method and applying transformation groups to this small subset .the finding of 95 generators for the entire set of 7040 fourth - order normal magic squares improves the previous lower bound of 220 found in literature[2 ] . furthermore , while many of the transformation groups were correctly categorized by staab [ 5 ] , results show that previously unknown transformation groups exist in the space , and that the trigg classification requires revisions to reflect the true underlying structure . with these two results combined , by virtue of the methods used in this analysis , such a construction can be developed with ease for fifth order magic squares which has previously been limited by inefficient methods . with this paper as reference ,a curious individual in the field of recreational mathematics may use these tools to yield many fascinating results about fifth and sixth order normal magic squares .

Abstract: in recreational mathematics , a _ normal _ magic square is an square matrix whose entries are distinctly the integers , such that each row , column , and major and minor traces sum to one constant . it has been proven that there are 7,040 fourth order normal magic squares and 2,202,441,792 fifth order normal magic squares [ 4 ] , with higher orders unconfirmed [ 3 ] . previous work related to fourth order normal squares has shown that symmetries such as the dihedral group exist [ 5 ] and that ( under certain conditions ) normal magic squares can be categorized into four distinct subsets . [ 2][6 ] with the implementation of an efficient backtracking algorithm along with supervised machine learning techniques for classification , it will be shown that the entire set of fourth order normal magic squares can be generated by expanding the symmetry groups of 95 asymmetric parents . discussion will suggest that methods employed in this project could similarly apply to higher orders .


neural text embedding models have recently gained significant popularity for both natural language processing ( nlp ) and information retrieval ( ir ) tasks . in ir ,a significant number of these works have focused on word embeddings and modelling short - text similarities . in traditional web search ,the query consists of only few terms but the body text of the documents typically has at least few hundred sentences . in the absence of click information , such as for newly - published or infrequently - visited documents , the body text can be a useful signal to determine the relevance of the document for the query .therefore , extending existing neural text representation learning approaches to long body text for document ranking is an important challenge in ir .however , as was noted during a recent workshop , in spite of the recent surge in interests towards applying deep neural network ( dnn ) models for retrieval , their success on ad - hoc retrieval tasks has been rather limited .some recent papers report worse performance of neural embedding models when compared to traditional term - based approaches , such as bm25 .traditional ir approaches consider terms as discrete entities .the relevance of the document to the query is estimated based on , amongst other factors , the number of matches of query terms in the document , the parts of the document in which the matches occur , and the proximity between the matches .in contrast , latent semantic analysis ( lsa ) , probabilistic latent semantic analysis ( plsa ) and latent dirichlet allocation ( lda ) learn low - dimensional vector representations of terms , and match the query against the document in the latent semantic space .retrieval models can therefore be classified based on what representations of text they employ at the point of matching the query against the document . at the point of match , if each term is represented by a unique identifiers ( _ local _ representation ) then the query - document relevance is a function of the pattern of occurrences of the exact query terms in the document .however , if the query and the document text is first projected into a continuous latent space , then it is their distributed representations that are compared . along these lines , classify recent dnn models for short - text matching as either _interaction_-focused or _representation_-focused .they claim that ir tasks are different from nlp tasks , and that it is more important to focus on exact matching for the former and on learning text embeddings for the latter . , on the other hand , claim that models that compare the query and the document in the latent semantic space capture a different sense of relevance than models that focus on exact term matches , and therefore the combination of the two is more favorable .our work is motivated by the latter intuition that it is important to match the query and the document using both local and distributed representations of text .we propose a novel ranking model comprised of two separate dnns that model query - document relevance using local and distributed representations , respectively .the two dnns , referred to henceforth as the _ local model _ and the _ distributed model _, are jointly trained as part of a single neural network , that we name as a _ duet _ architecture because the two networks co - operate to achieve a common goal .figure [ fig : wordimportance ] demonstrates how each subnetwork models the same document given a fixed query .while the local model captures properties like exact match position and proximity , the distributed model detects property synonyms ( e.g. ` obama ' ) , related terms ( e.g. ` federal ' ) , and even wellformedness of content ( e.g. ` the ' , ` of ' ) .  in this paper, we show that the duet of the two dnns not only outperforms the individual local and distributed models , but also demonstrate large improvements over traditional baselines and other recently proposed models based on dnns on the document ranking task .unlike many other work , our model significantly outperforms classic ir approaches by using a dnn to learn text representation .dnns are known for requiring significant training data , and most of the state - of - the - art performances achieved by these deep models are in areas where large scale corpora are available for training .some of the lack of positive results from neural models in the area of ad - hoc retrieval is likely due to the scarce public availability of large quantity of training data necessary to learn effective representations of text . in section [ sec : discussion ], we will present some analysis on the effect of training data on the performance of these dnn models .in particular , we found that unsurprisingly the performance of the distributed model improves drastically in the presence of more data .unlike some previous work that train on clickthrough data with randomly sampled documents as negative examples , we train our model on human - judged labels . our candidate set for every queryconsists of documents that were retrieved by the commercial search engine bing , and then labelled by crowdsourced judges .we found that training with the documents that were rated irrelevant by the human judges as the negative examples is more effective than randomly sampling them from the document corpus .to summarize , the key contributions of this work are :  1 .we propose a novel duet architecture for a model that jointly learns two deep neural networks focused on matching using local and distributed representations of text , respectively .we demonstrate that this architecture out - performs state - of - the - art neural and traditional non - neural baselines .we demonstrate that training with documents judged as irrelevant as the negative examples is more effective than randomly sampling them from the corpus .before describing our ranking model , we first present three properties found across most effective retrieval systems .we will then operationalize these in our architecture in section [ sec : model ] .first , _ exact term matches _ between the query and the document are fundamental to all information retrieval models .traditional ir models such as bm25 are based on counts of exact matches of query terms in the document text .they can be employed with minimal ( or no ) need for training data , sometimes directly on new tasks or corpora .exact matching can be particularly important when the query terms are new or rare .for example , if new documents appear on the web with the television model number ` sc32mn17 ' then bm25 can immediately retrieve these pages containing precisely that model number without adjusting any parameters of the ranking model .a good ranking model needs to take advantage of exact matches to perform reliably on fresh and rare queries .second , _ match positions _ of query terms in the document not only reflect where potentially relevant parts of the document are localized ( e.g. title , first paragraph , closing paragraph ) but also how clustered individual query term matches are with each other .figure [ fig : interactionmatrix ] shows the position of matches on two different queries and a sample of relevant and non - relevant documents . in the first query , we see that query term matches in the relevant document are much more clustered than in the non - relevant documents .we observe this behavior in the second query but also notice that the clustered matches are localized near the beginning of the relevant document .match proximity serves as a foundation for effective methods such as sequential dependence models .finally , _ inexact term matches _ between the query and the document refer to techniques for addressing the vocabulary mismatch problem .the main disadvantage of term matching is that related terms are ignored , so when ranking for the query ` australia ' then only the term frequency of ` australia ' is considered , even though counting terms like ` sydney ' and ` koala ' can be good positive evidence . pointed out that under the probabilistic model of ir there is , in fact , no good justification for ignoring the non - matching terms in the document .furthermore , demonstrated that a distributed representation based retrieval model that considered _ all _ document terms is able to better distinguish between a passage that is truly relevant to the query `` cambridge '' from a passage on a different topic ( e.g. , giraffes ) with artificially injected occurrences of the term `` cambridge '' .any ir model that considers the distribution of non - matching terms is therefore likely to benefit from this additional evidence , and be able to tell `` cambridge '' apart from `` an african even - toed ungulate mammal '' .  in practice , the most effective techniques leverage combinations of these techniques .dependence models combine exact matching with proximity .lda - based document models combine exact matching with inexact matching .query hypergraphs capture all three .our method also combines these techniques but , unlike prior work , jointly learns all free parameters of the different components .figure [ fig : architecture ] provides a detailed schematic view of the duet architecture .the distributed model projects the query and the document text into an embedding space before matching , while the local model operates over an interaction matrix comparing every query term to every document term .the final score under the duet setup is the sum of scores from the local and the distributed networks , where both the query and the document are considered as ordered list of terms ,  ] .each query term and document term is a vector where is the input representation of the text ( e.g. the number of terms in the vocabulary for the local model ) .we fix the length of the inputs across all queries and documents such that we consider only the first 10 terms in the query and the first 1000 in the document . if the either is shorter than the target dimension , the input vectors are padded with zeros .the truncation of the document body text to the first 1000 terms is performed only for our model variants .for all the neural and non - neural baseline models we consider the full body text .the local model estimates document relevance based on patterns of exact matches of query terms in the document . to this end , each term is represented by its one - hot encoding in a -dimensional space , where is the size of the vocabulary .the model then generates the binary matrix , capturing every exact match ( and position ) of query terms in the document .this interaction matrix , without the zero - padding , is analogous to the visual representation of term matches in figure [ fig : interactionmatrix ] , and therefore captures both exact term matches and match positions .it is also similar to the indicator matching matrix proposed previously by .while interaction matrix perfectly captures every query term match in the document , it does not retain information about the actual terms themselves .therefore , the local model can not learn term - specific properties from the training corpus , nor model interactions between dissimilar terms .the interaction matrix is first passed through a convolutional layer with filters , a kernel size of , and a stride of .the output corresponding to the convolutional window over is a function of the match between the term against the whole document , where is row of , is performed elementwise , and the matrix contains the learnable parameters of the convolutional layer . the output of the convolutional layer is a matrix of dimension .we use a filter size ( ) of 300 for all the evaluations reported in this paper .the output of the convolutional layer is then passed through two fully - connected layers , a dropout layer , and a final fully - connected layer that produces a single real - valued output .all the nodes in the local model uses the hyperbolic tangent function for non - linearity .the distributed model learns dense lower - dimensional vector representations of the query and the document text , and then computes the positional similarity between them in the learnt embedding space . instead of a one - hot encoding of terms , as in the local model, we use a character -graph based representation of each term in the query and document .our -graph based input encoding is motivated by the trigraph encoding proposed by . for each term , we count all of the -graphs present for .we then use this -graph frequency vector of length to represent the term .  instead of directly computing the interaction between the matrix and the matrix , we first learn a series of nonlinear transformations to the character - based input . for both query and document, the first step is convolution .the convolution window has filter size of 300 .it projects 3 consecutive terms to a 300 dimensional vector , then takes a stride by 1 position , and projects the next 3 terms , and so on . for the query ,the convolution step generates a tensor of dimensions . for document it generates one of dimensions .  following this, we conduct a max - pooling step . for the querythe pooling kernel dimensions are . for documentit is . as a result, we get one matrix for the query and a matrix for the document .the document matrix can be interpreted as separate embeddings , each corresponding to different equal - sized spans of text within the document .our choice of a window - based max - pooling strategy , instead of global max - pooling as employed by cdssm , is motivated by the fact that the window - based approach allows the model to distinguish between matches in different parts of the document .as posited in section [ sec : motivation ] , a model that is aware of match positions may be more suitable when dealing with long documents , especially those containing mixture of many topics .the output of the max - pooling layer for the query is then passed through a fully - connected layer .for the document , the dimensional matrix output is operated on by another convolutional layer with filter size of 300 , kernel dimensions of , and a stride of 1 .the combination of these convolutional and max - pooling layers enable the distributed model to learn suitable representations of text for effective inexact matching .  in order to perform the matching, we conduct the element - wise or hadamard product between the embedded document matrix and the extended or broadcasted query embedding ,    after this , we pass the matrix through fully connected layers , and a dropout layer until we arrive at a single score .similar to the local model , we use hyperbolic tangent function here for non - linearity .each training sample consists of a query , a relevant document and a set of irrelevant documents .we use a softmax function to compute the posterior probability of the positive document given a query based on the score .  and we maximize the log likelihood using stochastic gradient descent .we conducted three experiments to test :  the effectiveness of our duet model compared to the local and distributed models separately , and  the effectiveness of our duet model compared to existing baselines for content - based web ranking ,  the effectiveness of training with judged negative documents compared to random negative documents .  in this section, we detail our experiment setup and baseline implementations .   the training dataset consisted of 199,753 instances in the format described in section [ sec :experiment : training ] .the queries in the training dataset were randomly sampled from bing s search logs from a period between january , 2012 and september , 2014 .human judges rated the documents on a five - point scale ( _ perfect _ , _ excellent _ , _ good _ , _ fair _ and _ bad _ ) .the document body text was retrieved from bing s web document index .we used proprietary parsers for extracting the body text from the raw html content .all the query and the document text were normalized by down - casing and removing all non - alphanumeric characters .we considered two different test sets , both sampled from bing search logs .the _ weighted _ set consisted of queries sampled according their frequency in the search logs . as a result , frequent queries were well - represented in this dataset .queries were sampled between october , 2014 and december , 2014 . the _ unweighted _ set consisted of queries sampled uniformly from the entire population of unique queries .the queries in this samples removed the bias toward popular queries found in the weighted set .the unweighted queries were sampled between january , 2015 and june , 2015 .  because all of our datasets were derived from sampling real query logs and because queries will naturally repeat, there was some overlap in queries between the training and testing sets .specifically , 14% of the testing queries in the weighted set occurred in the training set , whereas only 0.04% of the testing queries in the unweighted set occurred in the training set .we present both results for those who may be in environments with repeated queries ( as is common in production search engines ) and for those who may be more interested in cold start situations or tail queries .table [ tbl : testsets ] summarizes statistics for the two test sets .lrrr & queries & documents & + training & 199,753 & 998,765 & 5 +  + weighted & 7,741 & 171,302 & 24.9 + unweighted & 6,808 & 71,722 & 10.6 +   besides the architecture ( figure [ fig : architecture ] ) , our model has the following free parameters : the maximum order of the character - based representation for the distributed model ( ) , the number of negative documents to sample at training time ( ) , the dropout rate , and the learning rate .we used a maximum order of five for our character -graphs in the distributed model . instead of using the full 62,193,780-dimensional vector , we only considered top 2,000 most popular -graphs , resulting 36 unigraphs ( a - z and 0 - 9 ) , 689 bigraphs , 1149 trigraphs , 118 4-graphs , and eight 5-graphs .  when training our model ( section [ sec : training ] ) , we sampled four negative documents for every one relevant document .more precisely , for each query we generated a maximum of one training sample of each form ,  one _ excellent _ document with four _ fair _ documents  one _ excellent _ document with four _ bad _ documents  one _ good _ document with four _ bad _ documents  .pilot experiments showed that treating documents judged as _ fair _ or _ bad _ as the negative examples resulted in significantly better performance , than when the model was trained with randomly sampled negatives . for training , we discarded all documents rated as _ perfect _ because a large portion of them fall under the navigational intent , which can be better satisfied by historical click based ranking signals .the dropout rate and the learning rate were set to 0.20 and 0.01 , respectively , based on a validation set .we implemented our model using cntk and trained the model with stochastic gradient descent based optimization ( with automatic differentiation ) on a single gpu .it was necessary to use a small minibatch size of 8 to fit the whole data in gpu memory .our baselines capture the individual properties we outlined in section [ sec : motivation ] .exact term matching is effectively performed by many classic information retrieval models .we used the okapi bm25 and query likelihood ( ql ) models as representative of this class of model .we used indri for indexing and retrieval .match positions are handled by substantially fewer models .metzler s dependence model ( dm ) provides an inference network approach to modeling term proximity .we used the indri implementation for our experiments .inexact term matching received both historic and modern treatments in the literature .originally presented latent semantic analysis ( lsa ) as a method for addressing vocabulary mismatch by projecting words and documents into a lower - dimension latent space . the dual embedding space model ( desm ) computes a document relevance score by comparing every term in the document with every query term using pre - trained word embeddings .we used the same pre - trained word embeddings dataset that the authors made publicly available online for download .these embeddings , for approximately 2.8 m words , were previously trained on a corpus of bing queries . in particular, we use the model , which was reported to have the best performance on the retrieval task , as a baseline in this paper . both the deep structured semantic model ( dssm ) and its convolutional variant cdssm consider only the document title for matching with the query . while some papers have reported negative performances for title - based dssm and cdssm on the _ ad hoc _ document retrieval tasks , we included document - based variants appropriately retrained on the same set of positive query and document pairs as our model .as with the original implementation we choose the irrelevant documents for training by randomly sampling from the document corpus . for the cdssm model , we concatenated the trigraph hash vectors of the first terms of the body text followed by a vector that is a sum of the trigraph hash vectors for the remaining terms .the choice of was constrained by memory requirements , and we pick 499 for our experiments .the drmm model uses a dnn to perform term matching , with few hundred parameters , over histogram - based features .the histogram features , computed using exact term matching and pre - trained word embeddings based cosine similarities , ignoring the actual position of matches .we implemented the variant of the model on cntk using word embeddings trained on a corpus of 341,787,174 distinct sentences randomly sampled from bing s web index , with a corresponding vocabulary of 5,108,278 words . every training sample for our modelwas turned into four corresponding training samples for drmm , comprised of the query , the positive document , and each one of the negative documents .this guaranteed that both models observed the exact same pairs of positive and negative documents during training .we adopted the same loss function as proposed by guo _et al . _all evaluation and empirical analysis used the normalized discounted cumulative gain ( ndcg ) metric computed at positions one and ten .all performance metrics were averaged over queries for each run . whenever testing for significant differences in performance , we used the paired -test with a bonferroni correction .table [ tbl : results - main ] reports ndcg based evaluation results on two test datasets for our model and all the baseline models .our main observation is that the duet model performs significantly better than the individual local and distributed models .this supports our underlying hypothesis that matching in a latent semantic space can complement exact term matches in a document ranking task , and hence a combination of the two is more appropriate .note that the ndcg numbers for the local and the distributed models correspond to when these dnns are trained individually , but for the ` duet ' the two dnns are trained together as part of a single neural network .  among the baseline models , including both traditional and neural network based models , cdssm and desm achieve the highest ndcg at position one and ten , respectively , on the weighted test set .on the unweighted test set drmm is our best baseline model at both rank positions .the duet model demonstrates significant improvements over all these baseline models on both test sets and at both ndcg positions .we also tested our independent local and distributed models against their conceptually closest baselines . because our local model captures both matching and proximity , we compared performance to dependence models ( dm ) .while the performance in terms of ndcg is statistically indistinguishable , both of the ndcg results are statistically significant ( ) .we compared our distributed model to the best neural model for each test set and metric .we found no statistically significant difference except for ndcg for the weighted set .we were interested in testing our hypotheses that training with labeled negative documents is superior to training with randomly sampled documents presumed to be negative .we conducted an experiment training with negative documents following each of the two protocols .figure [ fig : negsampling ] shows the results of these experiments .we found that , across all of our models , using judged nonrelevant documents was more effective than randomly sampling documents from the corpus and considering them as negative examples .our results demonstrated that our joint optimization of local and distributed models provides substantial improvement over all baselines .although the independent models were competitive with existing baselines , the combination provided a significant boost .we also confirmed that using judged negative documents should be used when available .we speculate that training with topically - similar ( but nonrelevant ) documents allows the model to better discriminate between the confusable documents provided by an earlier retrieval stage .this sort of staged ranking , first proposed by , is now a common web search engine architecture .in section [ sec : baselines ] we described our baseline models according to which of the properties of effective retrieval systems , that we outlined in section [ sec : motivation ] , they incorporate .it is reasonable to expect that models with certain properties are better suited to deal with certain segments of queries .for example , the relevant web page for the query `` what channel are the seahawks on today '' may contain the name of the actual channel ( e.g. , `` espn '' or `` fox '' ) and the actual date for the game , instead of the terms `` channel '' or `` today '' . a retrieval model that only counts repetitions of query terms is likely to retrieve less relevant documents for this query compared to a model that considers `` espn '' and `` fox '' to be relevant document terms .in contrast , the query `` pekarovic land company '' , which may be considered as a tail navigational intent , is likely to be better served by a retrieval model that simply retrieves documents containing many matches for the term `` pekarovic '' .a representation learning model is unlikely to have a good representation for this rare term , and therefore may be less equipped to retrieve the correct documents .these anecdotal examples agree with the results in in table [ tbl : results - main ] that show that on the weighted test set all the neural models whose main focus is on learning distributed representations of text ( duet model , distributed model , desm , dssm , and cdssm ) perform better than the models that only look at patterns of term matches ( local model and drmm ) .we believe that this is because the dnns are able to learn better representations for more popular queries , and perform particularly well on this segment .figure [ fig : slicendice ] provides further evidence towards this hypothesis by demonstrating that the distributed model has a larger ndcg gap with the local model for queries containing more popular terms , and when the number of terms in the query is small .the duet model , however , is found to perform better than both the local and the distributed models across all these segments .  in order to better understand the relationship of our models to existing baselines , we compared the per - query performance amongst all models .we conjecture that similar models should performance similarly for the same queries .we represented a retrieval model as a vector where each position of the vector contains the performance of the model on a different query .we randomly sample two thousand queries from our weighted test set and represent all ranking models as vectors of their ndcg values against these two thousand queries .we visualized the similarity between models by projecting using principal component analysis on the set of performance vectors .the two - dimensional projection of this analysis is presented in figure [ fig : modelspace ] .the figure largely confirms our intuitions about properties of retrieval models .models that use only local representation of terms are closer together in the projection , and further away from models that learn distributed representations of text .interestingly , the plot does not distinguish between whether the underlying model is based on a neural network based or not , and only focuses on the retrieval properties of the model .  ) , proximity ( ) , and inexact matches ( ) are presented .our models are presented as black squares.,width=288 ]  another interesting distinction between deep neural models and traditional approaches is the effect of the training data size on the performance of the model .bm25 has very few parameters and can be applied to new corpus or task with almost no training .on the other hand , dnns like ours demonstrate significant improvements when trained with larger datasets .figure [ fig : trainingdata ] shows that the effect of training data size particularly pronounced for the duet and the distributed models that learns representations of text .the trends in these plots indicate that training on even larger datasets may result in further improvements in model performance over what is reported in this paper .we believe this should be a promising direction for future work .  a last considerationwhen comparing these models is runtime efficiency .web search engines receive tens of thousands of queries per second . running a deep neural model on raw body text at that scaleis a hard problem .the local sub - network of our model operates on the term interaction matrix that should be reasonable to generate using an inverted index . for the distributed model ,it is important to note that the dimensional matrix representation of the document , that is used to compute the hadamard product with the query , can be pre - computed and stored as part of the document cache . at runtime, only the hadamard product and the subsequent part of the network needs to be executed .such caching strategies , if employed effectively , can mitigate large part of the runtime cost of running a dnn based document ranking model at scale .representations of data can be local or distributed . in a local representationa single unit represents an entity , for example there is a particular memory cell that represents the concept of a grandmother . that cell should be active if and only if the concept of a grandmother is present .by contrast , in a distributed representation , the concept of grandmother would be represented by a pattern of active cells . provides an overview contrasting distributed and local representations , listing their good and bad points . in a distributed representation , an activation pattern that has some errors or other differences from past datacan still be mapped to the entity in question and to related entities , using a similarity function .a local representation lacks this robustness to noise and ability to generalize , but is better at precisely storing a large set of data .this paper considers local and distributed representations of queries and documents for use in web page ranking .our measure of ranking quality is ndcg , which rewards a ranker for returning documents with higher gain nearer to the top , where gain is determined according to labels from human relevance assessors .we describe different ranking methods in terms of their representations and how this should help them achieve good ndcg .exact term matching models such as bm25 and query likelihood tend to rank a document higher if it has a greater number of query term matches , while also potentially employing a variety of smoothing , weighting and normalization approaches .such exact matching is done with a local representation of terms .exact match systems do not depend on a large training set , since they do not need to learn a distributed representation of queries and documents .they are useful in cases where the relevant documents contain exactly the query terms entered by the user , including very rare or new vocabulary , since new terms can be incorporated with no adjustments to the underlying model .they can also be extended to reward matches of query phrases and proximity .  to deal with the vocabulary mismatch problem that arises with local representations , it is possible to do document ranking using a distributed representation of terms . developed the popular word2vec embedding approach that has been used in a number of retrieval studies . use term embeddings as evidence for term weighting , learning regression models to optimize weighting in a language modeling and a bm25 retrieval model . used term embeddings for smoothing in the language modeling approach of information retrieval . used dual embeddings , one for document terms and one for query terms , then ranked according to the all - pairs similarity between vectors . used term embeddings to generate query expansion candidates in the language modeling retrieval framework , also finding better performance when training a specialized term embedding .other papers incorporating word embeddings include . propose the use of matching matrices to represent the similarity of short texts , then apply a convolutional neural network inspired by those in computer vision .they populate the matching matrix using both local and distributed term representations . in the local representation , an exact matchis used to generate binary indicators of whether the term of one text and term of the other are the same , as in our local model . in the distributed representation , a pre - trained term embeddingis used instead , populating the match matrix with cosine or inner product similarities .the method works for some problems with short text , but not for document ranking . however , by using the match matrix to generate summary statistics it is possible to make the method work well , which is our drmm baseline .these term embeddings are a learned representation of language , but in most cases they are not learned on query - document relevance labels. more often they are trained based on a a corpus , where a term s representation is learned from its surrounding terms or other document context .the alternative , learning a representation based on ndcg labels , is in keeping with recent progress in deep learning .deep models have multiple layers that learn distributed representations with multiple levels of abstraction . this kind of representation learning , along with other factors such as the availability of large labeled data sets , has yielded performance improvements on a variety of tasks such as speech recognition , visual object recognition and object detection .this paper learns a text representation end - to - end based on query - document ranking labels .this has not been done often in related work with document body text , but we can point to related papers that use short text such as title , for document ranking or related tasks . learn a distributed representation of query and title , for document ranking .the input representation is character trigraphs , the training procedure asks the model to rank clicked titles over randomly chosen titles , and the test metric is ndcg with human labels . developed a convolutional version of the model .these are our dssm and cdssm baselines .other convolutional models that match short texts using distributed representations include , also showing good performance on short text ranking tasks .  outside of document ranking , learning text representations forthe target task has been explored in the context of other ir scenarios , including query classification , query auto - completion , next query prediction , and entity extraction .we propose a novel document ranking model composed of two separate deep neural networks , one that matches using a local representation of text , and another that learns a distributed representation before matching .the duet of these two neural networks demonstrated a higher performance than the solo models on the document ranking task as well as significant improvements over all baselines , including both traditional ir baselines and other recently proposed models based on shallow and deep neural networks .our analysis indicate that these models may achieve even more substantial improvements in the future with much larger datasets .the authors are grateful to abdelrahman mohamed , pushmeet kohli , emine yilmaz , filip radlinski , david barber , david hawking , and milad shokouhi for the insightful discussions and feedback during the course of this work , and to frank seide and dong yu for their incredible support with cntk .

Abstract: models such as latent semantic analysis and those based on neural embeddings learn _ distributed _ representations of text , and match the query against the document in the latent semantic space . in traditional information retrieval models , on the other hand , terms have discrete or _ local _ representations , and the relevance of a document is determined by the exact matches of query terms in the body text . we hypothesize that matching with distributed representations complements matching with traditional local representations , and that a combination of the two is favorable . we propose a novel document ranking model composed of two separate deep neural networks , one that matches the query and the document using a local representation , and another that matches the query and the document using learned distributed representations . the two networks are jointly trained as part of a single neural network . we show that this combination or ` duet ' performs significantly better than either neural network individually on a web page ranking task , and also significantly outperforms traditional baselines and other recently proposed models based on neural networks .


nano / micro - mechanical resonators are fascinating devices , having microscopic dimensions and high quality factors ( ) that allow them to be used in precise experiments .they are therefore ideal for many measurement applications , including biosensing , charge measurements , mass sensing , accelerometry , temperature sensing , torque or force transduction , single spin detection , and observation of qubits .in addition , nano / micro - mechanical resonators provide an excellent platform for investigating scientific phenomena , such as vortices , quantum zero - point motion , molecular structure , gravity , superfluidity , and the casimir force .as nano / micro - mechanical resonator systems become capable of performing ever more precise experiments , the sensitivity of the detection methods used to observe the devices must increase commensurately .current state of the art techniques used for this detection include capactive measurements , free - space optics , piezoresitivity , and optomechanics .these methods offer extremely senstive transduction , achieving sub - attometer displacement sensitivity , yoctogram mass resolution , and force measurements on the order of attonewtons .  in order to quantify such sensitive measurements, it is crucial that the calibration of these devices be performed accurately . as the dimensions of nano / micro - mechanical resonators continue to decrease, this can prove to be a difficult task .however , there exists a powerful method by which the motion of these resonators can be calibrated via observation of their random thermal motion .this is known as _thermomechanical calibration _ and is performed by invoking the equipartition theorem , which establishes a relationship between the thermal energy of a device and its mean squared amplitude of motion .many of the detection methods mentioned above are sensitive enough to measure the thermal motion of devices on the micro or nanoscale .in addition , it has been shown that this calibration scheme agrees well with other methods , such as physically pushing on the device , but it has the advantage of being noninvasive .thermomechanical calibration has been used to measure the motion of many systems , including cantilevers , doubly clamped beams , strings , torsional resonators , and membranes .however , to the best of our knowledge , no single standalone document exists that describes the method of thermomechanical calibration in its entirety .we have noticed that a number of mutually inconsistent approaches to performing such a calibration can be found in the literature .this manuscript intends to elucidate the process of thermomechanical calibration by providing a full , self - consistent derivation of the procedure and ensuring that all pertinent equations are explained and defined . in this way, a general process is developed by which nano / micro - mechanical resonators can be calibrated the only difference between varying geometries being reduced to a single parameter , the device s effective mass . in this paper , the effective massis calculated both analytically and numerically , for a number of common device geometries .these calculations are then benchmarked against the well - known methods of sader _ et al . _ and cleveland _ et al . _ , which have been developed to calibrate atomic force microscopy cantilevers .in addition , we extend the methods of sader _ et al . _ and cleveland _ et al ._ to encompass other resonator geometries , such as doubly clamped beams and torsional resonators .this document which includes both a general overview of the process and specific examples provides a clear , straightforward recipe for carrying out thermomechanical calibration for any nano / micromechanical resonator system , regardless of its complexity .we begin by describing the motion of an extended resonator structure in the linear regime . in general , the complete , three - dimensional motion will be given by a displacement function that can be broken down into an infinite number of independent modes . to each modewe ascribe a label , , allowing for the displacement function to be expressed as     where is the mode shape of the mode , and is a function that describes the time dependence of the resonator s motion . herewe choose to normalize such that the maximum value of is unity .this choice of normalization ensures that has units of distance [ as the normalization leaves unitless ] and is in 1 - 1 correspondence with the resonator s true physical displacement at the point of measurement .it should also be noted that by normalizing this way , the standard convention of orthonormality between the modes is lost .many devices can be described by simple models , for which we are able to determine analytically .when the device s motion is more complicated , however , we turn to fem modeling to determine the mode shape of the structure . in this paper , these two methods are compared in a number of situations where the analytical mode shapes of the resonator can be determined .  the function is determined by mapping each of the resonator s independent modes to a damped harmonic oscillator of the form   here , , , and are the angular frequency , quality factor , and effective mass for the mode of the resonator . is a time - dependent driving force .an effective spring constant for the mode can also be defined and is given by  . a general expression for , along with its determination for a number of simple models ,will be outlined in sec .[ effmasssec ] .often a nano / micro - mechanical resonator s motion can be reduced to a one - dimensional displacement function    where describes the one - dimensional mode shape for the mode of the resonator as a function of position , and is unchanged from eq . .as in the three - dimensional case , we choose to normalize such that the maximum value of is unity .this equation will be used when the displacement of a nano / micro - mechanical resonator is simple enough to be modeled by a one - dimensional function .equations  and fully describe the motion of these devices in the linear regime and will be used when explicating the thermal calibration process .the various methods used to detect the motion of a nano / micro - mechanical resonator share one major similarity : the motion of the resonator in question is encapsulated by some time - varying signal , typically a voltage . generally , this measured signal is transformed into the frequency domain , providing a spectral representation of the resonator s motion that is peaked ( up to small corrections in 1/ ) at its resonance frequency . by squaring this spectrum and dividing by the resolution bandwidth of the instrument ,the measured one - sided power spectral density of the signal is produced .  when thermally calibrating a mechanical resonator , we must be able to relate the device s time averaged motion to its thermal energy .to do this , the equipartition theorem is used to express the device s mean - square amplitude of motion in terms of its thermal noise psd . in this section ,we derive a theoretical expression for the psd corresponding to a damped harmonic oscillator undergoing random thermal motion .this function can be used to properly calibrate thermal psd data for any nano / micro - mechanical resonator .we introduce the autocorrelation function , which describes how the signal is related to itself at a later time and is defined as     the two - sided power spectral density function , can then be obtained from the fourier transform of the autocorrelation function     likewise , the autocorrelation function can be produced by performing the inverse fourier transform     is defined such that it spans all frequencies , both positive and negative , and will give the total power of the signal if integrated over this entire frequency range .for real - valued signals , as is the case in physical situations , is an even function .therefore , we introduce , which is defined on the frequency interval from to such that it produces the power in the signal when integrated over all positive frequencies . from these two definitions , we find that and are related by  .this relation allows the total power in the signal to be conserved when each spectral density function is integrated over the frequency interval on which it is defined . in the rest of the derivation , will be used , as it provides a theoretical spectral density that can be fit directly to physical data .we define the quantity known as the mean - square amplitude of our signal as   }^2 . \label{meandef}\ ] ]  upon inspection , we see that this quantity is equal to eq .  at zero time shift ,provided that we have sampled the signal over a sufficiently long time window ( i.e. ,  ) .using eq . , along with the relation between between and , we can then express in terms of its psd via     this result provides us with the relationship between the mean - square amplitude and psd for . by fourier transforming eq ., we can relate to , which is the psd due to the forcing function  :    the factor of proportionality comes from the frequency response function ,    here and are the fourier transforms of and . for thermomechanical calibration , is generated by random thermal noise , which is broadband in the region of interest , producing a that will be flat with respect to frequency .we can therefore replace by a constant thermal force noise .this factor can be determined by combining eqs . , which results in    computing this integral by contour integration , gives  .  invoking the equipartion theorem , can be related to the thermal energy of the resonator .the equipartition theorem states that every degree of freedom that contributes a quadratic term to the total energy of a system has an average energy of  . for the timeaveraged potential energy of the one - dimensional harmonic oscillator considered here , this theorem is mathematically represented as     it can be seen by investigation of eq .  that the magnitude of is arbitrarily determined by our choice of normalization for .therefore , our effective mass must be chosen very carefully to match our definition of in order to preserve the validity of the equipartition theorem .this subtlety will be discussed in detail in sec .[ effmasssec ] .combining the result from eq .  with the equipartition theorem gives     allowing us to express the oscillator s psd as   }. \label{psdtang}\ ] ]  however , since the frequency we read off our instrument is given in terms of a natural frequency , , and not an angular frequency , it is convenient to express in terms of .this is done by taking , resulting in  }. \label{psdtfm}\ ] ]  alternatively , it is sometimes useful to express in terms of the effective spring constant  :  }. \label{psdtfk}\ ] ]  equations provide the power spectrum for the thermal noise of our resonator .however , we expect a number of other sources of noise to be present in our detection system , such as detector dark current and optical shot noise . these signals will combine with the thermal noise of the resonator to produce the power spectrum read out for the undriven device .if we assume these other sources of noise to be white , we can express our total psd as     where is a constant offset to account for the white noise of the detection apparatus and is a factor with units to allow for conversion of our theoretical displacement psd ( in units of ) to the measured voltage psd ( in units of ) .in other words , tells us how efficiently the detection system converts the motion of the resonator into the voltage signal that is measured and is closely tied to the sensitivity of the detection system .the experimentally measured psd is given as  where is the voltage spectrum of the resonator s motion obtained with a measurement bandwidth .the experimental psd can be fit with eq .  near a thermal resonance using , , , and as fit parameters .once has been determined , it can be used to convert the psd data from units of to , the result of which we will call .the resonator s displacement spectrum is then determined from , given in units of  . from this data, we can determine the maximum displacement of the resonator , as well as the displacement sensitivity of the detection method , the latter of which is given by .the displacement sensitivity is often used as a figure of merit for a detection system , as it is a measure of the minimun detectable signal for that setup .it should be noted that is highly dependent on a number of factors , for instance photodiode gain or probe position , and therefore an value extracted from a specific set of data should be associated only with that data set itself and not the detection system .this being said , it is possible to conceive of an experiment where all parameters are carefully controlled such that the value for repeated data sets would be similar and could be combined to determine an average value .similarly , a number of data sets originating from a carefully controlled system could be combined into an averaged signal and fit with eq .  to provide a statistically significant value of .the theoretical psd functions obtained in this section are a general result for a damped harmonic oscillator excited by random thermal fluctuations and can be used for any nano / micro - mechanical resonator . the only difference in eq .[ eq . ] that exists between varying resonators is the effective mass ( spring constant ) associated with the resonator s geometry .therefore , once we have the psd function given by eqs .  and , we need only focus on calculation of the device s effective mass . for this reason , the calculation of the effective mass for a number of resonator geometries will be the focus of the remaining portion of the document . for simplicity , we deal only with the quantity , as is directly related to through the mode s experimentally determined resonance frequency .in general , the effective mass associated with the specific mode of a resonator will depend on its mode shape . for this reason ,it is useful to investigate analytical expressions for the mode shapes and resonance frequencies of some simple geometries that are commonly used for nano / micro - mechanical resonators .a large number of nano / micro - mechanical resonators possess beam - like geometries and can be effectively modeled as macroscopic beam - like structures .the general equation of motion for a long thin beam of uniform cross section that is oscillating in one dimension is     where , , , and are the young s modulus , moment of inertia , density , and cross - sectional area of the beam . is an arbitrary position - dependent loading function .we know that will be in the form of eq ., so we can solve eq .for each of the resonator s modes separately . the oscillatory motion of the beam is encapsulated by and , for high resonators ,is given approximately as    where and are mode dependent constants .inputting into eq .  with this choice of , we obtain for the free motion [ of a beam , a differential equation for the mode shape of the mode    where is the length of the beam and is a dimensionless parameter .this differential equation has the general solution     enforcing the beam s boundary conditions allows us to determine the values of , as well as , , , and ( up to a normalization constant ) , thus solving for the mode shape of each of the beam s modes .it is then possible to calculate the frequency corresponding to the mode of the beam using    a method of determining a cantilever s effective mass by applying a load to it and observing its deflection has been developed by sader _et al . _it is therefore useful to determine the deformed mode shape when a beam is subjected to a static loading function , which can be found using     where is the bending moment of the structure due to the specified loading .this can be solved analytically for a number of different loadings , some of which are very important in defining effective masses and will be discussed below .solving the above equations for the specific cases of a singly clamped beam ( cantilever ) and a doubly clamped beam will be very useful in providing an analytical value for the effective mass in these two simple cases .in the subsequent sections , we will treat each of these cases separately , determining both dynamic and static displacements .these will be used later to determine the effective mass of each device .the singly clamped beam , or cantilever , is subject to the boundary conditions     these boundary conditions tell us that the cantilever is fixed and flat at , while free at , as can be seen in fig .[ cantmode shapefig ] . inputting these conditions into eq . , we find that obeys the relation     the solutions of which are summarized in table [ lambdatab ] .the mode shape of a cantilever is then determined to be   \\ - s_n \left [ \sinh\left(\frac{\lambda_n x}{l}\right ) -\sin \left(\frac{\lambda_n x}{l } \right ) \right ] \biggr ) , \label{normmodecant}\end{gathered}\ ] ]  where , , and .notice that our choice of normalization constant ensures that .this satisfies the mode shape normalization condition found in sec .[ motionsec ] , as the maximum amplitude of the cantilever is always at its free end .the first six normalized mode shapes of a cantilever can be seen in fig .[ cantmode shapefig ] .   out of phase in time with respect to each other ) .each mode is normalized such that its maximum displacement is equal to one . ]it is also useful to determine the deformed mode shape of a cantilever when a static force is applied perpendicular to its free end [ , so that its effective mass can be approximated using the method of sader _ et al ._ by calculating the bending moment for such a loading force , we can solve for using eq ., resulting in     the maximum displacement of the cantilever due to this loading will clearly occur at , so that    this allows us to define a corresponding spring constant    which will be used later to calculate the effective mass of a simple cantilever using the method of sader _ et al . _the boundary conditions for a doubly clamped beam are     physically , this means that the beam is both flat and fixed at each of its endpoints , as can be seen in fig .[ beammode shapefig ] .inputting these conditions into eq ., we obtain the following condition on for doubly clamped beams     the solutions to this equation are summarized in table [ lambdatab ] . the mode shape for a beamcan also be solved and is found to have the form \\ - s_n \left [ \sinh\left(\frac{\lambda_n x}{l}\right )- \sin\left(\frac{\lambda_n x}{l}\right ) \right ] \biggr ) , \label{modebeam}\end{gathered}\ ] ]  where is a normalization constant chosen such that the maximum value of is 1 , , and .it is interesting to note the similarities between eqs .  and for a doubly clamped beam and the corresponding eqs .  and for a cantileverunfortunately , the position at which is maximized is not obvious in the case of the doubly clamped beam , aside from the fundamental mode where the maximum is located at .for this reason , the determination of must be carried out numerically .the first six normalized mode shapes for a simple doubly clamped beam can be seen in fig .[ beammode shapefig ] .    as with the cantilever , we are also interested in the static mode shapes of doubly clamped beams due to specific loadings .this allows us to extend the method of sader _ et al ._ to approximate the effective mass of this structure .for a point loading force applied perpendicularly to the center of the beam [ , eq .can be solved to determine the deflected mode shape as     the maximum displacement occurs at and is found to be    the corresponding spring constant due to this force is given by    .values of for cantilevers and doubly clamped beams .the last row provides an approximation for for , which can be obtained by the fact that for large , is so large that we require to be very close to zero to satisfy eqs .  and .[ cols="^,^,^ " , ]   this simple example demonstrates that it is possible to thermally calibrate the motion of a torsional resonator using an effective mass , provided that we keep track of which degree of freedom we are measuring .this is an important result as it reinforces the fact that the method described in the previous sections can be used for any arbitrary oscillatory motion , be it flexural , torsional , or a complex combination of both .this document provides a general procedure by which the thermal motion of any nano / micro - mechanical resonator can be properly calibrated via the equipartition theorem .it was shown that for any given resonator , a generic spectral density function can be used to describe its motion in the frequency domain .the only modification existing between different structural geometries was manifested by a single parameter , effective mass , which is a mode shape dependent quantity .therefore , the effective mass was investigated in great detail for a number of nano / micro - resonator geometries , including cantilevers , doubly clamped beams , strings , membranes and torsional devices .the end result was a universal method by which any nano / micro - mechanical resonator can be thermomechanically calibrated , regardless of the complexity of its motion or number of degrees of freedom . by applying this method , the motion of such devices can be very precisely determined , allowing for use of nano / micro - mechanical resonators for a number of different measurement applications , as well as in the investigation of fundamental scientific phenomena , with confidence and accuracy .the authors wish to thank university of alberta , faculty of science ; the canada foundation for innovation ; the natural sciences and engineering research council , canada ; and alberta innovates technology futures for their support of this research .we would also like to thank wayne hiebert for helpful discussions .j. w. ndieyira , m. watari , a. d. barrera , d. zhou , m. vgtli , m. batchelor , m. a. cooper , t. strunz , m. a. horton , c. abell , t. rayment , g. aeppli , and r. a. mckendry , nature nanotech .* 3 * , 691 ( 2008 ) .

Abstract: we provide a detailed description of a general procedure by which a nano / micro - mechanical resonator can be calibrated using its thermal motion . a brief introduction to the equations of motion for such a resonator is presented , followed by a detailed derivation of the corresponding power spectral density ( psd ) function . the effective masses for a number of different resonator geometries are determined using both finite element method ( fem ) modeling and analytical calculations .


what if hitler would have been killed as soldier during world war i ? could this had stopped national socialism in germany ? + a certain , shocking event like the eruption of a vulcano , an airplane crash or hitlers death usually initializes a fraction of people to change their opinion in some way .an airplane crash for example can cause people to avoid airplanes for the next time and to travel by train .this behaviour is often a temporary precaution and is abandoned after a while . the eruption of mount st .helens certainly unsettles people close to the vulcano more than in paris .this shows that the effect also decreases with increasing distance .thus , a shocking event has _temporal _ and _ spatial _ impacts .while most of the studies on the effect of changed opinions on the opinion formation in a population deal with events that change _ several _ opinions and are limited to their _ temporal _ effects , in this paper the limiting case of a _ single _persons opinion change is investigated and the often neglected spatial impact is also attached importance to . + in order to do this , a model of opinion formation is required .we work with the _ sznajd model _ of 2000 although the results should be independent of the special choice of model .the _ sznajd model _ assumes that the interacting people ( _ agents _ ) are located on the places of a square lattice of sidelength and have exactly one of two possible opinions .the sznajd rule is : + * a pair of _ nearest neighbours _ convinces its six nearest neighbours of its opinion if and only if they both share the same opinion .otherwise the opinions of all eight involved agents remain unchanged . * + here _ nearest neighbours _ are two agents whose places share a common side .+ in the simulation on a computer the two possible opinions are initially distributed randomly on the lattices places. then step by step one agent is chosen randomly as well as one of its nearest neighbours . then the sznajd rule is applied on this pair .this whole procedure is called a _sznajd process_. one goes through the lattice like a typewriter and at every position visited one sznajd process is performed .one _ timestep _ shall be over when one sweep through the lattice is made , i.e. sznajd processes are performed in average . + this model is a _ consensus model _ that always leads to a consensus .all agents share the same opinion at the end , one opinion dies out .the consensus opinion depends critically on the initial random distribution of the opinions .if the probability for opinion is more than 50 percent , opinion will vanish completely at the end of the simulation and vice versa . in the case=0.5 the possible consensus opinions and are reached with the same probability 0.5 .a fortran - program simulating the sznajd model in the mentioned variant on a square lattice is listed in .the method of _ damage spreading _ was first used by s. kaufmann in biology and is an useful tool to investigate the development of two systems that obey the same kind of dynamic rules and differ only in a slight modification .the strategy is very simple : + a replication of the initial system is created and a certain amount of elements is changed in ( _ initial damage _ ) .then both systems develop under exactly the same conditions ( i.e. the same sequence of random numbers ) towards consensus and one observes the impact the initial damage has . + here the systems are two lattices , the initial damage consists in a single persons opinion change and the rule for the dynamics is the sznajd rule .the effect of the opinion change can be measured by the fraction of different opinions on both lattices at a certain moment and is called the _ damage _ ( consistent to the already defined initial damage ) or _ hamming - distance_. the damage is determined by a site - by - site comparison of both lattices while going through the lattice like a typewriter .each site with a difference in opinions is summed up .the first used of _ damage spreading _ in the sznajd model was in where more than one opinion were changed .+ in the sznajd model two different szenarios are possible : + the damage dies out ( ) or the damage spreads over the whole lattice ( ) .the last case ends up in a reversal of the consensus opinion of the unchanged lattice and is called _total damage_. when one of these two _ equilibrum states _ is reached , the simulation can stop .+ as already mentioned , the state of consensus is determined by the probability for opinion 1 if .a single change of opinion does not alter this probability .therefore the most interesting case is where total damages are most probable ( see fig .[ tdl ] ) although the values are very low .the further investigation was therefore limited to this case and it was decided to change the opinion in the center of the lattice to know the initial location of the damage . furthermore it should be mentioned that lattice - sizes of a prime number are used to enable long periods and minimal correlations of the random - number - sequences generated by multiplication with 16807 .the _ distance _ of a damaged site to the initial damage ( in the center ) is measured in the _ manhattan - metrics _ and the _ range _ of the damage in every timestep is the maximal distance occurring in that timestep .all values were averaged over 10,000 simulations .one result of the simulations concerning the temporal spreading of the damage is that the timespans needed to reach equilibrium ( in the sense mentioned above ) of the lattice size obey a power law .this could be found in cases of extinction as well as in cases of total damage ( see figure [ power1 ] ) .similarly the differences in times needed to reach consensus as a function of the lattice size follow a power law ( figure [ power1 ] ) .this means that a consensus can be delayed or accelerated dependending on the size of the system and is a quite interesting effect : it could be sometimes quite crucial whether a consensus happens in this moment or ten years later . +furthermore it is supposed that in cases of extinction the time scales with the systems size and the damage is a function of this scaled time ( ) ( see figure [ scalet ] ) .the distance scales with and the value ( damage ) is a function of this scaled distance ( ) .this relation causes a comparison to a diffusion process for which such a -dependence is characteristic . if the range of the damage ( see fig .[ reichweite ] ) is averaged over only those runs in which the damage is still alive at the moment considered , the damage increases and therefore spreads like in a diffusion process ( see inset of fig .[ reichweite ] ) but finally dies out .finally : + can the change of an opinion into the consensus - opinion of the benchmark - simulation cause a total damage ? +the simulations show that it is indeed possible that an opinion changed into the consensus opinion causes the opposite consensus .+   the results confirm the intuitive assumption that there is hardly any chance for one person to change the consensus , that the effect of this change dies out after a certain time and its range decreases with time .the _ consensus times _ were compared and it turned out that the consensus can be delayed or accelerated by this slight modification and that the amount of the difference in the consensus times obeys a certain power law as well as the lifetime of the effect .+ moreover two _ scaling laws _ concerning temporal and spatial aspects could be observed up to a certain size of population .99 b. m. roehner , d. sornette und j.v .andersen , _ int .c 15,809 ( 2004 ) _r. geipel , r. hrta , j. pohl , risiken i m mittelrheinischen becken , _ deutsche idndr - reihe nr.4 _ s. fortunato , d. stauffer , computer simulations of opinions and their reactions to extreme events , in extreme events in nature and society to be edited by s. albeverio , k. jentsch and h.kantz , _ springer verlag berlin - heidelberg _ k. sznajd - weron , j. sznajd , opinion evolution in a closed community , _ int .j. of mod .c 11 , 1157 ( 2000 ) _d. stauffer , how to convince others ?monte carlo simulations of the sznajd model , aip conf .690 , 147 ( 2003 ) s.a .kaufmann , _ j. theor22 , 437 ( 1969)_. a.t .bernardes , u. m. s. costa , a. d. araujo und d. stauffer , _ int . j. modc 12,159 ( 2001 ) _

Abstract: the method of _ damage spreading _ was used to simulate the influence that a _ single _ persons change of opionion has on the consensus opinion built up in a population if one assumes opinions to form according to the _ sznajd model_. the results confirm the intuitive assumption that there is hardly any chance for one person to change the consensus , that the effect of this change dies out after a certain time and its range decreases with time . the _ consensus times _ were compared and it turned out that the consensus can be delayed or accelerated by this slight modification and that the amount of the difference in the consensus times obeys a certain power law as well as the lifetime of the effect . moreover two _ scaling laws _ concerning temporal and spatial aspects could be observed up to a certain size of population .  the influence of an individual opinion + in the sznajd model  nina klietsch + institute for theoretical physics , cologne university , d-50923 kln , germany  _ keywords _ : consensus model ; damage spreading ; monte carlo simulation ; sznajd model