some of the most important features of quantum mechanics involve a system s ability to be in a superposition of two classically distinguishable states and the impossibility to perfectly distinguish between two non - orthogonal states .these features correspondingly lead to quantum information processing schemes and protocols , such as quantum cryptographic key distribution , quantum byzantine agreement and quantum communication complexity reduction .interestingly , these problems have found their solutions not only with the help of entangled states , commonly seen as a basic and main resource in quantum information processing , but also with sequential protocols where a single quantum system is transferred from one partner to the other .each partner then performs an action that cumulates with others so that a final decision is reached .  in this report, we present and experimentally demonstrate a novel quantum solution to a problem of dining cryptographers .it was introduced by chaum and refers to three cryptographers who went out together for a dinner in a high - class restaurant . in the end, they learned that the cheque was paid anonymously , either by one of them or by the agency they work for .the cryptographers respected each others right for secret generosity , but wanted to know if the dinner was covered by the agency .this problem is also known as the unanimous voting problem .  a simple ,classical solution was proposed in the original paper and can be outlined as follows : the diners pairwise establish secret bits , , and each of them announces sum ( modulo 2 ) of his secret bits if he did not pay or the negation of this sum if he did . now calculating xor of the announced sumreveals the outcome of the problem where a result 1 shows that one of them has paid , while 0 suggests that it was at the expense of the agency .however for a generalized case , this solution poses a number of problems and drawbacks .first is a collision , which occurs when two ( or any even number ) of partners have paid as their alterations would cancel each other and the agency incorrectly turns out to be the sponsor .the second is complexity , which is a complication for a large group of users .finally , the last partner to reveal his outcome might forge the broadcast .note that without the collision issue , the problem of dining cryptographers is equivalent to the problem of unanimous veto voting where casting a veto is the admission of payment by a party .typically , unanimous vetoing conforms to an organizational situation when a certain decision should be made by a group of people unanimously but the people want to keep their individual decisions a secret .a classical solution is presented in ref . . in the protocol, the partners must agree on a schnorr group and demonstrate their knowledge of the discrete logarithms in each run without revealing them .using two broadcast rounds and the assumption that the broadcast channel available to each voter is reliable , the protocol preserves the sender s anonymity with acceptable security unless all the participants are compromised .however , as is the case with most classical solutions , this protocol relies on the difficulty in reversing of some calculations , which is an arbitrary assumption in the light of the growing available computational power .  in regards to the classical limitations ,quantum solutions have also been proposed where the protocol security does not depend on the assumptions about computational complexity .it was observed by hillery , ziman , buek , and bielinkov that correlations of multipartite , high - dimensional ghz states can be used for the quantum solution .this was later rediscovered independently in where the protocol security relies on the ghz paradox and the genuineness of the shared ghz states between the involved parties . for a three party protocol , upon satisfying this condition, each party performs a local unitary operation on the shared ghz state in the case of a payment .randomly selected copies of the shared states are then used to distinguish between the cases when an even and odd number of payments are made . once this is accomplished , the distinction between zero and double payment scenarios ( for the even case ) and a single against triple payment ( for the odd case ) is the final step in the realization of the protocol .this is possible as the ghz states are highly symmetrical , e.g. , under permutations of parties , so certain actions by one user can be compensated by others .the generalized protocol promises unconditional security along with the guarantee of no multiple payment scenarios .  despite the complete security of the protocol , for voters, the protocol would require -dimensional hilbert space .generating entangled states of many particles , even a simpler one as ghz , is a non - trivial task and suffers from low generation rate and low state fidelity .fortunately , the essential correlations of the ghz states can be effectively simulated by phase shifts performed sequentially by all users on a single particle , followed by a measurement carried out by a receiver .this greatly reduces the complexity of the experiment , which now utilizes a single quantum system .let us start with a simplified version of our protocol , which is sensitive to collisions .the protocol s feature is that it is based on a single qubit and is later generalized to a scheme without collisions .the protocol is based on the concept of mutually unbiased bases ( mubs ) . for a dimensional system ,consider states , where enumerates the basis and vector in the basis . shall always denote the computational basis and the number of other known bases depends on the dimension . for powers of primes , sets of mubsare informationally complete ( ) .in particular , for prime we have from eq .( [ allmubs ] ) we can observe that there are two important unitary transformations , cyclically permutes the vectors within any of the non - computational bases , while cyclically changes the basis ( neither of these operations affects the computational basis ) .since the first operation transforms any state with to an orthogonal one , it will be used for casting a veto . on the other hand , transforms a state to a completely unbiased one , so it will be used by all participants of the protocol to improve the voter s privacy . is a special case , where standard formulae ( [ allmubs ] ) for mubs fail , and we have note that so the not only changes the bases , but also the cycles through the states the basic version of the protocol for voters will involve parties : a sender ( s ) , voters and a receiver ( r ) .s has a state preparation source , r has a measurement device and the voters are provided with sets of phase shifters .this is a flying particle protocol where voters do not perform any measurement but rather execute local unitary transformations on a particle propagating among them ( as shown in the fig . for three participants ) .such schemes have found applications in experimental realizations of quantum secret sharing , communication complexity reduction and byzantine agreement .first consider a protocol for an arbitrary number of voters sequentially communicating a single qubit .we begin with establishing an infrastructure to protect the voter s privacy .the voters will privately decide about the power of applied by them , so that an auxiliary eavesdropper will not know in which basis she should measure the state , regardless of the place of its interception .however , to make it possible for r and s to conclude the voting result , the overall transformation of this kind should be null .this is done as follows : each voter sets his or her device to perform , where is a secret random integer , and labels the voter .s sends in a random state from one of the mubs , + and r measures in one , randomly chosen , of these bases .they then discuss their choices and if they have a strong match , they accept the choice of the secret numbers , otherwise the voters choose them again . if , the overall effect of is the unity . on the other hand , it is unknown , to which the distributed state belongs .  for the voting step , the sender prepares or , chosen at random .each voter chooses if he wants to veto or otherwise .the receiver measures in one of the bases with or 1 . if s has sent and r registers he can be sure that some voter has put a vetohowever , registering might mean any even number of vetoes , including 0 .this protocol is subject to collisions and works under the assumption that at most one voter poses a veto .what are the ingredients of the protocol with arbitrarily many vetoes ?first , there should be no collisions , so the decision should always be conclusive .the easiest way to guarantee the conclusiveness to parties is to count the votes modulo .the other aspect is that we may prefer not to reveal how many vetoes were posed ; the only information to be known is if there were any .  in general , can be taken as , a prime larger than .that way , we are certain , that there exist bases , between which the voters may switch , and within which they can change states avoiding collisions , even if all of them cast a veto .the voters can realize and as given by eqns .( [ vprime ] ) and ( [ uprime ] ) . in the first step, the voters will again randomly choose their secret numbers from .the sender will randomly choose state from eqns .( [ allmubs ] ) and the receiver the basis in which he would measure .they proceed until they verify that .then , in the second step , the voters together with apply an additional transformation if they want to pose a veto .s sends or from eq .( [ allmubs ] ) .if r registers the same state as sent by s , it is likely that no one vetoed .  as an example , for the case of three participants where the protocol can be run using a system with , lets consider the two following bases ( with vectors given by columns ) :  plays a role of and will serve to cast a veto .note that we only use two bases , so the private numbers used by the voters to protect their privacy are simply bits .nevertheless it does not compromise the security of the protocol , since and are complementary .one can straight - forwardly generalize these operators for squares of primes and even other power . is represented by and is an entangling operation - for pair of subsystem it will transform an equimodular product state into a maximally entangled one , for more constituents - to a ghz state .for example for , .starting with a state , they lead to mubs.  one potential loophole still refers to collisions . since a veto is anonymous , the only reason a voter to be willing to cheat is to cancel out all possible vetoes . to do this , he would need to know how many other voters ( say , ) would pose a veto and apply .this can be deterministically successful only if the dishonest voter knows the intentions of the honest ones .otherwise , he can apply a random power hoping to cancel out all other vetoes .a simple way to eliminate this possibility is to impose a hardware limitation allowing the voters to pose only a single veto .the other issue is honesty of s and r. one option is to integrate the role of s and r with the first and the last voter , respectively , and then repeat the procedure for various orders of voters .  alternatively , for each run of the experiment one can allow each voter to have random and private trit ( labels the run of the experiment .upon , the voters perform according to their chosen numbers in attempt to establish infrastructure .for they also cast the veto if they wish to do so , and finally for they do nothing .s sends any state involved in the protocol and r measures in the random basis . only those runs , where the voter had all their trits equal matter .first , a list of runs with .if for those runs there is a perfect correlation between states sent and received , r and s can be trusted , otherwise the protocol is aborted . then a list of runs with all is used to verify if the infrastructure was established .if this is the case , the voters find a run with all to confirm if a veto was cast .1 .  _ setup stage _ : s sends a four - level quantum system prepared in a random state from sets and , given in ( [ mubs4 ] ) , where the computational basis is given by . the voters , , choose if they would apply a unitary operation or not .r , in turn , randomly chooses to perform a measurement in basis or .afterwards , s firstly communicates the basis that was used to prepare the state and if it does not match the basis choice of r , the round is discarded .secondly , s communicates which state was produced in this run of the experiment .if r finds a match between the produced and the measured state , this validates the voters choice of even number of .however , if there are no such correlations , the round is repeated and voters have to make their choice for again .voting stage _ : s sends the following two ququart states + + in a random order ( each state is sent exactly once ) .each voter receives the particle and applies if he decided so in the previous step .if he wants to pose a veto , he also applies .r randomly chooses one of the two bases to measure for both runs .outcome stage _ : since s changes the initial state in each run and r measures in the same basis for both runs , they will have a match in their choices for one of the two runs . only after they reveal their actions , it will be known that which of the two runs is relevant . in this run , if the state detected by r matches the one sent by s , it is certain that no veto was cast and the parties unanimously agree that the agency has paid .if however , r detects another state from the same basis , to what was prepared by s , it is certain that a veto was made and the state measured by r gives information about the number of vetoes .we shall also consider the case in which s and r are likely to conspire to produce false outcomes .for example , r can give fictional outcomes to convince everybody that a veto was cast , or s can alter the input state to ensure that a correlation is not observed .the easy solution is to allow observers to randomly perform a null transformation .  1 .each voter decides if he applies and , of course , if he wants to pose a veto by applying .2 .  each voter produces a random trit .its values will correspond to the following actions .when 0 is drawn , they perform no action on the particle .this is to test the honesty of r and s. when 1 is drawn , they perform ( if they planned so ) , but not , which allows them to test if they have chosen the number of to be even . for 2they perform all operations they intended , which is the voting round .3 .  s prepares randomly one of eight states from set ( [ mubs4 ] ) , and r measures in or .then they both share the list of state they have sent / received with the voters . up to this stage , the choices of the observes and their random trits are kept secret .the observers reject all the runs of the experiment , where r chose to measure in the wrong basis .this is similar to the sifting procedure in cryptography .5 .  afterwards, the voters announce their random trits , but not if they applied or .they reject all the runs where the trits were not equal . from the remaining rounds ,the rounds with three zeros imply that the parties performed no operations on the state sent by s. if r measured the same state as well then they ( s and r ) are honest and dishonest otherwise . for the runs with three ones , if the results of s and r do not match then the voters choice of , that must be even , was wrong and the protocol has to be redone .finally , for the runs with three twos , they performed all the intended operations ( , u , v , uv ) .if r measures a given state , he is unable to infer the number of vetoes from that state without knowing the state that was prepared by s. similarly , if s chooses randomly between and bases , this leaves no scope for the voters to cheat .thus , the honesty of all the involved parties can be established in this way .additionally , now considering that their honesty is proven , if the state detected by r matches the one sent by s , it can be said with certainty that no veto was cast and the parties unanimously agree that the agency has paid .if however , r detected another state from the same basis , to what was prepared by s , a veto was made and the state measured by r gives information about the number of vetoes . the experimental implementation of the quantum solution for the party flying particle protocol is presented in fig . [ fig2 ] .sender s prepares a four level quantum system using 2-path and 2-polarisation encoding scheme for a single photon attained through a heralded single photon source . the heralded source utilizes a spontaneous parametric down - conversion ( spdc ) process where a non - linear crystal ( bbo ) is strongly pumped by a femtosecond pulsed laser operating at 390 nm .the spdc process results in the creation of a pair of twin photons , commonly known as the signal and idler photons .the idler photon is then used as a trigger as its subsequent detection by a single photon detector heralds the arrival of the signal photon .these signal photons with well - defined spectral and spatial characteristics are then used as single photons for our experiment .  for a -dimensional physical system, information can be encoded in the following four basis states : , , and .( ) and ( ) are the horizontal and vertical polarizations whereas ( ) and ( ) are the two spatial modes of a single photon . in this way, we can write a ququart state as .the single photons are initially prepared in using a polarizer oriented to horizontal polarization direction .s can prepare any state from the and bases using a 50/50 non - polarising beam splitter ( bs ) followed by suitably oriented half ( hwp ) and quarter - wave plates ( qwp ) alongwith a phase shift setting ( pp ) . for the basis , , the states in ( [ mubs4 ] ) are denoted such that is the state defined by the column . for a proof of principle experiment ,we have chosen to prepare the following states , as an example , and for the infrastructure establishment round and states , for the voting round .these four states are parametrized as and only a hwp ( ) in each port of the bs ( 50:50 ) is sufficient for this purpose .  the experimental setup is shown in fig .[ fig2 ] . for the purpose of interference stability and practicality , a sagnac interferometer with shifted pathsis implemented that allows us to manipulate each path individually .   through suitable orientations of and stationed in the two output ports ( 1 ) , ( 2 ) of the bs .the three voters implement their desired transformations through suitable tilting of hwps in path ( 1 ) and ( 2 ) and a phase shift setting pp in path ( 2 ) by mounting them on rotational stages .a hwp just before the bs in path ( 2 ) allows the receiver to choose the measurement basis . to implement the measurement ,the two paths combine at the bs followed by two measurement stations and .,width=9 ]  in order to perform unitary transformations , and ; alice , bob and charlie are each provided with two hwp oriented at in path ( 1 ) and ( 2 ) , to set the phase and between the horizontal and the vertical polarizations in each path respectively .pp in path ( 2 ) sets the required phase between the paths ( 1 ) and ( 2 ) .these three plates are mounted on rotational stages allowing the user to perform any of the desired transformation by precisely tilting them to the suitable position . the setting for , where , are , , and to perform , and respectively .a second pp is added in path ( 1 ) for compensation .the physical system flies through the parties before arriving at the receiver , where it is subjected to a measurement in either of or bases .in addition , any unwanted phase arising from a component in the interferometer was carefully compensated .the measurement part consists of a hwp in path ( 2 ) just before the bs , where the two paths recombine , followed by measurement stations and at the two output ports of the bs .the choice of the measurement basis is set by appropriately tilting the hwp to set the desired phase ( ) between the polarizations in this path , similar to the voting process . for this purpose ,the hwp is also mounted on a rotational stage and another hwp is used in path ( 1 ) for compensation . contains a hwp ( ) succeeded by a polarising beam splitter ( pbs ) .the photons arriving at each output port of the pbs are detected by multimode fiber coupled single photon detectors . is similar to but contains an additional qwp ( ) in the beginning .this enables the photon to be projected onto any state of the chosen measurement basis represented by single photon detectors . in our scheme , the detectors , , and correspond to states , , and when is chosen and to states ,  and for the basis .the detectors used are silicon avalanche photodiodes ( apds ) from excelitas technologies with an effective detection efficiency , dark count rate and a dead time of 50 ns .an fpga based timing system is used to record the number of coincidence events between the signal and idler photons with a detection time window of  ns .the respective probabilities are then estimated from the number of detection events at each detector . for each measurement setting , approximately photons were detected per second with a total measurement time of  s.  for the infrastructure establishment round , s prepares randomly one state from each basis ( and ) .alice , bob and charlie choose if they want to apply a operation or not .bearing in mind , the results of the corresponding measurements are presented in table [ tab1 ] .for the voting round , s randomly prepares one of the states ( and ) from the basis .if the voters choose to pose a veto , they apply with .the results of the corresponding measurements are presented in table [ tab2 ] .table [ tab3 ] shows that when the receiver chooses the wrong measurement basis , the measurement result is inconclusive as all the detectors click with equal probabilities .the experimental results for each measurement setting in either round are in good agreement with the expected predictions .moreover , the interferometric visibility of our setup in all measurements is above 89 .this leads to more than 93 success probability in each run of the protocol ( see table [ tab2 ] ) .the estimated errors include poissonian counting statistics and systematic errors .the intrinsic imperfections of the bss , pbss and hwps are the main sources of systematic errors in this experiment .the problem of dining cryptographers is a known problem in quantum communication where proposed classical solutions are not only limited but also complex in application .we have theoretically introduced and experimentally demonstrated a novel quantum solution to this problem using a one - way sequential protocol .the solution also extends to the anonymous veto problem and is simple yet highly efficient based on the distribution of the same photon among the involved parties .the obtained experimental results validate the legitimacy of the protocol while maintaining the anonymity of the involved participants .we believe that the simplicity and completeness of the protocol makes it very attractive from a quantum cryptographic perspective .the project was financially supported by the knut and alice wallenberg foundation , swedish research council and erc advanced grant qolaps .b.m is supported by the grant fapesp n 2014/27223 - 2 and m.w is supported by ncn grant no .umo-2015/19/b / st-2/01999 .99 c. h. bennett and g. brassard , _ quantum cryptography : public key distribution and coin tossing _ , in _ proceedings of the ieee international conference on computers , systems , and signal processing _ , bangalore , p. 175m. fitzi , n. gisin , and u. maurer , _ quantum solution to the byzantine agreement problem , phys .* 87 * , 217901 ( 2001 ) .g. brassard , quantum communication complexity , foundation of physics * 33 * , 1593 - 1616 ( 2003 ) .d. chuam , _ jour_ * 1 * , 65 ( 1998 ) .f. hao and p. zieliski , _ a 2-round anonymous veto protocol , proc .14th works .secur . prot .2006_. m. hillery , m. ziman , v. buek , and m. bielikov , _ phys .a _ * 349 * , 75 ( 2006 ) .r. rahaman and g. kar , arxiv:1507.00592 @ arxiv.org .w. wootters and b. d. fields , _ ann ._ , * 191 * , 363 ( 1991 ) .c. shmidt , p. trojek , h. weinfurter __ , _ phys .lett . _ * 95 * , 230505 ( 2005 ) .p. trojek , c. schmid , m. bourennane __ , _ natural computing _ * 12 * , 19 ( 2013 ) . m. bourennane , a. cabello , and m. ukowski , arxiv:101.1947 @ arxiv.org .s.brierley , s. weigert , and i. bengtsson , arxiv:0907.4097 @ arxiv.org .

Abstract: quantum resources such as superposition and entanglement have been used to provide unconditional key distribution , secret sharing and communication complexity reduction . in this letter we present a novel quantum information protocol for dining cryptographers problem and anonymous vote casting by a group of voters . we successfully demonstrate the experimental realization of the protocol using single photon transmission . our implementation employs a flying particle scheme where a photon passes by the voters who perform a sequence of actions ( unitary transformations ) on the photonic state at their local stations .


in 1929 erwin schrdinger complained `` if i had known we were going to go on having all this damned quantum - jumping , i would never have got involved in the subject '' . over twenty years later he was still not reconciled to the idea , posing the question `` are there quantum jumps ? '' as the title of a paper in the _ british journal for the philosophy of science_. after another 34 years the question was definitively answered in another title when nagourney , sandberg and dehmelt published their paper shelved optical electron amplifier : observation of quantum jumps " in _ physical review letters_. but in spite of the direct and visible evidence for the reality of quantum jumps provided by this beautiful experiment , their place in theory is still not clear .the mathematical formulation of quantum mechanics is just as continuous as schrdinger could have wished ; the jumps , if they are present in the theory at all , are there only as part of the interpretive framework . in this paperi want to examine the necessity for and the possibility of an element of discontinuity in the theoretical framework of quantum mechanics .  to a particle physicist one of the most paradoxical aspects of quantum mechanicsis that it is the only theoretical framework for one s subject and yet it does not acknowledge the fundamental empirical elements in which one is interested .the evidence which must be explained by theories of elementary particle physics consists of _ events _ like the decay of the particle captured in the bubble chamber photograph of figure 1 .but _ there are no events in quantum theory_. has value " . throughout this paperi will use event " with its primary english meaning of happening " . ]the nearest thing to an event described in the basic theory is the result of a measurement something which has been deliberately provoked by an experimenter s conscious action .there is no way that the theory can describe events , like the decay in figure 1 , which happen spontaneously and are passively recorded by the waiting experimenter which is how the photograph of figure 1 was obtained .event ( brookhaven national laboratory , 1964).,width=377 ]  nevertheless , textbooks do purport to derive decay rates for such events from the general principles of quantum mechanics .the derivation goes like this . to describe a decay start at time with a state in which the unstable particle is certain not to have decayed , and follow its time evolution , governed by a hamiltonian , to a superposition of the initial state and a state of the decay products and : the general principles of quantum mechanics are then supposed to yield the interpretation that is the probability that by time there has been a transition from particle to particles .but where does this notion of a `` transition '' come from ?it appears nowhere in the general principles of the theory , as usually stated .the literal application of these principles to the state gives only the statement that _ if a measurement is made at time _ ( to determine , say , whether a particle of type is present ) , then is the probability that the result of the measurement will be positive .the inference that something discontinuous ( a transition ) happened between time 0 and time is completely unwarranted . according to the official principles , quantum systems evolve continuously ( as the time dependence of shows ) , and quantum jumps occur only when provoked by the intervention of an experimenter . despite schrdingers famous satire in his story of the cat , this remains the only description that the conventional form of the theory can tolerate ; the idea of a transition must be a deplorable backsliding into classical habits of thought .  if this is what official quantum mechanics pronounces , then so much the worse for official quantum mechanics . in the actual physical situation that we are trying to describe , it is talk of `` a measurement at time '' that is unwarranted .if a measurement can be said to take place at all when a decay is observed , it is an extended process occupying an interval of time rather than an instant ; and the empirically meaningful time is not the time at which the experimenter decides to make a measurement on the system , but the time at which the system does something for the experimenter to observe .of course , there is no questioning the success of the theory in predicting the right transition rates .the procedure that starts with the time - dependent state vector , produces from it a time - dependent probability , interprets this as a probability of something _ having happened _ , and on the strength of this interpretation derives from a transition probability per unit time , certainly ends up with an empirically adequate result .the problem i want to discuss is that of formulating the theory so as to make this argument as sound as its conclusion .i will argue that the germ of a solution was provided by john bell in 1984 ( ; see also and ) , but that it needs to be transposed into a different interpretative framework in order to be fully satisfactory . in this paperi will do no more than suggest how this transposition might be achieved .  before discussing bell s formulation , however, i would like to sharpen the problem by reviewing ( a ) the result of taking the conventional interpretation literally and pushing it to what seems to be its logical conclusion , and ( b ) the experiments which demonstrate the reality of quantum jumps .a watched pot never boils  let us imagine ourselves in the situation of steadily watching the unstable particle in the process  suppose it is a radioactive nucleus surrounded by a spherical detector wired to a loudspeaker , and we are listening for the click which will announce that the decay has occurred . being attentive observers , we know at every instant of our watch whether we have heard a click or not .how can this situation be described in conventional quantum mechanics ?it seems to require that a measurement is made at every instant , our knowledge at each instant being the result of the corresponding measurement .but this involves an uncountable number of measurements in a finite interval , which , as well as being embarrassing to analyse , probably flatters our vigilance .let us compromise by considering a finite sequence of measurements and trying to capture the ideal of continuous awareness by letting the measurements get closer and closer in time , taking the limit as the separation between them tends to zero .we will replace steady watching with repeated looking .the result is disconcerting .we suppose that in the time interval  ] and |\phi_n(t){\rangle } ,   we can now use the fact that the spontaneously emitted photons quickly move away from the vicinity of the atom to eliminate the terms in from eqs . and the terms containing from .let be an eigenvector of the matrix with eigenvalue ; then eqs . can be solved for the combination as an integral over the interval of an integrand containing , , and .the last two can be removed by applying an annihilation operator or and using the rapid - dispersal approximations   where and stand for r or b , is a constant , is the dispersal time which is of the order of , is any of the -photon states of interest , and is any slowly varying function .this yields and a similar equation with .note that the right - hand side does not contain the eigenvalue .we can repeat this for each of the eigenvectors of the matrix ; then , using the fact that these eigenvectors satisfy we obtain in particular , for we obtain expressions for and in terms of and . putting these into eq . andignoring rapidly oscillating terms with the frequency gives a set of homogeneous equations for the amplitudes which are the same as those obtained by intuitive arguments assuming quantum jumps : where are the ( complex ) decay rates ( including energy shifts ) for the spontaneous transitions and .in general , using to eliminate from the equations for the -photon states yields inhomogeneous equations which have the same matrix as in their homogeneous part , with a source term for involving the -photon states and : ( after dropping rapidly oscillating terms ) .suppose the atom is in its ground state at , no photons having been emitted ; then the initial conditions are , and .the solution of the homogeneous equations with these initial conditions can then be used as a green s function for the inhomogeneous equations ( after taking account of the field hamiltonian by multiplying by ) .the result is with similar expressions for and .it is more meaningful to present these in terms of the actual -photon states , without the exponential factors that were introduced into the definition in order to simplify the calculations .thus , if we define quantities with a tilde so that the full state at time is then the -photon states , and are given by   as in the case of resonance fluorescence in two - state atoms , these expressions have a simple intuitive analysis as superpositions of states in which the atom was in an excited state at some time , having previously emitted photons , emitted the photon at time , and emitted no further photons between times and .( this description , however , does not account for the exponential factors and in and . )the rapid - dispersal property of the photons created by and ensures that the different states in the superpositions do not interfere with each other , so that the probabilities for the existence of photons follow classical laws ; for example , the probability that there are photons at time and the atom is in its red state is which is the probability that would be obtained by a naive argument assuming that the atom is always in one of its three states , that when it is in one of the two higher states it has a probability of decaying into the ground state in time , and that if it is in the ground state at time the probability that it will be in the first excited state at a later time is .contain the full paradox of the description of time development in quantum mechanics .they represent the development of the system as being continuous in time , and yet they describe an experience ( that of watching the trapped atom ) which is undeniably discontinuous .this emphasises the shadowy status of the state vector , whose probabilistic and ontological aspects coexist uneasily .insofar as the state vector has the nature of a probability , it is not surprising that it should change continuously while describing a discontinuous process ; this is normal for a discrete stochastic process .in such a process , however , the states are described quite separately from the probabilities .the formulation of quantum mechanics suggested in the body of the paper can be seen as restoring this separation in the distinction between the pilot state and the visible state , the former being part of the statement of probabilities while the latter is ( in classical terms ) a state specification .

Abstract: it is argued that the conventional formulation of quantum mechanics is inadequate : the usual interpretation of the mathematical formalism in terms of the results of measurements can not be applied to situations in which discontinuous transitions ( quantum jumps " ) are observed as they happen , since nothing that can be called a measurement happens at the moment of observation . attempts to force such observations into the standard mould lead to absurd results : a watched pot never boils " . experiments show both that this result is correct when the experiment does indeed consist of a series of measurements , and that it is not when the experiment consists of a period of observation : quantum jumps do happen . the possibilities for improving the formulation by incorporating transitions in the basic postulates are reviewed , and a satisfactory postulate is obtained by modifying a suggestion of bell s . this requires a distinction between the external description of the whole of a physical system and internal descriptions which are themselves physical events in the system . it is shown that this gives correct results for simple unstable systems and for the quantum - jump experiments .


there are many facets to the analysis of data in high dimensions , depending on the type of application , some relying on dimension reduction , others relying on variable selection and a few employing both tactics .there has been considerable interest in dimension - reduction methods for the regression of a real response on a random vector of predictors since the introduction of sliced inverse regression [ sir ; ] and sliced average variance estimation [ save ; ] .a common goal of these and many other methods is to reduce the dimension of the predictor vector without loss of information about the response .the aim is to estimate a reduction , , with the property that or , equivalently , [ ] . in this way is sufficient because it captures all the information about that is available from .sufficient reductions are not determined uniquely by this definition because any bijective transformation of is also sufficient .nearly all methods for sufficient dimension reduction ( sdr ) restrict attention to the class of linear reductions , which arise naturally in many contexts .linear reduction can be represented conveniently in terms of the projection of onto a subspace . if , then is called a dimension - reduction subspace . under mild conditionsthe intersection of any two dimension - reduction subspaces is again a dimension - reduction subspace and that being so the central subspace , defined as the intersection of all dimension - reduction subspaces , is taken as the inferential target [ cook ( , ) ] .a minimal sufficient linear reduction is then of the form , where is any basis for .sdr has a long history of successful application and is still an active research area .recent novel sdr methods include likelihood - based sufficient dimension reduction [ ] , kernel dimension reduction [ ] , shrinkage inverse regression estimation [ ] , dimension reduction for nonelliptically distributed predictors [ , ] , cumulative slicing estimation [ ] , dimension reduction for survival models [ ] and dimension reduction for spatial point processes [ ] .this body of work reflects three different but related frontiers in sdr : extensions that require progressively fewer assumptions , development of likelihood - based methods and adaptations for specific areas of application .almost all sdr methods rely on traditional asymptotic reasoning for support , letting the sample size with fixed .they nearly all require the inverse of a  sample covariance matrix and thus application is problematic when .since accurate estimation of a general covariance matrix can require observations , it has seemed inevitable that sdr methods would encounter estimation problems when is not sufficiently large . , and others circumvented these issues by performing reduction in two stages , first replacing the predictors with principal components and then applying an sdr method to the regression of the response on the selected components .however , recent results on the eigenvectors of sample covariance matrices in high - dimensional settings raise questions on the value of such two - stage methods [ see , e.g. , ] . proposed an sdr method that avoids computation of inverses and reduces to partial least squares in a special case . showed recently that the partial least squares estimator of the coefficient vector in the linear regression of on is inconsistent unless and this raises questions about the behavior of the cook et al .sdr estimator when is not large relative to . used the least squares formulation of sliced inverse regression originated by to develop a regularized version that allows and achieves simultaneous predictor selection and dimension reduction .this seems to be a promising method , but its asymptotic properties are unknown and it may not work well when the regression is not sparse . studied the asymptotic properties of a family of sdr estimators , using a scad - type penalty for variable selection in sparse regressions where the number of relevant variables is fixed as .their method requires that for consistency .  while sparsity is an important concept in high - dimensional regression , not all high - dimensional regressions are sparse .for example , near - infrared reflectance is often measured at many wavelengths to predict the composition of matter , like the protein content of a grain .there is not normally an expectation that only a few wavelengths are needed to predict content . while some wavelengths may be better predictors than others , it is the cumulative information provided by many wavelengths that is often relevant .partial least squares has been the dimension - reduction method of choice in this type of regression .the regressions implied by this and other nonsparse applications share similar characteristics : ( 1 ) the predictor vectors are high - dimensional and typical area - specific analyses have employed some type of dimension reduction ; ( 2 ) while assessing the relative importance of the predictors may be of interest , prediction is the ultimate goal ; ( 3 ) information on the response is thought to accumulate , albeit perhaps slowly , as predictors are added , and ( 4 ) sparsity is not a driving notion .  in this articlewe introduce a family of sdr methods for studying high - dimensional regressions that differs from past approaches in at least three important ways .first , we do not require sparsity but rather we emphasize _ abundant regressions _ where most of the predictors contribute some information about the response . in the logic of ,the bet - on - sparsity principle arose because , to continue the metaphor , there is otherwise little chance of a reasonable payoff .we show in contrast that reasonable payoffs can be obtained in abundant regressions with prediction as the ultimate goal , leading to a contrasting _ bet - on - abundance _ principle .second , sdr studies have largely focused on properties of estimators of .we bypass this step and instead consider the limiting behavior of estimators of the sufficient reduction itself , assuming that the dimension of is fixed . more specifically , letting denote an estimated reduction, we establish rates of convergence in the following sense .let denote a new observation on  . if and if as , then is consistent for and its convergence rate is at least .third , we integrate recent work on the estimation of high - dimensional covariance matrices into our approach .in particular , we estimate a critical matrix of weights by using sparse permutation invariant covariance estimation ( spice ) as developed by .  in sum , by considering the reduction itself rather than the central subspace , we both introduce a novel viewpoint for addressing dimension reduction and develop theoretically grounded sdr methodology for regressions where other methods either have no asymptotic support or must necessarily fail .we describe the model for our study and its sufficient reduction in section [ secmodelstate ] . the class of estimators that we use is described in section [ secestimation ] , and stabilizing restrictions are presented in section [ secsignal ]. sections [ secwhat ] and [ secnormalerrors ] contain theoretical conclusions for selected estimators from the class described in section [ secestimation ] .simulation results are presented in sections [ secsimw ] and [ secsimulations ] .we turn to a spectroscopy application in section [ secdata ] and a concluding discussion is given in section [ secdiscussion ] . all proofs andadditional simulation results are available in a supplemental article [ ] .the following notational conventions will be used in our exposition .we use to denote the collection of all real matrices .we use and to denote the spectral and frobenius norms of .the largest and smallest eigenvalues of are denoted and .if , then is the diagonal matrix with diagonal elements equal to those of . is the operator that maps to  by stacking its columns .if and is symmetric and positive definite , then the operator that projects in the inner product onto , the subspace spanned by the columns of , has the matrix representation , and . indicates the projection onto in the usual inner product .a basis matrix for a subspace of dimension is any matrix whose columns form a basis for . for nonstochastic sequences and , we write if there are constants , and such that for all .similarly , for stochastic sequences and  , we write if and . for , means that is positive definite . denotes the kronecker product , and means and are equal in distribution . deviating slightly from convention , we do not index quantities by and , preferring instead to avoid notation proliferation by giving reminders from time to time .we assume throughout that the data consist of independent observations , , and that is increased by taking additional measurements on each of sampling units .we treat the process of adding predictors as conditional on the response and so our approach is based on inverse reduction , , which is a standard sdr paradigm .limits as are thus conditional on the responses unless specifically indicated otherwise .   the model that we employ is engendered by standard sdr assumptions .we first review those assumptions briefly and then give a statement of the model .classical methods like sir require two predominant and well - known conditions for estimation of .the first , called the _ linearity condition _, insists that be a linear function of , where is a basis matrix for  .it must be valid only at a true basis and not for all .this condition is generally regarded as mild since it holds to a good approximation when is large [ ] .let denote the subspace spanned by as varies in the sample space of .then the linearity condition implies that , which is used as a basis for estimating a subspace of .the second , called the _ coverage condition _ , requires that and it too is generally regarded as mild in many applications . while the linearity and coverage conditions are standard requirements for root- consistent methods based on the inverse mean function , the actual performance of those methods depends also on the inverse variance function .for instance , concluded based on simulations that sir works best when is nonstochastic , and demonstrated analytically that sir can be quite inefficient when varies .we refer to the requirement that be nonstochastic as the _ covariance condition_. if the linearity and covariance conditions hold , then must be nonstochastic as well .this is related to the usual covariance condition is constant used by save and other second - order methods .see , and for further discussion of these conditions .we assume the linearity , coverage and covariance conditions as a basis for our study . under these conditionsit can be shown straightforwardly that , where . consequently , letting be a basis matrix for , we are led to the model where , the error vectors are independent copies of a random vector with mean 0 and variance , and is the instance of an unknown vector - valued function with that gives the coordinates of in terms of and is independent of . in this representation .neither nor is identified in model ( [ origmodel ] ) , because for any conforming nonsingular matrix , , leading to a different parameterization .this nonuniqueness has been mitigated in past studies with fixed by requiring .however , that parameterization is not workable when allowing , and for this reason we adopt different restrictions .the next step is to reparameterize model ( [ origmodel ] ) to satisfy constraints that will facilitate our development .specifically , we construct an equivalent model where , is the coordinate vector relative to the new basis matrix , and we center the s in the sample so that .let the rows of and be and , .without loss of generality , we impose on model ( [ vecmodel ] ) the constraints that ( 1 ) , where is a _ scaling matrix _ that is defined in section [ scaling ] , and ( 2 ) is a diagonal matrix where is a symmetric population _ weight matrix _ that is discussed in section [ secestimation ] . to see how thisis done starting from model ( [ origmodel ] ) , let and let be an orthogonal matrix so that is a diagonal matrix .then , where and satisfy the constraints by construction . model ( [ vecmodel ] ) can be represented also in matrix form as where and have rows and , and has mean and variance .since bijective transformations of a sufficient reduction are themselves sufficient , we define to be the coordinates of the projection of onto in the inner product : where the first factor stabilizes as .without further structure it is not possible to use model ( [ vecmodel ] ) to estimate since the coordinate vectors are unknown . however , estimation is possible by approximating the coordinate vectors as , where , , and is the realization of a known user - selected vector - valued function . without loss of generalitywe center the sample .let be the matrix with rows , and assume that and , unless and then of course is nil .we next describe the class of estimators we use , postponing discussion of possible choices for until section [ choose - f ] .the class of reduction estimators that we study is based on using estimators of from a subclass of the family of inverse regression estimators proposed by : where the minimization is over , and subject to the constraints that and that is a diagonal matrix .a particular estimator is determined by the choice of the sample weight matrix with being a population version .some instances of  that we introduce later correspond to , and .we next report the estimators from this family .a sketch of the derivation is available in the supplemental article [ ] .it is easily seen that .let , and let denote the matrix of regression coefficients from the least squares fit of on , assuming that .also , let the columns of be the first eigenvectors of assuming that the eigenvalue of is strictly larger than the eigenvalue .this assumption will be true with probability 1 for the choices of  considered here .then the estimators are , , and the diagonal elements of the diagonal matrix consist of the first ( largest ) eigenvalues of , and is the coefficient matrix from the ols fit of on  .   clearly , we should strive to choose an so that , for some , is well approximated by for .this intuition is manifested in various calculations via the requirement that for all  . as an extreme first instance , suppose that . then and consequently  provides no information on .since , this requirement is essentially that the true coordinate vector is sufficiently correlated with its approximation , and it is equivalent to the condition derived by under a pfc model with normal errors and fixed . in the remainder of this articlewe assume that , for all , where the order is nonstochastic because we condition on the observed values of in our formal asymptotic calculations .there are many ways to choose an appropriate in practice . under model ( [ vecmodel ] ) , each coordinate , , of follows a univariate linear model with predictor vector .when is quantitative we can use inverse response plots [ , chapter 10 ] of versus , , to gain information about suitable choices for .when is too large for a thorough graphical investigation , which is the case in the context of this article , we can consider s that contain a reasonably flexible set of basis functions , like polynomial terms in . in some regressionsthere may be a natural choice for .for example , suppose that is categorical , taking values in one of categories , .we can then set and specify the coordinate of to be the indicator function .another option consists of `` slicing '' the observed range of a continuous into bins ( categories ) , . a rudimentary set of basis functionscan then be constructed by specifying the coordinate of as for the case of a categorical .this has the effect of approximating each conditional mean as a step function of with steps .this option is of particular interest because it exactly reproduces sirs estimate of in the traditional large- setting [ ] .cubic splines with the endpoints of the bins as the knots are a more responsive option that is less prone to loss of intra - slice information .our goal is to study the limiting behavior of , where and are given by ( [ popred ] ) and ( [ samplered ] ) .this difference depends on the choice of and the behavior of the true reduction as . in this sectionwe measure and constrain the interaction between  and the model .we also place weak constraints on to help ensure well - behaved limits .the context described in this section will be assumed to hold throughout this article , without necessarily being declared in formal statements .we will revisit these constraints occasionally during the discussion , particularly when some of them are implied by other structure .since bijective transformations of sufficient reductions are also sufficient , we need to have and in the same scale to ensure that their difference is a useful measure of agreement .this can be accomplished by choosing the scaling matrix so that the first model constraint stated following ( [ vecmodel ] ) becomes , which is a rank- matrix by ( [ rank ] ) .this choice ensures that is the lead term in the asymptotic expansion of , which places and on the same scale .we define the _ signal rate _ to be a positive monotonically increasing function that measures the rate of increase in the population signal : let , which is a diagonal matrix because is diagonal by construction .then is chosen to meet the requirement where is a diagonal matrix that is assumed to be positive definite with finite elements .the signal rate is not needed for computation of but it will play a key role in later developments .it depends via on the specific estimator selected , although this is not indicated notationally . when the bounds can be used to aid intuition on the magnitude of by presuming properties of and .for example , consider regressions in which and are bounded away from 0 and as and a positive fraction , , of the rows of is sampled from a multivariate density with finite second moments and the other rows of are all zero vectors .then the number of nonzero rows , , and we say that the regression has an _ abundant signal_. regressions with _ near abundant signals _ like may often perform similarly in practice to regressions with abundant signals .it is technically possible to have regressions in which , but we would not normally expect this in practice . on the other extreme , sparse regressions are those in which , so only finitely many predictors are relevant or the signal accumulates very slowly as .regressions with _ near sparse signals _ have , say , . typically we will use in definitions and formal statements , but use the abbreviated form otherwise .we assume in the rest of this article that ( [ h ] ) holds .our second requirement is that , as , the spectral norm converges to a finite constant , which may be 0 .if this is not so , then the variance of some linear combinations will diverge as , and the very notion of dimension reduction for large- regressions becomes problematic .a reduction is stable if .the following lemma gives a sufficient condition for stability that incorporates the signal rate . in preparation , define which is manifested in various asymptotic expansions as a measure of the agreement between and .[ lemmastable ] if , then reduction ( [ popred ] ) is stable .  according to lemma [ lemmastable ] , if , then the reduction is stable regardless of .all regressions in which is bounded yield stable reductions .bounded eigenvalues have been required in various studies to avoid ill - conditioned covariance matrices [ see , e.g. , and ] . more generally , the requirement for a stable reduction is that must increase at a rate that is no less than the rate at which increases . in particular , if , then the sufficient condition of lemma [ lemmastable ] requires that is bounded .the rates that we develop depend on the functions ^{1/2}$ ] and .the interpretation and roles of these functions will be discussed later ; for now we state across - the - board requirements that , as , these functions will nearly always be written without their arguments .we assume in the rest of this article that all regressions are stable and that the orders given in ( [ kappa ] ) hold .so far we have assumed only that the errors are independent copies of the random vector  which has mean 0 and variance .we impose two additional constraints to ensure well - behaved limits : as ,   , and   .  condition ( w.1 ) , which is equivalent to , seems quite mild .for example , if we use unweighted fits with , then this condition is simply that the average error variance is bounded .condition ( w.2 ) can be seen as placing constraints on and on the fourth moments of  .it too seems mild , although it is more constraining than the first condition .the following lemma describes a few settings in which condition ( w.2 ) holds .[ lemmaw ] let so that and .let , . then :  if , then condition implies condition .if and if as , then condition holds .  if the elements of have symmetric distributions or are independent and if as , then condition implies conditionwe assume in the rest of this article that conditions ( w.1 ) and ( w.2 ) hold .in this section we describe our results for the limiting reduction when converges in the spectral norm . specifically ,we require the following two conditions :  there exists a population weight matrix so that the spectral norm of converges to 0 in probability at rate at most as ; equivalently , as .   as .these conditions are implied by the stronger condition that , ( s.1 ) following from chebyshev s inequality and ( s.2 ) arising since .all of the weight matrices discussed in this article can satisfy these conditions , as well as other weight matrices that we have considered , so these conditions do not seem burdensome . for ease of reference we denote the corresponding estimator as .   we state and discuss one of our main results in this section . in preparation , let where is as defined in ( [ h ] ) , let and define the random vector , where is the population reduction using applied to the error of a new observation and is the targeted population reduction ( [ popred ] ) applied to the same error .the vector measures the population - level agreement between the user - selected reduction and the ideal reduction .it has mean and variance where the semi - orthogonal matrix .the next proposition describes asymptotic properties of and  , where was defined in ( [ khat ] ) .[ propwhat1 ] [ propwhat2 ] assume conditions and .then :  if , then . .  if , in addition , , then where and were defined in ( [ kappa ] ) .this proposition shows that the asymptotic properties of and depend on four quantities , each with a different role .we defer discussion of the order that measures the asymptotic behavior of to later sections .in the rest of this section we concentrate on the remaining and assuming that is nonstochastic , so , and .this involves no loss since none of these terms depends on .terms involving affect both estimation and prediction .if were to diverge , then , from proposition [ propwhat1](i ) , eventually will look like a diagonal matrix and there will be little signal left , which is why we imposed the universal condition ( [ kappa ] ) that .if diverges , then again will resemble a diagonal matrix , but this is prohibited by universal condition ( w.1 ) .here we also see the role of the rank condition ( [ rank ] ) introduced at the beginning of section [ choose - f ] . if , then and again some signal will lost . in the extreme ,if , then and all information on is lost .if , then increases faster than increases . in effectthere is a synergy between the signal rate and the sample size . if the regression is abundant , so , then .useful results can also be obtained when , because then , which may be small in some regressions . in sparse regressions where , , and we are back to the usual requirement that for consistency of . equally important , since does not depend on the weight matrix , the results indicate that it may not be possible to develop from the family of estimators considered rates of convergence that are faster than .the and terms arise from prediction . the first term has a characteristic that is different from the others because it does not depend on . since , the sufficient stability requirement of lemma [ lemmastable ] guarantees that is bounded as .while the contribution of will be negligible if is sufficiently small , for the best results we should have as .let be an orthogonal matrix . using a result from cook and forzani [ ( ) , eq .( a.4 ) ] , consequently , when is a reducing subspace of , even if .this result is similar to zyskind s ( ) classical findings about conditions for equality of the best and simple least squares linear estimators in linear models .if , then and is trivially a reducing subspace of . if is a generalized inverse of , , and if , then again reduces and .turning to , since , it follows that .consequently , if , then and . in a worst - case scenario ,if and diverges , then because of the requirement in proposition [ propwhat1](iii ) , and these terms reduce to the usual condition that .  in sparse regressions, the condition reduces to and means that must be bounded .these two conditions imply that .consequently , the best results in sparse regressions will be achievedwhen ( a ) is a reducing subspace of , ( b ) and ( c ) is bounded .in nonsparse settings will yield the best results when is bounded and the regression is abundant or near abundant .we summarize some implications of these conditions in the following corollary .[ corw21 ] assume that and that .then :   .if the regression is abundant , then .if is a reducing subspace of , then . if , in addition , the regression is abundant , then .the conclusions of proposition [ propwhat2 ] and its corollary [ corw21 ] hold as and in the required relationships and as with fixed . in the latter case the results simplify to those given in the next corollary .[ corw22 ] if and , then :   .  if , in addition , is a reducing subspace of , then .it may be clear from the previous discussion that the best possible rate is achieved when and then .consequently , we refer to as the _ oracle rate_.   we conducted a simulation study with to show the importance of , to demonstrate that proposition [ propwhat2 ] and corollaries [ corw21 ] and [ corw22 ] give good qualitative characterizations of the limiting behavior of , and to provide some intuition into the nonasymptotic behavior of .the simulated data were generated using a simple version of model ( [ vecmodel ] ) : , , , was constructed as a vector of standard normal random variables and is a diagonal matrix .specific scenarios may differ on , and the choice of . in any case, the true model then has .we confined attention to regressions with because we found that there is nothing in principle to be gained from settings with . for each sample size we used the estimators described at the end of section [ secestimation ] with and , , as the elements of , so and .  for each data set constructed in this way , we determined  at new data points generated from the original model .we summarized each data set by computing the absolute sample correlation between and and the sample standard deviation of the difference over the 100 new data points .this process was repeated at least 400 times for , 200 times for and 50 times for .the average absolute correlation and average standard deviation were used as overall summary statistics . our decision to use a diagonal matrix for  was based on the observation that the asymptotic results depend on  largely via and with , .  [ cols="^,^ " , ]   since there are only predictors it was straightforward , albeit somewhat tedious , to inspect inverse response plots of versus , , to gain information about the likely structure of  .we performed two analyses ; the first consisted of 54 pork samples and the second consisted of 103 meat samples of beef and pork . for a specified value of , we assessed the fitted model by using the residual mean square , where the fitted values were determined using ( [ ehat ] ) with the indicated combination of , and reduction  .we selected the value of by adapting the permutation scenario developed by .the hypothesis was tested sequentially , starting at and estimating as the first hypothesized value that was not rejected .the test statistic was compared to the distribution of induced by random permutations  applied to the rows of the predictor matrix as follows : where and were computed under the null hypothesis .this scheme leaves the signal intact while permuting the uninformative part of the predictors .the multicollinearity of the predictors made computing the weight matrix estimator for difficult for small values of its tuning parameter , values for which our cross - validation procedure recommended .we subsequently set its tuning parameter to for both analyses , since this was the smallest value for which a numerically stable solution was available .   in this casewe concluded that a cubic polynomial would be adequate ; a representative plot is shown in figure [ figporkinvresp](a ) . performing the permutation test to select ,the test statistic for using was which was smaller than the smallest value of observed among the random permutations under the hypothesis that .since the test statistic is much smaller than can be accounted for by chance under the null hypothesis , we concluded that . similarly , to test , we observed which fell at the 80th quantile of the permutation distribution of under the hypothesis .consequently , we used for the model .comparisons with other values of figured in our choice .cross - validation using as the criterion might also be used to select .=-1  @  ,  ,  and , using the pork samples.,title="fig : " ] & ,  ,  and , using the pork samples.,title="fig : " ] + ( a ) & ( b ) + ,  ,  and , using the pork samples.,title="fig : " ] & ,  ,  and , using the pork samples.,title="fig : " ] + ( c ) & ( d )  fitting each of the four estimators with , we found that , , and for , , and .plots of versus are shown in figure [ figpork](a)(d ) .the predictors in this illustration are highly collinear , as is typical in spectral data , and the relative performance of the four estimators is qualitatively similar to that shown in previous simulations ; however , numerical instability degraded the performance of .the relative signal rates for the four estimators are reflected by the values of , , and for , , and .in this case we concluded that a second - order polynomial and the indicator function of beef would be adequate , ; a representative plot is shown in figure [ figporkinvresp](b ) . using the permutation test approach to select ,the test statistic for using was which fell at the 0.003 quantile of observed among the random permutations under the hypothesis that .since the test statistic is much smaller than can be accounted for by chance under the null hypothesis , we concluded that .similarly , to test , we observed which fell at the 0.70 quantile of the permutation distribution of under the hypothesis .consequently , we used for the model .@  ,  ,  and .circles represent pork and plus signs represent beef.,title="fig : " ] & ,  ,  and .circles represent pork and plus signs represent beef.,title="fig : " ] + ( a ) & ( b ) + ,  ,  and .circles represent pork and plus signs represent beef.,title="fig : " ] & ,  ,  and .circles represent pork and plus signs represent beef.,title="fig : " ] + ( c ) & ( d )  fitting each of the four estimators with , we found that , , and for , , and .plots of  versus  are shown in figure [ figboth](a)(d ) .the relative signal rates for the four estimators are reflected by the values of , , and for , , and . the relatively large signal for reflected in the plots of figure [ figboth ] .the class of estimators that we studied is limited in scope relative to the range of sdr methods presently available for regressions . however , in the broader context of this article , which does not require , we introduced the concept of an abundant regression and presented what may be the first asymptotic analysis of a class of sdr methods when the focus is on estimating a reduction rather than on the underlying central subspace .these ideas can in principle be extended for other sdr methods , and we expect that the same general issues will be encountered . while each of the methods that we studied can perform usefully in the right situation , we judge the spice and weight matrices to be the best overall , although improvements for nonsparse weight matrices and for regressions with very highly correlated predictors are still needed .our simulation results were all based on normal errors to focus the presentation and save space .however , we have also conducted a variety of parallel simulations using uniform , and errors , each centered and appropriately scaled .the results were essentially the same as those with normal errors [ ] .alternative penalty functions may be used for estimating the weight matrix , particularly in scenarios when the inverse error covariance matrix is not sparse .for example , the sparse - seeking penalty function in ( [ optomega2 ] ) , , could be replaced with the quadratic penalty function , where controls whether or not the diagonal of , the inverse error correlation matrix , is penalized . if , the general spice algorithm developed by can efficiently solve ( [ optomega2 ] ) with the penalty defined in ( [ quadpenalty ] ) . if , derived an noniterative solution to an equivalent problem to ( [ optomega2 ] ) with the penalty defined in ( [ quadpenalty ] ) .recalling that denotes the eigenvalue of a matrix , let .witten and tibshirani showed that the eigenvectors of are equivalent to the eigenvectors of and that .the general rates given in proposition [ propwhat1 ] are not very sensitive to the choice of since they hold when satisfies the minimal rank condition ( [ rank ] ) .nevertheless , assuming normality and a correct , we obtained the oracle rates of proposition [ propwdelta1 ] , which indicates that there are advantages to pursuing good choices .the methods sketched in section [ choose - f ] are often useful in practice , but it is also possible to develop semiparametric methods to estimate directly rather than passing through approximations  .this might be accomplished iteratively : choose an initial and construct the corresponding estimates , and .a new estimate of can be obtained by smoothing the coordinates of against  , leading to a second reduction estimate .the process can now be continued until some convergence criterion is met .while we did not incorporate screening or variable selection into our reduction methodology , the potential benefits of those procedures are manifested in our results .consider , for instance , a regression in which of the predictors are inactive .then the oracle rate .however , if we remove of the inactive predictors , the oracle rate is increased to , which should be worthwhile in most applications .work along these lines is in progress .the authors are grateful to the referees whose helpful comments led to significant improvements in this article .

Abstract: we study the asymptotic behavior of a class of methods for sufficient dimension reduction in high - dimension regressions , as the sample size and number of predictors grow in various alignments . it is demonstrated that these methods are consistent in a variety of settings , particularly in abundant regressions where most predictors contribute some information on the response , and oracle rates are possible . simulation results are presented to support the theoretical conclusion .  ,  .  .


functional data analysis has grown in popularity for the last years due to the increasingly data availability for continuous time processes .typical examples of functional data include the temperature evolution , stock prices and path trajectories for objects in movement .new statistical methods have been developed to deal with the richer nature of functional data , being , and some of the main reference books in this area . + in many situations , the functional data is related to a scalar variable . for this cases, it is interesting to assess the relation of the variables via a regression model , which can be used to predict the scalar response from the functional input . analogue to the multivariate situation , the simplest functional regression model corresponds to the functional linear model with scalar response ( see for a review ) .+ an interesting methodology approach to deal with functional data is the use of random projections .the objective is to characterize the behaviour of a functional process , which has infinite dimension , via the behaviour of the one dimensional inner products of the functional process with suitable random functions .this method has interesting applications for the goodness of fit of the distribution of the process , as it can be seen in .more recently , provide a projection based test for functional covariate effect in a functional regression model with scalar response . in their paper, the authors adapt the tests of and , based on smoothing techniques , to the context of functional covariates .+ in this work , a first goodness of fit test for the null hypothesis of the functional linear model , , being the hilbert space of square integrable functions , is proposed .the statistic test is of a cramr von mises type and is based on a generalization of a previous test of , designed for the case of a regression model with multivariate covariates .the test statistic is easy to compute using geometrical arguments and simple to calibrate in its distribution by a wild bootstrap on the residuals .further , although the test is given for the functional linear model , it can be extended to other functional models with scalar response , as it is based on the residuals of the model . +this work is organized as follows .some background on functional data , the functional linear model and the random projections paradigm are introduced in section [ back ] .the main part of this work is section [ thetest ] , where the theoretical arguments of the test , jointly with the bootstrap calibration procedure , are presented .the finite sample properties of the test are illustrated by a simulation study in section [ simu ] .section [ appli ] illustrates the application of the test to two datasets and introduces a graphical tool to evaluate the goodness of fit of the functional linear model with scalar response .final comments and conclusions are given in section [ final ] .an appendix attached contains omitted proofs , tables and figures .the main goal of this paper is to propose a goodness of fit test for the null hypothesis of the functional linear model with scalar response .bearing in mind the different nature of the functional variables , some background on functional data , the functional linear model and the use of random projections is introduced .one of the first and most important problems when we deal with functional data is to choose a suitable functional space to work .the most used functional spaces are the metric , the banach and the hilbert spaces .this is a sequence of functional spaces with increasing richer structure , where the tools available for the former space are included in the latter .specifically , in a metric space we can measure distances between functions ; in addition , in a banach space we can also measure the functions and cauchy sequences are convergent ; and finally , in a hilbert space we have inner product , which allows to consider functional basis . + while there are a lot of types of metrics and norm spaces , the spaces are one of the most used .the  ] such that their norm is finite .the choice of the interval  ] , their inner product is defined as   in what follows we will consider as our working space the hilbert space  ] can be trivially replaced by another interval .the inner product allows for a basis representation of the elements of and , given a functional basis of , then any function in can be expressed by the linear combination , where , .a basis is said to be orthogonal if , and orthonormal if , in addition , , .typical examples of basis of are the fourier basis and the b splines basis ( see ) . + for the development of the test statistic, we will also need to introduce a basis , which corresponds to the first elements of the infinite basis .the representation of in this truncated basis is denoted by .the choice of the number of basis elements is crucial to have a reliable representation of the function by .although there exists several methods to select an appropriate , we will refer to the gcv criteria ( see , page 97 ) to select and represent adequately the function in .this criteria will be used in section [ simu : simp ] to select a suitable for the case of the simple hypothesis .+ to deal with functional random projections we will need to define the functional analogue of the euclidean  . in the functional case we have the _ functional sphere _ of , defined as , and the _ functional sphere of dimension _ , which is the set of functions of that , expressed in the basis , have unit norm : .+ the relationship between and is particularly interesting to develop the test .let be the matrix of inner products of the basis , the generated by this matrix and the cholesky decomposition of ( a semi positive matrix ) .first of all , we have the trivial isomorphism that maps elements of to elements of by means of the functional coefficients : .recall that functions and are well defined because .we must consider also a linear transformation from to , which is given by and whose jacobian is , the determinant of the matrix .+ using these two transformations , the integration of a functional operator with respect to a functional covariate in can be reduced to a real integration on the : in the case where the basis is orthonormal , and are the identity matrix of order . then the coefficients of in the basis belong to without any transformation .suppose that is a functional random variable in and is a real random variable .if both variables are centred , i.e. , }}=0 ] and }}=0 ] .the prediction of is done with the conditional expectation of given : }}={\left<{\mathcal{x}},\beta\right>}.\end{aligned}\ ] ] saying that share the functional linear model is equivalent to saying that the regression function of on , , belongs to the family .+ given a sample , the estimation of the functional parameter can be done by minimising the residual sum of squares ( rss ) : a possible method to search for the parameter that minimises the rss is representing the functional data and the functional parameter in the truncated functional basis and , respectively : using the vector notation , , , and , the previous representation can be expressed as and .the functional linear model results in where .then , basis representation allows to express the flm as a standard linear regression , where the estimated coefficients of in the basis are given by .although different combinations of and are possible , the usual choice is , being an orthogonal basis because in that case the matrix is diagonal .+ there are several alternatives to represent the functional process and estimate the parameter in a truncated basis .for instance , a general review of the estimation based on the use of basis expansions such as fourier series or b splines can be found in the book by . the so called functional principal component ( fpc ) regression estimation , proposed by , provide an orthogonal data driven basis that gives the most rapidly convergent representation of the functional dataset predictor in a sense ( see ) . proposed the functional partial least squares ( fpls ) regression method that produces iteratively a sequence of orthogonal functions , as the fpc are , but with maximum predictive performance . to implement any of the methods shown before ,it is required to fix the number of basis elements ( or components ) that are used in the estimation .+ the optimal number of components , , has to be fixed based on the information provided by the data .to do this , and use the predictive cross validation criterion ( pcv ) , and consider the generalized cross validation criterion ( gcv ) and and consider those methods based on the aic , aicc and bic information approaches .+ let denote by and the prediction of using components with the whole sample and with the whole sample excluding the element , respectively .the pcv is defined as : which is computationally expensive because it involves the estimation of the  times .this is especially expensive in the case of data driven basis ( fpc , fpls ) because the basis has to be recalculate for every datum . as an alternative , gcv avoids recalculating the for every datum by introducing a penalty term .the gcv is defined as where is the number of degrees of freedom consumed by the model , typically given by the trace of the matrix .gcv is closely related with aic , aicc and bic although they come from different perspectives .random projections are becoming quite popular when dealing with high dimensional data , as a way to overcome the well known _curse of the dimensionality_. the main idea behind is to reduce the dimension , and characterize the original distribution of the multidimensional data by the distribution of the randomly projected univariate data . + in the goodness of fit field , this is specially interesting , as the test procedures tend to become less efficient , less powerful , when the dimension of the model increases . used this technique to develop a goodness of fit test for multivariate regression models based on random projections . according to his simulation study, the test has an excellent power performance and has the best empirical power for most situations when comparing to their competitors .+ in the functional framework , it is also possible to consider random projections .usually , this is achieved by considering the inner product of the functional variable of and a suitable family of projectors , i.e. random functions in .for example , using with this approach developed some goodness of fit tests for parametric families of functional distributions , which includes goodness of fit tests for gaussianity and for the black scholes model . +a very interesting result on projections can be found in . in their paper , the authors provide a characterization of the conditional expectation of a scalar variable with respect to a functional variable given in terms of the conditional expectation of with respect to the projected .the result is stated here in the following lemma .[ l1 ] let be a random variable and a functional random variable in the functional space .the following statements are equivalent :  1 .  }}=0 ] , for a.e . and .[ l1:2 ] 3 .}}=0 ] , for a.e . .[ p1:2 ] 3 .}}=0 ] , for a.e . and , .[ p1:3b ] 5 .}}=0 ] , for a.e . and , .[ p1:4b ]  then is characterized by the null value of the moment }} ] .for example , if is a discrete random variable with distribution weights  , we have the _ golden section bootstrap_.[boot2 ] 3 .construct the bootstrap residuals .[boot3 ] 4 .set and estimate for the sample .[boot4 ] 5 .obtain the estimated bootstrap residuals .[boot5 ]  then , the procedure to calibrate the test is the following . in step [ boot1 ]we compute the test statistic with the residuals under using the implementation ( [ efficpcvm ] ) of the previous section. then repeat steps [ boot2][boot5 ] for , computing each time the bootstrap statistic and estimate the of the test by monte carlo : . for computational efficiency, it is important to note that we do not have to compute again the matrix in the bootstrap replicates . +a very interesting fact of the flm is that step [ boot5 ] can be easily performed using the properties of the estimation of . from ( [ modlin ] ) it is clear that the vector of coefficients of is estimated throughout .then , the estimated bootstrap residuals , represented by the vector , can be obtained as , where is the vector of bootstrap responses given in step [ boot4 ] and is the identity matrix of order .the projection matrix remains the same for all the bootstrap replicates , so it can be stored without the need of computing it again .obtaining the residuals in this way implies a significative computational saving .+ the bootstrap resampling in the case of the simple hypothesis is easier : just replace by and omit steps [ boot4 ] and [ boot5 ] , considering .to illustrate the finite sample properties of the proposed test , a simulation study was carried out for the simple and the composite hypotheses . the functional process considered for the functional covariate is an ornstein uhlenbeck process in  ] .see figure b2 in appendix for further details . +all the functional data in this simulation study is represented in equidistant points in the interval  ] .this is equivalent to test that the functional covariate has no effect on the scalar response , i.e. , test the null hypothesis .although there is an extensive collection of goodness of fit tests for finite dimensional covariates ( see ) , the literature for the case of functional covariates is more limited . therefore , we will focus on the competing procedures of and to compare the different tests in terms of level and power .let us describe briefly these two test statistics .+ propose a test statistic for , deriving its asymptotic law and giving a bootstrap procedure based on the residuals .the statistic , inspired in the propose of , is where is a kernel function , is a semimetric and is the bandwidth parameter . represents the probability distribution of the functional process and is a suitable weight function .the test used in our implementation results from considering no functional effect , i.e. , and from approximating the integral with respect to by the empirical mean of the sample .we have also considered the kernel , being the density of a , the distance in for and the uniform weight function .the bandwidth parameter is given by the pcv criterion and bootstrap resampling was done using golden wild bootstrap .+ the other competing test is the one proposed by and is based on the idea of extending the covariance to functional scalar data : where is the functional mean of and is the usual scalar mean of .the authors extend the ideas of the classical to the functional framework , resulting a statistic to test the null hypothesis of no interaction _ inside _ the functional linear model .the test is consistent and the authors derived the asymptotic distribution of the process , resulting in a brownian motion with mean }} ] .for block 1 the snr s of the alternatives are , and , respectively for , . for block 2 ,the snr s are , and . for block 3, we have , and .+ in the case of the simple hypothesis there is no estimation of the parameter , as it is known .however , it is necessary to express the functional process and the function in a suitable basis in order to compute the test statistic . up to this end , we consider a b splines basis and we choose automatically its number of elements by the gcv criteria commented in section [ func ] .+ the results of the study for the simple hypothesis are collected in table [ tab1 ] , which shows the empirical sizes and powers of the functional , the test of delsol et al . and the pcvm test for simple hypothesis , for the models previously commented .all of the tests seem to calibrate the significance level .with respect to the power , the functional has in average a superior behaviour in the alternatives , which represents deviations from the null _ inside _ a linear model . the test of delsol et al .performs also well with the cross validatory bandwidth , being the most competitive for the block .the pcvm test performs worse than the functional for alternatives and and similarly to the test of delsol et al .. nevertheless , for alternatives that are not inside the linear model , the pcvm test results the most powerful .similar results are obtained with a noise given by a recentred exponential distribution with parameter ..empirical power of the competing tests for the simple hypothesis , and significance level .noise follows a and a recentred .[tab1 ] [ cols="^,^,^,^,^,^,^",options="header " , ]

Abstract: in this work , a goodness of fit test for the null hypothesis of a functional linear model with scalar response is proposed . the test is based on a generalization to the functional framework of a previous one , designed for the goodness of fit of regression models with multivariate covariates using random projections . the test statistic is easy to compute using geometrical and matrix arguments , and simple to calibrate in its distribution by a wild bootstrap on the residuals . the finite sample properties of the test are illustrated by a simulation study for several types of basis and under different alternatives . finally , the test is applied to two datasets for checking the assumption of the functional linear model and a graphical tool is introduced . supplementary materials are available online .   * keywords : * functional data ; goodness of fit ; functional linear model ; bootstrap calibration .