the principles of thermodynamics were established in the last century as the universal laws characterizing the thermal and mechanical behavior of macroscopic systems .the fact that we can not control all the details of energy transfer leads to the concept of _ heat _ as a form of energy flow , and the _ carnot cycle _ has played a crucial role in the course of investigation leading to the introduction of entropy as a state variable in addition to energy . on the other hand , brownian motion and the stochastic dynamics of mesoscopic systems in general have also been studied for many years , and projection methods have allowed for the derivation of langevin dynamics from microscopic hamiltonian mechanics . in a properly defined langevin equation ,the influence of the unpredictable microscopic dynamics , which essentially represent the heat , is taken into account by markovian random forces obeying the fluctuation - dissipation relationship . in this manner, such an equation describes the canonical equilibrium distribution of the variables in question .very recently , the concept of the heat on mesoscopic scales has been unambiguously defined in terms of langevin dynamics .we refer to the formalism providing this definition as _stochastic energetics_.the essential point of the thinking behind this formalism is that the heat transfered to the system is nothing but the microscopic work done by both frictional forces _ and _ the random force in the langevin equation .the theoretical framework resulting from this realization widens the scope of application of langevin dynamics to the extent that it can be used to describe not merely equilibrium states of system in contact with heat baths , but also general thermodynamic _ processes _ connecting different equilibrium states . as a result, we can derive the first and the second laws of thermodynamics from stochastic energetics .this formalism , together with projection methods , bridge a longstanding gap between microscopic hamiltonian mechanics and macroscopic thermodynamics . in this paperwe apply the method of stochastic energetics to the investigation of the carnot cycle in the context of small systems . to make this paperself - contained , we briefly summarize the framework of stochastic energetics in  [ sec : steng ] .stochastic energetics has also been applied to the study of thermodynamic processes under non - equilibrium conditions , such as processes including two heat baths and processes in the presence of steadily driving forces . in particular ,feynman s ratchet model has been analyzed . regarding this model ,the doubt has been cast by parrondo , and later by sekimoto independently , about the attainability of reversible energy conversion with the ` carnot efficiency ' , where and are the temperatures of cool and hot heat baths .analysis using stochastic energetics has shown explicitly that the efficiency of feynman s ratchet is much less than that of the carnot efficiency mentioned above .   with the descriptive power of stochastic energetics in hand, we wish to reconsider the carnot cycle .we consider the carnot cycle as an _ object _ of analysis within the theoretical framework of stochastic energetics .note that since we can derive the laws of thermodynamics directly using the stochastic energetics based on the langevin description , the role of the carnot cycle in our study is _ not _ a source of theoretical results from which one derives the laws of thermodynamics as was its historical role .( of course both the langevin description and thermodynamics have a microscopic basis in mechanics . )we now describe our viewpoint in more detail . usually the carnot heat engine is considered in an ideally macroscopic context , working in the thermodynamic limit . there, the small relative fluctuations of the variables , typically on the order of the inverse square root of the system size , are neglected .also , the cost involved in the operations of attaching / detaching the system under study to / from heat baths is neglected , since this is not an the extensive quantity .it is important to notice that the second law of thermodynamics , which is consistent with such a macroscopic carnot engine , can exclude _ only marginally _ the existence of perpetual machine of the second kind whose cycles yield positive work in an isothermal environment .thus we may gain a deeper insight into the nature of statistical thermodynamics and mechanics if we can formulate a method to take account of the finiteness of the the system under study as well as the cost involved in operations of changing its interaction with heat baths , in particular considering reversibility and the second law of thermodynamics .the approach of the present work is to construct the simplest model of the carnot heat engine with a finite number ( actually only three ) of degrees of freedom , including the apparatus connecting / disconnecting it with heat baths , and to determine the effect of the finiteness of the system and the change resulting from operations of the type mentioned above .as the system of study ( or the ` working material ' ) we choose a single harmonic oscillator . we show that there is an inevitable source of dissipation due to the _ intrinsically _ irreversible nature of the operations of connecting and disconnecting it with heat baths , and that , with the exception of such loss , our model can attain the carnot maximal efficiency defined as a properly defined average over infinitely many cycles , each of which is performed infinitely slowly . at the same time this study reveals several basic problems which should be further scrutinized in the future: one regards the smooth connection between the adiabatic process and the isothermal processes , and the other regards the irreversibility of adiabatic processes . in the last section we discuss these problems as well as the problem involving energy conversion with no help from external operations .in the remaining part of this section we give a qualitative description of the aspects of the carnot cycle that we study in detail in the later sections .the operations of connection to and disconnection from the heat bath .+ we ask first how we can describe _ mechanically _ the connection and disconnection of the system with the heat baths . in an idealized picture ,this basically consists of the switching on and off of the interaction between the system and each heat bath . in  [ sec : model ] we describe explicitly a model that realizes these operations .we represent the influence of the heat baths by a frictional force and the random force of a langevin equation , and we control the strength of the coupling between the system and the heat baths by controlling the values of the corresponding interaction potentials .we call these interaction potentials ` couplers ' .( in an actual mechanical system , the control of such a coupler could be exercised by a system of clutches . )one could also imagine such control exercised through the change of the friction constants that appear in the langevin equation .in consideration of the absence of a definition of the required work to change these friction constants , however , this idea is not pursued in the present paper . .+ _ 2._the reversible and irreversible work of operating the couplers + the operation of the couplers can , _ in principle _, never be carried out quasi - statically , but , at the same time , that accompanying irreversible work can be made arbitrarily small .the former part of this assertion is based on the following argument : when the interaction between the system and a heat bath is strong , the energy transfer between them occurs with a short relaxation time .however , if we gradually weaken this interaction , this relaxation time increases more and more until it diverges when the system is completely detached from the heat bath .as long as the time - scale of the operation ( i.e. the switching - off ) is finite , this operation can never remain ` slow ' in comparison to the diverging relaxation time .thus the switching - off process is by no means quasi - static , or quasi - equilibrium .( this is analogous to the non - adiabaticity encountered in chemical reactions ; the born - oppenheimer approximation is inevitably invalid when the distance between nuclei is neither sufficiently large nor sufficiently small . )inevitable irreversibility can also exist in the process of strengthening the interaction between the system and the heat bath .such extreme strengthening of the interaction leads to the freezing of some degree(s ) of freedom involved in the interaction , and the mean first - passage time associated with these degrees of freedom may become larger than the timescale of the operation ( i.e. the strengthening ) . in such a situation also , the operation can never be carried out quasi - statically .unlike the switching - off process , however , the indefinite strengthening of the interaction is not necessarily a part of the carnot cycle .despite this fact , we consider the latter process in the sections that follow , because this allows us to minimize the calculations needed to reach a general conclusion .we should , however , note that the inevitably irreversibile nature of the operations described above does not necessarily imply an associated large amount of irreversible work . in [ sec : clutch ] we analyze the work involved in operating the coupler and show that the amount of irreversible work resulting from these inevitably irreversible operations can be made arbitrarily small in the limit that the timescale of the operation becomes large .we also show that the reversible part of the work associated with these operations remains finite in this limit , but that it cancels out within a cycle .+ _ 3._the condition for reversible contact between the system and a heat bath + temporarily putting aside the concept of irreversibility in the sense described in 2 above , we can scrutinize the remaining part of the cycle and ask if and how the carnot maximum efficiency can be attained .with regard to a macroscopic carnot cycle , according to textbook descriptions , in order to realize a reversible cycle , ` the temperature of the system should be the same as that of the heat bath with which the system is to make contact after an adiabatic process . ' strictly speaking , however , the energy , rather than the temperature , takes a definite value in a thermally isolated system , and the above statement needs to be refined in terms of the language of probability .we argue in  [ sec : match ] that reversible contact requires the probability distribution of the energy of the system just before interaction with a heat bath to be identical to the canonical distribution at the temperature of the heat bath. this condition can be satisfied if the system consists of harmonic oscillators , as the model described below . generally , however , this is not the case , and in the general situation an irreversible process takes place when system contacts the heat bath , even though there occurs no net irreversible energy transfer between the two (  [ subsec:1 ] ) . in [ sec : efficiencymax ] we summarize the necessary conditions for carnot cycle to realize the maximal efficiency without assuming the thermodynamic limit .we show at the same time that this actually is the case for the model described in  [sec : model ] ._ statistics of the efficiency of a finite number of cycles + the efficiency of the energy conversion of an _ individual _ cycle is statistically distributed , because the energy possessed by the system is different each time the system is disconnected from a heat bath .this fact reflects the indeterminate nature of the details of the microscopic states of the system and of the heat bath upon disconnection.as a consequence , if we define the cumulative ` bonus ' work as the difference between the cumulative work obtained over cycles and what we would expect from the carnot maximal efficiency , this bonus work takes the form of a discrete random walk as a function of .we show in  [ sec : efficiency ] and the appendix that the so - called null - recurrence property of a one - dimensional random walk insures that , although , if we actually carry out a sequence of these cycles , the cumulative bonus work we obtain will with probability 1 first become positive after a finite number of repetitions , the statistical average of is infinite ._ irreversibility of adiabatic processes + the carnot cycle includes an adiabatic process that is purely mechanical .we are interested in determining what work can be obtained through the cycle including _ non_-quasi - static adiabatic processes .if the efficiency in this case is increased in comparison to the quasi - static case , the existence of a perpetual machine of the second kind is inspired , because our carnot cycle can attain the maximal ( reversible limit ) efficiency under certain conditions specified in [ sec : efficiencymax ] . in [ subsec : adiab ] we show that , in relation to the impossibility of a perpetual machine , there emerges the concept of the irreversibility of purely mechanical processes ( with no assumption of the thermodynamic limit or mixing properties necessary ) under the influence of ` macroscopic ' operations by an external agent . here, designation of an operation as ` macroscopic ' implies that ( i ) we are ignorant of the initial phase point on a given energy surface , and ( ii ) we are interested only in the statistical average over such an initial ensemble at a given energy .we consider a langevin equation that represents a system in contact with a heat bath at temperature , here we denote by and the dynamical variable of the system and its conjugate momentum , while represents the mass , the friction constant , and the potential energy for .we assume that may depend on , in addition to , the variable ( or variables ) , which is controlled by an external agent ( or agents ) .the function represents , as usual , gaussian white noise obeying the relations ( hereafter we adopt units in which ) the second relation ( einstein s relation ) insures a canonical distribution of and at temperature if the parameter is held fixed for an infinitely long time .multiplication of each term in the second equation in ( [ eq : langevin2 ] ) by the displacement yields the equation where we have used the first equation of eq.([eq : langevin2 ] ) and also the identity .we note that is the _ reaction _ force exerted by the system against the heat bath , since the frictional force and the random force are both due to the heat bath .we identify the work done by the reaction force as the _ heat _ transferred from the system to the heat bath , which we denote by : ( the minus sign in front of is included to conform to the convention of thermodynamics textbooks . )the key point of introducing the concept of heat is that , although the heat bath is idealized and not affected by the system s dynamics , the heat bath can still be subject to a reaction force exerted by the system . adding the total differential to both sides of eq.([eq : balance2 ] ) , we obtain the general expression for the _ energy balance _ as now , because the l.h.s .is the total increase of the energy , and is the energy input to the system as a heat , the first term on the r.h.s . of eq.([eq :first ] ) must be identified as the _ work _ done by the external system , , on the system through the change of the variable , we conclude that the law of energy balance expressed as is satisfied for any _ single _ realization of the stochastic process described by eq.([eq : langevin2 ] ) .  for a quasi - static process ,in which is arbitrarily small , the work is reversible and is equal to the change in the helmholtz free energy with probability 1 .that is , in an ensemble of infinitely many realizations of such a process , the probability distribution of the work becomes a point distribution concentrated at the value of . with % .\label{eq : helmholtz}\ ] ] the derivation of eq.([eq : second0 ] ) is as follows .we first note that , for to change by any small but finite amount , it takes a time , which is indefinitely large in the quasi - static limit . during this time intervalthe state point comes arbitrarily close to almost all possible values , and its empirical distribution becomes asymptotically equal to the canonical distribution , . ] includes a point at which the equilibration time diverges .see 2 of  [ subsec : problems ] and  [ sec : clutch ] below . )we can then evaluate using its average with respect to in the quasi - static limit . using the identity , we reach the result eq.([eq : second0 ] ) .in fact eq.([eq : second0 ] ) is a stronger statement than the usual second law of thermodynamics for extensive systems , note that for a thermodynamic system constituted by an ensemble of a large number of independent stochastic systems obeying eq.([eq : langevin2 ] ) , the first law of thermodynamics is obtained from eq.([eq : balance ] ) , and the second law of thermodynamics for quasi - static processes is obtained from eq.([eq : second0]) , these relations are concerned with only the ensemble averages denoted by .it has also been shown that , for a finite rate of change of , the clausius inequality holds , and an explicit formula for the _ irreversible _ work , , has been obtained up to the second order in .figure [ fig : notion ] schematizes the idea of our model .we employ a single harmonic oscillator with mass and spring constant as the system under study , which we call simply the `` system '' .we denote by and the position and momentum of the system .correspond to compressing [ decompressing ] the ideal gas .below , we consider to be a quantity that can be controlled , as the volume of a gas system is controlled in macroscopic carnot cycles . in order to allow independent and variable interaction with each heat bath , we represent each such interaction in the form of a mechanical force , which subsumes the corresponding frictional and gaussian random forces .such mechanical forces should be related in some way to the degrees of freedom that directly interact with the heat baths , which we denote by and . for simplicity, we do this by writing the mechanical forces as interaction forces , and . as interaction potentials , we choose functions and , where and are the control parameters .we call and the ` couplers ' , because their values directly indicate the strength of the coupling between the system and the respective heat baths .we use the expressions like ` control the coupler(s ) ' in reference to changes made in the values of these control parameters .we assume that the functions ( =h , l ) are -periodic functions of .( see , for details ,  [ sec : clutch ] and fig . [fig : clutch ] . )  the degrees of freedom and are subject to the frictional forces and and the random forces and exerted by the heat baths at temperatures and , respectively , as well as the interaction forces from the system , and . here , and are the friction constants , and and are the white gaussian random forces satisfying , , a nd .the equations of motion for , , , and are given as follows :  [ eq_of_mot ]    we consider the gears of the heat bath and the system to be ` tightly connected ' ( i.e. completely engaged ) for , that is , the interaction is so strong that the difference is fixed except for a small thermal fluctuation around its mean value , while these gears are ` disconnected ' ( i.e. completely disengaged ) for , that is , we have neglected the inertia effect related to and , as they would play only secondary role for our analysis .the protocol by which we control the parameters is represented in the space of as shown in fig .[ fig : protocol ] .  in the figure, the paths along the axis correspond to the adiabatic processes , while the vertical paths with and correspond to the isothermal processes .the values of corresponding to these four horizontal paths are the parameters .as we discussed in [ sec : intro ] , the operation of the couplers can never be made quasi - static , because the time - scale of this operation inevitably becomes shorter than the equilibration time for the system when the coupling between the system and a heat bath becomes either absent or extremely tight .in addition to the work due to these irreversible processes , there is also reversible work associated with operation of coupler .figure [ fig : clutch ] illustrates the generic features of the potential for three different values of .here , , and represent the height of each potential profile .we assume that the height of is a monotonically increasing function of and satisfies , , , and with .there are two situations in which the time - scale of measurement and/or operation can not exceed the equilibration time of the system .one is when the height of is very small , and the other is when the height of is very large .let us assume that the regime corresponds to the former case ; that is , for , the interaction is so weak that the equilibration time of the system with the heat bath ( ) is beyond the timescale of measurement and/or operation .we call this the _loose regime_. then , we assume that the regime corresponds to the latter case ; that is , for , the interaction is so strong that the equilibration time characterized by the over - barrier transition of ( see fig [ fig : clutch ] ) is again beyond the timescale of measurement and/or operation . in particular, we assume that for there occur essentially no thermal activation events over the barrier .we call this the _ tight regime_. in the remaining regime , we assume that the operation can be carried out in a manner that arbitrarily closely approximates the quasi - static limit .note that what kramers calls the ` small viscosity ' and ` large viscosity ' cases in ref. correspond , respectively , to the limits of the loose and tight regimes .we now evaluate the work for the loose and tight regimes using eq.([eq : work ] ) .we will describe the case of for concreteness .( the case of can be treated similarly . ) in the loose regime ( the processes near b and c in fig .[ fig : protocol ] ) , is of order and is small , although this work may be mostly irreversible . since is associated with the lower limit of quasi - static operation , the timescale of the operation is required to be large enough to satisfy the condition . in the tight regime ( the processes near and in fig .[ fig : protocol ] ) , the situation is more subtle .the contribution to consists of _ ( i ) _ the contribution produced when moves around the valley regions of with and _ ( ii ) _ the contribution produced when visits , by _ rare _ thermal excitation , the barrier regions of . to simplify the analysis we exclude the former contribution by assuming that for in the valley regions of ( fig .[ fig : clutch ] ) in the tight regime .( this assumption is only technical ; one can reach the conclusion of this paragraph without it . )the evaluation of the contribution _ ( ii ) _ above is carried out as follows .the probability to find in the barrier region is , and for such values of , the change of from to 1 results in an amount of work .thus we have  . because the timescale of operation is sufficiently large to allow large values of .for this reason , the conditions and can be satisfied simultaneously . in conclusion ,the irreversible part of the work associated with both the loose regime and the tight regime can be made as small as we wish by making the timescale of the operation sufficiently long in these regimes .the same conclusion holds for the case of .the quasi - static work associated with the change of within the region can be evaluated using eq.([eq : second0 ] ) . belowwe show that such quasi - static work cancels exactly when summed over the consecutive operations of connection and disconnection with a heat bath .again , considering the case of , we denote by the helmholtz free energy of the composite system of the harmonic oscillator and the couplers , : \right\}.  \label{helmholtz }  \end{aligned}\ ] ] note that here and the factor in front of the integration on the r.h.s. comes from the phase integration over . performing the integration over first, we have with defined by the first term on the r.h.s of eq.([eq : clutchf ] ) is independent of , while is independent of .using the notation of eq.([eq : clutchwork ] ) , we find from eq.([eq : clutchf ] ) that  along b in fig . [fig : protocol ] , and  along c .these two cancel exactly : the actual time interval required to change between and is finite , say , . the irreversible work due to this finiteness has been shown to be quite generally .thus the irreversible work associated with the process in which is changed between and can be made as small as we wish by making the timescale of operation sufficiently long . for later use, we now also estimate the heat exchanged upon the operation of the couplers .as we have shown above , the amount of work in the loose and tight regimes can be made arbitrarily small . also , changes in the parameters lead to only small changes of the internal energy of the composite system .these facts together with the energy balance principle ( see eq.([eq : balance ] ) ) lead to the conclusion that the heat exchanged in these two regimes can be made as small as we wish .next , the heat exchanged during the quasi - equilibrium processes with is assessed as follows . from eq.([eq : clutchf ] ) the ensemble average of the internal energy of the composite system is given by .if we define by the average heat influx to the composite system during the quasi - equilibrium operation along b the condition of energy balance , eq.([eq : firstav ] ) , yields .$ ] this heat cancels exactly the average heat input similarly defined along c : in the same manner , we can show the cancellation of both the work and the heat during the quasi - static part of the operation of coupler along d and a .here we study the meaning of the idea of matching the ` temperature ' of the small system with that of a heat bath . for comparison , we note that this meaning is unambiguous for an isolated macroscopic thermodynamic system , for which the energy and the temperature are simultaneously well - defined quantities , and in order to realize a reversible carnot cycle , the temperature of the system should be the same as that of the heat bath with which it makes contact .contrastingly , for isolated small systems , the energy takes a definite value , while the temperature is not generally well - defined .we will show in this section that if the small system is a harmonic oscillator , the concept of the temperature is still useful , and reversibility can be obtained as in macroscopic systems .discussion regarding the general case is given in [ subsec:1 ] .suppose that a coupler is operated quasi - statically up to the edge of a loose regime ( along or along in fig .[ fig : protocol ] ) .the energy of the oscillator in this situation fluctuates as a function of time and though the temporal fluctuation of the energy is very slow , it still obeys the canonical distribution at the temperature of the heat bath ( for a and for c ) up to a small error of . by the definition of the loose regime , further weakening the connection results in a situation in which there is no appreciable exchange of energy between the system and the heat bath ( see  [ sec : intro ] and [ sec : clutch ] ) .then , the complete disconnection leaves an isolated system whose energy is distributed according to the canonical distribution corresponding to the temperature of the heat bath .this is in fact true even if the small system is not a harmonic oscillator .  for a harmonic oscillator, however , both the energy distribution in the canonical ensemble and the transformation of the system s energy through the quasi - static adiabatic process have special features .the energy distribution in the canonical ensemble at temperature , is independent of the parameter : ( see appendix [ sec : derivation ] for a derivation . ) then , at a and c , the energy of the oscillator is distributed according to and , respectively . if the oscillator has a specific ( initial ) energy and undergoes the quasi - static adiabatic process represented as a or c in fig .[ fig : protocol ] , its energy , , changes so that the value given by ( [ eq : harmonicj ] ) remains constant : the relation ( [ invariant ] ) determines the change of the energy distribution when the system undergoes a quasi - static adiabatic process through the change of : for a change from to , the altered distribution must obey with .thus we obtain the energy distribution of a canonical ensemble _ after _ the change in , with being defined by .however , we note that this simple situation is due to the special nature of the harmonic oscillator system . the general case is discussed in  [ subsec:1 ] .with these facts in mind , we can now characterize the condition for a quasi - equilibrium transition between adiabatic processes and subsequent isothermal processes :  ( 1 ) : :  the energy distribution of the system before the connection with the  heat bath must be that of a canonical ensemble .( 2 ) : :  the temperature characterizing this canonical ensemble must be the  same as that of the heat bath in question .the reason is that , under these conditions , the system upon connection behaves statistically _ as if _ it had been in contact with the heat bath for a long time .note that the connection should be begun through a sufficiently weak interaction ( see  [ sec : clutch ] ) . in our case with the protocol described by fig .[ fig : protocol ] , we require that at b and at d hold , respectively . denoting the value of at that between and as , that between and as , etc . , the condition for a quasi - equilibrium connection to the heat bath is given explicitly as follows : now evaluate the maximal overall efficiency . we must take into account ( i ) the operation of the couplers , ( ii ) the isothermal processes , and ( iii ) the adiabatic processes .\(i ) we assume that by making the time in the loose and tight regimes sufficiently long , the intrinsically irreversible work and heat flow can be made as small as we wish .the remaining part of the operation of couplers is assumed to be made under quasi - equilibrium conditions .the accompanying work and heat flow cancel exactly when summed over an infinite number of consecutive connection and disconnection with the heat bath ( see eqs.([eq : workadbc ] ) and ( [ eq : heatadbc ] ) ) .( ii ) for the isothermal parts of the cycle , b and d , we assume a quasi - static change of . the accompanying work is then given using the general formula eq.([eq : second0 ] ) .for the part b , in eq.([eq : second0 ] ) is replaced by of eq.([eq : clutchf ] ) , and the work done by the system is which we denote by .similarly , for the part d the work is then , the relation ( [ matching1 ] ) , we have ( iii ) for the adiabatic part of the cycle , a and c , we also assume a quasi - static change of .the energy of the system then obeys the law ( [ invariant ] ) and the amounts of work done by the system in the adiabatic processes and are given by and as the energies and obey the distribution eq.([eq : canonical ] ) with and , respectively , their statistical averages are and [ up to a small error of . using ( [ matching1 ] ), we then have + t_{\rm h}\left[1-\sqrt{\frac{k_{\rm d}}{k_{\rm c}}}\right]=0 . \label{sumcd}\end{aligned}\ ] ] while we obtain this simple result in the present case , it is important to note that the cancellation of the contributions from the adiabatic processes on the average is not a generic feature of the carnot cycle ( consider , for example , non - ideal gases ) .the heat influx from the high temperature heat bath is evaluated as follows . while the work during the isothermal quasi - equilibrium processes is a non - fluctuating quantity ( see eq.([eq : second0 ] ) ) , both the energy influx from the heat bath and the system s energy fluctuate subject to the constraint of energy balance described by eq.([eq : balance ] ) . from eqs.([eq : clutchf ] ) and ( [ eq : etildef ] ) we can show that the average internal energy of the composite system with degrees of freedom is independent of the parameter .therefore , the statistical average of the heat influx during the isothermal process , which we denote by , satisfies thus we have as we have seen in [ sec : clutch ] , there is no net heat flow due to the quasi - static part of the operation of coupler ( see eq.([eq : heatadbc ] ) ) , while the heat transfer associated with the loose and tight regimes can be made as small as we wish .  collecting the above results ,the maximal overall efficiency of the cycles is reduced to the following formula : then , using ( [ sumbcda ] ) and ( [ eq : input ] ) we have   we would like to stress that the attainment of this efficiency ( whose expression is familiar from textbook treatments ) is not due to the quasi - static operation of the whole system .in the situation we consider , we have seen that some parts of the cycle can never be carried out quasi - statically , due to the intrinsically irreversible operation of the couplers (  [ sec : clutch ] ) , as well as the intrinsic irreversibility resulting from the non - canonical energy distribution of the systems caused by the adiabatic processes( [ sec : match ] ) .the maximal efficiency eq.([eq : etamax ] ) obtained above represents exclusively the ratio of the total work done by the system to the total energy influx from the high temperature heat bath through infinite number of cycles . herewe consider the efficiency for a single cycle , , which can be written as with and we first note several properties of   _ 1 ._ the deviations and do not vanish , even in the quasi - equilibrium limit , since the system continues to exchange energy with a heat bath until the moment that it is disconnected from the heat bath ._ if we choose the initial point of an individual cycle to be somewhere between d and a , then the values of for different cycles are statistically independent .in fact , the statistical deviations of and are mutually uncorrelated because of the intervening isothermal markov processes and , while and are statistically correlated through the shared point b , as are and through the point c ._ one can define the ` excess output ' , , by \cr & = & -\delta_w-\eta_{\rm max}\,\delta_q .\label{eq : bonus}\end{aligned}\ ] ] a positive value of implies that we happened to get more work than that expected from the carnot maximal efficiency ( i.e. , ) .such a situation can result through fluctuations , and it is not in contradiction with the second law of thermodynamics .we may then , however , ask how many cycles on average we must carry out before we first obtain a cumulative excess output , , where is the excess output of the -th cycle and is the total number of consecutive cycles .the point of this question can be understood in terms of the following apparent paradox : suppose one monitors as a function of and stops when it becomes positive for the first time .if one could repeat such a procedure of monitor - and - stop indefinitely many times , one could construct a perpetual machine of the second kind .this , of course , would be in contradiction with the second law of thermodynamics .a pitfall of this false argument is that , although for a given sequence of cycles , the condition will be satisfied at some finite with probability 1 , the _ average _ over separate sequences of the smallest value of with positive is divergent .this fact is closely related to the fact that the one - dimensional random walk is ` null - recurrent ' ( see appendix [ sec : nullrecurrent ] ) .in  [ sec : match ] we have used the fact that for the harmonic oscillator system , as a result of the quasi - static adiabatic process , the energy changes in such a manner that the energy distribution remains in canonical form , with simply a change of the temperature , .however , the harmonic oscillator represents a special system , and this is not generically the case with any hamilton . more generally , the energy distribution is distorted into some non - canonical form , , as a result of the quasi - static adiabatic process .when an ensemble of systems following the distribution are brought into ( weak ) contact with a heat bath of arbitrary temperature , the energy distribution _ irreversibly _ relaxes to the canonical form .this is the case even if is chosen so that , i.e. even in the case that no net heat is transferred on the average from the bath to the system .the relevance of this irreversible relaxation to the energetics of small systems requires further scrutiny .this will be discussed in more detail in a separate paper .the adiabatic process in the carnot cycle is a mechanical process .the ergodic invariant theorem tells us that under quasi - static and adiabatic change of a system parameter , say , by a finite amount , the phase volume enclosed by the energy surface defined by the system s energy at each moment , ( see ( [ eq : defj ] ) for definition ) , remains constant .the theorem does not assume the thermodynamic limit nor the presence or absence of chaotic trajectories .contrastingly , for a non - quasi - static process can either increase or decrease , depending on both the nature of the change of and the initial conditions of the system .  in the context of the present paper , however , it is most meaningful to confine ourselves to only ` macroscopic ' external operations , excluding ` demonic ' ones which depend on detailed information of the system .more precisely , we focus on an unprejudiced choice of the initial conditions among those with a given energy , and also focus only on statistical averages of the energy , rather than consider particular results obtained from particular initial conditions .sato has recently studied a harmonic oscillator under a time - dependent force , , as the simplest non - trivial example of a system with only macroscopic external operations . here ,the change of amounts to a horizontal displacement of the potential .he showed analytically , as one can easily confirm , that for an arbitrary function , the energy of the oscillator , , is strictly non - decreasing _ if _ the average is taken with respect to the initial condition over all initial conditions represented by states with a given energy , i.e. , over a micro - canonical ensemble . only in the limit of a quasi - static process is the energy unchanged .this example demonstrates the irreversibility of a mechanical system of non - macroscopic size with properly defined macroscopic operation .a recent numerical work investigating a harmonic oscillator with time - dependent spring constant , , reveals the same phenomena when is constrained to return to its initial value . in the present context of the analysis of the efficiency of carnot cycle (  [ sec : efficiencygen ] ) , these findings are both natural and important , since , if the case were different , one could find functional form for for which the adiabatic work on the system would be less than what we expect for a quasi - static process , and the whole cycle could be used to construct a perpetual machine of the second kind .( note that in the analysis of [ sec : efficiencygen ] we have excluded only marginally the existence of such a perpetual machine if the loss due to the other sources is made arbitrarily small . ) it is desirable to obtain a general proof ( or counter - evidence ) of the irreversibility resulting from non - quasi - static processes . a related analysis using path probabilities has been performed for chaotic systems .however , we suspect that the essential mechanism of the irreversibility related to the characterization of macroscopic operators , which are ignorant of the initial microscopic state of the system , can be elucidated without resort to chaotic statistics . it is also a future problem to scrutinize the case in which the topology of the energy contour surface in the phase space ( ) changes at some parameter value ,say .in such cases , quasi - static processes can not be extended across .as in the case of the ordinary macroscopic carnot cycle , we have introduced control parameters , .we assumed that the values of these parameters are changed by some external agent whose dynamics are external to the equation of motion of the system .in fact , however , there are many ` self - controlled ' energy transducers which contain their own control systems . in such cases ,the identification of the control system is more or less a matter of interpretation . in the macroscopic world , dc electric motors and steam engines are examples , while motor proteins , such as myosins , kinesins and dyneins , etc . , are microscopic examples .theoretically , the so - called feynman ratchet and pawl system has been proposed as a microscopic energy transducer working by itself between hot and cool heat baths . in this model ,the role of the control system is played either by the pawl or by the ratchet , depending on which of these two is in direct contact with the cool heat bath .the stochastic energetics of this model have been analyzed .model is another self - controlled microscopic transducer . in this modela massive particle moves while in contact with a heat bath of position - dependent temperature , or . in this model ,the inertia of the particle serves to switch the particle s environment from to , or vice versa .the stochastic energetics of this model have also been analyzed .some people have claimed that the carnot limiting efficiency can be attained in feynman s ratchet and pawl system ( see and ) and in btticker s model ( see ) . with the exception of the original work by feynman , where no implementation details are given , these studies introduced into their analyses some ` gate ' mechanism .the study of the energetics of such systems , including the action of these gates , has not yet been made .discussions with k. sato , s. sasa , t. komatsu and t. chawan - ya on the adiabatic processes of harmonic oscillator are gratefully acknowledged .the helpful comments by g. paquette on the manuscript are also acknowledged .this work is supported in part by the inamori foundation ( t.h . ) , and a grant in aid by the ministry of education , culture and science ( priority area , nos.11156216 and 12030215 ) ( k.s . ) .for a general hamiltonian with a parameter , the energy distribution corresponding to the canonical ensemble at temperature t is with  where denotes the phase integral .here , , or , is an adiabatic invariant .  in the text, is that of an isolated harmonic oscillator , , and we take its spring constant as .the calculation of is straightforward , yielding  is , therefore , independent of .thus from ( [ eq : pcangeneral ] ) we reach ( [ eq : canonical ] ) . in general , however , depends on , and is not simply an exponential .as are statistically independent of each other , constitutes a one - dimensional discrete random walk . to simplify the argument we assume that takes only the values randomly . if we denote by the probability that at the -th step that the random walker comes to the position for the first time , it is known that the fact that is normalized ( ) implies that this event occurs with probability 1 at some . on the other hand , it is also true that which is referred to as the null - recurrence property .thus if we are to wait until the position of the walker becomes positive for the -th time , with , then the ` waiting time ' is , _ on the average _ , infinite .see , for example , e. fermi , _ thermodynamics _( dover publ .new york , 1956 ) .landau and e.m .lifshitz , _ statistical physics _ , 3rd ed ., ( pergamon press , new york 1980 ) .k. sekimoto , j. phys .66 * , 1234 ( 1997 ) .k. sekimoto and s. sasa , j. phys .. jpn . * 66 * , 3326 ( 1997 ) .k. sekimoto : progr .. suppl . *130 * , 17 ( 1998 ) .feynman , r.b .leighton , and m. sands , _ the feynman lectures on physics _ ( addison - wesley , reading , ma , 1966 ) , vol.i , chap.46 .j.m.r.parrondo and p.espanol , am.j.phys . * 64 * , 1125 ( 1996 ) .hereafter , multiplication of a fluctuating quantity by an other quantity , either fluctuating or non - fluctuating , such as , should be understood in the sense of stratonovich calculus .see , for example , c. w. gardiner , _ handbook of stochastic methods _ , ( springer 1985 ) .we have noticed that the relation ( [ eq : second0 ] ) to hold without averaging over the ensemble thanks to discussion with chris jarzynski . until now , only its weak form ( [ eq : second ] ) had been shown ( see ref. ) .h. a. kramers , physica * 7 , * 284 ( 1940 ) .landau and e.m .lifshitz , _ mechanics _ , 3rd ed . , ( pergamon press , new york 1976 ) , 49 . ( the description in the other editions are less detailed . ) k. sato , k. sekimoto , t. hondou and f. takagi , cond - mat/0008393 .the general mathematical proof of the adiabatic theorem can be found in : t. kasuga , proc .japan academy , tokyo , * 37 * , 366 ( 1961 ) , _ ibid ._ 372 , _ ibid .katsuhiko sato , private communication ( oct . 1999 ) .t. komatsu and s. sasa , annual meeting of the physical society of japan ( sept . 1999 ,oral presentation in japanese ) .s. sasa and t. komatsu , progr .j. r. cray , d. f. escande , j. l. tennyson , phys . rev .a , * 34 * , 4256 ( 1986 ) . m. o. magnasco and g. stolovitzky , j. stat . phys . * 93 * , ( 1998 ) 615 .m. bttiker , z. phys .b * 68 * , 161 ( 1987 ) . m. matsuo and s. sasa , physica * a 276 * , 188 - 200 ( 2000 ) . t. hondou and k. sekimoto , to be published in physe. ; cond - mat/0008176 . h. sakaguchi , j. phys . soc* 67 * ( 1998 ) 709 . a langevin simulation for the feynman ratchet model i. derenyi and r. d. astumian , phys . rev .e * 59 * , r6219 ( 1999 ) .see , for example , ` essential cell biology ' , b. alberts , d. bray , a. johnson , j. lewis , m. raff , k. roberts , and p. walter , ( garland pub ., new york & london , 1997), 5,12 , & 16 .r. d. vale and f. oosawa , adv .biophys . , ed . by m.kotani ( elsevier , limerick , 1990 ) * 26 * , 97 .

Abstract: in the thermodynamic limit , the existence of a maximal efficiency of energy conversion attainable by a carnot cycle consisting of quasi - static isothermal and adiabatic processes precludes the existence of a perpetual machine of the second kind , whose cycles yield positive work in an isothermal environment . we employ the recently developed framework of the energetics of stochastic processes ( called ` stochastic energetics ' ) , to re - analyze the carnot cycle in detail , taking account of fluctuations , without taking the thermodynamic limit . we find that both in this non - macroscopic situation , both processes of connection to and disconnection from heat baths and adiabatic processes that cause distortion of the energy distribution are sources of inevitable irreversibility within the cycle . also , the so - called null - recurrence property of the cumulative efficiency of energy conversion over many cycles and the irreversible property of isolated , purely mechanical processes under external ` macroscopic ' operations are discussed in relation to the impossibility of a perpetual machine , or maxwell s demon . this analysis may serve as the basis for the design and analysis of mesoscopic energy converters in the near future .


frequency and code division multiplexing ( ofcdm ) was recently proposed for multiuser access in the fourth generation ( 4 g ) wireless framework .the adoption of a two dimensional ( 2d ) spreading in ofcdm , by utilizing both frequency- and time - domain spreading , is shown to outperform the multicarrier code division multiple access ( mc - cdma ) and the direct sequence mc - cdma ( mc - ds - cdma ) schemes .this is achieved because each data stream is spread simultaneously over multiple subcarriers and multiple ofcdm symbols .hence , by exploiting both the time and the frequency diversity of the channel , a performance gain is straightforward .furthermore , ofcdm can facilitate the power limitation issue in multiuser environments , especially in the demanding reverse link scenarios present in modern wireless infrastructures ( i.e. cellular or _ad hoc _ networks ) .a great deal of attention has been given to the reduction of the bit - error - rate ( ber ) probability and/or the outage probability , thus striving for the performance improvement and the robustness enhancement of ofcdm systems - .one of the most effective approaches is the subcarrier allocation adaptation , which is based on a subcarrier grouping methodology , according to the fading characteristics of each user and the channel load . in order to further improve this methodology, we propose a _receiver - centric _ approach ( where the receiver rather than a potential sender controls the admission decision of a new transmission establishment ) , in order to exploit the diversity gain provided by the subcarrier grouping .it was shown in that such an approach significantly enhance the system performance of one dimensional ( 1d ) spreading - enabled networks , in comparison to a conventional sender - centric one .nevertheless , the receiver - centric approach has not been studied for 2d spreading schemes so far .we propose a novel receiver - centric 2d ofcdm approach with subcarrier grouping and we manifest its performance enhancement in terms of the system robustness in comparison to the conventional 2d ofcdm approach .the advantages of the proposed scheme are three - fold : 1 ) each sender - receiver active link does not need to have the knowledge of other ongoing transmissions within the same subcarrier group in terms of resource allocation and tolerable interference at the receiver ; 2 ) it is fully distributed since the presence of a centralized unit , coordinating the user transmissions , is not required ; 3 ) accurate interference estimation can be achieved at the receiver , while a low information exchange overhead between the potential senders and the receiver is required .we consider an ofcdm system with a total number of subcarriers , which are separated into noncontiguous groups denoted as where , is the subcarrier spacing and is the number of subcarriers in group . by applying a subcarrier grouping , an overall interference reduction caused by the frequency - domain spreadingis achieved .the system supports up to maximum number of users , e.g. users . upon the transmission of the _ _ k__th user , each data bit experiences a 2d spreading by using a unique pseudonoise ( pn ) code , where denotes the time - domain spreading length .the channel is assumed to be slowly varying with respect to the symbol duration and modeled as an independent rayleigh fading for different users , subcarriers and bit transmissions .  in this letter, we adopt a receiver - centric ofcdm system by defining two successive user transmission phases , namely the _ probing _ and the _ data phase_. the traffic arrives at each user in bursts of the same size .prior to the actual data transmission , a potential sender enters the probing phase , by sending a signaling tone at the receiver .this tone carries no data information and actually represents a replica of a probing code , , common to all the system users . the signaling toneis used by the receiver to experience a fraction of the upcoming interference that a potential sender would cause and to calculate the influence that would produce ( by entering the data phase ) to the ongoing transmissions of other users .furthermore , the statistics derived from the probing phase indicate each user s status at the receiver in terms of the signal fading and , thus , all the users can be appropriately allocated to the corresponding subcarrier group .  since we assume independent rayleigh fading coefficients for each user in the frequency - domain and a slowly varying channel within each ofcdm frame ( i.e. the _ _m__th fading entry of each user remains constant in the time - domain of the same ofcdm frame ) , we define the probing code as , where  ] denote a probing instance , the chip duration , the chip energy and a rectangular pulse shaping filter , respectively . assuming binary phase shift keying ( bpsk ) modulation schemes , .the term is the rayleigh fading gain for the __ m__th subcarrier of the _ _ k__th user , during the _ _ j__th transmitted bit .the term , while and denote the __ m__th subcarrier frequency and the respective phase shift ( produced by the channel ) of the received signal for the _ _ k__th user . the frequency - domain and the time - domain pn chips are ,respectively , expressed as and .the parameter equals one  and zero otherwise .finally , is the additive white gaussian noise ( awgn ) signal with a double - sided power spectral density of .  in order to accurately decode the data of a user , e.g. user 1 ,the received signal is restored to the baseband by multiplying it with , followed by a low - pass filtering .moreover , the interference term of user 1 is obtained as \\ & + \sum_{k ' \in g_{y}}\sum_{m \in g_{y}}\alpha^{(1)}_{j , m}\alpha^{(k')}_{j , m}c^{f(1)}_{j , m}c^{f(k')}_{j , m}\delta_{\phi}^{k',1}\\ \nonumber & \times\frac{1}{t_{c}}\int^{(j+\frac{1}{n})t_{b}}_{t = jt_{b}}\sum^{n}_{n=1}c^{t(1)}_{j , m , n}c^{t(k')}_{j , m , n}p[t-(jn+1)t_{c}]dt\big\},\end{aligned}\ ] ] where .since the bit information has zero mean , the power of the desired signal is derived by simply calculating its variance as where .finally , the noise power is expressed as   by using the _ central limit theorem _ , the power of can be approximated as a gaussian random variable , which produces good results for a sufficiently large number of users and is obtained as +\sum^{n}_{n=1}\epsilon_{c}v_{n}\textit{e}\left[(\alpha_{y})^{2}\right]\right\},\end{aligned}\ ] ] where =\frac{1}{\mathcal{b}_{y}m_{y}}\sum_{\beta\in \mathcal{b}_{y}}\sum_{m\in g_{y}}(\alpha^{(\beta)}_{y})^{2},\ ] ] and , .the terms and denote the total number of users within occurring in the probing phase and the data phase , respectively .hence , the signal - to - interference - plus - noise ratio ( sinr ) of the _ _ k__th user on , during the _ _j__th bit transmission , is expressed as +\sum^{n}_{n=1}\epsilon_{c}v_{n}\textit{e}\left[(\alpha_{y})^{2}\right]+nn_{\textit{o } } } , \label{4}\end{aligned}\ ] ] where equals one a signaling tone occurs within the __ n__th time instance of and zero otherwise .our main objective is to maximize .the proposed algorithm is analytically presented in the following steps :  _ step 1 _ : : :  during an ofcdm frame reception , the receiver checks if there is a  probing tone  ] and marks a potential  probing event as , identifying the respective   user at the same time ._ step 2 _ : : :  based on eq .( [ 4 ] ) , the receiver calculates   , ,  which is expressed as  +\sum^{n}_{n=1}\epsilon_{c}v^{(k)'}_{n}\textit{e}\left[(\alpha_{y})^{2}\right]+nn_{\textit{o}}},\ ] ]  : :  where equals one a  signaling tone occurs within the _ _ n__th time instance of   ( while excepting the under examination   one , i.e. ,   ) and zero otherwise .afterwards ,  the receiver finds   ] denotes the maximum operation ._ step 3 _ : : :  in the next ofcdm frame transmission duration , the receiver indicates   to the user , which  represents the most effective subcarrier group for this user ( and ,  hence , for all the ongoing transmitting users that exploit the data  phase ) , by sending a probing tone to ._ step 4 _ : : :  thereupon , the user initiates the data phase , by  sending packets within using its personal   at all the subsequent ofcdm frames ,  until its burst is completed .note that the receiver _ a priori _ knows  the latter spreading code and since it expects to collect information  from that user at , the decoding is accurately  performed .given the number of the consecutive ofcdm frames that are required for the completion of the data phase for each user , denoted as , the transmission duration of the entire burst can be expressed as .consequently , by adopting the proposed probing approach , the corresponding transmission duration of a given burst is extended to .the extra 2 ofcdm frames represent an outcome of the probing methodology , i.e. one for the probing transmission / reception ( _ step 1 _ ) and one for the optimization procedure ( _ step 3 _ ) .it is worth noting that for large values of , which is usually the case in realistic network implementations , the cost that is produced by the probing overhead can assumed to be negligible .the ber performance of the proposed approach is evaluated by using monte carlo simulations .the ofcdm system bandwidth is assumed to be 20 mhz , consisting of 128 subcarriers .the system supports up to 32 users , the burst arrival at each user is a poisson process , while the burst size is bits . since we focus on bpsk constellation alphabets , the ber probability for the _ _ k__th user , is expressed as where is obtained by averaging over , while is the gaussian _ q_-function . in fig .[ fig . 1 ] , along with the proposed _ probing _ scheme we also modeled the classical ofcdm receiver ( such as and ) , denoted as _ no probing _ scheme , for a cross - reference comparison . as an illustrative example , we consider two case studies , each with a different spreading gain , namely and .it is obvious that the latter spreading gain results to a superior ber performance for both schemes in comparison to the former one , due to the higher frequency and time diversity gain , since we apply a subcarrier grouping policy .the proposed scheme outperforms the conventional ofcdm scheme , as shown in fig .1 ] ( i.e. an improvement of approximately 2.5 db gain is achieved ) , which is an outcome of the proposed probing method .  , for 32 users and different spreading factors.,width=326 ]  note that the above - mentioned performance gain is directly associated with the structure of . the design of the _ column - wise _ instances , as explicitly defined in the previous section , counteracts the fragility between the multiuser diversity and the error resilience of the system , which is suitably effective under slowly varying fading channels .nevertheless , the structure of has to be appropriately re - defined in case of severe fading conditions ( e.g. when doubly selective channel fading is present ) .such a probing approach should compass for both frequency and time selectivity of the channel ( conditioned on the respective coherence time ) while maintaining the multiuser diversity gain in an acceptable level and it represents a challenging topic for further research .the outage probability denotes the probability given that the instantaneous sinr falls below a specified threshold , , and can be obtained as  where and denote , respectively , the probability density function ( pdf ) and the cumulative density function ( cdf ) of the instantaneous sinr . since is directly related to the sum of i.i.d .rayleigh parameters , ranging according to the size , we use a highly accurate closed - form approximation to estimate , which is expressed as where ^{1/m_{y}}$ ] , is the distribution shaping parameter and .the constants , and are used to enhance the estimation accuracy of the cdf approximation and depend on the length of .the particular values for these constants , utilized in this letter , are presented in table [ table i ] ( a detailed representation of the range of , and can be found in ( * ? ? ?* table i ) ) . as shown in fig .2 ] , the outage performance improves ( i.e. decreases ) more sharply with an increase of , especially at the low sinr regions , with respect to the normalized outage threshold ( ) .this is achieved due to the frequency - domain diversity gain , as the size of increases , at the cost of the multiuser diversity loss and vice versa .   vs. , for several values of and , based on the closed - form approximation given in eq .( [ 6]).,width=326 ]   c| c c c size & & & + 8 & 0.0257 & 0.1172 & 0.9491 + 16 & 0.0291 & 0.0133 & 0.9338 +a novel data transmission approach for multiuser 2d spreading ofcdm systems is presented .the receiver , rather than a potential sender , decides for the admission of a data transmission between each sender - receiver pair .a subcarrier grouping policy is established and the assignment of each user to the system is based on the channel load and the sinr status at each subcarrier group .this is accomplished by addressing two transmission phases for the potential system users , namely the probing and the data phase .we showed that the proposed approach outperforms the conventional ofcdm scheme with respect to the system robustness and the error resilience .chang and c .- h .wu , `` adaptive load balancing mdp - based approach of two - dimensional spreading for vsf - ofcdm in 4 g next - generation cellular communications , '' _ ieee trans . veh .1143 - 1156 , march 2009 .r. caldwell and a. anpalagan , `` adaptive subcarrier allocation in synchronous reverse links of a multicarrier cdma system with time and frequency spreading , '' _ ieee trans .1494 - 1501 , may 2008 .h. jiang , p. wang , w. zhuang and x. shen , `` an interference aware distributed resource management scheme for cdma - based wireless mesh backbone , '' _ ieee trans .wireless commun ._ , vol . 6 , no . 12 ,4558 - 4567 , dec . 2007

Abstract: in this letter , following a cross - layer design concept , we propose a novel subcarrier grouping technique for orthogonal frequency and code division multiplexing ( ofcdm ) multiuser systems . we adopt a two dimensional ( 2d ) spreading , so as to achieve both frequency- and time - domain channel gain . furthermore , we enable a receiver - centric approach , where the receiver rather than a potential sender controls the admission decision of the communication establishment . we study the robustness of the proposed scheme in terms of the bit - error - rate ( ber ) and the outage probability . the derived results indicate that the proposed scheme outperforms the classical ofcdm approach .  a receiver - centric ofcdm approach with subcarrier grouping  orthogonal frequency and code division multiplexing ( ofcdm ) , subcarrier grouping , bit - error - rate ( ber ) performance , outage probability , cross - layer design .


we live in a time where people increasingly turn to the web for help .our needs , however , often go far beyond mere information from existing webpages and we need help from real people . for example , we ask for answers to specific questions on _ stackoverflow.com_ , for donations on _ donorschoose.org_ , or for help on online social communities such as _ reddit.com_. in each of these cases a user performs a request , which we define as an act of asking formally for something .all these communities rely heavily on their members to help satisfy the request . yet, the factors that lead community members to satisfy a request remain largely unknown .understanding the dynamics and factors of successful requests has the potential to substantially improve such communities by educating users about better formulating requests and promoting likely - to - succeed requests .in addition to these practical benefits , understanding the factors that make a request successful has implications for questions in social psychology and linguistic pragmatics .  studies on the popular crowdfunding platform kickstarter have shown that the success of a request depends most crucially on _ what _ is being requested , that is , whether it is a small favor like an answer to a simple question or a large financial contribution .many other factors need to be controlled as well ; what the giver receives _ in return _ , _ when _ they are asking , and even group dynamics , since people are more likely to give to projects that others are already giving to .satisfying a request on peer - to - peer lending or crowd - funding platforms can also bring a reward , and this also can drive the selection process .it is extremely difficult to disentangle the effects of all these factors in determining what makes people satisfy requests , and what makes them select some requests over others .  in this paper, we develop a framework for controlling for each of these potential confounds while studying the role of two aspects that characterize compelling requests : _ social _ factors ( _ who _ is asking and how the recipient is related to the donor and community ) and _ linguistic _ factors ( _ how _ they are asking and what linguistic devices accompany successful requests ) . with the notable exception of ( ) ,the effect of language on the success of requests has largely been ignored thus far .our goal is to understand what motivates people to give when they do not receive anything tangible in return .that is , we focus on the important special case of _ altruistic _ requests in which the giver receives no rewards .this controls for the incentive to obtain attractive rewards commonly offered on crowdfunding sites such as kickstarter ; the absence of external factors such as tangible rewards also makes the language itself all the more important in persuading others to help . in this domainwe also do not need to consider crowdfunding - related marketing strategies such as emphasizing limited time offers ( scarcity ) or showing that other people made the same decision already ( social proof ) , which are known to manifest themselves in language .second , we focus on requests that a single user can fulfill , thereby additionally eliminating group behavior effects such as herding or completing donation biases . finally , we focus on one community in which _ what _ is being asked for is held constant .this allows us to explore a large number of different requests of different individual users , at different times , that all have the same goal .controlling for the request goal therefore allows us to study how to optimize a particular request solely by optimizing its presentation , and helps provide a direct practical benefit to the requester ( by contrast , advising a requester who needs something to instead ask for something different may be advice of limited practical use ) .we therefore chose to study donations in `` random acts of pizza '' , an online community devoted to giving away free pizza to strangers that ask for one .random acts of pizza ( raop ) is a community within the social news and entertainment website _reddit.com_. users can submit requests for free pizza and if their story is compelling enough a fellow user might decide to send them one , `` _ _ because ... who does nt like helping out a stranger ? the purpose is to have fun , eat pizza and help each other out .together , we aim to restore faith in humanity , one slice at a time ._ '' a typical post might sound something like this : `` _ _ it s been a long time since my mother and i have had proper food .i ve been struggling to find any kind of work so i can supplement my mom s social security ... a real pizza would certainly lift our spirits _ _ . ''this platform addresses many of the potential confounds that complicate other platforms or studies : all requests ask for the same thing , a pizza , there are no additional incentives or rewards , each request is satisfied by a single user , users and requests are embedded in a social network within reddit , and requests are largely textual .this dataset thus provides us with an unusually clear picture of the effect of language and social factors on success .the remainder of this paper is organized as follows : inspired by studies in crowdfunding , user - to - user evaluations in social networks , and helping behavior in social psychology , we introduce a variety of textual and social factors that are potentially associated with successful requests .we use topic modeling and automatic detection to extract a particularly complex factor , the _ narrative _ structure of requests .we employ a logistic regression framework to test what factors matter in the community , showing that narratives are significantly correlated with success , and that signaling gratitude , the intention to reciprocate in the future , supporting the narrative with additional evidence , as well as a high status of the user within the community further increase the chance of success .we do not find any support for theories predicting that positive sentiment , politeness , and user similarity are associated with success .thus , drawing from social psychology literature , our extracted high - level social features operationalize the relation between recipient and donor .we then demonstrate in a prediction task that the proposed model generalizes to unseen requests and significantly improves over several baselines .our dataset contains the entire history of the random acts of pizza subreddit from december 8 , 2010 to september 29 , 2013 ( 21,577 posts total ) . to compute user featureswe further crawled the entire lifetime history of posts and comments across all subreddits for all users involved in the raop subreddit ( 1.87 m submissions total ) .the community only publishes which _ users _ have given or received pizzas but not which _ requests _ were successful . in the case of successful users posting multiple times it is unclear which of the requests was actually successful .therefore , we restrict our analysis to users with a single request for which we can be certain whether or not it was successful , leaving us with 5728 pizza requests .we split this dataset into development ( 70% ) and test set ( 30% ) such that both sets mirror the average success rate in our dataset of 24.6% .all features are developed on the development test only while the test set is used only once to evaluate the prediction accuracy of our proposed model on held - out data .for a small number of requests ( 379 ) we further observe the identity of the benefactor through a `` thank you '' post by the beneficiary after the successful request .this enables us to reason about the impact of user similarity on giving .previous work on crowdfunding , helping behavior and user - to - user evaluations in social networks have pointed to a number of textual and social factors that could influence the success of a request .a person experiencing gratitude is more likely to behave prosocially towards their benefactor and others .however , gratitude is only one component of politeness .other indicators include deference , greetings , indirect language , apologizing and hedges .we ask a more general question : does a polite request make you more likely to be successful ?in social psychology , _ reciprocity _ refers to responding to a positive action with another positive action .people are more likely to help if they have received help themselves . since in altruistic domains, there is no possibility of direct reciprocity , we hypothesize that recipients might pay the kindness _ forward _ to another community member , a concept known as `` generalized reciprocity '' .feelings of gratitude can elicit this behavior .we hypothesize that the community would be more willing to fulfill the request of someone who is likely to contribute to the community later on .while many requests are fairly negative , talking about lost jobs , financial problems , or relationship breakups , some of them are positive , asking for pizza for birthday parties and other celebrations . helping behavior literature predicts that positive mood is associated with a higher likelihood of giving .while these studies refer to the sentiment or emotional state of the benefactor , the most closely related linguistic feature that is available in this setting would be the sentiment of the text .thus , the literature would predict that very positive requests are more likely to succeed .we additionally expect that very negative requests could be more successful , too , since they most likely describe very unfortunate situations of the requester .studies on the success of research grant proposals have shown that the simple factor of request length can be significantly related to funding success even when controlling for a variety of other factors .we hypothesize that longer requests will be interpreted as showing more effort on the side of the requester and giving them the opportunity to provide more evidence for their situation .studies on crowdfunding have shown that the size of the social network of the project creator is associated with success .work on user - to - user evaluations in online social networks suggests that the success of a request depends on who you are as a user and particularly that notions of user status and user similarity could be influential in the process .we study both status and similarity in this work .the textual part of a request , the narrative , has been shown to significantly influence the outcome in peer - to - peer lending platforms . in order to understand the nature and power of different narratives without coding them manually , we explore automatic methods of narrative extraction .consider the following two pizza requests :     to identify the different kinds of stories we draw on previous literature suggesting that narratives can be automatically extracted using topic modeling and related techniques .we therefore perform topic modeling through non - negative matrix factorization ( nmf ) of a tf - idf weighted bag - of - words representation of the requests in our dataset .we additionally enforce sparsity on the topic distribution for each request to shape the topics in a way that captures most of a given request , and restrict ourselves to nouns ( using the stanford part - of - speech tagger ) .we choose to use 10 topics and use a svd - based initialization for nmf .the resulting topics are shown in table [ table : topic_modeling ] along with descriptive names , the 15 highest - scoring terms and the success rate ( fraction of requests that successfully obtained pizza ) .we observe that many topic clusters follow a specific theme and that their success rates vary dramatically ( the average success rate is 24.6% ) .topics money1 and money2 focus on money , and the high success rate of topic money1 ( 32.3% ) suggests that this is a particularly successful narrative . topic job is similarly successful ( 31.9% ) and features job related terms . a large number of requests further seem to come from college students talking about studying for classes and finals , their roommates , and the university ( topic student ) .another narrative in the data are requests for and about family ( topics time&family and money2 ) .this narrative can be identified by the usage of words indicating family relationships like kid , husband , family , mother , wife , and parents ( not all of them are included in the top 15 terms ) .topic friend stands out since it noticeably worse than any other topic ( 17.0% ) .it captures requests asking for pizza for a friend that is in town , to cater for parties , or to provide culinary support for a night of movie watching with the girlfriend .we hypothesize that stories of this topic display little actual need for pizza ( particularly compared to stories talking about money and job problems ) and simply communicate a pizza craving by the requester .we further recognize that many requests employ previously defined factors such as gratitude ( `` thanks in advance '' in topic gratitude ) , providing pictures as additional evidence , and the intention to `` pay it forward '' .online platforms have created a new mechanism for people to seek aid from other users .many online communities such as question & answer sites and online philanthropy communities are created for the express purpose of facilitating this exchange of help .it is of critical importance to these communities that requests get addressed in an effective and efficient manner .this presents a clear opportunity to improve these online communities overall as well as improving the chance of success of individual requests .however , the factors that lead to requests being fulfilled are still largely unknown .we attribute this to the fact that the study of _ how _one should ask for a favor is often complicated by large effects of _ what _ the requester is actually asking for . we have presented a case study of an online community where all requests ask for the very same contribution , a pizza , thereby naturally controlling for this effect and allowing us to disentangle what is requested from textual and social factors .  drawing from social psychology literaturewe extract high - level social features from text that operationalize the relation between recipient and donor and demonstrate that these extracted relations are predictive of success .we show that we can detect key narratives automatically that have significant impact on the success of the request .we further demonstrate that linguistic indications of gratitude , evidentiality , and reciprocity , as well as the high status of the asker , all increase the likelihood of success , while neither politeness nor positive sentiment seem to be associated with success in our setting .we link these findings to research in psychology on helping behavior ( see table [ table : helping_behavior_link ] ) . for example , our work extends psychological results on offline communities to show that people behave pro - socially in online communities toward requestors who are of high status , display urgency , and who offer to pay it forward .other novel contributions of our work include the finding that linguistic indications of gratitude lead to pro - social behavior , and the result that higher status users are more likely to demonstrate generalized reciprocity .our results thus offer new directions for the understanding of pro - social behavior in communities in general , as well as providing a basis for further analysis of success in social media systems .we must recognize a number of limitations : a shortcoming of any case study is that findings might be specific to the scenario at hand . while we have shown that particular linguistic and social factors differentiate between successful and unsuccessful requests we can not claim a causal relationship between the proposed factors and success that would guarantee success .furthermore , the set of success factors studied in this work is likely to be incomplete as well and excludes , for instance , group behavior dynamics . despite these limitations , we hope that this work and the data we make available will provide a basis for further research on success factors and helping behavior in other online communities .we thank christina brandt and jure leskovec for many helpful discussions , niloufar salehi and tuan nguyen for helping with computing user similarity and statistical analysis , and julian mcauley for the original idea to use the raop dataset .we also thank reddit user `` jimwll '' who generously provided pizza when we most needed it and the anonymous reviewers for their altruistic comments .this work was supported in part by nsf iis-1016909 .gore , k. y. ; tobiasen , m. a. ; and kayson , w. a. 1997 .effects of sex of caller , implied sexual orientation of caller , and urgency on altruistic response using the wrong number technique .80(3):927930 .yinon , y. , and dovrat , m. 1987 .the reciprocity - arousing potential of the requester s occupation , its status and the cost and urgency of the request as determinants of helping behavior .

Abstract: requests are at the core of many social media systems such as question & answer sites and online philanthropy communities . while the success of such requests is critical to the success of the community , the factors that lead community members to satisfy a request are largely unknown . success of a request depends on factors like _ who _ is asking , _ how _ they are asking , _ when _ are they asking , and most critically _ what _ is being requested , ranging from small favors to substantial monetary donations . we present a case study of _ altruistic _ requests in an online community where all requests ask for the very same contribution and do not offer anything tangible in return , allowing us to disentangle what is requested from textual and social factors . drawing from social psychology literature , we extract high - level social features from text that operationalize social relations between recipient and donor and demonstrate that these extracted relations are predictive of success . more specifically , we find that clearly communicating need through the narrative is essential and that linguistic indications of gratitude , evidentiality , and generalized reciprocity , as well as high status of the asker further increase the likelihood of success . building on this understanding , we develop a model that can predict the success of unseen requests , significantly improving over several baselines . we link these findings to research in psychology on helping behavior , providing a basis for further analysis of success in social media systems .


error exponent analysis has been an active area of research for quite a few decades .the vast literature in this area can be categorized based on ( i ) whether the channel is memoryless or with memory ; ( ii ) whether there is or is not channel output feedback to the transmitter ; ( iii ) whether the employed coding is fixed - length or variable - length ; and ( iv ) whether upper ( converse ) or lower ( achievable ) bounds are analyzed .  in the case of memoryless channels with noiseless feedabck schalkwijk and kailath proposed a transmission scheme for the additive white gaussian noise ( awgn ) channel with infinite error exponent . on the other hand ,dobrushin  and later harutunian  , by deriving an error upper bound for discrete memoryless channels ( dmcs ) showed that at least for symmetric channels there is no gain to be expected through feedback when fixed - length codes are employed .this was a strong negative result since it suggested that for dmc channels , noiseless feedback can neither improve capacity ( as was well known ) nor can it improve the error exponent when fixed - length codes are used .a remarkable result was derived by burnashev in  , where matching upper and lower bounds were derived for dmcs with feedback and variable - length codes .the error exponent has a simple form , where is the average rate , is the channel capacity and is the maximum divergence that can be obtained in the channel for a binary hypothesis testing problem .several variable - length transmission schemes have been proposed in the literature for dmcs and their error exponents have been analyzed  .  in the case of channels with memory and feedback ,the capacity was studied in  , and a number of capacity - achievable schemes have been recently studied in the literature  . the only work that studies error exponents for variable - length codes for channels with memory and feedback is  where the authors consider a finite state channels with channel state known causally to both the transmitter and the receiver .  in this work, we consider channels with memory and feedback , and derive a straight - line upper bound on the error - exponent for variable - length codes .we specifically look at unifilar channels since for this family , the capacity has been characterized in an elegant way through the use of markov decision processes ( mdps )  .our technique is motivated by that of  , i.e. , studying the rate of decay of the posterior message entropy using martingale theory in two distinct regimes : large and small message entropies . a major difference between this work as compared to  is that we analyze the multi - step drift behavior of the communication system instead of the one - step drift that is analyzed for dmcs .this is necessitated by the fact that one - step analysis can not capture the memory inherent in the channel and thus results in extremely loose bounds .it is not surprising that the parameter in our case also relates to the maximum discrimination that can be achieved in this channel in a binary hypothesis testing problem . in order to evaluate this quantity ,we formulate two mdps with decreasing degree of complexity , the solutions of which are upper bounds on the the quantity , with the former being tighter than the latter .the tightness of the bounds is argued based on the fact that asymptotically this is the expected performance of the best system , and by achievability results presented in the companion paper  .an additional contribution of this work is a complete reworking of some of the more opaque proofs of  resulting in significant simplification of the exposition .we finally provide some numerical results for a number of interesting unifilar channels including the trapdoor , chemical , and other two - input / output / state unifilar channels . the main difference between our work and that in  is that for unfilar channels , the channel state is not observed at the receiver .this complicates the analysis considerably as is evidenced by the different approaches in evaluating the constant in these two works .furthermore , our results indicate that optimal policies for achieving the maximum divergence are very different when the receiver knows or does not know the channel state .the remaining part of this paper is organized as follows . in section [ sec :model ] , we describe the channel model for the unifilar channel and the class of encoding and decoding strategies . in section [ sec : bound ] , we analyze the drifts of the posterior message entropy in the large- and small - entropy regime . in section [ sec : c_1 ] , we formulate two mdps in order to study the problem of one - bit transmission over this channel .section [ sec : example ] presents numerical results for several unifilar channels .final conclusions are given in section [ sec : conclusions ]consider a family of finite - state point - to - point channels with inputs , output and state at time , with all alphabets being finite and initial state known to both the transmitter and the receiver .the channel conditional probability is for a given stochastic kernel and deterministic function , where denotes the space of all probability measure on , and is the kronecker delta function .this family of channels is referred to as unifilar channels  .the authors in  have derived the capacity under certain conditions in the form of in this paper , we restrict our attention to such channels with strictly positive for any and ergodic behavior so that the above limit indeed exists .let be the message to be transmitted .in this system , the transmitter receives perfect feedback of the output with unit delay and decides the input based on at time .the transmitter can adopt randomized encoding strategies , where with a collection of distributions .the decoding policy consists of a stopping time w.r.t .filtration and estimated message at every time .the average rate and error probability of this scheme are defined as } ] . since transmission schemes with result in the trivial error exponent }=0 ] which is exactly what we would do in the z channel .in this paper , we derive an upper bound on the error - exponent of unifilar channels with noiseless feedback and variable length codes .we generalize burnashev s techniques by performing multi - step drift analysis and deriving a lower bound on the stopping time together with a proposed submartingale . the constant which is the zero rate exponentis evaluated through an mdp and furher upper bounded through a more computationally tractable mdp .numerical results show that for some unifilar channels the two mdps give different results .a future research direction is the analytical solution of these mdps .in addition , the presented analysis can be easily generalized to channels with finite state and inter - symbol interference ( isi ) with the state known only to the receiver .given any and , \nonumber\\  & = -i(w;y_{t+1}|y^{t}=y^{t},s_1=s_1)\nonumber \\  & = -h(y_{t+1}|y^{t}=y^{t},s_1=s_1 ) +  h(y_{t+1}|y^{t}=y^{t},s_1=s_1,w ) \nonumber \\  & \overset{(a)}{\geqslant } -h(y_{t+1}|y^{t}=y^{t},s_1=s_1 ) +  h(y_{t+1}|y^{t}=y^{t},s_1=s_1,w , s_{t+1},x_{t+1 } ) \nonumber \\  & \overset{(b)}{= } -h(y_{t+1}|y^{t}=y^{t},s_1=s_1 ) +  h(y_{t+1}|y^{t}=y^{t},s_1=s_1,s_{t+1},x_{t+1 } ) \nonumber \\  & = -i(x_{t+1},s_{t+1};y_{t+1}|y^{t}=y^{t},s_1=s_1),\end{aligned}\ ] ] where ( a ) is due to conditioning reduces entropy , and ( c ) is due to the channel properties . note that the last term is the mutual information between and conditioning on , which is different from conditional mutual information .now the -step drift becomes \nonumber \\  & = -e[\sum_{k = t}^{t+n-1}\sum_{y^k_{t+1 } } p(y^k_{t+1}=y^k_{t+1}|y^t = y^t , s_1=s_1)e[h_{k+1 } - h_{k}|y^k = y^k , s_1=s_1]|y^t = y^t , s_1=s_1 ] \nonumber \\  & \overset{(a)}{\geqslant } -e[\sum_{k = t}^{t+n-1 } \sum_{y^k_{t+1 } } p(y^k_{t+1}=y^k_{t+1}|y^t = y^t , s_1=s_1 ) i(x_{k+1},s_{k+1 } ; y_{k+1}|y^{k}=y^k , s_1=s_1)|y^t = y^t , s_1=s_1]\nonumber \\  & = -e[\sum_{k = t}^{t+n-1 } i(x_{k+1},s_{k+1 } ; y_{k+1}|y^{k}_{t+1},y^{t}=y^t , s_1=s_1)|y^t = y^t , s_1=s_1]\nonumber \\  & \overset{(b)}{\geqslant } -n(c+\epsilon),\end{aligned}\ ] ] where ( a ) is due to  and ( b ) is due to .given any and , = \nonumber \\ & e[\log\frac{-\sum_{i}p(w = i|y^{t+n}_{t+1},y^t , s_1 )  \log p(w = i|y^{t+n}_{t+1},y^t , s_1)}{-\sum_{i}p(w = i|y^{t},s_1 )  \log p(w = i|y^t , s_1)}|y^t = y^t , s_1=s1].\end{aligned}\ ] ] for convenience , we define the following quantities    since , there exits a such that while for .we further define for .the following approximations are valid for close to 1 .  substituting these approximate expressions back to the drift expression we have \nonumber \\  & = \sum_{y^{t+n}_{t+1 } } p(y^{t+n}_{t+1}|y^t , s_1 ) \log \frac{\sum_i f_i(y^{t+n}_{t+1})\log f_i(y^{t+n}_{t+1})}{\sum_i f_i\log f_i }  \nonumber \\  & = \sum_{y^{t+n}_{t+1 } } \hat{q}(y^{t+n}_{t+1}|k ) \log \frac{(1-f_k ) ( \log(1-f_k ) + o(\log(1-f_k)))\sum_{j\neq k}\frac{\hat{f}_j\hat{q}(y^{t+n}_{t+1}|j)}{\hat{q}(y^{t+n}_{t+1}|k)}}{(1-f_k)(\log ( 1-f_k ) + o(\log(1-f_k ) ) }  \nonumber \\  & = -\sum_{y^{t+n}_{t+1}}\hat{q}(y^{t+n}_{t+1}|k ) \log \frac{\hat{q}(y^{t+n}_{t+1}|k)}{\sum_{j\neq k}\hat{f}_j\hat{q}(y^{t+n}_{t+1}|j ) } + o(1)\nonumber \\  & \geqslant -n(c_1 + \epsilon),\end{aligned}\ ] ] where the last inequality is due to the definition of .we can always choose a sufficiently small positive such that    we first consider the case . where ( a ) is due to .therefore we have & = e[z_{t+1}1_{\{h_{t } > h^*\}}-z_t1_{\{h_{t } > h^*\}}|\mathcal{f}_t ] \nonumber\\ & \geqslant e[(\frac{h_{t+1}-h^*}{k_1}+t+1)1_{\{h_{t } > h^*\}}-(\frac{h_{t}-h^*}{k_1}+t)1_{\{h_{t } > h^*\}}|\mathcal{f}_t ] \nonumber \\ & \geqslant 0 , \label{ineq : subhtbig}\end{aligned}\ ] ] where the last equation is due to .similarly , for the case , from we have and therefore \nonumber \\  & \geqslant e[\frac{\log \frac{h_{t+1}}{h^*}}{k_2}+t+1+f(\log \frac{h_{t+1}}{h^*})-\frac{\log \frac{h_{t}}{h^*}}{k_2}-t - f(\log \frac{h_{t}}{h^*})|\mathcal{f}_t]\nonumber \\ & \overset{(a)}{= } e[(\frac{1}{k_2}+f'(\log\frac{h_{t+1}}{h^*}))(\log\frac{h_{t+1}}{h^*}-\log\frac{h_{t}}{h^*})+1+\frac{f''(z(h_{t+1},h_t))}{2}(\log\frac{h_{t+1}}{h^*}-\log\frac{h_{t}}{h^*})^2|\mathcal{f}_t]\nonumber \\ & \overset{(b)}{\geqslant } e[-k_2f'(\log\frac{h_{t+1}}{h^*})+\frac{f''(z(h_{t+1},h_t))}{2}(\log\frac{h_{t+1}}{h^*}-\log\frac{h_{t}}{h^*})^2|\mathcal{f}_t ]  \nonumber \\ & = e[e^{\lambda\log\frac{h_{t+1}}{h^*}}+\frac{-\lambda e^{\lambda ( z(h_{t+1},h_t)-\log\frac{h_{t+1}}{h^*}+\log\frac{h_{t+1}}{h^*})}}{2k_2}(\log\frac{h_{t+1}}{h^*}-\log\frac{h_{t}}{h^*})^2|\mathcal{f}_t ] \nonumber \\ & \overset{(c)}{\geqslant }  e[e^{\lambda\log\frac{h_{t+1}}{h^*}}-\frac{\lambda e^{\lambda ( k_3+\log\frac{h_{t+1}}{h^*})}}{2k_2}(\log\frac{h_{t+1}}{h^*}-\log\frac{h_{t}}{h^*})^2|\mathcal{f}_t ] \nonumber \\ & \overset{(d)}{\geqslant } ( 1-\frac{\lambda e^{\lambda k_3}}{2k_2}k_3 ^ 2 ) e[e^{\lambda\log\frac{h_{t+1}}{h^*}}|\mathcal{f}_t ] \nonumber \\ & \overset{(e)}{\geqslant}\ 0,\label{ineq : subhtsmall}\end{aligned}\ ] ] where ( a ) is from the second - order taylor s expansion of at , ( b ) is due to and , ( c ) is due to that is between and , ( d ) is due to , and ( e ) is due to . from and , we have \geqslant 0 $ ] and thus is a submartingale .the proof essentially applies lemma [ lemma : newsubmartingale ] to the `` block '' submartingale .given any , there exists an such that by lemma [ lemma : driftentropy ] and lemma [ lemma : driftlogentropy ] ,    define , where is defined in , and filtration . is a submartingale w.r.t . by lemma [ lemma : newsubmartingale ] .notice that the quantity here indicates the order of the block of consecutive transmissions .furthermore , define the stopping time w.r.t . by . by definition of , we have now we essentially apply the optional sampling theorem on the submartingale as follows \nonumber \\  & = e[(\frac{\log h_{n\hat{t}}-\log \epsilon}{n ( c_1 + \epsilon ) } + f(\frac{\log h_{n\hat{t}}}{\log \epsilon}))1_{h_{n\hat{t}}\leqslant\epsilon } ] + e [ ( \frac{h_{n\hat{t}}-\epsilon}{n(c+\epsilon ) } ) 1_{h_{n\hat{t}}>\epsilon } ] + e[\hat{t } ] \nonumber \\  & \leqslant e[\frac{\log h_{n\hat{t}}+|\log \epsilon|}{n ( c_1 + \epsilon ) } + f(\frac{\log h_{n\hat{t}}}{\log \epsilon } ) ] + e [ \frac{h_{n\hat{t}}+\epsilon}{n(c+\epsilon ) } ] + e[\hat{t } ] \nonumber \\  & \overset{(a)}{= } e[\frac{\log h_{t}+|\log \epsilon|}{n ( c_1 + \epsilon ) } + f(\frac{\log h_{t}}{\log \epsilon } ) ] + e [ \frac{h_{t}+\epsilon}{n(c+\epsilon ) } ] + e[\hat{t } ]  \nonumber \\  & \overset{(b)}{\leqslant } \frac{\log e[h_{t}]+|\log \epsilon|}{n ( c_1 + \epsilon ) } + e[f(\frac{\log h_{t}}{\log \epsilon } ) ] +  \frac{e[h_{t}]+\epsilon}{n(c+\epsilon ) }  + e[\hat{t } ] \nonumber \\  & \overset{(c)}{\leqslant }  \frac{\log e[h_{t}]+|\log \epsilon|}{n ( c_1 + \epsilon ) }  + e[f(\frac{\log h_{n\hat{t}}}{\log \epsilon } ) ] + \frac{e[h_{t}]+\epsilon}{n(c+\epsilon)}+\frac{e[t]}{n}+1  \nonumber \\  & \overset{(d)}{\leqslant }  \frac{\log e[h_{t}]+|\log \epsilon|}{n ( c_1 + \epsilon ) } + \frac{e[h_{t}]+\epsilon}{n(c+\epsilon)}+ \frac{e[t]}{n}+1 + \frac{1}{\lambda n c_1 } \nonumber \\  & \overset{(e)}{\leqslant }  \frac{\log pe + \log(k-\log pe)+|\log \epsilon | } { n ( c_1 + \epsilon ) } + \frac{-pe\log pe-(1-pe)\log(1-pe ) + pek+\epsilon}{n(c+\epsilon ) } + \frac{e[t]}{n}+1 + \frac{1}{\lambda nc_1},\end{aligned}\ ] ] where ( a ) is due to that the receiver no longer performs actions after time , ( b ) is due the the concavity of , ( c ) is due to , ( d ) is due to that is upper - bounded by , and ( e ) is due to the fano s lemma .multiplying on the both sides the above inequality , we get } & \leqslant c_1(1-\frac{\overline{r}}{c } ) + \frac { \log(k-\log pe)+ |\log \epsilon|}{k/\overline{r } }  + \epsilon ( 1+\frac{|c_1-c|}{c(c+\epsilon)}\overline{r})\nonumber \\ & \qquad + \frac{c_1+\epsilon}{k/\overline{r } } ( n(\epsilon)+\frac{1}{\lambda c_1}+\frac{-pe\log pe -(1-pe)\log(1-pe)+pek+2\epsilon}{c+\epsilon } ) , \label{ineq : result}\end{aligned}\ ] ] which proves the result .

Abstract: the reliability function of memoryless channels with noiseless feedback and variable - length coding has been found to be a linear function of the average rate in the classic work of burnashev . in this work we consider unifilar channels with noiseless feedback and study upper bounds for the channel reliability function with variable length codes . in unifilar channels the channel state is known to the transmitter but is unknown to the receiver . we generalize burnashev s analysis and derive a similar expression which is linear in average rate and depends on the channel capacity , as well as an additional parameter which relates to a sequential binary hypothesis testing problem over this channel . this parameter is evaluated by setting up an appropriate markov decision process ( mdp ) . furthermore , an upper bound for this parameter is derived using a simplified mdp . numerical evaluation of the parameter for several binary input / state / output unifilar channels hints at the optimal transmission strategies . such strategies are studied in a companion paper to provide lower ( achievable ) bounds on the channel reliability function .