a distributed processing platform provides a method of making a remote procedure call , an rpc , almost transparent to the application developer see , for example , _ _ tivoli__ and _ orbix _  the relatively mature industry standard for both of which is the common object request broker architecture , _ corba _ , from the object management group , omg ; this defines an object request broker , orb , which is an infrastructure for rpcs .recently _ sun microsystems _ have enhanced the remote method invocation package , , for _ java_ as part of the _ java_development kit , jdk , 1.2 .it now allows connections to be created between a client and a server which can have a different data transfer representation . as pointed out in the documentation for this feature , this is particularly suitable for implementing the secure socket layer , ssl , and could also be used to implement the proposed successor to ssl transport level security , .[ [ open - distributed - processing - architecture ] ] open distributed processing architecture + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  a suitable architecture to exploit this new functionality in and in other distributed processing platforms has been proposed in the open distributed processing standards , iso odp , .[ [ bindings - and - channels ] ] bindings and channels  the prescriptive model , , generalizes the concept of connection between client and server to be a logical binding which is realized by both client and server using a channel , see also .figure [ fig : odp - channel ] illustrates the concepts of bindings and channels .application objects have bound with one another probably using a naming service ; they have negotiated a channel configuration that requires two objects : a data presentation conversion object and a data transport object .the channel configuration is fixed by bindings .the client sends data ; the server receives .the data is passed from the application object to the channel which carries out the conversion and the transport to the server .  binding : :  a binding is a contract between the client and the server stating the  parameters by which they will communicate .a binding between two  transport objects would usually specify :  +  * transport layer protocol to be used , _e.g._udp / ip or tcp / ip or  pipe / unix .* transport layer parameters  * network addresses for chosen transport layer protocol  +  the presentation layer object would convert from the the local data  representation to a network representation .the only specification  needed here is the source and target representations . in this case ,  the binding is implicit in the implementation of the objects , they  need not be initialized with parameters .+  more sophisticated systems will have higher demands and will insist  that other services be used as well , for example :  +  * data security  * transaction management  * call billing  * data compression  * relocation manager  +  these would all require that configuration parameters be specified and  may also demand that they are operate together .there is no need for channels to be symmetric , the server could implement a call logging object in its channel without having an object of the same type in the client s channel .bindings do need to be current .a transport object might close a connection , in which case , it would no longer be current , but if it were to leave enough information to allow a re connection , then it is , in effect , still current .  [[ tripartite - bindings ] ] tri partite bindings  bindings need not be bi partite .an application service that would require a tri partite binding is relocation management , illustrated in figure [ fig : odp-3binding ] .the idea of which is that should the server choose to relocate , it would notify a relocation manager of its new addresses and move there . when the client calls the server at its old address and fails to reach it , the relocator object in the client s channel would call the third party and ask for the new address of the server , establish a new set of bindings with it , _ i.e._construct a new channel and destroy the old , and send the message again .this would all be transparent to the application object .the relocation object in the channel would need to call the relocation manager for the new addresses and would thus become an application object ; it would need to establish its own channel with the relocation manager .although the iso odp defines a flexible architecture , most implementors of distributed processing platforms provide application programmers with inflexible systems : the channel can only contain a presentation and a transport object .  [ [ presentation - stubs - and - skeletons ] ] presentation : stubs and skeletons + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  both _ corba _ orbs and the rmi defined in _ java_make use of `` stubs '' and `` skeletons '' and a stub compiler .the term infrastructure will be used for the engineering that realizes an orb or rmi .stubs : :  these are used by a client when invoking a method on a remote server .  the client does not have a local implementation for the service the  remote server has , but it will have a definition of its interface  which can be used as if it were the implementation of the service .it  can present the interface definition to a stub compiler which will  generate code to invoke each method on the remote server .the stub  acts as a proxy for the remote server in the client s address space.  the stub for each method needs to do the following things :  +  1 .  construct a request  +  a request object is a container entity that carries the invocation to  the server ; it contains :  * the address of the remote server  * the name of the method being invoked , possibly with version control  information for the interface .  * the parameters for the method  * the return address  +  the stubs can also contain the code to __ marshall _ _ the parameters into a universal transfer  presentation . in this form, the stubs also comprise the presentation  object .  2 .invoke the request  +  the request , now just a sequence of bytes with an associated address  for the server and a return address for the sender , is passed to the  infrastructrure which sends it to the network socket for the server .get the reply  +  the server will return a reply in another container type .construct the reply  +  the reply is then re constructed or , rather , its contents are  _ un marshalled _ and returned to the client . again, this may be part of  the code in the stubs .infrastructure : :  it might be best now to explain how the infrastructure manages to  provide the rpc service.  +  1 .  at the server on creation+  an application programmer defines an interface and produces an  implementation for it .a program that effectively acts as a loader  issues instructions to the infrastructure to create a socket to  receive requests for that server and will associate the remote server  implementation with that socket .  2 .at the naming service  +  the application programmer will ensure that the address of his  newly created remote server is put into a well known naming service .the client will then collect the address from the naming service .3 .  at the client on invocation+  the address collected at the naming service will contain enough  information to allow the client s infrastructure to send the request  container as a stream of bytes to the socket that the server s  infrastructure has associated with the remote server s implementation .4 .  at the server on invocation+  the network socket will be activated by the client s infrastructure ( a  connect and a data send ) and the server s infrastructure will collect  the data at the socket and , because it has recorded the object  responsible for that socket , it can activate the server skeleton .skeletons : :  [ sec : inflexible:1 ] these are invoked by the server s infrastructure  when the network socket for a server is activated and the data  comprising a call has been collected .it invokes the implementation of  the method the client wants to use at the server .a skeleton can  consist of one method that unmarshalls enough of the request container  to be able to look up the method to invoke this is known as  _dispatching_.  +  this partial unmarshalling has to be done in this way , because each  method will unmarshall the remainder of the request container  differently to obtain the parameters to pass to the server s method  implementation .+  after the invocation is made the results will be marshalled into a  reply container which is then sent back to the client .the stubs and the skeleton effectively form the presentation channel object .stubs have to be produced by a stub compiler , but skeletons can be made generic , if the underlying infrastructure supports a reflective invocation mechanism , see or the _ corba _ dynamic invocation interface .some stubs generated by the _ java _ , remote method interface compiler , are given in appendix [ cha : stubs ] .these demonstrate the use of reflective language features .[ [ transport ] ] transport + + + + + + + + +  when the request containers are passed to the infrastructure the transport object is eventually invoked .most orbs only provide one transport mechanism which sends data through tcp / ip sockets to its destination .although some systems do allow different protocols : udp / ip or pipe / unix .[ [ limitations ] ] limitations + + + + + + + + + + +  the problem with this is that if one wants to implement any useful application services encryption , billing and so forth the infrastructure does not help . for example , to encrypt and decrypt data sent as part of a remote procedure call , one would have to implement one s own stubs and skeletons , see figure [ fig : odp - channel-0 ] .the application programmer has to construct a call , marshall the data , encrypt it , send it using a generic method , which will marshall it again . at the server ,the data would be delivered to the generic method , and the application programmer would then have to decrypt it , unmarshall the decrypted data , reconstruct the call and dispatch it .after dispatching , collect the results , marshall , encrypt and return the reply .a more flexible implementation has been provided by _sun _ in _ java_. it allows a different type of socket to be used as the transport object .[ [ method ] ] method + + + + + +  the custom socket factories have to implemented in the following way :  1 .derive and implement classes for the new socket type s datastream from and , call them and .2 .  derive and implement classes for the new socket types and that use the new streams and .  then create socket factory implementations that can be used by rmi .  1 .  a client - side socket factory that implements and implement the method .2 .  a server - side socket factory that implements and implement the method .  then one has to ensure that the constructor for the remote server is told to use the new socket factories .the infrastructure creates the new type of socket when demanded and invokes the create socket methods .the at the client will then determine that a particular type of socket has to be used and will load the custom socket factory implementations .  [ [ possibilities - implementing - ssl ] ] possibilities : implementing ssl + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  using custom socket factories , it is possible to implement a secure socket layer .the would be used to perform the key exchange with the server and the custom input and output streams would apply the session key to encrypt on send and decrypt on receive .[ [ limitations-1 ] ] limitations + + + + + + + + + + +  unfortunately , using custom socket factories only increases the variety of the transport objects that can be employed , it does not allow different kinds of objects to be placed in the channel .what is needed is a means of placing objects in the channel before and after the presentation object .these objects should have a simpler instantiation and invocation procedure than using the custom socket method in _java_.   1 .different interests system configurable + the objects placed in the channel between client and server are the result of a negotiated agreement between the client , its server and their respective environments .it is well known that security requirements for messages depend on the workstation that the client is using , which may be connected to a secure local area network on which both the the client and server reside and so , for example , no security measures need be taken ; or , the client could be accessing the server remotely from the internet through a modem in which the server might require that the client use encryption . + it would be desirable if the client and the server could both specify their requirements and some negotiation take place that could create a mutually acceptable protocol stack in the software communications channel .2 .  different methods for different actions +a request and reply actually require that four different channels be traversed by messages , see figure [ fig : odp-4channels ] : 1 .request this channel is created by the infrastructure for the client and sends a message over the network .2 .  indication this channel is used to receive from the network and is created by the infrastructure for the server .3 .  response created by the infrastructure for the server to return the results of the client s request message .confirmation receives from the network and is created by the infrastructure for the client .+ should an error occur at the server it is returned via the response and confirmation messages .+ the stubs are responsible for managing the thread of execution of the application object : once a message is put onto the request channel , the application object s thread can be suspended whilst it waits for the reply to arrive on the confirmation confirmation channel .+ if the message is a cast of some kind ( broadcast , multi cast or one cast ) there will be no response or confirmation .3 .  different implementations + most channel objects are derived from the same source and will have the same implementation ._ java_allows classes implementing different objects to be loaded over the network , so it would be possible for the client and server to agree upon and load the same class implementation , which they would be able to do with the custom socket factories method .this may not always be the case , some channel objects may be optimized to make use of a different operating system , but provide the same functionality .audio and video data streaming are good examples of this need : some micro processors now have support for stream data .4 .  transparency 1 .management + it would be desirable if the additional services provided by the channel objects did not need to be initialized or managed by either the client or the server application programs but they were activated by their respective infrastructures .exception handling + one of the difficulties of developing applications in enterprise environments is that as messaging becomes more sophisticated supporting for example , confidentiality , authorization , call billing the number of possible errors increases because each of these sub systems introduces new ones . it must be possible for channel objects to clear down their own errors , so that errors returned to application objects only involve the application .3 .  using possession rather than inheritance for coupling +it would also be desirable if the classes for the channel objects did not extend the existing class hierachies of the message transmission sub system , in the way that custom socket and socket factories do .extending class hierachies is not as flexible as specifying an order of invocation .interface definitions suitable for reflection .+ it would also be desirable if the infrastructures could load channel objects classes remotely and have a simple enough invocation syntax so that reflection mechanisms could be used to invoke the channel objects _ without requiring a stub compiler_. _ java_already does this with its version 1.2 stubs , using package , and _ corba _ has a dynamic invocation interface which can achieve the same goal .5 .  efficiency + it would also be desirable if the channel objects did not create a large stack of calls ; primarily because some target environments for remote procedure call platforms will be embedded systems on smartcards .   with the aid of figure [ fig : odp-4channels ] ,it is possible to be more precise with the terms used :  * initiator : starts a four phase call sequence ; may also be called the requestor . *acceptor : accepts the call made upon it ; may also be called the responder , because it generates the response .  both an initiator and an acceptor will send and receive as part of four phase call sequence .ordinarily , the client will be the initiator of all calls , but some remote procedure call systems permit call backs , in which case the server is the initiator and the client the acceptor .there are basically two types of architecture that could support channel objects .the custom socket factory architecture is stream oriented .it acts upon the data being sent between client and server as a stream applying a data transformation to the data when it is sent and undoing this transformation at the other end .the data is treated as opaque and can be delivered in packets as small as one byte .the implementor of the stream handler does not know whether the data has just started or is about to finish .one of the attractions of this approach is that many of the operations that data networks perform on data can be implemented in software : segmenting , and its converse re assembling , can be implemented easily and this would allow remote procedure call systems to make use of packet oriented transmission , such as udp / ip .segmenting and re assembly could be implemented as channel objects and data would be segmented and then sent as packets on a udp socket for re assembly by another channel object at the server .streams can be chained the output of one providing the input for the next .most operating systems support _ pipes _ to do this : _ java_supports the , and output , classes .multi casting could be easily achieved with pipes : simply have a pipe that sends on a socket and echoes its input ; then connect a series of these together .  a stream oriented architecture is better for real time data delivery .all of the processing in the stream handler is applied to the data it is not expected to communicate with relocation or transaction managers and incur indeterminate time penalties .consequently , the emphasis in the design of stream handlers should be to ensure they introduce a constant latency in transmission and reception .this architecture implicitly appreciates that the data being delivered is a call .call handlers usually add parameters to a remote procedure call .they do not form part of the message sent by the application object , but set its context .for example :  * timestamps logging when messages are sent and received by adding an `` time sent '' parameter to the remote procedure call . *accounting adding an account identifier to a remote procedure call so that the server can log charges for calls to a particular account . * transactions a transaction might consist of a number of calls to different servers , they could all be identified with a transaction identifier , to synchronize committing and aborting transactions as a whole . *authorization a privilege certificate could be attached to the call so that the server could check what rights and privileges the caller is allowed to exercise within the server s work space .a call oriented architecture _ should _ be implemented so that it has access to the parameters passed as part of the request .if this is the case , then as well as being able to add parameters , it would be possible to perform data transformations on parameter values that are part of the call .for example :  1 .representation conversions 1 .wholly + the presentation objects implemented in remote procedure call systems change the data representation of the parameters of a call so that they can be transmitted over the network as a sequence of bytes , with the receiver being responsible for converting the byte sequence to its local representation .it may be more efficient to convert to the receiver s format before the data is sent so that the server can use conversion methods available from its native operating system .2 .  partly + it might be the case that a server has a different data context for a particular data type : internationalization of text strings and currency formats could be converted prior to transmission .2 .  pseudo objects + pseudo objects are usually legacy systems that can be directly controlled by the client s remote procedure call infrastructure , but for the sake of uniformity , and to simplify re engineering and relocation of services , they are provided with the same interface as remote servers .the operating system used by a distributed processing platform is itself a legacy system .+ an example of a pseudo object that application programmers use is the database driver provided in the _ java_database connectivity package , .this pseudo object implementation establishes and drives a connection with the database .other examples of services that could be implemented as pseudo objects are directory and naming services that are available through native operating systems .3 .  stream + a call oriented architecture could also be used to transform data in the same way that a stream oriented architecture could .encryption , compression and checksum insertion could all be performed by marshalling the parameters using a data representation object to produce a sequence of bytes and then applying the stream operation to it .the output would be opaque and would replace the parameters .a call oriented architecture is better suited to recovering from errors , since it is possible to determine which channel object is at fault and it can take measures to recover from the error .   the stream oriented architecture is ideal for delivering data at high speed with a determinate latency , the call oriented architecture is ideal for communicating control information .this is a similar design problem that faced the developers of telephone networks and it was resolved , in the integrated services digital network , isdn , by having a control channel manage the use of two bearer channels .figure [ fig : odp-2sources ] illustrates a request made to the server on a control channel and a reply being received on a broadband data channel .this sort of architecture could be used for controlling the delivery of `` pay per view '' television , where the rpc mechanism is used by an application to make a payment over a control network and the data is delivered on differently constructed channels over a broadband network possibly to different hardware .essentially the differences between the two architectures , with regard to the activation of the channel objects , are :  * synchronizing with the call , and * the opacity of the call s contents  a stream oriented architecture need not transmit data to the server in a contiguous block that represents the marshalled bytes of a message .a call oriented architecture would have each channel object invoked with each call sent .a stream oriented architecture only has access to the marshalled bytes that represent a message .a call oriented structure sees the method that is invoked and the parameters for it .the most flexible architecture is the call oriented one .but for implementing the transport objects , a stream oriented architecture should be preferred .this means that the two architectures fall above and below the marshalling channel object , see figure [ fig : odp - call - stream ] .this figure attempts to place the channel objects in an open systems interconnection model , .objects in the call oriented architecture provide presentation layer services and , after the marshalling object , which reduces a call to a sequence of bytes , the session objects and finally the network transport object , a socket driver , can operate upon the data as a stream .the session layer objects would also perform stream oriented encryption , but a presentation layer object would negotiate keys .channel objects that are call oriented will be called _ call handlers _ and those that are stream oriented will be called _ stream the marshalling object is a call handler and , if need be , it could invoked a number of time to render the data as a sequence of bytes .this would be useful for data security , since it may be necessary to encrypt the data and make it unintelligible to other call handlers ._ sun _ with _ java_and the socket factory technique have implemented what is described later as a simplex system , [ sec : simplex ] : each stream has two channel objects , one for sending the output stream and one for receiving the input stream .there are two kinds of pairs : the client s , or , more precisely , the initiator s , pair and the server s , or acceptor s , pair .although there is no explicit code to synchronize the two streams so that only one may be used at a time , it is not expected that an application can simultaneously send and receive .this limitation could be removed by a suitably designed channel object which could specify a different return address .as pointed out above , a message passed between a sender and receiver would negotiate four different channels :  1 .request 2 .indication 3 .response 4 .confirmation  but if the call is a cast of some kind , it need only have two :  1 .request 2 .indication  and some parts may not do anything , for example a request handler could log all calls made by the client , but the server need not record all the calls that are made upon it .  when an application programmer makes use of a remote procedure call the infrastructure sends the message to the server and blocks the thread pending the arrival of the confirmation .one could also implement the channel objects in this way .this leads to two different architectures :  * either : one channel object for each of request , indication , response and confirmation _ simplex_. * or : two channel objects : one that requests and blocks pending the arrival of a confirmation ; one that receives indications , invokes the service ( and blocks waiting for it ) and then sends the response _duplex_.  the latter will be discussed first .one channel object for the initiator performs request and confirmation , another channel object performs indication and response for the acceptor .this means that every channel object has a bi directional data flow with the channel object below it , rather like the application object has with the distinct channels in figure [ fig : odp-4channels ] .this has the attraction that should the initiator s channel objects for the request and the confirmation need to share state , then this is achieved implicitly because they are the same object .this has a problem with broadcasts , ( or one casts ) .the initiator s channel object ( request and confirmation ) needs to determine if it is to block or not and that would require this information be made available to the channel object , perhaps best supplied as a parameter to the method invocation .this information is required in _ corba _ , the interface definition language allows methods on interfaces to be marked as , but is not part of the _ java_specification ; although it could be inferred if the definition of the remote method returns no result , in which case , the method can be invoked as a one cast . however ,if exceptions are returned by the remote server then the method is a call : the exception , whether raised or not , is a response .   if a channel object is implemented for each stage of the communication , then there are , at most , two pairs of objects : a request and confirmation pair at the initiator ; an indication and response pair at the acceptor . figure [ fig : odp-4channels ]illustrates the engineering of this .  because the objects are distinct they will need to bind with one anothershould they need to share state : an example of the difficulties this might lead to can be seen when communicating and attempting to correct errors .the infrastructure would block the application programmer s thread of control pending the arrival of the confirmation message from the server containing the results of the call .ordinarily , the confirmation channel objects will unravel the message returned by the server , instantiate the result for the application programmer s thread and unblock it .if there is an error in any of the channel objects , then it should be possible for the channel objects to attempt to clear the error and resend the message if need be .the problem then is that the channel objects in the request channel need to be synchronized with the error results received in the confirmation channel .if there is a failure in one of the channel objects in the indication channel , then they must be propagated via the response and confirmation channels .this is a difficult issue and is discussed later at greater length , [ sec : exceptions ] .the important difference between the duplex and simplex methods is the relationship to the state of the call . with the duplex model the request channel retains the state of the call awaiting an acknowledgement from the confirmation channel .what makes the duplex model retain state is that the channel objects invoke one another and form a stack of calls which can be associated with a thread .this can be emulated in the engineering of a simplex model without requiring the use of the stack , by passing a call identifier .this would only be of use if the channel objects were also to retain their internal state when they release control .[ [ example - a - secured - message - delivery - service ] ] example : a secured message delivery service + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +  as an example , an encryption and decryption service would require :  1 .request encrypt 2 .indication decrypt 3 .response encrypt 4 .confirmation decrypt  there are only two functions encrypt and decrypt but performed at four locations .it should be possible to provide just one pair of implementations an and a located differently .  1 .* request channel object initiator * response channel object acceptor 2 .  * indication channel object acceptor * confirmation channel object initiator  and would both be implemented as stream handler channel objects .encryption is subject to replays of old messages unless the messages are time stamped and sequenced . usually , time stamping and sequencingare implemented as part of the encryption and decryption objects , but with channel objects they can be provided separately .stamping has two functions :  1 .request timestamp issue 2 .indication timestamp check 3 .response timestamp issue 4 .confirmation timestamp check  two functions : , ; four locations :  1 .* request channel object initiator * response channel object acceptor 2 .  * indication channel object acceptor * confirmation channel object initiator  similarly for a sequence number generator and checker .  andwould both be implemented as call handler channel objects .  because it is difficult to synchronize clocks in a distributed networks , some secure message delivery systems allow some skew on the clocks and use checksums to detect replayed messages . only the acceptor s indication channel and the initiator s confirmationneed deploy a object .it would be implemented as a call handler generating a checksum from the marshalled data incoming as an indication or as a confirmation .there is a greater similarity in the simplex architecture to the engineering underlying the messaging system than in the duplex architecture , but the duplex architecture has some attractive state retention properties which should be emulated in a simplex architectureq .the rest of this discussion will concern itself with a simplex architecture that attempts to retain state across all four phases of a call .the other great attraction of the simplex architecture is that if the channel objects reside in different channels , then it is easier to relocate the channel .this would be especially useful when a call uses two different media as the example system in figure [ fig : odp-2sources ] illustrated .from what has been said above , the _ java_rpc mechanism has a stream oriented architecture and what follows is a proposed call oriented architecture for it .it should serve as an example of how channel objects could be deployed in other _corba _ rpc systems .the distributed processing infrastructure will have to determine the order in which the channel objects will be invoked .there are now two ways in which the methods of the channel objects can be invoked and how they should bind with one another .* either : have the request object invoke a method on the indication object and block waiting for the response : _ peer to peer _ invocation . * or : have the request object perform its work and return a modified call object and return immediately : _ service _ invocation .  the peer to peer option requires the duplex architecture which has been dismissed , so only the service method of invocation is left . peer to peer is very attractive , it would allow channel objects to be client server pairs and thus be able to use the stub compiler .unfortunately , it would demand too much memory to stack all of the calls required to navigate complicated protocol stacks .the approach taken by _ sun _ in their own implementation of , shipped as as part of the _ java_runtime environment , is suitable for the simplex architecture proposed . referring to the the code fragment given in the appendix [ sec : stub ] , the key method is .it is implemented along the lines given in the next code fragment : note the two comments indicating when the two types of channel objects should be invoked ..... package sun.rmi.server ;  public class unicastref implements remoteref {   public object invoke(remote r , reflect.method m ,             object [ ] p , long l ) throws exception   {    // * request channel objects should be invoked now *     // establish a connection using information in remote r     // get the streams associated with the connection    // marshall data onto the stream    // execute the call        //* confirmation channel objects should be invoked now *     //unmarshall    //release the connection    //return the result   } } ....   the channel objects would be invoked serially and would be passed the same parameters as , collectively call these a object .the request channel objects would return a object , but these would usually have the original message as one of its parameters .this is a simple encapsulation procedure and is illustrated in figure [ fig : odp - channel - message-1 ] , where a time stamping channel object has been passed the application object s message .the application object wants to invoke method , the channel object passes this message as a parameter of its own message .that message invokes method .the two message objects might duplicate target and return addresses , it should be possible to remove this redundancy , but it should still be possible to specify a different target address and a different return address if need be .the objects located in the indication and confirmation channels would need a skeleton rather like that described above , [ sec : inflexible ] , either a dispatcher or use a reflective invocation on the channel object s service implementation .after the service has completed processing , it would return its results to the skeleton which would then return the message object to the infrastructure .some channel objects will have counterparts in the remote channel , for example , a time stamping request channel object should have a time stamp checking indication channel object , but some may not , for example a request logging channel object placed in a server s indication channel .if a channel object sends then its counterpart receives and _ vice versa_.  a channel object can have an associate in a local channel .a request channel object could have an associate in the confirmation channel .if a channel objects sends then its associate receives and _ vice versa_.  figure [ fig : odp - counterparts ] should help to clarify this .  1 .marshalling and unmarshalling + looking at the marshalling and unmarshalling objects : the marshalling object in the client s request channel has a counterpart in the server s indication channel and an associate in the client s confirmation channel .2 .  timing + the top layer of channel objects are used for timing : time stamping and time checking , they comprise a full complement , where each channel object has an associate and a complement .usage + the second layer of channel objects is used to record usage statistics and only logs requests and responses , so they have no associates but a counterpart .clearly , this might prove to be cumbersome , it might be simpler to insist that all channel objects have a counterpart and an associate and have a default implementation which just copies the message over .any of the channel objects can raise an exception , but part of the function of channel objects is to attempt to clear exceptions , for example :  * relocation objects would obtain new addresses for servers , * key management objects would obtain new keys in the event of expiry . * authorization managers could obtain new privileges .the channel objects have to retain some state that would allow them to act upon exceptions .this would require that they place state in a common object that associated channel objects can access .the state would need to be stored with an identifier unique to the call .1 .  exception handling in one channel 1 .clear + when one channel object raises an exception , the infrastructure signals the other channel objects in the same channel , in the reverse order in which they were invoked , asking them to attempt to clear the exception .2 .  uncleared : undo + if the exception can not be cleared then the channel objects should be signalled to undo their previous actions .cleared : undo and redo + if the exception can be cleared then the channel objects may need to undo their previous actions and be allowed to redo them .+ redo is a distinct operation , because it may allow the implementation to be optimized for error recovery .+ co ordinating this is a little difficult .there are two sequences : 1 .attempt to clear then undo and then redo + each object in turn in ascending order ( _ i.e._reverse order to invocation ) attempts to clear the exception , if any one succeeds then the undo operation is invoked in ascending order to the top of the channel . then the redo operation is invoked in descending order .2 .  attempt to clear and undo and then redo + each object in turn in ascending order ( _ i.e._reverse order to invocation ) attempts to clear the exception and performs an undo .if any one succeeds then the undo is invoked on the remaining objects in ascending order and then the redo operation is invoked in descending order .+ the former might prove more efficient if errors are expected to be cleared , the latter if not .the idea is communicated in figure [ fig : odp - exceptions ] , but the details of invocation are not .+  + if the latter scheme ( attempt to clear and undo simultaneously ) is used : channel object raises an exception , it undoes its action , passes back the original message it received to object which also undoes its action and returns the original message it received to . clears the exception and redoes its action and passes the message on to which also redoes its action and thence to .exception handling across channels + if a receiving counterpart raises an exception , it should be signalled to the sender : this would be achieved by the receiving counterpart sending a message to its sending associate to raise an exception with its receiving counterpart ._ java_already has a proven mechanism for this , objects of class can be contained in objects of class .1 .  exception raised in the indication channel + as an example , see figure [ fig : odp - exceptions-1 ] , a request channel object sends a message which raises an exception in the indication channel , the channel object in the indication channel raises an alert with its associate in the response channel , which sends an exception to its counterpart in the confirmation channel .+ points to note in figure [ fig : odp - exceptions-1 ] are : 1 .the indication channel is cleared down 2 .the server receives no message 3 .the response channel propagates the exception 4 .the objects in the confirmation channel can signal their associates in the request channel .+ it might be possible for the request channel objects to invoke the same procedure as portrayed in figure [ fig : odp - exceptions ] , clear the exception and re send the message , this would require that they have access to the message as issued by the clientexception raised in the confirmation channel + if a response channel object sends a message which raises an exception in the confirmation channel , then the confirmation channel object would signal its associate in the request channel , which may , if it has retained state , _i.e._the message it sent , be able to clear the exception and re send without intervention by the client application object .a simple message class is needed to contain the parameters passed to the method and to all the channel objects .  .... package java.lang.rmi.channel ;  import java.rmi . * ; import java.lang.reflect . * ;  public class message {  remote remote ;  method method ;  object [ ] parms ;  long hash ;   public message(remote r , method m , object [ ] p , long h ) {   remote = r ;   method = m ;   parms = p ;   hash = h ;  } } ....   [ [ basic - interface ] ] basic interface + + + + + + + + + + + + + + +  channel objects then all have the same interface and it is their location which determines their function , _ i.e._whether they wrap or unwrap ..... package java.lang.rmi.channel ;  public interface handler {  public message clear(message m , exception e ) throws exception ;  public message todo(message m ) throws exception ;  public message undo(message m , exception e ) throws clearedexception ;  public message redo(message m ) throws exception ; } ....  both methods of clearing exceptions are covered in this interface because there is a separate method .[ [ exceptions ] ] exceptions + + + + + + + + + +  the method throws an exception to indicate it has cleared the exception it was passed ..... package java.lang.rmi.channel ;  public class clearedexception extends exception {    public clearedexception(string s ) {    super(s ) ;   }    public clearedexception(string s , exception ex ) {    super(s , ex ) ;   }  } ....  other exceptions which might make processing more decisive :  1 .unclearable + it might also prove useful to have raise an exception that the exception can not be cleared and this would allow the infrastructure to request a re send decision from the application object .2 .  rebind + it might also prove useful if the channel objects can demand a rebind and force the destruction of their channel .this would be useful for a relocation channel object .[ [ handler - specification ] ] handler specification + + + + + + + + + + + + + + + + + + + + +  the next is an abstract class that provides a container to hold the channel objects which would be loaded by the infrastructue .it might be wise , for security reasons , to implement the class more fully and make the method final .the class also provides a means of obtaining associates and counterparts ..... package java.lang.rmi.channel ;  import java.rmi .* ;  public abstract class handlers  {  public static final int request = 1 ;  public static final int indication = 2 ;  public static final int response = 3 ;  public static final int confirmation = 4 ;   public handlers ( ) {   ;  }   handler gethandler(int identity ) {   return null ;  }   remote getcounterpart(int identity ) {   return null ;  }   object getassociate(int identity ) {   return null ;  } } ....  [ [ channels ] ] channels + + + + + + + +  channels themselves would be simple containers probably implemented with a vector object which can be easily iterated in the operation .  .... package java.lang.rmi.channel ;  import java.rmi . * ;  public abstract class channel  {  private int identity = -1 ;  private handlers handlers ;   public channel(int i d , channels h ) {   identity = i d ;   handlers = h ;  } } ....   [ [ binding ] ] binding + + + + + + +  channel objects would need to bind with one another , so that they can exchange parameters .the channel object methods are defined on one interface , they can therefore suppport other methods on another service interface .[ [ invocation ] ] invocation + + + + + + + + + +  when a channel object is passed a message object , it will need to invoke a method on its counterpart . the easiest way for it to dothis is to use the same stub and skeleton mechanism for dispatching a call as remote procedure calls use .as can be seen in figure [ fig : odp - channel - message-1 ] , where one channel object adds enough parameters to the message so that when it is received by its counterpart , it can use a local method to pass the parameters and obtain the resulting message object from a local service interface .if channel objects have to invoke methods upon one another they can either send them with the message or they can go `` out of band '' and communicate them directly if they support the interface using their own channel ._ java_certainly has the functionality to implement channel objects .the author has already implemented a similar scheme ( using a duplex , peer to peer architecture ) for a variant of the _ ansaware _ , , known as _ dais _ , the distributed processing platform produced by _ icl _ , , which was _ corba _ compliant .that implementation , written wholly in _ , was able to support a _ kerberos _, , authentication and confidentiality service invisibly to the application programmer , the channels were constructed according to a template specified by an environment variable .the author has developed a prototype application which can support the transport level security protocol .( a fairly rigorous treatment of both it and the _ kerberos _ protocol is given in . ) a simple channel object is put in place that negotiates the keys , whilst a custom socket is used to perform the encryption ._ ansaware _ evolved to a product known as _ reflective java _ , , which supported channel objects , but was designed to provide support to application objects presentation of data and so forth and not to provide system services .the principal difficulty faced in these prototypes is that there is no simple or well defined mechanism for expressing how two parties should bind with one another ._ ansaware _ had a well developed binding model , , but its final implementation had a limited number of quality of service parameters .  _java_now has a very well developed security architecture , , but currently seems to have no means of specifying the degree of security _ required _ , it seems to be principally oriented towards setting permissions .( of course , a permission one does not have is a requirement . )the would appear to be the best placed component of the _ java_security architecture to negotiate and configure channels .it will be difficult for any distributed processing platform to achieve any degreee of acceptance in an enterprise wide data processing environment if it is not possible to implement many of the logging services that have long been available in mainframe systems .the most pertinent example of which is _ cics _ , the customer information control system , which was originally a messaging system , but was extended to become a transaction monitor that could manage database enquiries .hopefully , with channel objects in place , remote procedure calls could do the same across the world wide web .the remote method invocation package of _ java_needs to use a stub compiler to generate stubs .it no longer needs to generate skeletons for servers .there is , of course , no need to distribute the stubs , the client can collect them from the server .   [ [ interface ] ] interface + + + + + + + + +  this very simply returns a string , given a string ..... package rmi.demo ;  public interface answerer extends java.rmi.remote {  string answer(string mesg ) throws java.rmi.remoteexception ; } ....  [ [ implementation ] ] implementation + + + + + + + + + + + + + +  this is the implementation of the interface .most of the code involves posting the server s reference to the naming service ..... package rmi.demo ;  import java.rmi .* ; import java.rmi.server.unicastremoteobject ;  public class answererimpl  extends unicastremoteobject  implements answerer {  private string name ;   public answererimpl(string s ) throws remoteexception {   super ( ) ;   name = s ;  }   //begin  //remotely accessible methods  // parameters have to be implementations of serializable    public string answer(string message ) throws remoteexception {   system.out.println("received : " + message ) ;   return new string("you said : " + message ) ;  }   //end   public static void main(string args [ ] )  {   // create and install a security manager   system.setsecuritymanager(new rmisecuritymanager ( ) ) ;   // without it no new classes can be loaded    try {    system.out.println("construct " ) ;    answererimpl obj = new answererimpl("answererserver " ) ;    system.out.println("bind " ) ;    naming.rebind("answererserver " , obj ) ;    system.out.println("bound " ) ;   } catch ( exception e ) {    system.out.println("answererimpl err : " + e.getmessage ( ) ) ;    e.printstacktrace ( ) ;   }  } } ....   this is the version 1.2 stub for the service interface .  1 .reflective language features + the most important feature of these stubs is that they make use of the capabilities provided by . when the class is loaded by the client , and it is designed to be loaded from a remote location , the initialization code instantiates the attribute for the stub using the method of .2 .  instantiating objects that implement + because the client has to use the naming service to locate an object that implements the , the naming service loads the stub class and calls the constructor in it .the client casts the object the naming service returns to be .3 .  invoking methods+ the stub compiler generates proxy implementations of the methods that are declared in .they use the reflective method variable and the method .  .... //stub class generated by rmic , do not edit . //contents subject to change without notice .package rmi.demo ;  public final class answererimpl_stub  extends java.rmi.server.remotestub  implements rmi.demo.answerer , java.rmi.remote {  private static final long serialversionuid = 2 ;     private static java.lang.reflect.method method_answer_0 =     rmi.demo.answerer.class.getmethod("answer " ,                      new java.lang.class [ ]                      { java.lang.string.class } ) ;    } catch ( java.lang.nosuchmethodexception e ) {    throw new     java.lang.nosuchmethoderror("stub class initialization failed " ) ;   }  }     //constructors  public answererimpl_stub(java.rmi.server.remoteref ref ) {   super(ref ) ;  }     //methods from remote interfaces     // implementation of answer(string )  public java.lang.string answer(java.lang.string result =      ref.invoke(this , param_string_1 } ,            -8351992698817289230l ) ;     return ( ( java.lang.string ) $ result ) ;    } catch ( java.lang.runtimeexception e ) {     throw e ;    } catch ( java.rmi.remoteexception e ) {     throw e ;    } catch ( java.lang.exception e ) {     throw new      java.rmi.unexpectedexception("undeclared checked exception " ,                     e ) ;    }   } } ....research was funded by the engineering and physical sciences research council of the united kingdom .thanks to malcolm clarke , russell wynn jones and robert thurlby ._ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ walter eaves + department of electrical engineering , + brunel university + uxbridge , + middlesex ub8 3ph , + united kingdom _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _       tim dierks and christopher allen .the tls protocol : version 1.0 . in postel ,november 1997 .ftp://ietf.nri.reston.va.us / internet - drafts / draft - ietf - tls - protocol-05% .txt.z[ftp://ietf.nri.reston.va.us/ internet - drafts / draft - ietf - tls - protoc% ol-05.txt.z[ftp://ietf.nri.reston.va.us / internet - drafts / draft - ietf - tls - protoc% ol-05.txt.z ] ] , expires may 12 , 1998 .walter d eaves .ransport level security : a proof using the gong - needham - yahalom logic .technical report , e print archives , april 1999 , http://xxx.lanl.gov / abs / cs.cr/9904005[http://xxx.lanl.gov / abs / cs% .cr/9904005[http://xxx.lanl.gov / abs / cs% .cr/9904005 ] ] .alan o. freier , philip karlton , and paul c. kocher .the ssl protocol : version 3.0 . in postel , november 1995 .http://www.netscape.com / eng / ssl3/draft302.txt[http://www.netscap% e.com / eng / ssl3/draft302.txt[http://www.netscap% e.com/eng/ssl3/draft302.txt ] ] , expired .basic reference model : the basic model . in jday , editor , _ information technology - open systems interconnection _ , number 200 in series x recommendations x.200 to x.900 .international telecommunication union , international telecommunication union ( itu ) , place des nations , ch-1211 geneva 20 , switzerland , 1994 .http://info.itu.ch / itudoc / itu - t / rec / x / x200up.html[http://info.it% u.ch / itudoc / itu - t / rec / x / x200up.html[http://info.it% u.ch/itudoc/itu-t/rec/x/x200up.html ] ] .reference model : architecture ( 10 ) . in herbert , number 903 in series x recommendations x.900 to x.1000 , 1995 .http://info.itu.ch / itudoc / itu - t / rec / x / x500up.html[http://info.it% u.ch / itudoc / itu - t / rec / x / x500up.html[http://info.it% u.ch/itudoc/itu-t/rec/x/x500up.html ] ] .  .series x recommendations x.900 to x.1000 .international telecommunication union , international telecommunication union ( itu ) , place des nations , ch-1211 geneva 20 , switzerland , 1997 .http://info.itu.ch / itudoc / itu - t / rec / x / x500up.html[http://info.it% u.ch / itudoc / itu - t / rec / x / x500up.html[http://info.it% u.ch/itudoc/itu-t/rec/x/x500up.html ] ] .dave otway .the ansa binding model .technical report ar1392.01 , architectural projects management , poseidon park , cambridge , uk , january 1995 .ftp://ftp.ansa.co.uk / phase3-doc - root / apm.1392.01.ps.gz[ftp://ftp% .ansa.co.uk/ phase3-doc - root / apm.1392.01.ps.gz[ftp://ftp% .ansa.co.uk / phase3-doc - root / apm.1392.01.ps.gz ] ] .    creating a custom rmi socket factory .world wide web , 1998 .http://java.sun.com / products / jdk/1.2/docs / guide / rmi / rmisocketfactory.d% oc.html[http://java.sun.com / products / jdk/1.2/docs / guide / rmi / rmisocketfac% tory.doc.html[http://java.sun.com / products / jdk/1.2/docs / guide / rmi / rmisocketfac% tory.doc.html ] ] .java security .world wide web , 1998 .http://java.sun.com / products / jdk/1.2/docs / guide / security/[http:/% /java.sun.com / products / jdk/1.2/docs / guide / security/[http:/% /java.sun.com / products / jdk/1.2/docs / guide / security/ ] ] .zhixue wu and scarlet schwiderski .reflective java : making java even more flexible .technical report ar1936.02 , architectural projects management , poseidon park , cambridge , uk , january 1997 ./ phase3-doc - root / apm.1936.02.ps.gz[ftp://ftp% .ansa.co.uk/ phase3-doc - root / apm.1936.02.ps.gz[ftp://ftp% .ansa.co.uk / phase3-doc - root / apm.1936.02.ps.gz ] ] .

Abstract: this paper describes an architecture for a distributing processing system that would allow remote procedure calls to invoke other services as messages are passed between clients and servers . it proposes that an additional class of data processing objects be located in the software communications channel . the objects in this channel would then be used to enforce protocols on client server applications without any additional effort by the application programmers . for example , services such as key management , time stamping , sequencing and encryption can be implemented at different levels of the software communications stack to provide a complete authentication service . a distributing processing environment could be used to control broadband network data delivery . architectures and invocation semantics are discussed , example classes and interfaces for channel objects are given in the _ java_programming language .


in the era of big data , traditional statistical learning methods are facing two significant challenges : ( 1 ) how to scale current machine learning methods to the large - scale data ? and( 2 ) how to obtain accurate inference results when the data are noisy and may even contain malicious outliers ?these two important challenges naturally lead to a need for developing _ scalable robust _ learning methods .traditional robust learning methods generally rely on optimizing certain robust statistics  or applying some sample trimming strategies  , whose calculations require loading all the samples into the memory or going through the data multiple times  .thus , the computational time of those robust learning methods is usually at least linearly dependent on the size of the sample set , .for example , in rpca  , the computational time is where is the intrinsic dimension of the subspace and is the ambient dimension . in robust linear regression  ,the computational time is super - linear on the sample size : .this rapidly increasing computation time becomes a major obstacle for applying robust learning methods to big data in practice , where the sample size easily reaches the terabyte or even petabyte scale .online learning and distributed learning are natural solutions to the scalability issue .most of existing online statistical learning methods propose to optimize a surrogate function in an online fashion , such as employing stochastic gradient descent to update the estimates , which however can not handle the outlier samples in the streaming data .similarly , most of existing distributed learning approaches ( _ e.g. _ , mapreduce  ) are not robust to contamination from outliers , communication errors or computation node breakdown .  in this work, we propose an online robust learning ( orl ) framework to efficiently process big data with outliers while preserving robustness and statistical consistency of the estimates .the core technique is based on two - level online learning procedure , one of which employs a novel median filtering process .the robustness of median has been investigated in statistical estimations for heavy - tailed distributions . however , to our best knowledge , this work is among the first to employ such estimator to deal with outlier samples in the context of online learning .the implementation of orl follows mini - batch based online optimization which is popular in a wide range of machine learning problems ( _ e.g. _ , deep learning , large - scale svm ) from large - scale data . within each mini - batch, orl computes an independent estimate .however , outliers may be heterogeneously distributed on the mini - batches and some of them may contain overwhelmingly many outliers . the corresponding estimate will be arbitrarily bad and break down the overall online learning .therefore , on top of such streaming estimates orl performs another level of robust estimationmedian filteringto obtain reliable estimate .the orl approach is general and compatible with many popular learning algorithms . besides its obvious advantage of enhancing the computation efficiency for handling big data, orl incurs negligible robustness loss compared to centralized ( and computationally unaffordable ) robust learning methods .in fact , we provide analysis and demonstrate that orl is robust to a constant fraction of `` bad '' estimates generated in the streaming mini - batches that are corrupted by outliers .we specify the orl approach for two concrete problemsonline robust principal component analysis ( pca ) and linear regression .comprehensive experiments on both synthetic and real large scale datasets demonstrate the efficiency and robustness advantages of the proposed orl approach .in addition , orl can be adapted straightforwardly to distributed learning setting and offers additional robustness to corruption of several computation nodes or communication errors , as demonstrated in the experiments .in short , we make following contributions in this work .first , we develop an outlier robust online learning framework which is the first one with provable robustness to a constant fraction of outliers .secondly , we introduce two concrete online robust learning approaches , one for unsupervised learning and the other for supervised learning .other examples can be developed in a similar way easily .finally , we also present the application of the orl approach to distributed learning setting which is equally attractive for learning from large scale data .we consider a set of observation samples , which contains a mixture of authentic samples and outliers . the authentic samples are generated according to an underlying model ( _ i.e. _ , the ground truth ) parameterized by .the target of a statistical learning procedure is to estimate the model parameter according to the provided observations . throughout the paper , we assume the authentic samples are sub - gaussian random vectors in , which thus satisfy that for some . here denotes the unit sphere . in this work ,we focus on the case where a constant fraction of the observations are outliers , and we use to denote this outlier fraction . in the context of online learning , samples are provided in a sequence of mini batches , each of which contains observations .denote the sequence as .the target of online statistical learning is to estimate the parameter , only based on the observations revealed so far .we first introduce the _ geometric median _ herea core concept underlying the median filtering procedure that is important for developing the proposed online robust learning approach .[ def : median ] given a finite collection of i.i.d .estimates , their geometric median is the point which minimizes the total distance to all the given estimates , _i.e. _ ,   an important property of the geometric median is that it indeed aggregates a collection of independent estimates into a single estimate with strong concentration guarantees , even in presence of a constant fraction of outlying estimates in the collection .the following lemma , straightforwardly derived from lemma 2.1 in  , characterizes such robustness property of the geometric median .[ lemma : median ] let be the geometric median of the points .fix and .suppose there exists a subset of cardinality such that for all and any point , .then we have .  in words , given a set of points, their geometric median will be close to the `` true '' as long as at least half of them are close to .in particular , the geometric median will not be skewed severely even if some of the points deviate significantly away from .in this section , we present how to scale up robust learning algorithms to process large - scale data ( containing outliers ) through online learning without losing robustness .we term the proposed approach as online robust learning ( orl ) .the idea behind orl is intuitiveinstead of equally incorporating generated estimates at each time step , orl aggregates the sequentially generated estimates by mini - batch based learning methods via an _ online _ computation of the robust geometric median .basically , orl runs online learning at two levels : at the bottom level , orl employs appropriate robust learning procedures with parameter ( _ e.g. _ , robust pca algorithms on a mini - batch of samples ) to obtain a sequence of estimates of based on the observation mini - batch ; at the top level , orl updates the running estimate ( ) through a geometric median filtering algorithm ( explained later ) over and outputs a _ robust _ estimate after going through all the mini - batches .intuitively , according to lemma [ lemma : median ] , as long as a majority of mini - batch estimates are not skewed by outliers , the produced would be robust and accurate .this new two - level robust learning gives orl stronger robustness to outliers compared with ordinary online learning .  to develop the top level geometric median filtering procedure , recall definition of the geometric median in .a natural estimate of the geometric median is the minimum of the following empirical loss function : the empirical function is differentiable everywhere except for the points , and can be optimized by applying stochastic gradient descent ( sgd ) . more concretely , at the time step ,given a new estimate ( based on the -st mini - batch ) and the current estimate , orl computes the gradient at point of the empirical function in eqn .evaluated only at : then orl updates the estimate by following filtering : here is a predefined step size parameter which usually takes the form of with a constant characterizing convexity of the empirical function to optimize . besides , controls contribution of each new estimate conservatively in updating the global estimate .details of orl are provided in algorithm [ alg : rol ] .* input * : mini - batch sequence , convexity parameter , robust learning procedure parameter . * initialization * : . call the robust learning procedure ; compute weight with . update the estimate : .* output * : final estimate .another level of filtering is important .certain mini - batches may contain overwhelming outliers .therefore , even though a robust learning procedure is employed on each mini - batch , the resulted estimate can not be guaranteed to be accurate .in fact , a mini - batch containing over outliers would corrupt any robust learning procedurethe resulted estimate can be arbitrarily bad and breakdown the overall online learning . to address this critical issue , orl performs another level of online learning for updating the `` global '' estimate with adaptive weights for the new estimate and `` filters out '' possibly bad estimates .we provide the performance guarantees for orl in this section . throughout this section ,we use following asymptotic inequality notations : for positive numbers and , the asymptotic inequality means that where is a constant only depending on .suppose samples , a constant fraction of which are authentic ones and have sub - gaussian distributions as specified in for some , are evenly divided to mini - batches and outlier fractions on the mini - batches are respectively .let be a collection of independent estimates of output by implementing the robust learning procedure on the mini - batches independently .we assume an estimate or the robust learning procedure provides following composite deviation bound , where is the size of each mini - batch whose value can be tuned by the desired accuracy ( _ e.g. _ , through data augmentation ) .we will specify value of the constant depending on and explicitly in concrete applications .the above bound indicates the estimation error depends on the standard statistically error and the outlier fraction .if is overwhelmingly large , the estimate will be arbitrarily bad .we now proceed to demonstrate that the orl approach is robust to outlierseven on a constant fraction of mini - batches , the obtained estimates are not good , orl can still provide reliable estimate with bounded error . given a sequence of estimates produced internally in orl , we analyze and provide guarantee on performance of the orl through following two steps .we first demonstrate the geometric median function is in fact strongly convex and thus geometric median filtering provides a good estimate of the `` true '' geometric median of .then we derive following performance guarantee for orl by invoking the robustness property of geometric median .[ prop : rol ] suppose in total samples , a constant fraction of which have sub - gaussian distribution as in , are divided into sequential mini batches of size with outlier fraction . herewe run a base robust learning algorithm having a deviation bound as in on each mini batch .denote the ground truth of the parameter to estimate as and the output of orl ( alg .[ alg : rol ] ) as . then with probability at least , satisfies : here and denotes the smallest outlier fraction in with .the above results demonstrate the estimation error of orl consists of two components .the first term accounts for the deviation between the solution and the `` true '' geometric median of the sequential estimations .when is sufficiently large , _i.e. _ , after orl seeing sufficiently many mini batches of observations , this error vanishes at a rate of .the second term explains the deviation of geometric median of estimates from the ground truth .the significant part of this result is that the error of orl only depends on the smallest outlier fraction among mini - batches , no matter how severely the other estimates are corrupted .this explains why orl is _ robust _ to outliers in the samples .in this section , we provide two concrete examples of the orl approach , including one unsupervised learning algorithmprincipal component analysis ( pca ) and one supervised learning onelinear regression ( lr ) .both algorithms are popular in practice but their online learning versions with robustness guarantees are still absent .finally , we also discuss an extension of orl for distributed robust learning .classical pca is known to be fragile to outliers and many robust pca methods have been proposed so far ( see  and references therein ) .however , most of those methods require to load all the data into memory and have computational cost ( super-)linear in the sample size , which prevents them from being applicable for big data . in this section ,we first develop a new robust pca method which robustifies pca via a robust sample covariance matrix estimation , and then demonstrate how to implement it with the orl approach to enhance the efficiency .  given a sample matrix \in \mathbb{r}^{p \times n} ] , subspace dimension , outlier fraction .compute the trimmed covariance matrix : .perform eigen decomposition on and take the eigenvectors corresponding to the largest eigenvalues  ] and response , outlier fraction ( set as if unknown ) . for ,compute . .  due to the blessing of online robust learning framework , orl - lr has the following performance guarantee .[ theo : rol_sr ] adopt the notations in theorem [ theo : rol_pca ] .suppose the authentic samples have the sub - gaussian distribution as in with noise level , are divided into sequential mini - batches .let be the output of orl - lr and be the ground truth .then , with probability at least , the following holds :    following the spirit of orl , we can also develop a distributed robust learning ( drl ) approach .suppose in a distributed computing platform , machines are usable for parallel computation .then for processing a large scale dataset , one can evenly distribute them onto the machines and run robust learning procedure in parallel .each machine provides an independent estimate for the parameter of interest . aggregating these estimates via geometric median ( ref .eqn . ) can provide additional robustness to the inaccuracy , breakdown and communication error for a fraction of machines in the computing cluster , as stated in lemma [ lemma : median ] . of particular interest, drl can provide much stronger robustness than the commonly used averaging over the estimates , as average or mean is notoriously fragile to corruption .even a single corrupted estimate out of the estimates can make the final estimate arbitrarily bad .in this section , we investigate robustness of the orl approach by evaluating the orl - pca and orl - lr algorithms and comparing them with their centralized and non - robust counterparts .we also perform similar investigation on drl ( ref .section [ sec : drl ] ) considering robustness is also critical for distributed learning in practice . in the simulations , we report the results with the outlier fraction which is computed as .  * data generation * in simulations of the pca problems , samples are generated according to . herethe signal is sampled from the normal distribution : .the noise is sampled as : .the underlying matrix is randomly generated whose columns are then orthogonalized .the entries of outliers are i.i.d .random variables from uniform distribution  ] .the response of outlier is generated by .we use to measure the error . here is the output estimate .  *online setting * results shown in figure [ subfig : pca_non_uniform ] give following observations .first , orl - pca converges to comparable performance with batch rc - pca with accesses to the entire data set .this demonstrates the rapid convergence of orl - pca .it is worth noting that orl - pca saves considerable memory cost than batch rc - pca ( mb _ vs. _ mb ) and computation time ( seconds vs.  hours ) since orl - pca performs svd on much smaller data .secondly , orl - pca offers much stronger robustness than naively averaged aggregation when outlier order is adversarial to corrupt a fraction of mini - batches .as shown in figure [ subfig : pca_non_uniform ] , when some batches have overwhelming outliers ( outlier fraction ) , base rc - pca fails on these batches and outputs completely corrupted estimations .the corruption of mini - batches also fails online averaging rpca .in contrast , orl - pca still offers correct estimation , even when a fraction of of estimates from mini batches are corrupted .we also report the results of orl - lr and comparison with online - averaging baselines in figure [ subfig : online_lr ] .similar to orl - pca , one observes that orl - lr offers outperforming robustness to the sample outliers and batch corruptions , in contrast to the naive averaging algorithm .  *distributed setting * all the simulations are implemented on a pc with quad cpu and gb ram .it takes centralized rpca around seconds to handle samples with dimensionality of .in contrast , distributed rpca only costs seconds by using parallel procedures .the communication cost here is negligible since only eigenvector matrices of small sizes are communicated . for rlr simulations, we also observe about efficiency enhancement .  as for the performance , from fig .[ fig : drpca ] , we observe that when , drl - rpca , rpca with division - averaging ( div .- avg .rpca ) and centralized rpca ( _ i.e. _ , the rc - pca ) achieve similar performances , which are much better than non - robust standard pca .when , _i.e. _ , when there are no outliers , the performances of drl - rpca and div .- avg .rpca are slightly worse than standard pca as the quality of each mini - batch estimate deteriorates due to the smaller sample size .however , distributed algorithms of course offer significant higher efficiency .similar observations also hold for lr simulations from fig . [fig : drlr ] . actually , standard pca and lr begin to break down when .these results demonstrate that drl preserves the robustness of centralized algorithms well .when outlier fraction increases to , centralized ( blue lines ) and division - averaging algorithms ( green lines ) break down sharply , as the outliers outnumber their maximal breakdown point of .in contrast , drl - rpca and drl - rlr still present strong robustness and perform much better , which demonstrate that the drl framework is indeed robust to computing nodes breaking down , and even enhances the robustness of the base robust learning methods under favorable outlier distributions across the machines .* comparison with averaging * taking the average instead of the geometric median is a natural alternative to drl .here we provide more simulations for the rpca problem to compare these two different aggregation strategies in the presence of different errors on the computing nodes .  in distributed computation of learning problems , besides outliers ,significant deterioration of the performance may result from unreliabilities , such as latency of some machines or communication errors .for instance , it is not uncommon that machines solve their own sub - problem at different speed , and sometimes users may require to stop the learning before all the machines output the final results . in this case ,results from the slow machines are possibly not accurate enough and may hurt the quality of the aggregated solution .similarly , communication errors may also damage the overall performance .we simulate the machine latency by stopping the algorithms once over half of the machines finish their computation . to simulate communication error , we randomly sample estimations and flip the sign of of the elements in these estimations .the estimation errors of the solution aggregated by averaging and drl are given in table [ tab : avg - compare ] .clearly , drl offers stronger resilience to unreliability of the computing nodes ..comparisons of the estimation error for pca between division - averaging ( div .- avg . ) and drl , with machine latency and communication errors . under the same parameter setting as figure [ fig : drpca ] .outlier fraction .the average and std of the error from repetitions are reported . [cols="^,^,^",options="header " , ]   we perform experiments with the online learning setting , and compare the performance of the proposed orl - lr with the online averaging lr .we also implement a non - robust baseline stochastic gradient descent to solve the lr problem .the size of min - batch is fixed as images . from the results in table [ tab : flickr - acc_orl ], one can observe that orl - lr achieves significantly higher accuracy than non - robust baseline algorithms , with a margin of more than .[ lemma : hoeffding ] let be independent random variables taking values in  ] such that , for all , and let be a random variable with a binomial law .there exists a coupling such that has the same distribution as , has the same distribution as and such that . in particular , for all , .the following lemma demonstrates that aggregating estimates via their geometric median can enhance the confidence significantly .[ lemma : median_aggregation ] given independent estimates of satisfying , for all .let , then we have , here and are defined in lemma [ lemma : median ] .  according to lemma [ lemma : median ] , we have let , and let have binomial distribution , then according to lemma [ lemma : coupling ] . applying the hoeffding s inequality in lemma [ lemma : hoeffding ] ( with , , and )gives     we suppose from now on following conditions hold .[ cond1 ] assume that is the parameter of interest .let be a collection ofindependent estimates of , which are not concentrated on a straight line : for all , there is such that and with .as noted in  , condition [ cond1 ] ensures that geometric median of the estimates is uniquely defined .[ cond2 ] the distribution of the independent estimates of is a mixing of two `` nice '' distributions : .here is not strongly concentrated around single points : if is the ball , and is a random variable with distribution , then for any constant , \leq c_a.\ ] ] in addition , is a discrete measure , . here is a dirac measure at point .we denote by the support of and assume that the median .conditions [ cond1 ] and [ cond2 ] are only technical conditions to avoid pathologies in the convergence analysis for algorithm [ alg : rol ] . in practical implementations , we can simply set the sub - gradient of at as zero ( a valid sub - gradient as proved in ) when .given the definition of geometric median in , we can define following population geometric median loss function , , that we want to minimize to compute the geometric median : .\ ] ]  in this subsection , we first show that the geometric median function in is indeed strongly convex under conditions [ cond1 ] and [ cond2 ] .thus the sgd optimization is able to provide solutions with a convergence rate of to the true geometric median , given independent estimates .[ def : strong - convex ] a function is -strongly convex , if for all and any sub - gradient of at , we have   the following theorem establishes the strong convexity of the geometric median function in .[ theo : convexity ] let be the sub - gradient of at . under conditions [ cond1 ] and [ cond2 ] , there is a strictly positive constant , such that : and thus is -strongly convex .the proof can be derived from the proof for the proposition 2.1 in  straightforwardly and we omit details here .given the strong convexity property of geometric median function , we can apply the convergence argument of sgd for strongly convex functions ( _ e.g. _ , proposition 1 in ) , and obtain the following convergence rate for online geometric median filtering .[ theo : converge_rate ] assume conditions [ cond1 ] and [ cond2 ] hold , and .then with probability .assume .let .pick in algorithm [ alg : rol ] and let denote the output at time step . furthermore , let be the geometric median of . then for any , with probability at least . here .the bound on the gradient is from the definition of the gradient in , condition [ cond2 ] and the assumption that all the estimates are bounded .   from now on ,we slightly abuse the notation and use to denote the geometric median of a collection of estimates .proposition [ prop : rol ] can be derived by following triangle inequality : where denotes the `` true '' geometric median of estimates .we now proceed to bound the above two terms separately . based on theorem [ theo : converge_rate ] , we have with a probability at least .the second term can be bounded as follows by applying lemma [ lemma : median_aggregation ] : where and denotes the smallest outlier fraction in with .combining these two bounds together gives :    before proving the performance guarantee for orl - pca and orl - lr , we provide robustness analysis for the base robust learning procedurethe rc - pca and rotr .[ theo : rpca ] suppose in total samples are provided with authentic samples and outliers .let .assume the authentic samples follow sub - guassian design with parameter .let , where denotes the largest eigenvalue of ground - truth sample covariance matrix .let be the output -dimensional subspace projector from rc - pca .then for a constant , we have with probability ,   according to the proof of theorem 4 in and deviation bound on the empirical covariance matrix estimation in , when the authentic samples are from sub - gaussian distribution with parameter , we have , for the covariance matrix constructed in algorithm [ alg : rpca ] , with a probability at least . here is a constant , is the number of authentic samples and is the number of outliers .let be the eigenvalue gap , where denotes the -th largest eigenvalue of . then , applying the davis - kahan perturbation theorem , we have , whenever , .thus , with a probability at least .   theorem [ theo : rol_pca ] can be derived directly from following triangle inequality : and we bound the above two terms separately .the first term can be bounded by theorem [ theo : converge_rate ] as , with a probability .the second term can be bounded as in theorem [ theo : rpca ] that with a probability , combining the above two bounds ( with union bound ) proves the theorem .   before proving theorem [ theo : rol_sr ] ,we first show the following performance guarantee for rotr algorithm from . the estimation error of the rotr is bounded as in lemma [ lemma : rotr ] .[ lemma : rotr ] suppose the samples are from sub - gaussian design with , with dimension and noise level , then with probability at least , the output of rotr satisfies the bound : here is a constant independent of .based on the results in the lemma [ lemma : rotr ] and lemma [ lemma : median_aggregation ] , it is straightforward to get : where with being the constant in lemma [ lemma : rotr ] , , and with being the smallest outlier fraction in .  as proving theorem [ theo : rol_pca ] , theorem [ theo : rol_sr ]can be derived based on the results in theorem [ theo : converge_rate ] .for simplicity , we omit the details here .we developed a generic online robust learning ( orl ) approach with provable robustness guarantee and we also demonstrate its application for distributed robust learning ( drl ) .the proposed approaches not only significantly enhance the time and memory efficiency of robust learning but also preserve the robustness of the centralized learning procedures . moreover , when the outliers are not uniformly distributed , the proposed approaches are still robust to adversarial outliers distributions .we provided two concrete examples , online and distributed robust principal component analysis and linear regression .

Abstract: we consider the problem of learning from noisy data in practical settings where the size of data is too large to store on a single machine . more challenging , the data coming from the wild may contain malicious outliers . to address the scalability and robustness issues , we present an _ online _ robust learning ( orl ) approach . orl is simple to implement and has provable robustness guaranteein stark contrast to existing online learning approaches that are generally fragile to outliers . we specialize the orl approach for two concrete cases : online robust principal component analysis and online linear regression . we demonstrate the efficiency and robustness advantages of orl through comprehensive simulations and predicting image tags on a large - scale data set . we also discuss extension of the orl to distributed learning and provide experimental evaluations .


so - called numerical relativity ( computational simulations in general relativity ) is a promising research field having implications for ongoing astrophysical observations such as gravitational wave astronomy .many simulations of binary compact objects have revealed that mathematically equivalent sets of evolution equations show different numerical stability in the free - evolution scheme .there are many approaches to re - formulate the einstein equations for obtaining a longterm stable and accurate numerical evolution ( e.g. see references in ) . in a series of our works, we have proposed the construction of a system that has its constraint surface as an attractor . by applying eigenvalue analysis of constraint propagation equations, we showed that there _ is _ a constraint - violating mode in the standard arnowitt - deser - misner ( adm ) evolution system when it is applied to a single non - rotating black - hole space - time .we also found that such a constraint - violating mode can be compensated for if we adjust the evolution equations with a particular modification using constraint terms like the one proposed by detweiler .our predictions are borne out in simple numerical experiments using the maxwell , ashtekar , and adm systems .there are also several numerical experiments to confirm our predictions are effective .however we have not yet obtained definite guidelines for specifying the above adjusting terms and their multipliers .  in this article, we show the essential steps in analyzing constraint amplification factors ( defined in [ seccafdef ] ) . in [ sec3 ] , we show that degeneracy of eigenvalues sometimes leads constraint evolution to diverge .this observation suggests the importance of checking the diagonalizability of characteristic matrices , and gives further insights for constructing an asymptotically constrained system .we begin by reviewing our proposal for an adjusted system " .suppose we have a dynamical system of variables , which has evolution equations , and the ( first class ) constraints , note that we do not require ( [ ueq ] ) to form a first - order hyperbolic form .we propose to investigate the evolution equation of ( constraint propagation ) , for evaluating violation features of constraints .the character of constraint propagation , ( [ ceq ] ) , will vary when we modify the original evolution equations .suppose we modify ( adjust ) ( [ ueq ] ) using constraints then ( [ ceq ] ) will also be modified as therefore , finding a proper adjustment is a quite important problem .hyperbolicity analysis may be a way to evaluate constraint propagation , ( [ ceq ] ) and ( [ ceqadj ] ) . however , this requires ( [ ceq ] ) to be a first - order system which is easy to be broken .( see e.g. detweiler - type adjustment in the adm formulation ) .furthermore hyperbolicity analysis only concerns the principal part of the equation , that may fail to analyze the detail evaluation of evolution .alternatively , we have proceeded an eigenvalue analysis of the whole rhs in ( [ ceq ] ) and ( [ ceqadj ] ) after a suitable homogenization , which may compensate for the above difficulties of hyperbolicity analysis .we propose to transform the constraint propagation equation , ( [ ceq ] ) and ( [ ceqadj ] ) , into fourier modes , then to analyze the eigenvalues , say , of the coefficient matrix , .we call and the constraint amplification factors ( cafs ) and constraint propagation matrix ( cp matrix ) , respectively .so far we have proposed the following heuristic conjectures :  * if the caf has a _ negative real - part _ ( the constraints are necessarily diminished ) , then we see more stable evolution than a system which has a positive caf . *if the caf has a _ non - zero imaginary - part _ ( the constraints are propagating away ) , then we see more stable evolution than a system which has a zero caf .we observe that this eigenvalue analysis requires the fixing of a particular background space - time , since the cafs depend on the dynamical variables , .the cafs indicate the evolution of constraint violations ( definitely its fourier modes ) .it is natural to assume that a divergence of constraint norm is related to the numerical blow - ups .therefore we classify the fundamental evolution property of constraint propagation equation ( [ ceqf ] ) as follows :  * _ asymptotically constrained : _ violation of constraints decays ( converges to zero ) . * _ asymptotically bounded : _ violation of constraints is bounded at a certain value . * _ diverge : _ at least one constraint will diverge .  note that ( c1 ) ( c2 ) .we will derive the necessary and sufficient conditions for ( c1 ) and ( c2 ) in the next section .hereafter , we consider a set of evolution equations , where is a complex - valued vector , is a complex - valued matrix , and is assumed to have finite - valued initial data .  without loss of generality , the cp matrix can be assumed to be a jordan normal form , since within complex - valued operations all the matrices can be converted to this form .suppose that has different eigenvalues , where .let the multiplicity of as , where . can be expressed as where the cell size of is .the jordan matrix is then expressed using a jordan block , note that is , , and .the minimum polynomial of is written as if is diagonal ( i.e. ) , then for that . if is diagonalizable ( i.e. for ) , then for all .we then have the following statement .the solution of can be expressed formally as [ propa ]  a proof is available by mathematical induction .suppose that is which is the maximal size . by direct calculation, we have that yields ( [ kino ] ) with -polynomial of degree . then we see that ( [ kino ] ) is satisfied in general .  from this proposition , the highest power in in ( [ kino ] )is bounded by .the matrix in ( [ cp2 ] ) can be directly extended to the full cp matrix , , in ( [ cp ] ) .therefore the highest power in all constraints is bounded by    the following propositions [ prop1 ] and [ prop2 ] give us the next theorem .asymptotically constrained evolution ( violation of constraints converges to zero ) is obtained if and only if all the real parts of the cafs are negative .[ theoremc1 ]  all the real part of cafs are negative asymptotically constrained evolution .[ prop1 ]  proof ) we use the expression ( [ kino ] ) .if for , then will converge to zero at no matter what the terms are .  asymptotically constrained evolution all the real parts of the cafs are negative .[ prop2 ]  proof ) we show the contrapositive .suppose there exists an eigenvalue of which the real - part is non - negative .then we get of which the solution is . does not converge to zero .   the following propositions [ prop3 ] and [ prop4 ] give us the next theorem .   asymptotically bounded evolution ( all the constraints are bounded at a certain value ) is obtained if and only if all the real parts of cafs are not positive and is diagonal when .[ theoremc2 ]  * corollary * _ asymptotically bounded evolution is obtained if the real parts of cafs are not positive and the cp matrix is diagonalizable ._  all the real parts of cafs are not positive and is diagonal when  asymptotically bounded evolution .[ prop3 ]  proof ) we use the expression ( [ kino ] ) .when , will converge to zero no matter what the terms are .when , we see from the assumption of diagonality of .so we see the terms are constant and is bounded .   asymptotically bounded evolution all the real parts of the cafs are not positive and is diagonal when .[ prop4 ]  proof ) we show the contrapositive .if there exists an eigenvalue of which the real - part is positive , then constraints will diverge no matter what the terms are. therefore we try to show that constraints will diverge when all the real - parts of eigenvalues are non - positive , and there exists such that and its jordan matrix is not diagonal .since jordan matrix is not diagonal , we see the power of  is greater than 1 in the expression ( [ kino ] ) .thus we have that ( [ kino ] ) will diverge in . theorems will give us a guideline to analyze a constraint - violating mode of the system .the result supports our previous heuristic conjecture ( a ) , but also suggests an ill - behaving case when cafs are degenerated and its real - part is zero , when the associated jordan matrix is not diagonal .this indicates the importance of checking the diagonalizability of constraint propagation matrix .  along the line of our evaluation of constraint propagation equations ( [ cp ] ), we propose a practical procedure for this classification in figure [ fig : flowchart ] .we think that this diagram will provide systematic predictions for obtaining a robust evolution system in any constrained dynamics .( 85,75 ) ( -5,00 )  the present classification is only on the fixed background spacetime and only for .it is still not clear at what value the constraints are bounded if a limiting value exists .thus further modifications are underway .we are also applying the present classification scheme to various adjusted systems of the einstein equations ( adjusted adm , and further modified versions ) , together with numerical experiments .we hope to report on them in the near future .the current constraint analysis only concentrates to the evolution equations and does not include the effect of the boundary treatments . since the eigenvalues are evaluated locally , it will be possible to include the effect of numerical boundary conditions if they are expressed apparently in a part of the evolution equations .this is also the one direction to proceed our future research .meanwhile , we would like to remark that one of our proposed adjustments in contributes to enforce the computational ability of the black - hole excision boundary treatment .  by extending the notion of norm " or compactness " of constraint violations, it might be interesting to define a new measure which monitors a distance " between the constraint surface and an evolution sector in constraint dynamics .hs thanks the caltech visitors program for the numerical simulation of gravitational wave sources for their hospitality , where a portion of this work was completed .hs is supported by the special postdoctoral researchers program at riken .this work was supported partially by the grant - in - aid for scientific research fund of japan society of the promotion of science , no .14740179 .h. shinkai and g. yoneda , reformulating the einstein equations for stable numerical simulations " , in _ progress in astronomy and astrophysics _ ( nova science publ ) to be published .the manuscript is available upon request .

Abstract: in order to obtain stable and accurate general relativistic simulations , re - formulations of the einstein equations are necessary . in a series of our works , we have proposed using eigenvalue analysis of constraint propagation equations for evaluating violation behavior of constraints . in this article , we classify asymptotical behaviors of constraint - violation into three types ( asymptotically constrained , asymptotically bounded , and diverge ) , and give their necessary and sufficient conditions . we find that degeneracy of eigenvalues sometimes leads constraint evolution to diverge ( even if its real - part is not positive ) , and conclude that it is quite useful to check the diagonalizability of constraint propagation matrices . the discussion is general and can be applied to any numerical treatments of constrained dynamics .   = 11