fig1 illustrates the global clock system for a 4 - node cluster . as shown in the figure , the central oscillator generates pulses that increment synchronously the local counters in the processing nodes . the figure shows the reset signal , which can be generated by any of the processors , that is used to initialize all the counters simultaneously . after the reset is issued , the counters of all nodes will be incremented simultaneously by the central oscillator . the global clock can support an increasing number of processors by using a hierarchical structure of pulse generator modules as shown in fig2 . as can be seen in the figure , each module contains a fixed number of connections to the processors that compose the cluster and an extra connection to the other modules , forming a tree - like structure . the modules in the intermediate levels do not use their oscillators , but receive pulses from the module above in the hierarchy . in addition , intermediate modules propagate the reset signals they receive from the modules above in the hierarchy to the modules below , and vice - versa . each module uses a circuit for pulse detection associated with an electronic key to self identify as either an intermediate or the top module . an electro - mechanic key can also be used for this purpose . in fig2 , each intermediate pulse generator module uses an or gate to join all the incoming reset signals received from the points below in the hierarchy . the or gate &# 39 ; s output is transmitted up in the hierarchy to allow the reset of the global clock . an incoming reset signal that comes from either other pulse generator module or processing node in another point in the hierarchy goes up to until reaching the pulse generator module at the top of the hierarchy . the module at the top of the hierarchy uses an or logical gate to join the reset signals that arrive to it . the or gate has as many inputs as the number of incoming reset signals that arrive to it ( or it uses an and gate , if the reset signals were active on zero ). the output of this logic gate is the reset signal that goes down in the hierarchy of pulse generator modules until arriving simultaneously into all the processing nodes in the cluster . in each pulse generator module , the reset signal that goes down in the hierarchy is amplified by using buffers to ensure scalability . like the hardware of the local time counters , the hardware of the modules can also be connected to the i / o busses of the cluster &# 39 ; s processors , taking advantage of the available power supply . the cables that interconnect the pulse generator modules that form the hierarchical tree , and these to the counter modules of the processing nodes , can vary in size provided such variation is kept within certain limits that guarantee that the time difference to propagate the reset signal between any pair of counters does not exceed the cycle time of the central oscillator . thus , given a certain global clock frequency equal to f ( in hertz ), the difference d ( in meters ) between cable sizes from the top module to each counter cannot exceed the following value : where v ( in meters / second ) is the speed of propagation of electromagnetic signals through the cables . fig3 shows the diagram of our prototype of the pulse generator module of the global clock . the diagram is shown only for illustration purpose since other implementations following the specifications of the present invention are also possible . in the prototype , we use a pulse detector , identified as clock_detector in fig3 , to verify automatically whether or not the module is at the top of the hierarchy . if it is the top module then the ext_clock input of the module &# 39 ; s clock_detector will not receive pulses , since that input is used to receive pulses from modules higher in the hierarchy . the clock_detector circuit of the top module detects the absence of pulses in ext_clock , links its int_clock input , which receives pulses from the module &# 39 ; s local oscillator , to the sel_clock output , which is linked to the obuf buffer whose output sends the pulses to lower levels in the hierarchy . the clock_detector circuit generates a true logic level in its toplevel output also . if it is an intermediate module , there are pulses in the ext_clock input of the clock_detector . in this case , clock_detector detects the existence of pulses in the ext_clock input , links its ext_clock input to sel_clock output , and generates a false logic level in its toplevel output . reset signals that come from lower levels in the hierarchy are referred to as n 0 _reset , n 1 _reset , . . . , n 7 _reset , as shown in the diagram of fig3 ( note that in our prototype each pulse generator can be connected up to eight modules or processing nodes at lower levels of the hierarchy ). these signals pass through ibuf buffers and are linked to the inputs of the and gate of eight inputs and 8 . if any of these inputs is activated ( logic value zero ), then the and 8 output switches to zero . this zero value will be communicated to the higher levels through the obuf buffer linked to the and 8 output , in case of existing higher levels , and will also change the q output of the d - type synchronous flip - flop identified as fdc in fig3 to zero using its clr input ( via the inv inverter ). if it is a top module , then the logic level of the clock_detector &# 39 ; s toplevel output will be true . this true logic level will set the m 2 _ 1 multiplex , through its control input &# 39 ; s 0 , so that its d 1 input will be multiplexed to its o output . the q output of the fdc flip - flop will then pass through the multiplex , generating the reset signals no_reset 0 , n 1 _reset 0 , . . . , n 7 _reset 0 to the lower levels . if it is not the top module , the m 2 _ 1 multiplex will multiplex d 0 to its o output , enforcing reset signals generated to the lower levels to come from the ibuf linked to the top_reset signal , which brings the reset signal coming from higher levels of the hierarchy . a logical diagram of the pulse detector we used in our module prototype is shown in fig4 . in this figure , l 1 ( counter ) is a binary counter of three bits . this counter has an asynchronous reset input named async_ctrl . this input is linked through the inv inverter to the ext - clock signal , which brings pulses from modules at higher levels of the hierarchy . if a module is not the top module , the ext_clock signal does not have pulses and is kept at the high logic level ( as inactive in the prototype ). this logic level will be inverted by inv , so that it will not cause the counter to be reset by async_ctrl . in this case , the internal module &# 39 ; s pulses that come from int_clock will activate the l 1 counter , which will count until its output reaches the maximum allowed binary value ( 8 , since it is a 3 - bit counter ). once the counter &# 39 ; s value is maximum , its term_cnt output ( end - of - counting ) changes to the true logic value . this true logic value inhibits , through its clk_en input ( via the inv inverter ), extra increments to the counter . in addition , the term_cnt signal set the m 2 _ 1 multiplex connecting its d 1 input to its o output , in order to make the output signal sel_clock of this pulse detector equal to the pulses that come from the int_clock . in case the module is not the top one there are pulses in ext_clock coming from modules above in the hierarchy . thus , the term_cnt output will be always equal to zero , which sets the m 2 _ 1 multiplex connecting its d 0 input to its o output , making the output signal of the pulse detector equal to the pulses that come from the ext_clock . each one of these pulses will clear the l 1 counter , never allowing it to count until its maximum . in each processing node , the hardware of the global clock , or the counter module , is quite simple and its block diagram is shown in fig5 , inside of a box . the hardware is composed of an interface with the bus of the processing unit and a counter . the interface allows any processing node to initialize the global clock and to read the counter value , which can have as many bits as necessary ( 64 bits in the prototype we implemented ). under the command of the processor in the processing node , a reset signal can be issued by the interface ( fig5 ). this signal arrives at the first module through one of the lines n 0 _reset , n 1 _reset , etc ., as shown in fig3 , and goes up through the module hierarchy until it reaches the top module . the signal goes back synchronously through the modules arriving at all counters simultaneously . this ensures that , after initialization , simultaneous reads of counter values in different processing nodes will result in reading the same counting of time ( number of pulses counted ) since the initialization . to avoid race conditions in reading the counter value ( the counter can be in the middle of an increment at the moment of a read ) it must be synchronized with the global clock pulse .