In a normal application, a simple call to time.Now() in most languages will suffice for getting a timestamp, but what about in a web application spread over multiple servers? As I understand it time.Now() will use the current system time, but there's no reliable way of telling after the fact if that time was synchronised with the rest of the machines. By this point you're throwing data into persistent storage with inconsistent timestamps which is not such a problem for created/modified times when inserting into a database, but a bigger problem when tracking event timing).The best way I see to mitigate this is to have all machines request a timestamp from one specific 'master' machine, but then I thought what about latency? What if the network is busy and the response is delayed? There's several flaws with this method but I can't think of a more reliable one other than using CPU and network resources synchronising the time every few seconds.What is the accepted best practice/industry standard here?
What is an accepted method of timestamping in a distributed system?
distributed computing;time
The usual practice is to set the servers to all keep their time updated using NTP.  There are limitations to the accuracy when using NTP time syncing which means that you should only rely on the time stamp to give a general idea of when events occurred, which is likely to be good enough for identifying the set of events you are interested in.Timestamps are no good for determining the sequence of events, even on a single server at a precision level, as even with microsecond timing, each CPU core will process 1000's of assembler instructions during the same timestamp.  Your server may have 40 such cores.The best approach I know of, to keep track of what is happening in a distributed system is to use transaction ID's and log files. A single log file written sequentially allows you to determine event order on a single server.  Anything that passes work between servers will need to appear in the logs on both servers recording the transfer of the work with a shared transaction ID.  