So I'm a senior in college and a couple of weeks ago, I interviewed at Microsoft for an entry-level SDE position. While most of the interviews (out of the total 7) were on the general principles of data structures and algorithms, one of the interviewers asked me if I knew anything about distributed computing.I replied that, while I haven't had a course on 'distributed computing' per se, I'd like to take a stab at the question. He proceeded to ask a question on mutexes and semaphores that was covered in our operating systems course, and I thought was more about parallelism and concurrency than distributed computing.Having gone through this experience, I am led to wonder as to what exactly constitutes distributed computing? Do multiple parallel threads trying to synchronize for access to a resource constitute a problem in the domain of distributed computing?
What exactly is distributed computing?
distributed computing;operating systems
Having gone through this experience, I am led to wonder as to what exactly constitutes distributed computing?Distributed computing is an inherently parallel collection of processing elements that communicate with one another to tackle one or more problems. Those processing elements are sufficiently separated from each other that it is not practical to build a reliable and timely messaging fabric between them, and so it becomes impossible for there to be a global knowledge of the state of the system. Particular features of messaging with distributed systems are that messages will get lost, will get garbled, will get delayed  solutions in this space have to take account of this.  Thus, distributed programming is about dealing with networks and messages and parallelism and a lack of global information.The easiest method of working around the problems is to make a single processing element be special, i.e., authoritative for a particular piece of information. Then the other elements can either refer back to it every time, or cache the information and hope that it doesn't go out of date (since they can't count on being told of changes). This is the classic client/server architecture.Internet computing is distributed computing, but without the ability to control what most of the distributed nodes really do.Do multiple parallel threads trying to synchronize for access to a resource constitute a problem in the domain of distributed computing?They constitute a possible solution that is useful when building the client/server model, but at a cost of a potentially dramatic increase in resource contention. For reads, that's not a very big deal (providing there's enough hardware) but for writes it's a big problem indeed.What you try to avoid though is distributed locks. The lack of reliable timely messaging absolutely slays distributed decision protocols (unless you use something like the Paxos Commit protocol, but that's got a lot of caveats). The fundamental problem with distributed computing is that Bad Stuff Happens To Messages. (Relatively-low level protocols like TCP lessen the problems, but you can still come badly unstuck.)