I use Intel MKL for BLAS and use the Intel MKL Link Line Advisor for help with the command line options.The advisor provides options for Static, Dynamic and Single Dynamic Library. What do these terms mean?There have been numerous posts on SO regarding this but all of them have been focused on developing softwares rather than scientific/mathematical codes. (This one was especially useful)How are these terms relevant to Parallel Computing?Is one better than the other? (I know that better would need to be defined more precisely but I don't know how to as of now)What is used more commonly and why?What does the programmer need to consider when linking?
What does Static, Dynamic and Single Dynamic linking mean?
compiling
The terms static linking and dynamic linking are not directly related to parallel computing, though it has been known for quite some time that dynamic loading (as opposed to loading a statically-compiled executable) does not scale well on network file systems due to the heavy metadata load caused by the dynamic loader searching load paths for target libraries.It is hard to make general statements about whether static or dynamic libraries are better in high performance computing.  Certainly for most supercomputing applications, it is simpler and preferred to statically link.  Why is this?  On the current generation of supercomputers, there is usually only one job running per node, which strongly reduces any benefits of reduced memory consumption due to shared libraries.  Additionally, scientific computing codes tend not to be very sophisticated in terms of language features or program design, and they rarely make use of language features that require dynamic loading (such as plugin modules).  Dynamic libraries have the additional difficulty of being much less portable across operating systems than static libraries.  This has lead to the creation of several tools for automating the installation of dynamic libraries, which can sometimes create more problems than they solve.As a consequence of all of this, most HPC systems use static compilation when available.  Static libraries are viewed as faster, easier to install and maintain, and generally more robust.  HPC codes based on Python are one of the exceptions to this, but they are still subject to the performance problems associated with dynamic loading (several users on scicomp are actually working on this problem right now!).When you are choosing static vs. dynamic linking, you need to consider how and where your code will be deployed, whether the underlying libraries are likely to change or move, and the performance characteristics of your network file system.  You should also evaluate whether you need dynamic linking, either through a library dependence or to interoperate with a dynamic scripting language such as Python.Single Dynamic Library is an Intel-specific term.  It refers to the packaging of their dynamic libraries into a single meta-library to simplify the linking process.  If you will be using dynamic linking with Intel libraries, this form is probably preferred unless you are doing something complicated.