<commit_before>#+TITLE: cl-mpi

cl-mpi provides convenient CFFI bindings for the Message Passing
Interface (MPI). MPI is typically used in High Performance Computing to
utilize big parallel computers with thousands of cores. It features minimal
communication overhead with a latency in the range of microseconds. In
comparison to the C or FORTRAN interface of MPI, cl-mpi relieves the
programmer from working with raw pointers to memory and a plethora of
mandatory function arguments.

If you have questions or suggestions, feel free to contact me
(<EMAIL>).

cl-mpi has been tested with MPICH, MPICH2, IntelMPI and Open MPI.

** Usage
An MPI program must be launched with =mpirun= or =mpiexec=. These commands
spawn multiple processes depending on your system and commandline
parameters. Each process is identical, except that it has a unique rank that
can be queried with =(MPI-COMM-RANK)=. The ranks are assigned from 0 to
=(- (MPI-COMM-SIZE) 1)=.

A wide range of communication functions is avilable to transmit messages
between different ranks.

To get familiar with cl-mpi, see the [[file:examples/][examples directory]]. When [[http://cliki.net/CL-Launch][cl-launch]] is
installed, each test can be invoked like this:
#+BEGIN_SRC sh :results output
mpiexec -n 3 ./examples/ring.lisp
#+END_SRC

However it is usually faster and more resource friendly to first compile
your Lisp application to a standalone executable and than launch it with
=mpiexec=. To illustrate this, the [[file:scripts/][scripts directory]] contains a script
[[file:scripts/build-cl-mpi-application.sh][build-cl-mpi-application.sh]] (that is just a very thin layer on top of
cl-launch) that can be used like this:
#+BEGIN_SRC sh results output
./scripts/build-cl-mpi-application.sh ./examples/ring.lisp ring
mpiexec -n 4 ring
#+END_SRC

** Testing
To run the test suite:
#+BEGIN_SRC sh :results output
   ./scripts/run-test-suite.sh all
#+END_SRC

or

#+BEGIN_SRC sh :results output
   ./scripts/run-test-suite.sh YOUR-FAVOURITE-LISP
#+END_SRC

** Performance
cl-mpi makes no additional copies of transmitted data and has therefore the
same bandwidth as any other language (C, FORTRAN). However the convenience
of error handling, automatic inference of the message types and safe<commit_msg>Minor documentation improvements [skip ci]<commit_after>#+TITLE: cl-mpi

cl-mpi provides convenient CFFI bindings for the Message Passing
Interface (MPI). MPI is typically used in High Performance Computing to
utilize big parallel computers with thousands of cores. It features minimal
communication overhead with a latency in the range of microseconds. In
comparison to the C or FORTRAN interface of MPI, cl-mpi relieves the
programmer from working with raw pointers to memory and a plethora of
mandatory function arguments.

If you have questions or suggestions, feel free to contact me
(<EMAIL>).

cl-mpi has been tested with MPICH, MPICH2, IntelMPI and Open MPI.

** Usage
An MPI program must be launched with =mpirun= or =mpiexec=. These commands
spawn multiple processes depending on your system and commandline
parameters. Each process is identical, except that it has a unique rank that
can be queried with =(MPI-COMM-RANK)=. The ranks are assigned from 0 to
=(- (MPI-COMM-SIZE) 1)=.

A wide range of communication functions is available to transmit messages
between different ranks.

To become familiar with cl-mpi, see the [[file:examples/][examples directory]]. When [[http://cliki.net/CL-Launch][cl-launch]] is
installed, each test can be invoked like this:
#+BEGIN_SRC sh :results output
mpiexec -n 3 ./examples/ring.lisp
#+END_SRC

However, it is usually faster and more resource-friendly to first compile
your Lisp application to a stand-alone executable and than launch it with
=mpiexec=. To illustrate this, the [[file:scripts/][scripts directory]] contains a script
[[file:scripts/build-cl-mpi-application.sh][build-cl-mpi-application.sh]] (that is just a very thin layer on top of
cl-launch) that can be used like this:
#+BEGIN_SRC sh results output
./scripts/build-cl-mpi-application.sh ./examples/ring.lisp ring
mpiexec -n 4 ring
#+END_SRC

** Testing
To run the test suite:
#+BEGIN_SRC sh :results output
   ./scripts/run-test-suite.sh all
#+END_SRC

or

#+BEGIN_SRC sh :results output
   ./scripts/run-test-suite.sh YOUR-FAVOURITE-LISP
#+END_SRC

** Performance
cl-mpi makes no additional copies of transmitted data and has therefore the
same bandwidth as any other language (C, FORTRAN). However the convenience
of error handling, automatic inference of the message types and safe
{'fraction_non_alphanumeric': 0.07631751585392521, 'fraction_numerical': 0.002186748305270063, 'mean_word_length': 4.674937965260546, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1237595', 'n_tokens_mistral': 1362, 'n_tokens_neox': 1256, 'n_words': 614}