<commit_before>---
layout: dark-post
title: Building a pip-installable Python package that invokes custom CUDA code [Python/C++/CUDA]
description: "Accessing CUDA code through Python"
tags: [Python, C++, CUDA, NVIDIA, GPU]
---

*The code for this post is located [here](https://github.com/apryor6/pipcudemo). To run the code you will need a CUDA-enabled GPU and an installation of the [CUDA toolkit](https://developer.nvidia.com/cuda-downloads)*

Python is easy. GPUs are fast. Combining Python and CUDA is a fantastic way to leverage the performance benefits and parallelism inherent in GPUs with all of the syntactic niceties of Python. However, there is a potential snag. If the package is written purely in Python, C, or C++, distribution is easily accomplished using `setuptools` and users can install your package and its dependencies with either `pip install` or through the `setup.py` script. Unfortunately, `setuptools` is not currently compatible with NVIDIA's compiler, `nvcc`, which is necessary for compiling CUDA code. So how can we make a Python package that involves CUDA? One solution to this problem is to compile the CUDA code into a shared library which is then linked against by the Python package just like any other C/C++ library. In this tutorial I will walk through all of the steps necessary to write a C++/CUDA library and then to create a Python extension package that can be uploaded to PyPi and subsequently installed by users with `pip`. I'm a big fan of [CMake](https://cmake.org/), and will use it to build the shared library.   

The functions implemented in this example will be simple to illustrate the methodology, but if you are interested in "real-world" example, I used this technique when developing `PyPrismatic`, a GPU-accelerated package for image simulation in electron microscopy as part of my Ph.D. Full details of this can be found at [www.prism-em.com](www.prism-em.com), which includes a walkthrough of the source code (you can also read a free-print version of the associated academic paper [here](https://arxiv.org/abs/1706.08563)).

<!-- more -->

## The CUDA/C++ Code

Our library is going to be a simple calculator that can add, subtract, multiply, and divide two integers. Be aware that this is actually a bad application of GPUs, but it is intended to be a simple example. If you were to instead add two arrays of integers, GPUs are great.  

What we will do is expose some C++ functions that our Python package will be able to invoke, and the implementation of those functions will invoke CUDA code. The key is that Python never directly sees CUDA. It sees a C++ function signature (which it knows how to work with), but the actual implementation of that function is delayed until the linking process, and by that point the CUDA code will have been compiled.  

First, let's prototype our C++/CUDA functions

~~~ c++
// "myCuda.cuh"

#ifndef MYCUDA_H
#define MYCUDA_H

namespace pipcudemo{<commit_msg>pipcudemo<commit_after>---
layout: dark-post
title: Building a pip-installable Python package that invokes custom CUDA code [Python/C++/CUDA]
description: "Accessing CUDA code through Python"
tags: [Python, C++, CUDA, NVIDIA, GPU]
---

*The code for this post is located [here](https://github.com/apryor6/pipcudemo). To run the code you will need a CUDA-enabled GPU and an installation of the [CUDA toolkit](https://developer.nvidia.com/cuda-downloads)*

Python is easy. GPUs are fast. Combining Python and CUDA is a fantastic way to leverage the performance benefits and parallelism inherent in GPUs with all of the syntactic niceties of Python. However, there is a potential snag. If the package is written purely in Python, C, or C++, distribution is easily accomplished using setuptools and users can install your package and its dependencies with either `pip install` or through the `setup.py` script. Unfortunately, `setuptools` is not currently compatible with NVIDIA's compiler, `nvcc`, which is necessary for compiling CUDA code. So how can we make a Python package that involves CUDA? One solution to this problem is to compile the CUDA code into a shared library which is then linked against by the Python package just like any other C/C++ library. In this tutorial I will walk through all of the steps necessary to write a C++/CUDA library and then to create a Python extension package that can be uploaded to PyPi and subsequently installed by users with `pip`. I'm a big fan of [CMake](https://cmake.org/), and will use it to build the shared library.   

The functions implemented in this example will be simple to illustrate the methodology, but if you are interested in "real-world" example, I used this technique when developing `PyPrismatic`, a GPU-accelerated package for image simulation in electron microscopy as part of my Ph.D. Full details of this can be found at [www.prism-em.com](www.prism-em.com), which includes a walkthrough of the source code (you can also read a free-print version of the associated academic paper [here](https://arxiv.org/abs/1706.08563)).

<!-- more -->

## The CUDA/C++ Code

Our library is going to be a simple calculator that can add, subtract, multiply, and divide two integers. Be aware that this is actually a bad application of GPUs, but it is intended to be a simple example. If you were to instead add two arrays of integers, GPUs are great.  

What we will do is expose some C++ functions that our Python package will be able to invoke, and the implementation of those functions will invoke CUDA code. The key is that Python never directly sees CUDA. It sees a C++ function signature (which it knows how to work with), but the actual implementation of that function is delayed until the linking process, and by that point the CUDA code will have been compiled.  

First, let's prototype our C++/CUDA functions

~~~ c++
// "myCuda.cuh"

#ifndef MYCUDA_H
#define MYCUDA_H

namespace pipcudemo{
{'fraction_non_alphanumeric': 0.06495263870094722, 'fraction_numerical': 0.0033829499323410014, 'mean_word_length': 4.409881061299177, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 8, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6525927', 'n_tokens_mistral': 1600, 'n_tokens_neox': 1493, 'n_words': 901}