Porting Signal processing algorithms to CuPy for precision measurement
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Shukla, Mamta
At European Organization for Nuclear Research(CERN), for the alignment of large superconducting magnets and cryogenics, an interferometry based system is being devised to identify the position of their elements. This technique uses interferometry principle and uses sweeping laser to identify the distance of multiple points using Fourier Analysis. The data acquired from photo-detection module, received after a sweep of laser source, needs to undergo sophisticated post processing to obtain the final results. The system must monitor position of a large number of elements every second. Dealing with 1000s of target points in less than 1 second required time-optimized and precise calculation. Thus, GPU was employed to provide faster and precise results. This required to use signal processing algorithms like: Butterworth Filter, Hilbert Transform, Savitzky-Golay smoothing Filter in GPU. This talk will cover steps involved in adopting signal processing algorithm to GPU to achieve better performance and understand the effects of parallelism achieved. For the initial development, CuPy library for NVIDIA GPUs is used and later moved to implementation in C. CuPy provides wrapper for most of the CUDA toolkit in Python. We will also provide highlight about performance metrics with respect to increase in the data size and possible optimizations of its processing.
 Good afternoon everyone. Today I will be talking about porting signal processing algorithm to QPy for precision measurement. First I would like to thank all the people associated with this project at CERN. This is the outline of my talk. First I would like to talk about frequency scanning interferometry system device for precision measurement. Next I would like to highlight about signal processing algorithm being used in this system. Then I would like to highlight about few of the algorithms and how they are ported to QPy and in the end my final thoughts about this project. Frequency scanning interferometry system is a distance measurement technique which can measure absolute distance up to micrometer precision between the laser source and the target which is a reflective surface or a retro reflecting a surface or glass balls. This technique is used in metrology and the system at CERN is one of the first kind which will be used for accelerators and cryogenics. It will be used for monitoring components inside the cryostat and provide precision alignment under harsh conditions like high radiation, low temperature and ultra high vacuum. On the left you can see photo of cryostat mounted with laser source and the reflective targets are actually inside. FSI is based on Michelson interferometry principle where source of laser that is a fixed laser source is divided into two arms by using a beam splitter. So one goes to the target that is the retro reflector and one goes to the reference mirror. So at the end that is at photo detector we obtain the interference of both the signal. The interference signal is represented by this equation where A is the magnitude of the signal and T tau determines the time delay between the signal and alpha is the sweep rate of the laser. So instead of using a fixed laser source we use a sweeping laser source in which here in this other system the fiber tip actually acts as a reference mirror and it reflects back 4% of the light and rest of the 96% of the light is reflected back from the target. So based on the number of cycles of the signal which we receive at the photo detection module we can actually determine the distance of the target. You can check out this presentation to know more details about frequency scanning interferometry system. But we needed multiple target points to compute the distance for huge components and to align there. So for this the interference signal can be depicted by this equation when it is received at the photo detector end. To deal with the multiple points Fourier analysis is used where FFT of the measured signal is calculated to obtain the corresponding beat frequency for each of the different distances. And so by knowing the sweep rate and different frequencies the different beat frequencies we can actually calculate the distance for multiple targets as well. But it is not easy to get the sweep rate of the laser being used. So we use a reference interferometer system for which we already know the length L and the laser is passed through the HEN gas cell where we observe the absorption of the peak through the spectrum of the gas cell obtained to get the actual value of alpha. So by having a reference interferometer system of length L and the unknown distances are acquired through the measurement channels we can actually relate these two formulas to find the distance for the unknown distance for different targets. So this is the fine actual system since we have to deal with thousands of measurement channels for different target points we use multiple photo detection modules are used inside a measurement chassis which actually acquire the analog signal with the help of ADC it is converted into digital signal and then it is transferred to the server side where we have GPU and it take care of all the massive calculation and post processing required. Now I would like to talk about the algorithm and the routines and the signal processing algorithms involved in the post processing of a post processing to obtain the final distance. So here you can see how the signal is acquired in the different stages and here the part where the GPU is responsible for doing all the calculation and where actual spectral and actual signal processing and the analysis happen. So the algorithm is basically divided into three main steps first is the data linearization where we pass the reference signal through a Butterworth filter high pass filter and we calculate the instantaneous frequency and the phase information which is used in the later two stages by gas cell and the measurement cell to linearize them. The second step is to process the gas cell data to obtain the sweep rate and last we need to do fast Fourier transform of the measurement cell to actually find the beat frequencies and subsequently calculate distance based on the alpha and the different beat frequencies in the FFT. So this is the complete view of signal processing algorithm involved. So how does our raw data looks like? So this is a 2.5 million simple points which were obtained at 100 megahertz and all these needs to undergo the processing in GPU to obtain the final distance. Now we know our algorithm and required signal analysis so we will be dealing with large set of data and for this let's see how Qpy maps to this picture and how it helps in speeding up the process. First I would like to highlight about Qpy. It is an open source library written in Python to provide high performance computation on GPU. It uses the underlying architecture of GPU and lot of CUDA libraries are already being used. So for example you can see for doing the linear algebra it uses Q plus for doing computation on sparse matrix and there is Q sparse library already present and similarly we can do more things like multi GPU data transfer and even create our own customized kernel. Along with this it provides a drop in replacement for NumPy and hence it is very easy to start for proof of concept and validation on GPU. It is an open source as I previously mentioned and it is distributed under MIT license and it's very easy to start and scale and also to develop customized kernel and run them using just in time compiler. So what does Qpy provides for signal processing since we already have a massive library which is used on CPUs. So along with a good coverage for NumPy there are scipy routines which can be adopted out of the box. For example the discrete Fourier transform for doing the operations related to linear algebra like linear decomposition or calculating eigenvalues. Then there are also different filters available for doing image processing and for calculation of sparse matrices as well. One of the interesting projects to look forward which uses Qpy and signal processing is Q signal but since it is a very new a new project so it will be very interesting to look forward to it and there is already a very interesting developments associated with this project. So just a few considerations that I observe or I thought of when porting any algorithm to GPU. First of all check the data format because Qpy provides with floating point precision that is float 64 and float 32 type of data. So it is wise to choose your data accordingly. Most of the time consuming or cycle consuming process is copying data from CPU RAM to GPU global memory or shared memory and this is one of the places where the computations can be reduced massively and it's always better to avoid the recursive functions on GPUs and GPU is more effective if you have to deal with computation for large data set and have have possibility for data and task parallelism. So now I would like to talk about few of the routines are the functions from scipy which are not present in Qpy and how they are adapted for the how they're adopted and how their performance is measured. So the first is Butterworth filter and signal filter and signal processing we use filters to allow specific range of signal. So Butterworth filter is basically a monotonic filter which allows minimum ripple in the passband and the stopband and it is represented by this transfer function and where cutoff frequency actually decides the range in which you want to allow your signal or not to allow your signal and here on the right you can see a low pass filter where the green line depicts the cutoff frequency so it passes all the signal till this point and it tries to stop rest of the signal. So here we are using Butterworth filter for reference signal and here it is you being used as a high pass filter to allow components all out to allow all the sample points above 100 kilohertz. So the algorithm is pretty much similar to scipy or to say it's more like a matlab style. So for creating the Butterworth digital filter we have to first create the analog prototype. So the filter is represented with zeros poles and gains. So first we design the analog filter based on based on order and cutoff frequency mentioned then we can actually map analog filters amplitude directly for digital filter but for frequencies we need to go through frequency warping and then since we are designing a high pass filter the slow pass analog prototype needs to be converted into a high pass prototype first and by using the bilinear transformation we can actually get the digital filter represented in forms of zeros poles and the gain and then you can convert into b into transfer function that is the b by a form. So this is how this function is called and it is pretty much exactly similar like scipy calling and to use this to apply this filter on a data set you can apply using l filter but since it's a recursive filter and it's in the recursive form so it is not adopted for GPU so the best way is to use the FFT since we have a very fast computation available for fast Fourier transform on GPU so we can apply this filter coefficients on a impulse response and do element wise multiplication with our actual real data and obtain the filtered output by using the inverse transform. Next I would like to talk about Hilbert transform so for any real signal if you want to want to know the instantaneous frequency or the amplitude Hilbert transform is used so to obtain the Hilbert transform the real signal is shifted to 90 degree and it is very easy to obtain it by using a fast Fourier transform and the analytic signal is actually the combination of real signal and the imaginary part which is basically the Hilbert transform and with this we can actually calculate the instantaneous frequency so the idea behind calculating the Hilbert transform is to first get the fast Fourier transform of the real signal and then all these Fourier coefficients have to be shifted to 90 degree phase shift and then do the element wise multiplication with the actual signal and by obtaining the inverse Fourier transform we get the analytic signal. So here Hilbert transform is used after filtering the reference cell to obtain the parameters which can be used for interpolation of the gas cell and the measurement cell so this is how the Hilbert transform has been applied on reference cell and we obtained the time analysis for this transform since we are using a fast Fourier transform and there is a very fast implementation on GPU with the CUDA so it drastically reduces the time and here you can see the phase shift in the reference cell like the orange one depicts actually the analytic signal obtained corresponding to the reference cell not only the time signal but we can actually check more like how our FFT kernel is exactly behaving by the time timeline analysis using NVIDIA N-SYSTEMS and this actually depicts some of the interesting information like how many threads are being created, how much memory pool is being used and how much exact time is being taken by this kernel. Next I would like to talk about Savitski Goli filter so it is basically a smoothing filter it keeps the signal our tries to retain the signal in its original form and removes the noise so it is designed based on the window that is the number of segment of the data you want to smooth or you want to calculate at a time and for all these points we obtain polynomial coefficients and it is calculated based on the order you provide so these polynomial coefficients are calculated using least square fit but if the signal is equally spaced we can calculate this least square fit and we can find analytical solution or a true solution as we say and use this as a convolution coefficient and convolve with the actual signal to get the smooth data. So here you see one of the implementation on one of the noisy signal and how the shape is mostly retained after filtering through Savitski Goli filter. So as I explained how we can do this and it is not exactly same as the sci-pi one but the idea is same that to calculate first the coefficients and then convolve with the actual signal. So we are using this FIR filter on gas cell data to reduce the noise and to reduce the noise in the signal so this is the spectrum of the gas cell data which is used to identify the alpha and this is the filtered output after applying the gas after applying the filter and here you can actually see the zoomed out view of the signal and how it is more smooth and the noise is reduced. So this is the time analysis of the filter and it is calculated on GPU and CPU respectively and for CPU the actual implementation available in sci-pi was used. One of the interesting facts is when we implementation through NVIDIA inside systems or by checking the kernel a different CUDA kernel statistics in the command line you can actually observe the areas where where a lot of memory copying is happening and check in the code where it can be avoided. Next after obtaining the filtered gas cell data it actually we actually need to find the peaks after a certain threshold but since there is no actual implementation to find the peaks in QPY yet so on GPU it was developed considering that a peak has it can be identified just by knowing the neighbors but the performance is not that impressive. So here are some of the more routines which can be used which are used some of the NumPy routines which were applied on the reference cell which is actually which consists of 2.5 million data points and here you can see how drastically the speed has improved with QPY and it can be more better as well. So one of the most used computational technique is fast Fourier transform in signal processing and thanks to the QPY library there is an implementation of FFT which is actually quite fast and it provides a flexibility to plan out the FFT as well and so here in this analysis we apply fast Fourier transform on the measured signal measurement cell to obtain the beat frequencies which are later used to get the final distance with the help of alpha and here you can see the FFT on GPU is very fast and you can also view how much time is being taken for each of the FFT kernel by knowing the kernel statistics and also what is happening parallelly inside the GPU at the same time. So my final thoughts about using QPY and signal processing. So QPY is a very good library to start with and it also gives the idea whether your algorithm is good to be used on GPU and how well you can use the GPU to offer large set of data. Since we know performance improving performance is like a continuous learning process so more benchmarking and more test analysis are needed to ultimately increase the performance and there is also ID to use custom kernels to improve the performance of drent algorithms and if this is useful I will be happy to upstream some of the developments to QPY repository and in the end I would like to mention about one of the famous code from Winston Churchill and and actually yes this is this is actually the beginning of doing signal processing on GPU there has been already a lot of work going on and it's a very interesting area to look forward to as well. So thank you for your attention and now I am open for questions and any feedback anything is welcome thank you. And thanks for being here at Fozdem in the HPC dev room to answer questions without the talk. I'll start with one from Jeffrey who says isn't this Mickelson interferometry a similar system to that used to detect gravity waves? Yes I would like to answer that since actually I'm not a physicist and expert with the gravity waves but I have heard that it is actually used for detecting the gravity waves as well and in our application it is more associated with accelerators and just for identifying the distance of components inside a cryogenic so it's more associated with the accelerators. Cool the QPY software that you've built all of this measurement system on top of is open sourced with the MIT license is the FSI software that you've developed on top is soon open sourcing that? Yes it's completely available on GitLab of Sun and later the only part of this project including the hardware it's available on open hardware repository so even the design or photo detection module to the card used for acquisition it's open sourced and then since it is our first prototype so this code will be also available in open hardware repository and also you can access through the GitLab link that I mentioned so it's completely open sourced. So there's a lot of low-level signal processing routines in QPY is there anything you think that's missing is there any of the stuff that you've developed do you think orps to be in QPY for everyone? Yes actually that's why I explicitly mentioned about those three algorithms that was butter filter and Hilbert transform and Savitski Goli filter so these are actually missing currently in QPY library and I will definitely want to upstream it in the QPY so that anyone else who wants to use it can use it and also improve the performance as well and the another thing to check out is Q signal project as well it is also very initial one where they are developing signal processing algorithms and they are also using QPY so it's a very good approach as well to adopt these filters for GPU so I'll definitely try to upstream it in the mainline QPY repository. Have you talked to the QPY developers? Not yet. So you said that this is a prototype system how long has this been in development? So it's been like a year I can say and it is targeted to be deployed for high luminosity that is when the accelerators restart in 2025 not the next year but after the another stop so it's meant for like after 2025 like completely deployed in the production level. So you've got examples of the performance on CPU and GPU and did you start with a CPU only implementation? Yes actually first we started with the CPU only implementation but for us the deadline or the higher like stop like it should be done within one second and at that moment we have to deal with thousands of channel at the moment and this actually was taking a lot of time in CPU so that's why we went for a GPU for like making this computation faster and it's also easy to scale as well so that's why we adopted for GPUs. Cool are there any tests to make sure that the GPU implementation gets the same answer within some level of precision as the CPU implementation? For the like initial phases we just compare what we are getting as an output from both the systems like just comparing it with the values we are getting and yeah so that was kind of a benchmarking we did for this system so later we are looking for more number of channels like going up to thousands of them so that will be a more faster test for this one. To understand like GPU is actually efficient for this kind of system. One last question before the thing cuts off Kenneth asks about AMD GPUs what would you do? Yes yes so first because we wanted to like prototype this and get to know like the idea we are thinking about the...
 Good afternoon everyone. Today I will be talking about porting signal processing algorithm  to QPy for precision measurement. First I would like to thank all the people associated  with this project at CERN. This is the outline of my talk. First I would like to talk about  frequency scanning interferometry system device for precision measurement. Next I would like  to highlight about signal processing algorithm being used in this system. Then I would like  to highlight about few of the algorithms and how they are ported to QPy and in the end  my final thoughts about this project. Frequency scanning interferometry system is a distance  measurement technique which can measure absolute distance up to micrometer precision between  the laser source and the target which is a reflective surface or a retro reflecting  a surface or glass balls. This technique is used in metrology and the system at CERN  is one of the first kind which will be used for accelerators and cryogenics. It will be  used for monitoring components inside the cryostat and provide precision alignment under harsh  conditions like high radiation, low temperature and ultra high vacuum. On the left you can  see photo of cryostat mounted with laser source and the reflective targets are actually inside.  FSI is based on Michelson interferometry principle where source of laser that is a fixed laser  source is divided into two arms by using a beam splitter. So one goes to the target that  is the retro reflector and one goes to the reference mirror. So at the end that is at  photo detector we obtain the interference of both the signal. The interference signal  is represented by this equation where A is the magnitude of the signal and T tau determines  the time delay between the signal and alpha is the sweep rate of the laser. So instead  of using a fixed laser source we use a sweeping laser source in which here in this other system  the fiber tip actually acts as a reference mirror and it reflects back 4% of the light  and rest of the 96% of the light is reflected back from the target. So based on the number  of cycles of the signal which we receive at the photo detection module we can actually  determine the distance of the target. You can check out this presentation to know more  details about frequency scanning interferometry system. But we needed multiple target points  to compute the distance for huge components and to align there. So for this the interference  signal can be depicted by this equation when it is received at the photo detector end.  To deal with the multiple points Fourier analysis is used where FFT of the measured  signal is calculated to obtain the corresponding beat frequency for each of the different distances.  And so by knowing the sweep rate and different frequencies the different beat frequencies  we can actually calculate the distance for multiple targets as well. But it is not easy  to get the sweep rate of the laser being used. So we use a reference interferometer system  for which we already know the length L and the laser is passed through the HEN gas cell  where we observe the absorption of the peak through the spectrum of the gas cell obtained  to get the actual value of alpha. So by having a reference interferometer system of length  L and the unknown distances are acquired through the measurement channels we can actually relate  these two formulas to find the distance for the unknown distance for different targets.  So this is the fine actual system since we have to deal with thousands of measurement  channels for different target points we use multiple photo detection modules are used  inside a measurement chassis which actually acquire the analog signal with the help of  ADC it is converted into digital signal and then it is transferred to the server side  where we have GPU and it take care of all the massive calculation and post processing  required. Now I would like to talk about the algorithm and the routines and the signal  processing algorithms involved in the post processing of a post processing to obtain  the final distance. So here you can see how the signal is acquired in the different stages  and here the part where the GPU is responsible for doing all the calculation and where actual  spectral and actual signal processing and the analysis happen. So the algorithm is basically  divided into three main steps first is the data linearization where we pass the reference  signal through a Butterworth filter high pass filter and we calculate the instantaneous  frequency and the phase information which is used in the later two stages by gas cell  and the measurement cell to linearize them. The second step is to process the gas cell  data to obtain the sweep rate and last we need to do fast Fourier transform of the measurement  cell to actually find the beat frequencies and subsequently calculate distance based  on the alpha and the different beat frequencies in the FFT. So this is the complete view of  signal processing algorithm involved. So how does our raw data looks like? So this is a  2.5 million simple points which were obtained at 100 megahertz and all these needs to undergo  the processing in GPU to obtain the final distance. Now we know our algorithm and required  signal analysis so we will be dealing with large set of data and for this let's see  how Qpy maps to this picture and how it helps in speeding up the process. First I would  like to highlight about Qpy. It is an open source library written in Python to provide  high performance computation on GPU. It uses the underlying architecture of GPU and lot  of CUDA libraries are already being used. So for example you can see for doing the linear  algebra it uses Q plus for doing computation on sparse matrix and there is Q sparse library  already present and similarly we can do more things like multi GPU data transfer and even  create our own customized kernel. Along with this it provides a drop in replacement for  NumPy and hence it is very easy to start for proof of concept and validation on GPU. It  is an open source as I previously mentioned and it is distributed under MIT license and  it's very easy to start and scale and also to develop customized kernel and run them  using just in time compiler. So what does Qpy provides for signal processing since we already  have a massive library which is used on CPUs. So along with a good coverage for NumPy there  are scipy routines which can be adopted out of the box. For example the discrete Fourier  transform for doing the operations related to linear algebra like linear decomposition  or calculating eigenvalues. Then there are also different filters available for doing  image processing and for calculation of sparse matrices as well. One of the interesting projects  to look forward which uses Qpy and signal processing is Q signal but since it is a very  new a new project so it will be very interesting to look forward to it and there is already  a very interesting developments associated with this project. So just a few considerations  that I observe or I thought of when porting any algorithm to GPU. First of all check the  data format because Qpy provides with floating point precision that is float 64 and float 32  type of data. So it is wise to choose your data accordingly. Most of the time consuming or cycle  consuming process is copying data from CPU RAM to GPU global memory or shared memory and this is one  of the places where the computations can be reduced massively and it's always better to avoid the  recursive functions on GPUs and GPU is more effective if you have to deal with computation for  large data set and have have possibility for data and task parallelism. So now I would like to talk  about few of the routines are the functions from scipy which are not present in Qpy and how they  are adapted for the how they're adopted and how their performance is measured. So the first is  Butterworth filter and signal filter and signal processing we use filters to allow specific  range of signal. So Butterworth filter is basically a monotonic filter which allows minimum ripple  in the passband and the stopband and it is represented by this transfer function and where  cutoff frequency actually decides the range in which you want to allow your signal or not to  allow your signal and here on the right you can see a low pass filter where the green line depicts  the cutoff frequency so it passes all the signal till this point and it tries to stop rest of the  signal. So here we are using Butterworth filter for reference signal and here it is you being used  as a high pass filter to allow components all out to allow all the sample points above 100 kilohertz.  So the algorithm is pretty much similar to scipy or to say it's more like a matlab style. So for  creating the Butterworth digital filter we have to first create the analog prototype. So the filter  is represented with zeros poles and gains. So first we design the analog filter based on  based on order and cutoff frequency mentioned then we can actually map analog filters amplitude  directly for digital filter but for frequencies we need to go through frequency warping and then  since we are designing a high pass filter the slow pass analog prototype needs to be converted into  a high pass prototype first and by using the bilinear transformation we can actually get  the digital filter represented in forms of zeros poles and the gain and then you can convert into  b into transfer function that is the b by a form. So this is how this function is called  and it is pretty much exactly similar like scipy calling and to use this to apply this filter on  a data set you can apply using l filter but since it's a recursive filter and it's in the recursive  form so it is not adopted for GPU so the best way is to use the FFT since we have a very fast  computation available for fast Fourier transform on GPU so we can apply this filter coefficients  on a impulse response and do element wise multiplication with our actual real data and  obtain the filtered output by using the inverse transform. Next I would like to talk about  Hilbert transform so for any real signal if you want to want to know the instantaneous  frequency or the amplitude Hilbert transform is used so to obtain the Hilbert transform the real  signal is shifted to 90 degree and it is very easy to obtain it by using a fast Fourier transform  and the analytic signal is actually the combination of real signal and the imaginary part which is  basically the Hilbert transform and with this we can actually calculate the instantaneous frequency  so the idea behind calculating the Hilbert transform is to first get the fast Fourier  transform of the real signal and then all these Fourier coefficients have to be shifted to 90  degree phase shift and then do the element wise multiplication with the actual signal and by obtaining  the inverse Fourier transform we get the analytic signal. So here Hilbert transform is used after  filtering the reference cell to obtain the parameters which can be used for interpolation of the gas  cell and the measurement cell so this is how the Hilbert transform has been applied on reference cell  and we obtained the time analysis for this transform since we are using a fast Fourier transform and  there is a very fast implementation on GPU with the CUDA so it drastically reduces the time  and here you can see the phase shift in the reference cell like the orange one depicts  actually the analytic signal obtained corresponding to the reference cell not only the time signal  but we can actually check more like how our FFT kernel is exactly behaving by the time  timeline analysis using NVIDIA N-SYSTEMS and this actually depicts some of the interesting  information like how many threads are being created, how much memory pool is being used and  how much exact time is being taken by this kernel. Next I would like to talk about Savitski  Goli filter so it is basically a smoothing filter it keeps the signal our tries to retain the signal  in its original form and removes the noise so it is designed based on the window that is the  number of segment of the data you want to smooth or you want to calculate at a time and for all  these points we obtain polynomial coefficients and it is calculated based on the order you provide  so these polynomial coefficients are calculated using least square fit but if the signal is equally  spaced we can calculate this least square fit and we can find analytical solution or a true  solution as we say and use this as a convolution coefficient and convolve with the actual signal  to get the smooth data. So here you see one of the implementation on one of the noisy signal  and how the shape is mostly retained after filtering through Savitski Goli filter.  So as I explained how we can do this and it is not exactly same as the sci-pi one but the idea is  same that to calculate first the coefficients and then convolve with the actual signal. So we are  using this FIR filter on gas cell data to reduce the noise and to reduce the noise in the signal  so this is the spectrum of the gas cell data which is used to identify the alpha  and this is the filtered output after applying the gas after applying the filter and here you can  actually see the zoomed out view of the signal and how it is more smooth and the noise is reduced.  So this is the time analysis of the filter and it is calculated on GPU and CPU respectively and for  CPU the actual implementation available in sci-pi was used. One of the interesting facts is when we  implementation through NVIDIA inside systems or by checking the kernel a different CUDA kernel  statistics in the command line you can actually observe the areas where where a lot of memory  copying is happening and check in the code where it can be avoided.  Next after obtaining the filtered gas cell data it actually we actually need to find the peaks  after a certain threshold but since there is no actual implementation to find the peaks in QPY yet  so on GPU it was developed considering that a peak has it can be identified just by  knowing the neighbors but the performance is not that impressive. So here are some of the more  routines which can be used which are used some of the NumPy routines which were applied on the  reference cell which is actually which consists of 2.5 million data points and here you can see  how drastically the speed has improved with QPY and it can be more better as well.  So one of the most used computational technique is fast Fourier transform in signal processing  and thanks to the QPY library there is an implementation of FFT which is actually quite  fast and it provides a flexibility to plan out the FFT as well and so here in this analysis  we apply fast Fourier transform on the measured signal measurement cell to obtain the beat  frequencies which are later used to get the final distance with the help of alpha and here you can  see the FFT on GPU is very fast and you can also view how much time is being taken for each of the  FFT kernel by knowing the kernel statistics and also what is happening parallelly inside the GPU  at the same time. So my final thoughts about using QPY and signal processing. So QPY is a  very good library to start with and it also gives the idea whether your algorithm is good to be used  on GPU and how well you can use the GPU to offer large set of data. Since we know performance  improving performance is like a continuous learning process so more benchmarking and more test  analysis are needed to ultimately increase the performance and there is also ID to use custom  kernels to improve the performance of drent algorithms and if this is useful I will be  happy to upstream some of the developments to QPY repository and in the end I would like to  mention about one of the famous code from Winston Churchill and and actually yes this is  this is actually the beginning of doing signal processing on GPU there has been already  a lot of work going on and it's a very interesting area to look forward to as well. So thank you for  your attention and now I am open for questions and any feedback anything is welcome thank you.  And thanks for being here at Fozdem in the HPC dev room to answer questions without the talk.  I'll start with one from Jeffrey who says isn't this Mickelson interferometry a similar system  to that used to detect gravity waves? Yes I would like to answer that since actually I'm not a  physicist and expert with the gravity waves but I have heard that it is actually used for  detecting the gravity waves as well and in our application it is more associated with accelerators  and just for identifying the distance of components inside a cryogenic so it's more associated with  the accelerators. Cool the QPY software that you've built all of this measurement system on top of  is open sourced with the MIT license is the FSI software that you've developed on top  is soon open sourcing that? Yes it's completely available on GitLab of Sun and later the only  part of this project including the hardware it's available on open hardware repository so even the  design or photo detection module to the card used for acquisition it's open sourced and then since  it is our first prototype so this code will be also available in open hardware repository and also  you can access through the GitLab link that I mentioned so it's completely open sourced.  So there's a lot of low-level signal processing routines in QPY is there anything you think  that's missing is there any of the stuff that you've developed do you think orps to be in QPY for  everyone? Yes actually that's why I explicitly mentioned about those three algorithms that was  butter filter and Hilbert transform and Savitski Goli filter so these are actually missing currently  in QPY library and I will definitely want to upstream it in the QPY so that anyone else who  wants to use it can use it and also improve the performance as well and the another thing to  check out is Q signal project as well it is also very initial one where they are developing signal  processing algorithms and they are also using QPY so it's a very good approach as well to  adopt these filters for GPU so I'll definitely try to upstream it in the mainline QPY repository.  Have you talked to the QPY developers? Not yet. So you said that this is a prototype system how  long has this been in development? So it's been like a year I can say and it is targeted to be  deployed for high luminosity that is when the accelerators restart in 2025 not the next year but  after the another stop so it's meant for like after 2025 like completely deployed in the production  level. So you've got examples of the performance on CPU and GPU and did you start with a CPU only  implementation? Yes actually first we started with the CPU only implementation but for us  the deadline or the higher like stop like it should be done within one second and at that  moment we have to deal with thousands of channel at the moment and this actually was taking a lot  of time in CPU so that's why we went for a GPU for like making this computation faster and it's  also easy to scale as well so that's why we adopted for GPUs. Cool are there any tests to make sure  that the GPU implementation gets the same answer within some level of precision as the CPU  implementation? For the like initial phases we just compare what we are getting as an output  from both the systems like just comparing it with the values we are getting and yeah so that was  kind of a benchmarking we did for this system so later we are looking for more number of channels  like going up to thousands of them so that will be a more faster test for this one.  To understand like GPU is actually efficient for this kind of system.  One last question before the thing cuts off Kenneth asks about AMD GPUs what would you do?  Yes yes so first because we wanted to like prototype this and get to know like the idea we  are thinking about the...
signal_processing_cupy_0_179.png signal_processing_cupy_180_527.png signal_processing_cupy_528_887.png signal_processing_cupy_888_1283.png signal_processing_cupy_1284_1391.png signal_processing_cupy_1392_5171.png signal_processing_cupy_5172_8831.png signal_processing_cupy_8832_10631.png signal_processing_cupy_10632_10847.png signal_processing_cupy_10848_11351.png signal_processing_cupy_11352_11399.png signal_processing_cupy_11400_11699.png signal_processing_cupy_11700_12287.png signal_processing_cupy_12288_12791.png signal_processing_cupy_12792_20171.png signal_processing_cupy_20172_21707.png signal_processing_cupy_21708_22631.png signal_processing_cupy_22632_23375.png signal_processing_cupy_23376_25631.png signal_processing_cupy_25632_26015.png signal_processing_cupy_26016_27347.png signal_processing_cupy_27348_29147.png signal_processing_cupy_29148_29819.png signal_processing_cupy_29820_32879.png signal_processing_cupy_32880_35783.png signal_processing_cupy_35784_37799.png signal_processing_cupy_37800_38171.png signal_processing_cupy_38172_39587.png signal_processing_cupy_39588_40403.png signal_processing_cupy_40404_40727.png signal_processing_cupy_40728_41579.png signal_processing_cupy_41580_41867.png signal_processing_cupy_41868_42287.png signal_processing_cupy_42288_42683.png signal_processing_cupy_42684_43115.png signal_processing_cupy_43116_43238.png