<jupyter_start><jupyter_text>https://github.com/awthomp/cusignal-icassp-tutorial/blob/main/notebooks/raw_cupy_cuda/raw_cupy_cuda_examples.ipynb  Hardware DGX StationV100 SMX2 16GBIntel Xeon E5-2968 v4 @ 2.20GHz SoftwareCUDA 11.0cuSignal 0.19SciPy 1.6.3CuPy 9.0.0 Profiling[Nsight Systems](https://docs.nvidia.com/nsight-systems/UserGuide/index.html)  CPU BaselineBaseline Scipy's Signal Lombscargle function<jupyter_code>!nsys profile -s none -o scipy_v1 -f true --stats=true python3 icassp_scipy_v1.py float32 5<jupyter_output>Warning: LBR backtrace method is not supported on this platform. DWARF backtrace method will be used.
WARNING: Backtraces will not be collected because sampling is disabled.
Collecting data...
Processing events...
Capturing symbol files...
Saving temporary "/tmp/nsys-report-76d9-cd22-0527-57d3.qdstrm" file to disk...
Creating final output files...

Processing [==============================================================100%]
Saved report file to "/tmp/nsys-report-76d9-cd22-0527-57d3.qdrep"
Exporting 643307 events: [================================================100%]

Exported successfully to
/tmp/nsys-report-76d9-cd22-0527-57d3.sqlite

Generating CUDA API Statistics...
CUDA API Statistics (nanoseconds)




CUDA trace data was not collected.


Generating Operating System Runtime API Statistics...
Operating System Runtime API Statistics (nanoseconds)

Time(%)      Total Time       Calls         Average         Minimum         Maximum  Name                                             [...]<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       |           |          |             |          || CuPy (User Cache)     |           |          |             |          || CuPy (Data Type)      |           |          |             |          || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       |           |          |             |          || CuPy (User Cache)     |           |          |             |          || CuPy (Data Type)      |           |          |             |          || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 11. CuPy implementation of Lombscargle function<jupyter_code>!nsys profile -s none -o cupy_v1 -f true --stats=true python3 icassp_cupy_v1.py float32 2000 0
#!nsys stats --report gpukernsum --report nvtxppsum --report gputrace cupy_v1.qdrep # Use to get register usage
!nsys stats --report gpukernsum --report nvtxppsum cupy_v1.qdrep<jupyter_output>Using cupy_v1.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v1.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average      Minimum    Maximum          Name       
 -------  ---------------  ---------  ------------  ---------  ----------  -----------------
   100.0   20,053,721,658      2,001  10,021,849.9  9,562,272  10,849,989  _cupy_lombscargle

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v1.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances     Average       Minimum      Maximum            Range        
 -------  ---------------  ---------  -------------  -----------  -----------  ---------------------
    99.3   20,244,660,806      2,000   10,122,330.4    9,662,292   10,950,094  cupy_lombscargle_loop
     0.7      137,536,440          1  137,536,440.0  137,536,440  137,536,440  cupy_lombscargle<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       | 30        | 135.1    | 10.1        | 5062.2   || CuPy (User Cache)     |           |          |             |          || CuPy (Data Type)      |           |          |             |          || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       | 48        | 145.0    | 21.2        | 2396.5   || CuPy (User Cache)     |           |          |             |          || CuPy (Data Type)      |           |          |             |          || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 21. CuPy implementation of Lombscargle function2. Added user cache<jupyter_code>!nsys profile -s none -o cupy_v2 -f true python3 icassp_cupy_v2.py float32 2000 0
!nsys stats --report gpukernsum --report nvtxppsum cupy_v2.qdrep<jupyter_output>Generate SQLite file cupy_v2.sqlite from cupy_v2.qdrep
Exporting 635440 events: [================================================100%]
Using cupy_v2.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v2.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average      Minimum    Maximum          Name       
 -------  ---------------  ---------  ------------  ---------  ----------  -----------------
   100.0   20,070,227,792      2,001  10,030,098.8  9,565,922  11,214,888  _cupy_lombscargle

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v2.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances     Average       Minimum      Maximum            Range        
 -------  ---------------  ---------  -------------  -----------  -----------  ---------------------
    99.3   20,177,914,899      2,000   10,088,957.4    9,623,261   11,272,907  cupy_lombscargle_loop
    [...]<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       | 30        | 135.1    | 10.1        | 5062.2   || CuPy (User Cache)     | 30        | 137.2    | 10.1        | 5062.2   || CuPy (Data Type)      |           |          |             |          || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       | 48        | 145.0    | 21.2        | 2396.5   || CuPy (User Cache)     | 48        | 145.7    | 21.2        | 2396.5   || CuPy (Data Type)      |           |          |             |          || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 31. CuPy implementation of Lombscargle function2. Added user cache3. Specific functions per data type<jupyter_code>!nsys profile -s none -o cupy_v3 -f true python3 icassp_cupy_v3.py float32 2000 0
!nsys stats --report gpukernsum --report nvtxppsum cupy_v3.qdrep<jupyter_output>Generate SQLite file cupy_v3.sqlite from cupy_v3.qdrep
Exporting 648951 events: [================================================100%]
Using cupy_v3.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v3.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average      Minimum    Maximum              Name           
 -------  ---------------  ---------  ------------  ---------  ----------  -------------------------
   100.0   20,377,888,995      2,001  10,183,852.6  9,708,131  10,887,367  _cupy_lombscargle_float32

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v3.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances     Average       Minimum      Maximum            Range        
 -------  ---------------  ---------  -------------  -----------  -----------  ---------------------
    99.3   20,578,059,624      2,000   10,289,029.8    9,805,129   11,003,895  cu[...]<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       | 30        | 135.1    | 10.1        | 5062.2   || CuPy (User Cache)     | 30        | 137.2    | 10.1        | 5062.2   || CuPy (Data Type)      | 26        | 135.4    | 10.3        | 4963.9   || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       | 48        | 145.0    | 21.2        | 2396.5   || CuPy (User Cache)     | 48        | 145.7    | 21.2        | 2396.5   || CuPy (Data Type)      | 48        | 150.4    | 21.2        | 2396.5   || CuPy (Fast Math)      |           |          |             |          || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 41. CuPy implementation of Lombscargle function2. Added user cache3. Specific functions per data type4. Utilize fast_math compiler flag<jupyter_code>!nsys profile -s none -o cupy_v4 -f true python3 icassp_cupy_v4.py float32 2000 0
!nsys stats --report gpukernsum --report nvtxppsum cupy_v4.qdrep<jupyter_output>Generate SQLite file cupy_v4.sqlite from cupy_v4.qdrep
Exporting 609810 events: [================================================100%]
Using cupy_v4.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v4.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum             Name           
 -------  ---------------  ---------  -----------  ---------  ---------  -------------------------
   100.0    4,474,465,748      2,001  2,236,114.8  2,133,767  2,320,873  _cupy_lombscargle_float32

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v4.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances     Average       Minimum      Maximum            Range        
 -------  ---------------  ---------  -------------  -----------  -----------  ---------------------
    97.4    4,658,597,871      2,000    2,329,298.9    2,226,509    2,634,089  cupy_lom[...]<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       | 30        | 135.1    | 10.1        | 5062.2   || CuPy (User Cache)     | 30        | 137.2    | 10.1        | 5062.2   || CuPy (Data Type)      | 26        | 135.4    | 10.3        | 4963.9   || CuPy (Fast Math)      | 32        | 126.1    | 2.3         | 22229.7  || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       | 48        | 145.0    | 21.2        | 2396.5   || CuPy (User Cache)     | 48        | 145.7    | 21.2        | 2396.5   || CuPy (Data Type)      | 48        | 150.4    | 21.2        | 2396.5   || CuPy (Fast Math)      | 48        | 146.1    | 21.2        | 2396.5   || CuPy (Fatbin)         |           |          |             |          || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 51. CuPy implementation of Lombscargle function2. Added user cache3. Specific functions per data type4. Utilize fast_math compiler flag5. Precompile CUDA kernels into fatbins to be loaded at runtime<jupyter_code>!nvcc --fatbin -std=c++11 --use_fast_math \
    --generate-code arch=compute_70,code=sm_70 \
    --generate-code arch=compute_75,code=sm_75 \
    --generate-code arch=compute_80,code=[sm_80,compute_80] \
    _lombscargle.cu -odir .
!nsys profile -s none -o cupy_v5 -f true python3 icassp_cupy_v5.py float32 2000 0
!nsys stats --report gpukernsum --report nvtxppsum cupy_v5.qdrep<jupyter_output>Generate SQLite file cupy_v5.sqlite from cupy_v5.qdrep
Exporting 622705 events: [================================================100%]
Using cupy_v5.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v5.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum             Name           
 -------  ---------------  ---------  -----------  ---------  ---------  -------------------------
   100.0    4,493,516,577      2,001  2,245,635.5  2,137,543  2,597,897  _cupy_lombscargle_float32

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v5.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum           Range        
 -------  ---------------  ---------  -----------  ---------  ---------  ---------------------
    99.9    4,681,463,056      2,000  2,340,731.5  2,231,473  2,696,755  cupy_lombscargle_loop
    [...]<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       | 30        | 135.1    | 10.1        | 5062.2   || CuPy (User Cache)     | 30        | 137.2    | 10.1        | 5062.2   || CuPy (Data Type)      | 26        | 135.4    | 10.3        | 4963.9   || CuPy (Fast Math)      | 32        | 126.1    | 2.3         | 22229.7  || CuPy (Fatbin)         | 32        | 3.9      | 2.3         | 22229.7  || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       | 48        | 145.0    | 21.2        | 2396.5   || CuPy (User Cache)     | 48        | 145.7    | 21.2        | 2396.5   || CuPy (Data Type)      | 48        | 150.4    | 21.2        | 2396.5   || CuPy (Fast Math)      | 48        | 146.1    | 21.2        | 2396.5   || CuPy (Fatbin)         | 48        | 22.3     | 21.2        | 2396.5   || CuPy (Launch Bounds)  |           |          |             |          || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 61. CuPy implementation of Lombscargle function2. Added user cache3. Specific functions per data type4. Utilize fast_math compiler flag5. Precompile CUDA kernels into fatbins to be loaded at runtime6. Add launch bounds to CUDA kernels<jupyter_code>!nvcc --fatbin -std=c++11 --use_fast_math \
    --generate-code arch=compute_70,code=sm_70 \
    --generate-code arch=compute_75,code=sm_75 \
    --generate-code arch=compute_80,code=[sm_80,compute_80] \
    _lombscargle_lb.cu -odir .
!nsys profile -s none -o cupy_v6 -f true python3 icassp_cupy_v6.py float32 2000 0
!nsys stats --report gpukernsum --report nvtxppsum cupy_v6.qdrep<jupyter_output>Generate SQLite file cupy_v6.sqlite from cupy_v6.qdrep
Exporting 635144 events: [================================================100%]
Using cupy_v6.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v6.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum             Name           
 -------  ---------------  ---------  -----------  ---------  ---------  -------------------------
   100.0    4,481,171,056      2,001  2,239,465.8  2,143,303  2,488,841  _cupy_lombscargle_float32

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v6.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum           Range        
 -------  ---------------  ---------  -----------  ---------  ---------  ---------------------
    99.9    4,666,106,676      2,000  2,333,053.3  2,236,028  2,626,096  cupy_lombscargle_loop
    [...]<jupyter_text>Single Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 51,075.2 | 51,128.2    | 1.0      || CuPy (Baseline)       | 30        | 135.1    | 10.1        | 5062.2   || CuPy (User Cache)     | 30        | 137.2    | 10.1        | 5062.2   || CuPy (Data Type)      | 26        | 135.4    | 10.3        | 4963.9   || CuPy (Fast Math)      | 32        | 126.1    | 2.3         | 22229.7  || CuPy (Fatbin)         | 32        | 3.9      | 2.3         | 22229.7  || CuPy (Launch Bounds)  | 31        | 3.9      | 2.3         | 22229.7  || CuPy (Trig Functions) |           |          |             |          |  Double Precision|                       | Registers | JIT (ms) | Kernel (ms) | Speed Up ||:----------------------|:----------|:---------|:------------|:---------|| Scipy                 | -         | 50.799.1 | 50,806.4    | 1.0      || CuPy (Baseline)       | 48        | 145.0    | 21.2        | 2396.5   || CuPy (User Cache)     | 48        | 145.7    | 21.2        | 2396.5   || CuPy (Data Type)      | 48        | 150.4    | 21.2        | 2396.5   || CuPy (Fast Math)      | 48        | 146.1    | 21.2        | 2396.5   || CuPy (Fatbin)         | 48        | 22.3     | 21.2        | 2396.5   || CuPy (Launch Bounds)  | 48        | 22.8     | 21.2        | 2396.5   || CuPy (Trig Functions) |           |          |             |          |  GPU Scenario 71. CuPy implementation of Lombscargle function2. Added user cache3. Specific functions per data type4. Utilize fast_math compiler flag5. Precompile CUDA kernels into fatbins to be loaded at runtime6. Add launch bounds to CUDA kernels7. Use fast sin + cos functions<jupyter_code>!nvcc --fatbin -std=c++11 --use_fast_math \
    --generate-code arch=compute_70,code=sm_70 \
    --generate-code arch=compute_75,code=sm_75 \
    --generate-code arch=compute_80,code=[sm_80,compute_80] \
    _lombscargle_lb_trig.cu -odir .
!nsys profile -s none -o cupy_v7 -f true python3 icassp_cupy_v7.py float32 2000 0
!nsys stats --report gpukernsum --report nvtxppsum cupy_v7.qdrep<jupyter_output>Generate SQLite file cupy_v7.sqlite from cupy_v7.qdrep
Exporting 615887 events: [================================================100%]
Using cupy_v7.sqlite file for stats and reports.
Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/gpukernsum cupy_v7.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum             Name           
 -------  ---------------  ---------  -----------  ---------  ---------  -------------------------
   100.0    4,474,814,029      2,001  2,236,288.9  2,139,815  2,490,057  _cupy_lombscargle_float32

Exporting [/opt/nvidia/nsight-systems/2020.3.2/target-linux-x64/reports/nvtxppsum cupy_v7.sqlite] to console... 

 Time(%)  Total Time (ns)  Instances    Average     Minimum    Maximum           Range        
 -------  ---------------  ---------  -----------  ---------  ---------  ---------------------
    99.9    4,652,614,783      2,000  2,326,307.4  2,226,740  2,602,141  cupy_lombscargle_loop
    [...]
{'fraction_non_alphanumeric': 0.21507402422611036, 'fraction_numerical': 0.08241363840287125, 'mean_word_length': 1.1688071609262503, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 114, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '245715', 'license': 'Apache-2.0', 'n_tokens_mistral': 8089, 'n_tokens_neox': 6784, 'n_words': 1488, 'path': 'notebooks/raw_cupy_cuda/raw_cupy_cuda_examples.ipynb', 'repo_name': 'awthomp/cusignal-icassp-tutorial', 'size': '22290'}