<jupyter_start><jupyter_text>TSIM: Cycle Accurate Simulation for Custom HW in TVMTSIM uses [Verilator](https://www.veripool.org/wiki/verilator) to integrate accelerators, including VTA, into TVM and provides flexibility in the hardware language used to implement them.For example, one could use OpenCL, C/C++ or Chisel3 to describe a VTA design that would eventually be compiled down to Verilog, since it is the standard input language for FPGA/ASIC tools.Additionally, Verilator supports the Direct Programming Interface (DPI), which is part of the Verilog standard and provides a mechanism to support foreign programming languages.We leveraged these features available in Verilator and created DPI modules that provide interfaces to hardware and software. The following figure describes at higher level what TSIM can do. TSIM hardware interfaceNormally, a hardware accelerator interface can be simplified in two main components, one for control and another for data. The control interface is driven by a host CPU, whereas the data interface is connected to either external memories (DRAM) or internal memories in the form of scratchpads or caches. Currently, we support a shared-memory model between the host and accelerator. This implies that the host is in charge of passing values and addresses or pointers, including data and code if needed, to the accelerator.There are two hardware modules written in Verilog implementing these two interfaces called `VTAHostDPI.v` and `VTAMemDPI.v`. Accelerators implemented in Verilog can use these modules directly. However, we also provide Chisel3 wrappers `BlackBox` for accelerators described in this language.The following block diagram shows how to wire-up an accelerator to the host and memory interface. TSIM software interfaceThe software DPI module allows users to write drivers to handle the accelerator. For example, some accelerators may need to know memory addresses before issuing data or code requests to memory. This module provides this support via functions that write and read register in the accelerator such as:```c// Read an accelerator registeruint32_t ReadReg(int addr);// Write an accelerator registervoid WriteReg(int addr, uint32_t value);```In addition to accessing registers, users can manage the hardware simulation thread with launch and finish functions.```c// Launch hardware simulation until accelerator finishes or reach max_cyclesvoid Launch(uint64_t max_cycles);// Finish hardware simulationvoid Finish();```  Setup  Get TVM<jupyter_code>try:
  import google.colab
  IN_COLAB = True
except:
  IN_COLAB = False

if IN_COLAB:
    ! gsutil cp "gs://tvm-fcrc-binariesd5fce43e-8373-11e9-bfb6-0242ac1c0002/tvm.tar.gz" /tmp/tvm.tar.gz
    ! mkdir -p /tvm
    ! tar -xf /tmp/tvm.tar.gz --strip-components=4 --directory /tvm
    ! ls -la /tvm
    ! bash /tvm/package.sh
    # Add TVM to the Python path.
    import sys
    sys.path.append('/tvm/python')
    sys.path.append('/tvm/topi/python')
    sys.path.append('/tvm/nnvm/python')
    sys.path.append('/tvm/vta/python')
else:
    print("Notebook executing locally, skipping Colab setup ...")<jupyter_output><empty_output><jupyter_text>Vanilla acceleratorWe built a vanilla accelerator to showcase how TSIM works in TVM. The vanilla accelerator is implemented in two hardware backends, including Verilog and Chisel3, to demonstrate the flexibility of this infrastructure and help users understand how to add accelerators written in Verilog and "hardware languages" that can generate Verilog. The accelerator performs the operation **A = B + C**, where **A** and **B** are 1-D tensors and **C** just a constant. The following figure shows the hardware architecture.  Verilog backend  Source files<jupyter_code>%%bash
tree -C /tvm/vta/apps/tsim_example/hardware/verilog<jupyter_output><empty_output><jupyter_text>How to build<jupyter_code>%%bash
cd /tvm/vta/apps/tsim_example/hardware/verilog
make 2> /dev/null<jupyter_output><empty_output><jupyter_text>Chisel3 backend  Source files<jupyter_code>%%bash
tree -C /tvm/vta/apps/tsim_example/hardware/chisel/src<jupyter_output><empty_output><jupyter_text>How to build<jupyter_code>%%bash
cd /tvm/vta/apps/tsim_example/hardware/chisel
make 2> /dev/null<jupyter_output><empty_output><jupyter_text>Software driver  Source filesThe software driver for the accelerator can be found at `/tvm/vta/apps/tsim_example/src/driver.cc`. Here are some of the most important bits of this driver.The following method in the `Device` class execute all the steps during a accelerator run. ```cpp  uint32_t Run(uint32_t c, uint32_t length, void* inp, void* out) {    uint32_t cycles;    this->Launch(c, length, inp, out);    cycles = this->WaitForCompletion();    dpi_->Finish();    return cycles;  }```Also, the driver is registered as TVM function, so it can be invoked in Python.```cppTVM_REGISTER_GLOBAL("tvm.vta.driver").set_body([](TVMArgs args, TVMRetValue* rv) {    Module dev_mod = args[0];    DLTensor* A = args[1];    DLTensor* B = args[2];    Device dev_(dev_mod);    uint32_t cycles = dev_.Run(static_cast(args[3]), A->shape[0], A->data, B->data);    *rv = static_cast(cycles);});```  How to build<jupyter_code>%%bash
cd /tvm/vta/apps/tsim_example
make driver<jupyter_output><empty_output><jupyter_text>Create a test<jupyter_code>import tvm
import numpy as np
import ctypes
def tsim(hw_backend):
  def load_dll(dll):
    try:
      return [ctypes.CDLL(dll, ctypes.RTLD_GLOBAL)]
    except OSError:
      return []

  def run(a, b, c):
    if hw_backend in ["chisel"]:
      hw_lib = '/tvm/vta/apps/tsim_example/hardware/chisel/build/libhw.so'
    else:
      hw_lib = '/tvm/vta/apps/tsim_example/hardware/verilog/build/libhw.so'
    sw_lib = '/tvm/vta/apps/tsim_example/build/libsw.so'
    load_dll(sw_lib)
    f = tvm.get_global_func("tvm.vta.driver")
    m = tvm.module.load(hw_lib, "vta-tsim")
    cycles = f(m, a, b, c)
    print("cycles:{}".format(cycles))
  return run
def test_accel(n, c, hw_backend):
    ctx = tvm.cpu(0)
    rmax = 64
    a = tvm.nd.array(np.random.randint(rmax, size=n).astype("uint64"), ctx)
    b = tvm.nd.array(np.zeros(n).astype("uint64"), ctx)
    f = tsim(hw_backend)
    f(a, b, c)
    for i, (x, y) in enumerate(zip(a.asnumpy(), b.asnumpy())):
      print("i:{0:<4} c:{1:<4} a:{2:<4} b:{3:<4}".format(i, c, x, y))<jupyter_output><empty_output><jupyter_text>Run Accelerator in Verilog<jupyter_code># the number of clock cycles is directly proportional to the size of the array

n = 10 # size of array
c = 2 # constant value

# run test
test_accel(n, c, "verilog")<jupyter_output><empty_output><jupyter_text>Run Accelerator in Chisel<jupyter_code># the number of clock cycles is directly proportional to the size of the array

n = 5 # size of array
c = 2 # constant value

# run test
test_accel(n, c, "chisel")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09844938560561732, 'fraction_numerical': 0.010093622001170276, 'mean_word_length': 4.730930427493713, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 45, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '475129', 'license': 'Apache-2.0', 'n_tokens_mistral': 2264, 'n_tokens_neox': 2151, 'n_words': 746, 'path': 'notebook/05_TVM_Tutorial_TSIM.ipynb', 'repo_name': 'deepakbabel23/tutorial', 'size': '6836'}