FIELD

Embodiments of the present invention generally relate to the field of device testing. More specifically, embodiments of the present invention relate to methods and systems for simulating a device under test (DUT) during testing.

BACKGROUND

A device or equipment under test (DUT) is typically tested to determine the performance and consistency of the device before the device is sold. The device can be tested using a large variety of test cases, and the result of the test cases is compared to an expected output result. When the result of a test case does not match the expected output value, debugging is performed in an attempt to identify and correct any defects that result from the device.

A DUT is usually tested by automatic or automated test equipment (ATE), which may be used to conduct complex testing using software and automation to improve the efficiency of testing. The DUT may be a memory device or component that is intended to be integrated into a final product, such as a computer or other electronic device. Existing system level test software used in the NAND flash device development process specifically requires software to be tested and debugged after the development of the DUT is nearly complete. Even in an urgent case, the test program and the software product typically wait for the storage device provided by the customer to be ready for development and debugging, and therefore the productivity of the testing software development is constrained by the degree to which the device has been completed, which can lead to missed deadlines and products that must be launched before they are thoroughly tested.

Moreover, existing approaches to DUT testing are only able to modify certain properties of the DUT at a given time, and certain variations can only be tested in isolation. What is needed is a method for performing DUT testing that does not rely on the completion of the development of the device, that can improve the efficiency of production development by testing and debugging the testing software before the actual DUT is available, and that can be customized to test a vast number of variables to enhance the robustness and scalability of the testing.

SUMMARY

Embodiments of the present invention provide systems and methods for performing DUT testing that does not depend on the development of the device being complete, that can improve the efficiency of production development by testing and debugging the device simulated in software before the actual DUT is available, and that can be customized to test a vast number of variables to enhance the robustness and scalability of the testing. The testing can be performed at the API level according to the specification of the DUT. By performing testing and debugging earlier in the development process, less debugging and redesign is required when the actual device is available for testing, leading to an earlier delivery date of the device.

According to one embodiment, a method of simulating a memory-based communication protocol for testing a simulated device is disclosed. The method includes storing data in a host data buffer, where the host is a tester system, and where the host data buffer is implemented in a shared memory space, executing instructions of a tester system program to store a command in the shared memory space using a data structure including an index, an ID, and a memory location, executing instructions of a DUT simulation to read the command from the shared memory space, access a block of the shared memory space to perform an operation defined by the ID, where a location of the block of shared memory is indicated by the index, and send a completion indicator to the tester system program after the operation is complete.

According to some embodiments, the ID indicates that the command includes a program operation, where the access is reading the block to obtain data for writing to the memory location, where the operation includes performing the program operation, and further including the DUT simulation writing the data to a disk file responsive to the command.

According to some embodiments, the ID indicates that the command includes a read operation, where the access is storing data read from the memory location, where the operation includes performing the read operation, and further including executing instructions of the DUT simulation to access one or more disk files to read data at the memory location and store to the host data buffer in the block of shared memory.

According to some embodiments, the instructions of the tester system program and the instructions of the DUT simulation are executed as separate processes managed by a Linux-based operating system.

According to some embodiments, the method includes verifying the data after the operation to test the simulated device for errors after the program operation.

According to some embodiments, the verifying includes comparing the data to an expected result.

According to some embodiments, the command is stored in a base address register of the simulated device.

According to some embodiments, the instructions of tester program and the instructions of the DUT simulation are executed by a same processor.

According to some embodiments, the shared memory space includes memory-mapped files that emulate physical memory.

According to some embodiments, the memory-mapped files include files mapped using mmap.

According to some embodiments, the shared memory space includes memory-mapped files that emulate NAND flash memory.

According to some embodiments, the memory-based communication protocol utilizes a PCIe communication bus protocol.

According to another embodiment, a method of simulating a memory-based communication protocol for testing program operations of a simulated device is disclosed. The method includes storing data in a host data buffer, where the host data buffer is implemented in a shared memory space, executing instructions of a tester system program to store a program operation in the shared memory space using a data structure including a first index, an ID of the program operation, and a memory location, and executing instructions of a DUT simulation to, access the program operation in the shared memory space, read data from a block of the shared memory space defined by the first index, write the data to one or more disk files at the memory location, and send a first completion indicator to the tester system program after writing the data to the memory location. The method further includes executing instructions of the tester system program to store a read operation in the shared memory space using a data structure including a second index, an ID indicating that the read operation, and the storage location, and executing instructions of the DUT simulation to read the data from the memory location, write the data to a block of the host data buffer according to the second index, and send a second completion indicator to the tester system program after writing the data to the block defined by the second index.

According to some embodiments, the method includes verifying the data after the read operation to test the simulated device for errors, where the verifying includes comparing the data to an expected result.

According to some embodiments, the first completion indicator is stored in a queue managed by the tester system program.

According to some embodiments, the memory location references a memory location of a disk, and where the memory location is implemented as a disk file to simulate flash memory.

According to a different embodiment, a system for simulating a device under test is disclosed. The system includes a processor executing a Linux-based operating system and a memory in communication with the processor that provides a shared memory space for a tester system program and a DUT simulation program. Instructions of the tester system program and the DUT simulation program are executed by the processor, and the processor performs a method of simulating a memory-based communication protocol for testing a simulation of the device under test. The method includes storing data in a host data buffer, where the host is a tester system, and where the host data buffer is implemented in a shared memory space, executing instructions of the tester system program to store a command in the shared memory space using a data structure including an index, an ID, and a memory location, and executing instructions of the DUT simulation program to read the command from the shared memory space, access a block of the shared memory space to perform an operation defined by the ID, where a location of the block of shared memory is indicated by the index, and send a completion indicator to the tester system program after the operation is complete.

According to some embodiments, the method includes verifying the data after the operation to test the simulated device, and where the verifying includes comparing the data to an expected result after the operation.

According to some embodiments, the ID indicates that the command includes a program operation, where the operation includes performing the program operation, and where the DUT simulation program writes the data from a block of the shared memory defined by the index to a disk file responsive to the command.

According to some embodiments, the ID indicates that the command includes a read operation, where the operation includes performing the read operation, and further including executing instructions of the DUT simulation program to access one or more disk files to read data and to write the data from the disk file to the host data buffer at a location defined by the index.

According to some embodiments, the instructions of the tester system program and the instructions of the DUT simulation program are executed as separate processes and where the separate processes are managed by the Linux-based operating system.

According to some embodiments, the memory-based communication protocol is a PCIe communication bus protocol.

According to a different embodiment, a system for performing memory-based communication with a simulated device is disclosed. The system includes a processor, a memory coupled to the processor, and a tester system process executed by the processor, where the tester system process is assigned a shared memory address space including a host data buffer, and where the tester system process stores data in the host data buffer and issues commands to a device process by storing commands to the shared memory address space. The device process is executed by the processor for simulating a device, and where the device process includes a controller operable to read data from disk files, write data to the disk files, read the commands from the shared memory address space, and perform operations according to the commands. The commands include a program operation that reads indexed data from the host data buffer and writes the indexed data to the disk files, a read operation that reads data from the disk files and writes the data to the host data buffer according to an index value, and an erase operation that erases one or more of the disk files.

According to some embodiments, the tester system process simulates a tester system and the device process simulates a device under test.

According to some embodiments, the shared memory address space simulates a PCIe bus between the tester system and the device under test.

