We are excited to present Tetris, a project which demonstrates the edge computing capabilities of hardware accelerators using the IMX274 camera sensor. As a start, this article will go over the basics of this project which runs on a Zynq UltraScale+ MPSoC device on the ZCU104 platform. Also note that Tetris is similar to Xilinx’s reVISION but that no embedded OS is required here, this project is bare metal.
The image sensor output is displayed on the HDMI after being processed by the hardware accelerator implementing a common edge detection algorithm or in passthrough mode to show the captured image directly. The following image shows both views of the same scene.
Architecture
As mentioned above, this project starts from Xilinx’s ZCU102 MIPI CSI-2 RX subsystem example design. The image sensor (IMX274-MIPI-FMC) is attached to the ZCU104 using the FMC LPC connector.
The architecture for this project is shown in the illustration below. The data input pipeline records 4K video data into the shared memory. The data input pipeline is implemented using Xilinx’s pixel-level video IPs to debayer the raw bayered image and transform it into a RGB image.
The accelerator is implemented using a memory-to-memory (m2m) pipeline. A common image edge detector is implemented using OpenCV for Vivado HLS. The hardware accelerator takes images which were written to data by the data input pipeline, processes them and writes them back to memory.
The data readout pipeline is created using Xilinx HDMI Tx subsystem for live video visualization. It reads the processed frames from the shared memory and outputs them.
Hardware requirements
The following hardware will be required for this project to work:
ZCU104 platform (rev 1.0 was used for this project)
A IMX274-MIPI-FMC camera
A HDMI cable
A micro USB cable
Code
The code for this project is freely available at Six Metrix GitLab page:
About Six Metrix
With several years of experience in FPGA and digital signal processing design, Six Metrix will see to it that your hardware acceleration application performs as expected.
Contact us for more information on how we can support you!