Generative AI introduced on Raspberry Pi 5

A little over a year ago, Raspberry Pi introduced the AI HAT+, an add-on board for Raspberry Pi 5 featuring the Hailo-8 (26-TOPS variant) and Hailo-8L (13-TOPS variant) neural network accelerators. With all AI processing happening directly on the device, the AI HAT+ provided true edge AI capabilities to its users, supplying them with data privacy and security while eradicating the necessity to subscribe to expensive cloud-based AI services.

While the AI HAT+ provides best-in-class acceleration for vision-based neural network models, including object detection, pose estimation, and scene segmentation, it cannot run the increasingly popular generative AI (GenAI) models.

Featuring the new Hailo-10H neural network accelerator, the AI HAT+ 2 delivers 40 TOPS (INT4) of inferencing performance, ensuring generative AI workloads run smoothly on Raspberry Pi 5. Performing all AI processing locally and without a network connection, the AI HAT+ 2 operates reliably and with low latency, maintaining the privacy, security, and cost-efficiency of cloud-free AI computing that the company introduced with the original AI HAT+.

Unlike its predecessor, the AI HAT+ 2 features 8GB of dedicated onboard RAM, allowing the accelerator to efficiently handle much larger models than previously possible. This, along with an updated hardware architecture, enables the Hailo-10H chip to accelerate LLMs, VLMs, and other generative AI applications.

For vision-based models, such as Yolo-based object recognition, pose estimation, and scene segmentation, the AI HAT+ 2's computer vision performance is broadly equivalent to that of its 26-TOPS predecessor, thanks to the onboard RAM. It also benefits from the same tight integration with the company's camera software stack (libcamera, rpicam-apps, and Picamera2) as the original AI HAT+. For users already working with the AI HAT+ software, transitioning to the AI HAT+ 2 is mostly seamless and transparent.