Raspberry Pi Camera Uses Sound to Create Photos with AI

What's that sound? Let this blind camera show you!

They say a picture is worth a thousand words, but could a thousand words be worth a picture? Maker and developer Diego Trujillo Pisanty sought to find out with his Raspberry Pi-powered blind camera project. Instead of using a lens to capture light to make a picture, it listens for sound and constructs an approximation of what could be around it based on the audio detected.

The device works similarly to a regular camera in that you aim the camera at what you want to capture and press a button to generate an image. In this case, however, it has a giant horn on the front used to help amplify the capturing of sounds. Users should aim this horn in the direction of what they want to capture before pressing the button. The camera then parses the audio through an AI filter and generates an image.

According to Pisanty, he developed a custom artificial neural network (or ANN) just for this project. He trained the AI with his own models based on a set of videos taken around Mexico City.

The model was created by taking each frame of video and accompanying it with the last second of audio. This helped build an association of sound and video that the system can use to create images. Because it was trained in this way, everything it creates is loosely based on inner-city images of Mexico City.

For this project, Pisanty is using a Raspberry Pi 3B module. It would be possible to recreate it using a Raspberry Pi 4, however. As long as the Pi can handle Tensorflow, it should work. The camera also has a small screen that serves as a sort of viewfinder. It lets users know when images are processing and provides a preview of the generated images. Everything is housed inside of a custom 3D-printed shell.

The AI model used to train the blind camera was created using Python 3. It's designed to work with Tensorflow 2 and runs on the Raspberry Pi using TFLite. If you want to recreate this Raspberry Pi project or just get a closer look at how it goes together, check out the video shared by Pisanty to YouTube and read more about the project over at his website.

Join the experts who read Tom's Hardware for the inside track on enthusiast PC tech news — and have for over 25 years. We'll send breaking news and in-depth reviews of CPUs, GPUs, AI, maker hardware and more straight to your inbox.

Ash Hill is a Freelance News and Features Writer at Tom's Hardware US. She manages the Pi projects of the month and much of our daily Raspberry Pi reporting.

Raspberry Pi Night Vision Goggles Let You See at Night

Seeed Announces 10-inch reTerminal DM Raspberry Pi All-in-One

Runway's Powerful Gen-2 Text-to-Video Tool Now Available to Everyone for Free

By Ash HillJune 07, 2023

By Les PounderJune 07, 2023

By Stewart BendleJune 07, 2023

By Ash HillJune 07, 2023

By Jarred WaltonJune 06, 2023

By Aaron KlotzJune 06, 2023

By Anton ShilovJune 06, 2023

By Zhiye LiuJune 06, 2023

By Anton ShilovJune 06, 2023

By Mark TysonJune 06, 2023

By Aaron KlotzJune 06, 2023