IELS2003 Object detection for the blind and visually impaired
Wearable aid that finds fruit with camera-based object detection and shows the direction with a vibration belt and Norwegian speech. Group project of three.
The problem
People who lose their sight as adults have visual memories but lack access to information in real time. The project started from one scenario: making a fruit salad on your own. The need was mapped through conversations with two experts.
The solution
- A Raspberry Pi 5 runs YOLOv8n, which recognizes apple, banana and orange in real time.
- The position is sent over UDP to an ESP32-C3 that drives seven vibration motors in a belt, so the user feels the direction without using their hearing.
- Norwegian speech synthesis (Piper) announces detections, and the user can ask questions with a button via Whisper and Claude Haiku.
- Home-built battery pack from reused 18650 cells, and 3D-printed enclosures.
My role
I was responsible for the software and the integration between the modules: object detection on the Raspberry Pi 5, speech to text, text to speech, API connection and workflow with GitHub.
Iteration
The first concept was glasses with a camera and speech in the ear. The conversations showed that blind people do not necessarily turn their head toward what they are looking for, and that constant speech takes up their hearing. The solution was moved to the chest with haptic feedback.
Results and limitations
- 6 to 8 frames per second, confidence of 0.7 to 0.9 in good light, and answers to voice commands in 1 to 2 seconds.
- No formal user tests with the target group were carried out. That is the most important further work.
- No distance sensor and no AI accelerator, because the components were missing.
