Turning a Furby into an AI-Powered Talking Robot - No Internet Required скачать в хорошем качестве

Turning a Furby into an AI-Powered Talking Robot - No Internet Required

A vintage 1998 Furby was customized into an AI conversation robot, with its original components controlled by an ESP32 microcontroller. The Furby's original analog microphone was replaced with an upgraded I2S microphone module, while the speaker, optical rotary encoder (which includes a phototransistor and an infrared LED), and camshaft position sensor were retained as original components. It’s a fun project, and I’m excited to share it with you. Whether you're a tech enthusiast, a DIY electronics fan, or just someone who loves Furbies, I think you’ll enjoy this project. Feel free to leave your comments, share your thoughts, and if you enjoy the video, consider subscribing for more projects like this. I would be thrilled if you share this video with fellow tech enthusiasts and Furby fans. === 🔖 Chapters === 0:00 Intro 0:36 Building the Foundation: Core Speech Capabilities 1:12 Server Logs: Insights into Speech Processing 1:35 Listening Without Limits: Handling Long User Input 3:00 Capturing the Voice: Voice Activity Detection 3:26 Interrupt Handling: Silencing Furby 3:36 GPU Specifications and Error Handling 4:12 Maximizing Battery Life: Deep Sleep Mode 5:05 GPT-4o Voice Chat: Advanced Voice Interaction 5:24 Outro === 🛠️ How it works === Upon detecting voice activity, the ESP32 initiates recording the voice data to memory while continuously sending the data to the local hosted server. Once the server receives all the data, it begins transcribing the data and passes the transcription to a Large Language Model (LLM). The generated response is then converted into Furby's speaking voice and sent back to the ESP32. The only audio data saved on the ESP32 are yawning and snoring sounds. === 📜 Backstroy === While exploring various AI models and their applications, I came up with an idea for building an audio-based chatbot. I was amazed by incredible AI-powered robots like Ameca, Sophia and Pepper-kun. Although I like Amazon Alexa, I wanted a robot that would give me the experience of talking and engaging in a conversation. I am a big fan of circuit-bending projects, especially involving Furbies and Big Mouth Billy Bass. Some people have got Furbies to talk using ChatGPT, and it was absolutely brilliant. Furbies are cute, playful, humorous, and iconic. Because it’s a Furby, any LLM hallucinations can be forgiven, and I can stay alert for false information. === 🎁 Furby === Release date: 1998 Developer: Tiger Electronics (a division of Hasbro) Distributor (United States/International): Hasbro Distributor (Japan): TOMY Main Developers: Dave Hampton-san and Caleb Chung-san Voice actors: Tony Pope-san (English), Taeko Kawata-san (Japanese) === 🧩 Electronic Components === Microcontroller: Espressif Systems ESP32 Microphone: INMP441 Amplifier: MAX98357 Motor Driver: DRV8833 === 🚀 Tech Stack === (Special Thanks To) llama.cpp (MIT) [https://github.com/ggerganov/llama.cpp] Dolphin 2.6 Phi 2 (MIT) [https://huggingface.co/TheBloke/dolph...] Faster Whisper (MIT) [https://github.com/SYSTRAN/faster-whi...] ESP32 Firmware: C++, PlatformIO, ESP-IDF (Apache-2.0), Arduino-ESP32 (LGPL-2.1) Server Program: Python, Shell script (For daemonization) IDE: vim I'm not releasing the source code for now. I want to encourage others to work on similar projects and experience the satisfaction of building it themselves. === 🎤 Voices === (Special Thanks To) The AI-powered Furby's voice data (Attribution Required): Jenny [https://github.com/dioco-group/jenny-...] The AI-powered Furby's voice synthesizer: coqui-ai/TTS (MPL-2.0) [https://github.com/coqui-ai/TTS] The Creator's voice synthesizer: myshell-ai/OpenVoice (MIT) [https://github.com/myshell-ai/OpenVoice] The Creator's voice data: (self-recorded) My accent and recording inexperience might distract listeners. I replaced my voice with a clearer AI voice based on my own for a better experience. === 🎵 Background Music: Sourced from YouTube Audio Library === "Morning Mood" by Edvard Grieg "Bach Cello Suite No. 1 in G Major, Prelude" performed by Cooper Cannell === 🎥 Video Production Equipment === Canon EOS 60D with Magic Lantern Canon EF-S28-55mm f3.5-5.6 II (f/3.5, 28.0mm, 1/60, ISO320) BOYA BY-MM1 DaVinci Resolve 18 === 🏷️ Tags === #furby #esp32 #llm #ai