Skip to content

Home

Jump to bottom
Mətanət edited this page Jul 1, 2026 · 9 revisions

Project Overview

An AI-powered communication tool that "translates" dysarthric or difficult-to-understand speech into clear, humanoid audio in real-time. It allows users with speech impairments to participate in digital spaces like Zoom meetings or voice-activated tech.

Problem Statement

People with speech disabilities (e.g., from Stroke, CP, or ALS) are often misunderstood by humans and ignored by standard AI voice assistants. This tool gives them their "digital voice" back.

Core Features (MVP)

  1. Personalized Voice Training: A module where the user records specific sample sentences to help the app learn their unique speech patterns.
  2. Speech-to-Clean-Text: High-accuracy transcription based on the user’s speech profile.
  3. Text-to-Speech (TTS) Output: Converting the cleaned text into a natural-sounding voice.

Nice-to-Have Features

  1. Video Call Integration: A virtual microphone driver so the "clean" voice can be used directly in Zoom/Teams.
  2. Common Phrases Library: Quick-access buttons for frequently used sentences to reduce fatigue.

Stretch Goals

  1. Emotion Mapping: Analyzing the user's tone to ensure the "translated" voice sounds happy, serious, or urgent based on the original input.
  2. Mobile Companion App: A portable version for face-to-face grocery shopping or appointments.

Technical Stack

  • Front-end: React (Web)
  • Back-end: Python (FastAPI) — Python is essential for AI libraries.
  • Database: MongoDB (to store user profiles and training metadata)
  • Other: Groq SDK - whisper-large-v3-turbo (for speech-to-text transcription), Groq SDK - GPT OSS 20B (LLM post-processing for punctuation and capitalization), edge-tts library (for high-quality TTS)

Project Structure

  • Audio Engine: Handles microphone input and noise reduction.
  • Training Module: Manages the dataset of the user's "difficult speech" vs. "intended text."
  • Inference Pipeline: The sequence of capturing audio -> AI processing -> Audio output.

Success Criteria

  • User can speak an unclear sentence and get a clean, highly accurate contextual correction matching their personalized speech profile
  • Total network loop latency, from the exact moment the user stops the Single Take button to the moment the clean audio is ready to play, remains under 3-4 seconds
  • The system successfully converts the refined text into a natural, flowing audio playback stream within the browser environment
  • User can successfully track, save and retrain their voice profile

Clone this wiki locally