Conversational Voice AI/● Technical Preview / R&D

Ada

Real-Time Turkish AI Voice Sales Assistant

Voice Providers
OpenAI + Local
Local Turn
~5.9 s
STT Latency
−58.8%
Language
tr-TR
01 // CONTEXT & PROBLEM

The Operational Problem

Context & Domain

Most production voice AI is English-first and cloud-only. For Turkish businesses two things matter: natural Turkish speech, and the option to keep customer conversations off third-party APIs. Ada was built in staged increments (v0.1 → v0.2.2) to test both, as a browser-only technical preview before any telephony work.

Overview

A sales conversation is more than chat: the assistant has to move through stages, remember what the prospect said and score the lead, consistently, whichever speech and language engine is underneath. Small local models are far less reliable at strict instruction-following and tool calling than hosted realtime models, and a turn-based local pipeline adds latency at every stage.

02 // TOPOLOGY & ARCHITECTURE

System Architecture

Ada separates the sales logic from the voice engine. A provider-independent Sales Brain (stage machine, lead context, lead scoring) sits behind an IVoiceAgent abstraction with two implementations: an OpenAI Realtime agent that reports sales data through a native tool call, and a local agent that chains in-process Silero VAD, faster-whisper STT, Qwen3 8B on Ollama and Piper TTS, reporting sales data through a validated JSON envelope. The browser speaks one event protocol and needed zero changes when the local provider was added.

SYSTEM_DATA_FLOW_MAP
┌───────────────────────────────────────────────────────────────────────────┐
│ BROWSER CLIENT  (static HTML/JS, no build step)                           │
│ AudioWorklet mic capture · PCM16 mono 24 kHz both directions              │
│ Live pipeline stages: Mic → WS → VAD → STT → LLM → TTS                    │
└─────────────────────────────────────┬─────────────────────────────────────┘
                                      │ same-origin WebSocket  /ws/voice
                                      ▼
┌─────────────────────────────────────┴─────────────────────────────────────┐
│ ADA.API  (ASP.NET Core 8)                                                 │
│ ├── IVoiceAgent: provider picked by config, Local → OpenAI fallback       │
│ └── SALES BRAIN  (provider-independent)                                   │
│     stage machine · lead context · lead scoring                           │
│     one four-part prompt: identity · sales · behaviour · compliance       │
└─────────────────┬───────────────────────────────────────┬─────────────────┘
                  │                                       │
           OpenAI │                                       │ Local (no cloud)
                  ▼                                       ▼
┌─────────────────┴─────────────────┐   ┌─────────────────┴─────────────────┐
│ OPENAI REALTIME API               │   │ Silero VAD    in-process (ONNX)   │
│ streaming speech ⇄ speech         │   │ faster-whisper STT  (FastAPI)     │
│ sales data via tool call          │   │ Qwen3 8B on Ollama → JSON reply   │
│ update_sales_context              │   │ Piper TTS  (persistent process)   │
└───────────────────────────────────┘   └───────────────────────────────────┘
Browser Client01

Captures microphone audio with AudioWorklet, plays PCM16 replies, and shows live pipeline stages and per-turn latency

HTML/JS (no build step)AudioWorkletWebSocket
Ada.Api02

Same-origin WebSocket relay, health checks and provider selection with fail-fast configuration validation; API keys never reach the browser

ASP.NET Core 8WebSocketsSerilog
Sales Brain (Ada.Core)03

Stage machine, lead context and lead scoring shared by both providers, driven by one four-part system prompt

C#xUnit
Local AI Stack04

Silero VAD in-process via ONNX Runtime; a FastAPI sidecar for faster-whisper and a persistent Piper process; Qwen3 8B served by Ollama

ONNX RuntimePython / FastAPIfaster-whisperPiperOllama
03 // TRADEOFF ANALYSIS

Engineering Decisions

One Sales Brain, Two Voice Providers

Rationale

Sales behaviour must not depend on which speech engine is active; otherwise every provider change means re-testing the whole conversation.

Trade-offs Considered

Each provider maps its own protocol onto Ada's internal event model, but the sales logic is tested once and the browser client stays provider-agnostic.

JSON Envelope Instead of Local Tool Calling

Rationale

Tool-calling reliability of a small local model is not guaranteed.

Trade-offs Considered

The local LLM always replies with {response, context_updates}. Every field is trimmed, length-capped and validated before it reaches the tracker, and a malformed reply degrades to plain speech instead of ending the session.

Model Chosen by Live Conversations, Not Benchmarks

Rationale

qwen3:1.7b was about 2x faster than qwen3:8b in a scripted benchmark.

Trade-offs Considered

In a live five-turn conversation it repeated identical replies to different inputs, so qwen3:8b stayed the default: about 2 s slower per turn, but coherent. qwen3:4b leaked chain-of-thought and phi3.5 produced broken Turkish.

04 // COMPLEXITY RESOLUTION

Technical Challenges & Solutions

Silent Voice Activity Detection Failure

Silero VAD returned near-zero speech probability for real speech, without raising any error.

↳ Engineered Solution

Current Silero ONNX builds need each 512-sample call prefixed with a 64-sample rolling context window from the previous call. Ada's .NET implementation adds it and was cross-checked against the reference Python wrapper.

Per-Turn Latency of a Local Pipeline

The first local version took about 7.3 s per turn in steady state.

↳ Engineered Solution

Kept one Piper process alive instead of spawning one per utterance (TTS −32.8%), moved STT from Whisper small to base after benchmarking (−58.8%), and added per-stage timing logs and a live latency panel. Steady state dropped to about 5.9 s per turn (−19%).

05 // CODE & STACK

Technology Stack

Backend
  • • ASP.NET Core 8
  • • C#
  • • WebSockets
  • • Serilog
Cloud Voice
  • • OpenAI Realtime API
Local Voice
  • • Silero VAD
  • • faster-whisper
  • • Qwen3 8B (Ollama)
  • • Piper TTS
Sidecar
  • • Python
  • • FastAPI
  • • uvicorn
Testing
  • • xUnit

Key Implementation Highlights

  • One PCM16 24 kHz binary and JSON event protocol for both providers; the local agent resamples internally
  • Automatic Local → OpenAI fallback when the local stack is down, always logged, never silent
  • /health/local-ai endpoint reporting LLM, STT and TTS availability separately
  • No audio persisted anywhere; local STT audio stays in memory for a single utterance
06 // PRODUCTION UI & SCREENSHOT EVIDENCE

System Diagram & Measurements

System topology with real values measured in the development environment.

https://ada.mbftech.internal
SYSTEM TOPOLOGY // VOICE PIPELINE
Ada Voice Pipeline: Two Providers, One Sales Brain
Click to inspect full resolution

Ada Voice Pipeline: Two Providers, One Sales Brain

Measured local turn on a 6-core CPU with an RX 580: STT 824 ms, LLM 4,324 ms, TTS 739 ms, about 5.9 s per turn in steady state.

FIGURE 01
07 // VERIFICATION & RESULTS

Results & Current State

Technical Preview (v0.2.2)

End-to-end Turkish voice sales conversations work in the browser on both providers. Telephony (SIP/Asterisk), CRM and calendar integration are deliberately out of scope for this stage.

↳ Local steady-state turn: ~5.9 s (STT 824 ms · LLM 4,324 ms · TTS 739 ms)
↳ −19% total turn latency compared with the v0.2.1 baseline
↳ Four local LLMs compared on speed, JSON reliability and Turkish quality

Key Architectural Takeaways

  • 01.Keep business logic independent of the AI vendor; swapping engines should not mean rewriting the product.
  • 02.Small local models need defensive parsing, not trust: validate every field the model returns.
  • 03.Measure in real conversations; the fastest model in a benchmark was not the best one in a live call.