Audio Protection & Error Recovery

Invox Medical's streaming transcription client implements enterprise-grade audio protection mechanisms to ensure zero data loss even during network failures, browser crashes, or service disruptions. This guide explains the technical architecture that makes audio transcription reliable and resilient.

Overview

The transcription client employs multiple layers of protection:

  • Persistent Storage using IndexedDB for browser-native durability
  • Circuit Breaker Pattern to prevent cascading failures
  • Intelligent Retry Logic with exponential backoff
  • Dual Acknowledgment System for efficient confirmation
  • Automatic Recovery from network interruptions
  • Queue Management with flow control
  • Session Restoration after crashes or page refreshes

Storage Mechanisms

IndexedDB Integration

The client uses IndexedDB through the idb-keyval library for persistent storage that survives page refreshes and browser restarts.

What Gets Stored

Every audio chunk is immediately persisted with the following structure:

interface StoredAudioChunk {
  id: string; // Unique identifier (timestamp_sequence)
  sessionId: string; // UUID session identifier
  timestamp: number; // Creation timestamp
  sent: boolean; // Has been sent to server
  acknowledged: boolean; // Server confirmed receipt
  retryCount: number; // Number of retry attempts (max: 3)
  sequenceNumber: number; // Monotonic sequence for ordering
  bytesB64: string; // Base64-encoded PCM audio (16-bit, 16kHz)
  durationMs: number; // Audio duration in milliseconds
}

Storage Keys

// Audio chunks: audio_chunk_{sessionId}_{chunkId}
const AUDIO_STORAGE_PREFIX = "audio_chunk_";

// Session metadata: transcription_session_{sessionId}
const SESSION_STORAGE_KEY = "transcription_session_";

// Maximum chunks before cleanup
const MAX_STORED_CHUNKS = 1000;

When Storage Occurs

  1. Immediate Persistence - When audio chunk is created (before sending):
// Audio buffer reaches threshold (0.6 seconds)
await this.storeAudioChunk(audioChunk); // Persist first
this.enqueueAndProcess(audioChunk); // Then enqueue
  1. Update on Send - When chunk is transmitted:
await this.markChunkAsSent(chunkId); // Updates sent: true
  1. Update on Acknowledgment - When server confirms receipt:
await this.markChunkAsAcknowledged(chunkId); // Updates acknowledged: true

Storage Cleanup

To prevent unlimited storage growth, the client automatically cleans old chunks:

// Cleanup triggers when storage exceeds MAX_STORED_CHUNKS (1000)
private async cleanOldChunks() {
  // Priority: Delete acknowledged chunks first
  // Then delete oldest chunks by timestamp
  // Keeps 100 buffer slots available
}

Cleanup Strategy:

  • Prioritizes deletion of acknowledged chunks (already processed)
  • Preserves unacknowledged chunks (not yet confirmed)
  • Maintains 100-slot buffer for new chunks
  • Sorted by acknowledgment status, then by timestamp

Why IndexedDB?

Advantages over localStorage:

  • Larger capacity: 50MB+ vs 5-10MB
  • Asynchronous: Non-blocking operations
  • Native objects: No JSON serialization overhead
  • Transactional: Consistent state guarantees
  • Persistent: Survives crashes and restarts

Error Handling Strategies

Circuit Breaker Pattern

The client implements a circuit breaker to prevent cascading failures during service outages.

Configuration

const circuitBreaker = new CircuitBreaker({
  failureThreshold: 5, // Open after 5 consecutive failures
  timeout: 60000, // Wait 60s before retry
  name: "TranscriptionClient",
});

Circuit States

1. CLOSED (Normal Operation)

  • All operations allowed
  • Failures are counted
  • Transitions to OPEN after 5 failures

2. OPEN (Service Degraded)

  • Operations immediately rejected
  • Transcription paused automatically
  • User notified: "Service temporarily unavailable"
  • After 60 seconds, transitions to HALF_OPEN

3. HALF_OPEN (Testing Recovery)

  • Single test operation allowed
  • Success → CLOSED (full recovery)
  • Failure → OPEN (back to degraded)

Protected Operations

The circuit breaker wraps critical operations:

// WebSocket connection
await circuitBreaker.execute("connect", async () => {
  // Connection logic
});

// Audio chunk transmission
await circuitBreaker.execute("sendChunk", async () => {
  // Send logic
});

// Transcription retrieval
await circuitBreaker.execute("getTranscription", async () => {
  // Fetch logic
});

Connection Error Handling

WebSocket Failures:

// Tracks failure patterns
interface ConnectionMetrics {
  consecutiveFailures: number;
  lastFailureTime: number;
  networkLatency: number;
  throughput: number;
}

// On each failure
this.connectionMetrics.consecutiveFailures++;
this.connectionMetrics.lastFailureTime = Date.now();

Poor Network Adaptation:

// Automatically adjusts heartbeat interval
private handlePoorNetworkConditions() {
  // Doubles heartbeat interval (max: 30s)
  this.heartbeatMs = Math.min(this.heartbeatMs * 2, 30000)
}

Recovery Processes

Session Recovery on Reconnection

When the WebSocket reconnects, the client automatically recovers unsent audio:

private async recoverStoredChunks() {
  // 1. Load all chunks from IndexedDB for current session
  const chunks = await loadChunksFromStorage(this.sessionId)

  // 2. Filter chunks that need sending
  const pendingChunks = chunks.filter(chunk =>
    !chunk.sent ||
    (!chunk.acknowledged && chunk.retryCount < MAX_RETRY_ATTEMPTS)
  )

  // 3. Sort by sequence number to maintain order
  pendingChunks.sort((a, b) => a.sequenceNumber - b.sequenceNumber)

  // 4. Re-enqueue all pending chunks
  for (const chunk of pendingChunks) {
    this.enqueue(chunk)
  }

  // 5. Resume processing
  this.processQueueIfReady()
}

Recovery Triggers:

  • After successful WebSocket connection
  • Automatically restores unsent chunks
  • Resumes unacknowledged chunks (within retry limit)

Resending Pending Chunks

After network recovery, in-flight chunks are automatically resent:

private async resendPendingChunks() {
  let recoveredDuration = 0

  // Move all in-flight chunks back to queue
  for (const [chunkId, inFlightData] of this.inFlight) {
    this.clearAcknowledmentTimeout(chunkId)
    recoveredDuration += inFlightData.chunk.durationMs
    this.sendQueue.unshift(inFlightData.chunk)  // Priority: front
  }

  // Reset in-flight tracking
  this.inFlight.clear()
  this.inFlightAudioDurationMs -= recoveredDuration

  // Resume sending
  this.processQueueIfReady()
}

Triggered By:

  • Network reconnection (online event)
  • WebSocket state change to connected
  • Recovery from disconnected/error state

Exponential Backoff Reconnection

Failed connections retry with increasing delays:

private async reconnectLoop() {
  // Add jittered delay (prevents thundering herd)
  await sleep(jitter(this.backoffMs))

  // Exponential backoff: 500ms → 1s → 2s → 4s → ... → 30s
  this.backoffMs = Math.min(maxBackoffMs, this.backoffMs * 2)

  return this.connect()
}

// Jitter adds randomness (±20%) to prevent synchronized retries
function jitter(ms: number) {
  const r = Math.random() * 0.4 + 0.8  // Range: 0.8-1.2
  return Math.floor(ms * r)
}

Backoff Schedule:

  • Initial: 500ms ± 20%
  • Maximum: 30,000ms (30 seconds)
  • Multiplier: 2x per attempt
  • Jitter: ±20% randomization

Network Resilience Features

Online/Offline Detection

The client monitors browser network status in real-time:

// Setup network monitoring
window.addEventListener("online", () => {
  this.isOnline = true;
  this.handleNetworkReconnection(); // Immediate reconnection
});

window.addEventListener("offline", () => {
  this.isOnline = false;
  this.handleNetworkDisconnection(); // Pause operations
});

Benefits:

  • Prevents sending when offline
  • Automatic reconnection when online
  • User feedback on network status
  • Preserves audio during outages

WebSocket State Management

The client maintains detailed connection states:

enum ClientStates {
  IDLE = "idle", // Not yet connected
  CONNECTING = "connecting", // Establishing connection
  CONNECTED = "connected", // Active connection
  RECONNECTING = "reconnecting", // Attempting reconnection
  DISCONNECTED = "disconnected", // Intentionally disconnected
  ERROR = "error", // Error state
  PAUSED = "PAUSED", // Circuit breaker open or manual pause
}

State Transitions:

IDLE → CONNECTING → CONNECTED
         ↓              ↓
      ERROR ← → RECONNECTING
                        ↓
                  DISCONNECTED

Connection Health Checks

Heartbeat Mechanism:

// Send periodic ping to detect silent failures
private startHeartbeat() {
  this.heartbeatTimer = setInterval(() => {
    this.sendJson({ type: 'ping', t: Date.now() })
  }, this.heartbeatMs)  // Default: 10,000ms (10 seconds)
}

Adaptive Heartbeat:

  • Normal conditions: 10 seconds
  • Poor network: Doubles to 20s, then 30s (max)
  • Detects silent connection failures
  • Prevents idle connection timeouts

Automatic Reconnection

The client automatically reconnects on unexpected closures:

private async onCloseWebSocket(event?: CloseEvent) {
  console.log('WebSocket closed:', event?.code, event?.reason)
  this.stopHeartbeat()

  // Skip reconnection if intentionally ended
  if (this.isEndingTranscription) {
    this.updateState(ClientStates.DISCONNECTED)
    return
  }

  this.updateState(ClientStates.DISCONNECTED)

  // Auto-reconnect unless in ERROR or PAUSED
  if (this.state !== ClientStates.ERROR && !this.isPaused) {
    await this.reconnectLoop()
  }
}

Acknowledgment & Retry Logic

Dual Acknowledgment System

The client supports two acknowledgment methods for flexibility and efficiency:

1. Individual Chunk Acknowledgment

Each chunk receives explicit confirmation:

// Server response format
{
  "id": "1732579200000_000042",
  "chunkId": "1732579200000_000042"
}

// Client handling
private async handleChunkAcknowledgment(chunkId: string) {
  this.clearAcknowledmentTimeout(chunkId)

  const inFlightData = this.inFlight.get(chunkId)
  if (inFlightData) {
    // Free up in-flight budget
    this.inFlightAudioDurationMs -= inFlightData.chunk.durationMs

    // Remove from tracking
    this.inFlight.delete(chunkId)

    // Persist acknowledgment
    await this.markChunkAsAcknowledged(chunkId)

    // Try sending more chunks
    await this.processQueueIfReady()
  }
}

Benefits:

  • Fine-grained confirmation
  • Low-latency feedback
  • Simple to implement

2. Batch Acknowledgment

Multiple chunks confirmed with single message:

// Server response format
{
  "type": "AUDIO_CHUNK_ACK",
  "stream_id": "session-uuid",
  "ack_id": 42  // All chunks with sequenceNumber <= 42 confirmed
}

// Client handling
private async handleBatchAcknowledgment(streamId: string, ackId: number) {
  const chunksToAck = []

  // Find all chunks with sequenceNumber <= ackId
  for (const [chunkId, inFlightData] of this.inFlight) {
    if (inFlightData.chunk.sequenceNumber <= ackId) {
      chunksToAck.push(inFlightData)
    }
  }

  // Sort by sequence to maintain order
  chunksToAck.sort((a, b) => a.sequenceNumber - b.sequenceNumber)

  // Acknowledge all matched chunks
  for (const data of chunksToAck) {
    this.clearAcknowledmentTimeout(data.chunk.id)
    this.inFlight.delete(data.chunk.id)
    await this.markChunkAsAcknowledged(data.chunk.id)
    this.inFlightAudioDurationMs -= data.chunk.durationMs
  }

  await this.processQueueIfReady()
}

Benefits:

  • 90%+ reduction in ACK messages
  • Handles high-throughput scenarios
  • Prevents ACK storms

Timeout-Based Retry

Chunks without acknowledgment are automatically retried:

// Set 30-second timeout on send
private setAcknowledmentTimeout(chunkId: string) {
  const timeoutId = setTimeout(() => {
    this.handleChunkTimeout(chunkId)
  }, CHUNK_TIMEOUT_MS)  // 30,000ms

  this.acknowledgmentTimeouts.set(chunkId, timeoutId)
}

// Handle timeout
private handleChunkTimeout(chunkId: string) {
  const inFlightData = this.inFlight.get(chunkId)
  if (inFlightData) {
    this.inFlight.delete(chunkId)
    this.retryChunk(inFlightData.chunk)
  }
}

Timeout Duration: 30 seconds (conservative for high-latency networks)

Exponential Backoff Retry

Failed chunks retry with increasing delays:

private retryChunk(chunk: PendingChunk) {
  chunk.retryCount = (chunk.retryCount || 0) + 1

  if (chunk.retryCount <= MAX_RETRY_ATTEMPTS) {  // MAX = 3
    setTimeout(() => {
      this.sendQueue.unshift(chunk)  // Priority: front of queue
      this.processQueueIfReady()
    }, jitter(1000 * 2 ** (chunk.retryCount - 1)))
  } else {
    // Report failure after max attempts
    this.handleChunkFailure(chunk)
  }
}

Retry Schedule:

  • 1st retry: ~1s (2⁰ × 1000ms ± jitter)
  • 2nd retry: ~2s (2¹ × 1000ms ± jitter)
  • 3rd retry: ~4s (2² × 1000ms ± jitter)
  • After 3 failures: Report error via onError callback

Total Time: ~37 seconds from initial send to final failure

Queue Management

Queue Structure

The client maintains two data structures:

// FIFO queue for chunks awaiting transmission
private sendQueue: PendingChunk[] = []

// Map of chunks sent but not yet acknowledged
private inFlight: Map<string, { chunk: PendingChunk; timestamp: number }> = new Map()

Flow Control

To prevent memory overflow and buffer bloat:

const MAX_IN_FLIGHT_DURATION_MS = 10000  // 10 seconds max unacknowledged

private canSendNow(chunkDurationMs: number) {
  return (
    this.ws?.readyState === WebSocket.OPEN &&
    this.isOnline &&
    this.inFlightAudioDurationMs + chunkDurationMs <= MAX_IN_FLIGHT_DURATION_MS
  )
}

Mechanism:

  • Tracks total duration of unacknowledged audio
  • Blocks new sends when limit reached
  • Waits for acknowledgments to free budget
  • Prevents network congestion

Example:

  • Max 10 seconds in-flight
  • Chunks are ~600ms each
  • Max ~16 chunks simultaneously
  • 17th chunk waits until ACK received

Queue Processing

private async processQueueIfReady() {
  // Guard: Check connection state
  if (!this.ws || this.ws.readyState !== WebSocket.OPEN ||
      !this.isOnline || this.isPaused) {
    return
  }

  while (this.sendQueue.length > 0) {
    const chunk = this.sendQueue[0]

    // Respect flow control limits
    if (!this.canSendNow(chunk.durationMs)) {
      break  // Wait for ACKs
    }

    this.sendQueue.shift()

    try {
      await this.sendChunkInternal(chunk, 'sendChunk')
    } catch (error) {
      if (error instanceof CircuitBreakerError) {
        // Return to queue, stop processing
        this.sendQueue.unshift(chunk)
        break
      }

      // Other errors: retry with backoff
      this.retryChunk(chunk)
      break
    }
  }
}

Queue Discipline:

  • FIFO (First In, First Out)
  • Priority for retries (added to front via unshift)
  • Respects flow control limits
  • Breaks on error to prevent starvation

Error Recovery Scenarios

Scenario 1: Network Drop During Recording

Flow:

  1. Browser detects network loss (offline event)
  2. isOnline flag set to false
  3. canSendNow() returns false (blocks queue)
  4. Audio continues recording to buffer
  5. Chunks flushed and persisted to IndexedDB
  6. In-flight chunks timeout → moved back to queue
  7. Network returns (online event)
  8. handleNetworkReconnection() called
  9. resendPendingChunks() re-queues chunks
  10. connect() re-establishes WebSocket
  11. recoverStoredChunks() loads from IndexedDB
  12. processQueueIfReady() resumes sending

Result: ✅ Zero audio loss

Scenario 2: Browser Crash Mid-Session

Flow:

  1. Browser crashes, all RAM lost
  2. IndexedDB survives (disk-persisted)
  3. User relaunches application
  4. connect() called with same session
  5. Session ID loaded from transcription_session_current
  6. recoverStoredChunks() reads from IndexedDB
  7. Unsent chunks (sent: false) added to queue
  8. Unacknowledged chunks added to queue
  9. Queue processed in sequence order

Result: ✅ Session resumed with all audio intact

Scenario 3: Circuit Breaker Opens

Flow:

  1. Server returns 5 consecutive errors
  2. Circuit breaker transitions to OPEN
  3. isPaused set to true
  4. User shown: "Service temporarily unavailable"
  5. All sends fail immediately with CircuitBreakerError
  6. Chunks remain in queue and IndexedDB
  7. After 60s, circuit transitions to HALF_OPEN
  8. Single test operation attempted
  9. Success: Circuit closes, queue processed
  10. Failure: Circuit reopens, wait another 60s

Result: ✅ Graceful degradation, no data loss

Scenario 4: Chunk Timeout (No ACK)

Flow:

  1. Chunk sent at t=0
  2. No ACK by t=30s
  3. Timeout handler fired
  4. Chunk removed from in-flight
  5. Retry scheduled with backoff
  6. Retry 1 at t=31s (1s delay)
  7. Retry 2 at t=33s (2s delay)
  8. Retry 3 at t=37s (4s delay)
  9. handleChunkFailure() called
  10. Error reported via onError callback

Result: ✅ 4 total attempts over ~37s before reporting failure

Performance Characteristics

Storage Performance

  • Write latency: ~5-10ms per chunk (IndexedDB async)
  • Read latency: ~10-20ms per chunk recovery
  • Bulk recovery: ~100-200ms for 1000 chunks

Network Performance

  • Chunk size: ~14-20 KB (0.6s PCM16 @ 16kHz)
  • Send frequency: ~1.7 chunks/sec (600ms intervals)
  • Bandwidth: ~24-34 KB/sec (~200 Kbps)
  • In-flight limit: 10 seconds (~170 KB max buffer)

Memory Footprint

  • Send queue: 200 KB - 1 MB (10-50 chunks)
  • In-flight map: ~320 KB (~16 chunks)
  • Audio buffer: ~19 KB (~0.6s buffering)
  • Total RAM: ~500 KB - 1.5 MB (worst case)

Monitoring API

The client exposes methods for monitoring health and performance:

Client State

// Get current connection state
const state = client.getState();
// Returns: 'idle' | 'connecting' | 'connected' | 'reconnecting' | 'disconnected' | 'error' | 'PAUSED'

// Get current session ID
const sessionId = client.getSessionId();
// Returns: UUID string

Network Metrics

const metrics = client.getNetworkMetrics();
// Returns: {
//   consecutiveFailures: number,
//   lastFailureTime: number,
//   networkLatency: number,
//   throughput: number,
//   isOnline: boolean
// }

Circuit Breaker Status

const stats = client.getCircuitBreakerStats();
// Returns: {
//   state: 'CLOSED' | 'OPEN' | 'HALF_OPEN',
//   failureCount: number,
//   lastFailureTime: number,
//   retryAfter: number,
//   operations: Record<string, OperationStats>
// }

const state = client.getCircuitBreakerState();
// Returns: 'CLOSED' | 'OPEN' | 'HALF_OPEN'

Storage Information

const storageInfo = await client.getStorageInfo();
// Returns: {
//   totalChunks: number,
//   totalSessions: number,
//   currentSessionChunks: number
// }

Best Practices

1. Monitor Circuit Breaker State

client.onState = (state) => {
  if (state === "PAUSED") {
    // Show user-friendly message
    showNotification("Transcription temporarily paused. Retrying...");
  }
};

2. Handle Errors Gracefully

client.onError = (error) => {
  console.error("Transcription error:", error);
  // Log to monitoring service
  logError(error, { sessionId: client.getSessionId() });
};

3. Track Network Status

client.onNetworkStatus = (isOnline) => {
  if (!isOnline) {
    showWarning("Network connection lost. Audio being saved locally.");
  } else {
    showSuccess("Network reconnected. Resuming transcription.");
  }
};

4. Monitor Storage Usage

// Periodically check storage
setInterval(async () => {
  const info = await client.getStorageInfo();
  if (info.currentSessionChunks > 500) {
    console.warn("Large number of pending chunks:", info.currentSessionChunks);
  }
}, 30000);

5. Clean Up on Session End

// After transcription complete
await client.end();
await client.cleanLocalStorage();

Conclusion

The Invox Medical streaming transcription client provides enterprise-grade audio protection through:

  • ✅ Zero data loss guarantee via IndexedDB persistence
  • ✅ Intelligent retry logic with exponential backoff
  • ✅ Circuit breaker protection against cascading failures
  • ✅ Dual acknowledgment system for efficiency
  • ✅ Automatic recovery from crashes and network failures
  • ✅ Flow control to prevent buffer overflow
  • ✅ Session restoration after interruptions

This architecture ensures that audio transcription remains robust and reliable even under adverse conditions including network failures, server outages, browser crashes, and high-latency scenarios.

The system prioritizes data integrity above all else while maintaining optimal performance characteristics for real-world production environments.
Table of Contents