How EmotiVoice Works

Dive deep into the technology behind our advanced speech emotion recognition system. Understand the pipeline from raw audio to emotional insights.

The Processing Pipeline

From raw audio waveform to emotional insight in 6 steps

1

Audio Input

Raw speech signal at 16kHz sampling rate

  • Waveform representation
  • Time-domain signal
  • Variable length input
2

Pre-processing

Signal normalization and noise reduction

  • Amplitude normalization
  • Silence trimming
  • High-pass filtering
3

MFCC Extraction

13 Mel-frequency cepstral coefficients

  • Windowing (25ms)
  • FFT computation
  • Mel filterbank
4

Spectrogram

Mel spectrogram for frequency analysis

  • 128 Mel bands
  • Time-frequency map
  • Log compression
5

LSTM Network

Bidirectional LSTM with attention mechanism

  • 256 hidden units
  • 2 LSTM layers
  • Dropout 0.3
6

Emotion Output

Probability distribution across 7 emotions

  • Softmax activation
  • Confidence scores
  • Top-1 prediction

Core Technologies

The science powering every prediction

MFCC

Mel-Frequency Cepstral Coefficients

MFCCs capture the spectral characteristics of speech using a mel scale that mimics human auditory perception — the gold standard for speech feature extraction.

13 coefficients per frame
25ms window / 10ms hop
Captures vocal tract shape
Robust to noise variations
MFCC = DCT(log(|FFT(windowed signal)|))

Mel Spectrogram

Time-Frequency Representation

A visual representation showing how frequencies change over time, converted to the mel scale for better alignment with human pitch perception.

128 mel frequency bands
Log-scaled magnitudes
Time-frequency map
Captures prosodic features
Mel(f) = 2595 × log₁₀(1 + f/700)

LSTM

Long Short-Term Memory

A recurrent neural network capable of learning long-term temporal dependencies — ideal for analyzing emotional patterns across the full duration of speech.

Bidirectional processing
256 hidden units / layer
Attention mechanism
Variable-length input
h_t = o_t ⊙ tanh(c_t)

Classification

Softmax Emotion Output

The final layer produces a probability distribution across 7 emotion classes using softmax activation, enabling interpretable, confidence-scored predictions.

7 emotion classes
Softmax output layer
Cross-entropy loss
Class-weighted training
P(emotion) = softmax(Wx + b)

LSTM Architecture

Layer-by-layer breakdown with 2,448,903 trainable parameters

Input Layer

MFCC features (13 coefficients × T frames)

Shape(batch, T, 13)
Params0

Bidirectional LSTM 1

First recurrent layer — forward + backward passes

Shape(batch, T, 512)
Params542,720

Dropout (0.3)

Regularization to prevent overfitting

ShapeRate: 0.3
Params0

Bidirectional LSTM 2

Second recurrent layer for deeper temporal patterns

Shape(batch, T, 512)
Params1,576,960

Attention Layer

Self-attention to focus on emotionally salient frames

Shape(batch, 512)
Params262,656

Dense (ReLU)

Fully connected layer for feature compression

Shape(batch, 128)
Params65,664

Output (Softmax)

Probability distribution across 7 emotion classes

Shape(batch, 7)
Params903

Total Architecture

Optimized for real-time emotion recognition

2,448,903
Parameters
~9.5
MB Model
7
Layers

Performance Benchmarks

State-of-the-art results on standard emotion recognition datasets

Accuracy by Emotion

Per-class classification accuracy

94.2%
Overall Accuracy
Weighted average across all emotions
0.93
F1 Score
Harmonic mean of precision and recall
~250ms
Inference Time
Average prediction latency
10K+
Training Data
Diverse emotional speech samples

Training Configuration

OptimizerAdam
Learning Rate1e-4
Batch Size32
Epochs100
DatasetRAVDESS
Val Split20%