Multi-Client Facial Tracking Relay for VTubers

The Problem: One Stream, Many Applications

iFacialMocap is an excellent iOS app for facial tracking, but it only sends data to one destination at a time. As a VTuber, I wanted to use the same tracking data in multiple applications simultaneously:

• VTube Studio: For live streaming with my 3D model
• Warudo: For virtual world interactions and recording
• Future Tools: Any other application that accepts ARKit-style blendshape data

Running multiple instances of iFacialMocap or switching between applications wasn't practical. I needed a way to split the UDP stream without introducing latency or packet loss.

The Solution: Byte-for-Byte UDP Relay

VTuberHub is a lightweight Go application that listens on a UDP port, receives packets from iFacialMocap, and forwards identical copies to all configured targets in parallel. The relay makes zero modifications to the data—it's a pure traffic splitter.

Key Design Principles:

1. Zero Latency: Packets are forwarded immediately using Go's efficient UDP networking
2. Parallel Forwarding: All targets receive data simultaneously, not sequentially
3. No Packet Modification: Byte-for-byte forwarding ensures compatibility with all ARKit-compatible applications
4. Minimal Overhead: Go's compiled nature and efficient concurrency keep CPU usage under 0.5%

Architecture & Performance

The relay uses Go's net.UDPConn for efficient packet handling. Each received packet triggers parallel writes to all configured targets using goroutines, ensuring minimal latency. The main loop pre-allocates buffers to avoid garbage collection pauses during high-frequency streaming (30-60 FPS).

Performance Metrics:

• Average Latency: <1 ms (typically 0.1-0.5 ms)
• CPU Usage: <0.5% on modern hardware
• Memory Usage: <15 MB
• Packet Loss: <0.1% under normal conditions

Socket buffers are set to 64KB (read and write) to handle burst traffic and reduce packet drops. The relay includes comprehensive statistics reporting, showing uptime, packets received/forwarded/dropped, and average latency every 10 seconds.

Packet Dumping & Replay Tooling

One of the most powerful features is the packet dump functionality. When enabled via dump_packets: true in the config, the relay writes every received packet to timestamped files in the specified directory (default: raw_packets/).

This enables several use cases:

• Format Analysis: Study the exact ARKit blendshape format for building custom tools
• Replay Tooling: Create scripts that replay saved packets to test applications without live tracking
• Chat Reaction System: Build systems that trigger specific facial expressions based on chat messages by replaying pre-recorded blendshape sequences
• Debugging: Capture problematic packets for analysis when troubleshooting tracking issues

Each packet is saved as a separate file with a timestamp, making it easy to process them chronologically or filter by time range.

Configuration & Usage

The relay uses a simple JSON configuration file:

{
  "listen_port": 13121,
  "targets": [
    {"host": "127.0.0.1", "port": 49983, "name": "VBridger"},
    {"host": "127.0.0.1", "port": 39539, "name": "Warudo"}
  ],
  "buffer_size": 4096,
  "log_level": "info",
  "stats_interval": 10,
  "dump_packets": false,
  "dump_dir": "raw_packets"
}

Setup is straightforward:

1. Configure iFacialMocap to send to your PC's IP on port 13121
2. Start the relay: ./ifmrelay.exe -config relay_config.json
3. Configure target applications (VBridger, Warudo) to listen on their respective ports
4. Enable packet dumping if you want to capture data for replay tooling

Code Quality & Testing

VTuberHub follows enterprise-grade standards with comprehensive unit tests covering config loading, normalization, and packet forwarding logic. The codebase includes:

• Config Validation: Strict normalization ensures sensible defaults and prevents misconfiguration
• Error Handling: Graceful degradation when targets are unreachable—other targets continue receiving data
• Atomic Statistics: Thread-safe counters using Go's atomic package for accurate metrics
• Cross-Platform Support: Works on Windows (primary), macOS, and Linux with identical behavior

All tests pass with go test ./..., and the code is formatted with gofmt following Go best practices.

Windows Firewall Configuration

Windows Defender Firewall blocks incoming UDP connections by default. You'll need to create a firewall rule to allow the relay to receive packets from your iOS device. The README includes PowerShell commands and GUI instructions for both port-based and executable-based rules.

For quick setup (PowerShell as Administrator):

New-NetFirewallRule -DisplayName "iFacialMocap UDP Relay" `
  -Direction Inbound -Protocol UDP -LocalPort 13121 `
  -Action Allow -Profile Any

Ask Celeste

Q: Will this work with other facial tracking apps besides iFacialMocap?
A: Yes! Any application that sends ARKit-compatible blendshape data over UDP can use VTuberHub. The relay is protocol-agnostic—it forwards whatever it receives. Just configure the source app to send to the relay's listen port.

Q: Can I use this for non-VTuber applications?
A: Absolutely. The relay is useful for any scenario where you need to split a UDP stream to multiple destinations—motion capture, sensor data, game streaming, or any real-time UDP application.