Velum Studio
All guides

Guides › Make it yours

Face tracking with Velum Avatars

Move your avatar with the tracking app you already use: XR Animator, VSeeFace, Waidayo and more, over VMC.

Velum Avatars moves your avatar's mouth with your voice, and it breathes and blinks on its own. To follow your head, face and body too, it listens to the tracking app you already use. Most VTuber trackers can send what they see over VMC, a common way for these apps to talk to each other on your computer or your home network. This guide sets up each one.

Velum Studio doesn't track your face or body itself. The tracking happens in the app you choose, and Velum Avatars poses your model from what that app sends.

Velum Avatars' side

  1. Install Velum Avatars: the gear › Gallery… › Velum Avatars. It's free with Velum Studio.
  2. In Layers, click + (Add layer) › Plugins › Avatar (Velum Avatars), and choose your Avatar file (VRM 1.0, VRM 0.x or .glb).
  3. In the Inspector, open Motion capture and set VMC port to the port your tracking app sends to. 39539 is the usual one. 0 means off.

That's all most apps need. The rest of Motion capture is there for when something looks off:

Normalized sender bonesOff to start with. Turn it on if the eyes cross or go blank, or arms and feet look mirrored. Velum tells most apps apart by itself (XR Animator, for one), so you rarely need it.
Interpolation delay16 ms to start with. A little higher smooths out jittery tracking, at the cost of a little lag.
Accept blends without ApplyOff to start with. Turn it on if the head moves but the face never does: a few apps leave out the message that says "use these expressions now".
Sender prioritiesFor two apps at once. See Two apps at once.
Send VMC to, Send frames per secondSends your avatar's pose on to another app, as IP:port (for example 127.0.0.1:39540). Empty is off.

What moves, and what doesn't

  • Head, eyes and body follow whatever the app sends. A face-only app moves the head and face; a full-body app moves the rest too.
  • Expressions. Blinks, smiles and the other standard VRM expressions work on any VRM that has them.
  • Perfect sync. iPhone trackers can send all 52 ARKit face shapes (cheeks puffing, tongue out, one brow up). Velum passes every one through, but your model only shows them if it was made with those shapes. Without them, you still get the head, eyes and standard expressions.
  • The mouth. Your microphone in Studio moves the mouth (Lip sync from). When the tracking app sends mouth movement, the tracker's mouth wins while it's sending.
  • When tracking stops, or the app closes, your avatar eases back to its idle pose within a moment, instead of freezing.

Load the same model in your tracking app as in Velum Avatars. Most apps send each bone as it sits in their copy of the model, so a different model can come out twisted.

Addresses and ports

Every app below asks for two things: where to send (an IP address) and a port.

  • Tracking app on the same computer as Studio: send to 127.0.0.1.
  • Tracking app on a phone: send to your computer's address on your home network, something like 192.168.1.20. On Windows, run ipconfig in Command Prompt and use the IPv4 Address. On a Mac, it's in System Settings › Wi-Fi › Details. The phone and computer must be on the same network.
  • The port in the app and VMC port in Velum Avatars must match. Two apps on one computer can't listen on the same port, so if your tracker also receives on 39539, give one of them another number.
  • Windows Firewall. A phone's tracking is usually blocked the first time. Studio's plugins run in a helper called velum-plugin-host, and that's what Windows asks about: allow it on Private networks. If you missed the question, open Windows Security › Firewall & network protection › Allow an app through firewall, and allow velum-plugin-host.exe from Velum Studio's folder, or add an inbound rule for UDP on your port. Apps on the same computer don't need this.

The apps

AppRuns onCostTo Velum
XR AnimatorWindows, macOS, LinuxFreeDirectly
VSeeFaceWindowsFreeDirectly
Webcam Motion CaptureWindows, macOSFree, with paid plansDirectly
VMagicMirrorWindowsFree, with a paid Full EditionDirectly
WarudoWindowsFree for personal useDirectly
WaidayoiOSFreeDirectly
iFacialMocapiOSPaidThrough VSeeFace
MeowFaceAndroidFreeThrough VSeeFace

XR Animator

Free full-body and face tracking from a webcam, by Butz Yung. It runs in a browser or as a desktop app for Windows, macOS and Linux.

  1. Use the desktop app. In XR Animator, VMC is a desktop-app feature: a browser tab can't send it to Velum on its own.
  2. Load your avatar, start your webcam, and double-click VMC-protocol in its menu.
  3. Set App mode to Others, and turn VMC-protocol on. It sends to this computer on port 39539 unless you change it.
  4. In Velum Avatars, set VMC port to 39539.

XR Animator sends bones in a different form from most apps. Velum Avatars recognizes it by itself; if the eyes still cross or go blank, turn on Normalized sender bones. Newer versions of XR Animator can also run on a phone; we haven't tested sending from a phone to Velum yet, so if you try it, tell us how it goes.

VSeeFace

Free webcam face tracking for Windows, and a good hub for phone trackers: it can take tracking from iFacialMocap, MeowFace, Waidayo and others, and pass it on to Velum.

  1. Load your avatar and get tracking working in VSeeFace as usual.
  2. Open General settings and find the VMC protocol sender. Turn it on.
  3. Set its IP address to 127.0.0.1 and its port to 39539 (the usual one).
  4. In Velum Avatars, set VMC port to 39539.

If you also use VSeeFace's VMC receiver (for Waidayo, for example), keep it on a different port from the sender, such as 39540. VSeeFace's own window can sit behind Studio; it only needs to keep running.

Webcam Motion Capture

Webcam tracking of face, hands and body, for Windows and macOS. Free to start, with paid plans for more features; check its site for what each plan includes.

  1. Start tracking with your webcam as usual.
  2. Choose Send to External App, at the top center of the Webcam Motion Receiver screen.
  3. Make sure VMC is selected under Data Format.
  4. Enter 127.0.0.1 and port 39539, and click Start Sending.
  5. In Velum Avatars, set VMC port to 39539.

It sends the body and the standard VRM expressions, which every VRM has.

VMagicMirror

A desktop avatar app for Windows that follows your webcam, keyboard and mouse. The Standard Edition is free (or a donation) on BOOTH; the paid Full Edition lifts its limits.

  1. In VMagicMirror's Settings window, open the VMCP tab and turn on Show VMCP Tab on Main Window.
  2. On the VMC Protocol tab, check Enable VMC Protocol Data Send.
  3. Set Target Address to 127.0.0.1 and Target Port to 39539, then click Apply Changes.
  4. Turn on Send Bone Poses and Send BlendShape Data (and Send Finger Bone Poses if you want fingers).
  5. In Velum Avatars, set VMC port to 39539.

In the Standard Edition, VMagicMirror adds a visual effect to what it sends, and can't send while Game Locomotion is the body motion. The Full Edition doesn't have those limits.

Warudo

A 3D VTuber app for Windows, free for personal use on Steam. It takes tracking from webcams, phones and more, and can send your character's pose with its VMC Sender.

  1. Set up your character and its tracking in Warudo as usual.
  2. Add a VMC Sender asset.
  3. Set IP Address to 127.0.0.1, Port to 39539, and Character to your character. Leave Send to VirtualMotionCapture off.
  4. In Velum Avatars, set VMC port to 39539.

Warudo's own VMC receiver also uses 39539 to start with. If you use it, move one of them to another port.

Waidayo

Free face tracking on an iPhone or iPad with Face ID. It sends straight to your computer, no other app needed.

  1. On your computer, find its address on your network (above).
  2. In Waidayo, open Settings and enter that address in Send Motion IP Address.
  3. Leave Send Motion Port at 39540.
  4. In Velum Avatars, set VMC port to 39540.
  5. If nothing arrives, check that Waidayo is allowed Local Network in the iPhone's Settings, and the firewall on your computer.

Waidayo can send all 52 ARKit face shapes. They show on a perfect-sync model; on others, you get the head and standard expressions.

iFacialMocap

Face tracking on an iPhone or iPad with Face ID, from the App Store (paid; prices vary by country). The iPhone app doesn't send VMC itself, so it goes through VSeeFace:

  1. Put the phone and computer on the same network, and open iFacialMocap. It shows the phone's IP address.
  2. In VSeeFace's General settings, turn on the receiver for iFacialMocap and enter the phone's IP address.
  3. Turn on VSeeFace's VMC protocol sender to 127.0.0.1, port 39539, as in VSeeFace.
  4. In Velum Avatars, set VMC port to 39539.

iFacialMocap also has a desktop app that can send VMC. Its makers note that it sends bone positions as zero, which can bend some models oddly, so we suggest VSeeFace for now.

MeowFace

Free face tracking with an Android phone's camera. It isn't being updated any more, but it still works with VSeeFace, which passes it on to Velum.

  1. Put the phone and computer on the same network, and open MeowFace. Note the phone's IP address.
  2. In VSeeFace's General settings, turn on the receiver for phone tracking, choose VTube Studio's protocol (the one MeowFace speaks), and enter the phone's IP address.
  3. Turn on VSeeFace's VMC protocol sender to 127.0.0.1, port 39539.
  4. In Velum Avatars, set VMC port to 39539.

Two apps at once

One avatar can take its face from one app and its body from another, for example the face from Waidayo on your iPhone and the body from XR Animator on your webcam. Point both apps at the same VMC port. Velum Avatars keeps them apart by where each one sends from.

  • The easy way: let each app send only its part. Where an app lets you, turn off what the other one covers (face tracking in the body app, for example). With nothing overlapping, there's nothing to choose between.
  • When both send the same thing, use Sender priorities: one entry per app, written IP:port=priority. The higher number wins for each bone and expression; if that app stops, the other takes over. The IP is the device the app runs on (127.0.0.1 for this computer, the phone's address for a phone), and the port is the one the app sends from, which isn't the port it sends to. Many apps pick that port fresh each time they start, and then an entry won't keep working; if so, use the easy way.

If something's wrong

Nothing movesCheck that VMC port matches the port the app sends to exactly, and that no other app is listening on it. From a phone, check the IP address, the network, and the firewall.
Eyes crossed or blank, limbs mirroredTurn on Normalized sender bones. Also make sure both apps have the same model loaded.
Jittery or shakyRaise Interpolation delay a little at a time, say to 33 or 50 ms.
Head moves, face doesn'tTurn on Accept blends without Apply, and check the app is set to send expressions or blendshapes.
Doesn't blink or smile from trackingThe model probably doesn't have those expressions, or only has the standard ones while the app sends perfect-sync shapes. Try a model made for perfect sync.
The mouth ignores my voiceThe tracking app is sending the mouth, and the tracker's mouth wins. Turn off mouth tracking in the app to let your voice move it.

Help us test

We've built Velum Avatars to work with all of these apps, and we'd love to know how they do on your setup. Try any of them and post in the Velum Studio Testers Discord with:

  • the app, and its version if you know it
  • your computer, and your phone if one's involved
  • what worked, and what didn't
  • your model: VRM 0.x or VRM 1.0, and whether it has perfect sync

A screenshot or short clip helps a lot.