MateEngine Lip Sync & Expressions Patch for the SillyTavern integration mod on windows

A robust mod for MateEngine that adds fully dynamic text-based lip syncing and accurate facial expression handling (Joy, Fun, Angry, etc.) directly triggered by the messages from SillyTavern.

This repository provides a custom patch to optimize audio-driven lip-syncing for 3D VRM models operating in Mate Engine X while receiving conversational outputs from SillyTavern extensions. It includes the core C# implementation and an automated PowerShell script to easily compile and apply the modifications to your local environment.

Repository Contents Overview

• BetterLipSync.cs: The core C# logic overriding or improving the default avatar lip-sync behavior.
• MateEnginePatch.csproj: The .NET project file required to compile the C# patch.
• Patch-SillyTavern.ps1: A PowerShell automation script designed to handle the installation and configuration of the patch alongside your SillyTavern instance.

---

🎯 Complete Setup Guide

"By default, MateEngine's text lip-sync by tts from Sillytavern is not working. This patch completely fixes it." If you are starting from scratch and want to set up 3D animated characters interacting with SillyTavern on your Windows PC, follow this complete, step-by-step guide.

Step 1: Download the Base Programs

First, you need to download and install the two main programs.

1. Download SillyTavern:

   • Head over to the official repository: SillyTavern GitHub
   • Follow their official installation guide to set it up (requires Node.js).

2. Download MateEngine X:

   • This is the 3D application that renders the characters.
   • Download the latest Windows release directly here: MateEngineWin3.2.7z
   • Official Repository Link: LazyCodingKing/MateEngine-Sillytavern
   • Extract the .7z file using 7-Zip or WinRAR to a folder on your computer (for example, C:\MateEngine).

Step 2: Install This Lip-Sync Mod

By default, MateEngine's text lip-sync by tts is not working. This patch completely fixes it.

1. Download the latest MateEnginePatch.dll and Patch-SillyTavern.ps1 from this repository.
2. Open the folder where you extracted MateEngine.
3. Navigate into MateEngineX_Data\Managed.
4. Paste the MateEnginePatch.dll file there.

Step 3: Link SillyTavern & MateEngine (Automated Patcher)

For MateEngine to know when the character speaks, we need to link it to SillyTavern's Text-to-Speech system. I've included an automated tool that does this instantly for you!

1. Open Windows PowerShell on your computer.
2. Navigate to the folder where you downloaded Patch-SillyTavern.ps1, then copy and paste the following command. Make sure to replace the path with where your actual SillyTavern folder is located:

.\Patch-SillyTavern.ps1 -SillyTavernPath "C:\Your\Path\To\SillyTavern"

3. Hit Enter! The script will safely inject the necessary hooks.

You're completely done! Start SillyTavern, start MateEngine, load a character, and start chatting. You will now see perfect lip-syncing and facial expressions!.

---

🛠 For Devs

Compiling

• Build the .csproj targeting .NET Framework 4.7.1 (net471). Do not use .NET Standard or Unity will fail to load the assembly.

Technical Details

Unlike the original basic implementation which relied on AccumulateValue without ever flushing the buffers (causing faces to freeze or fail to animate), this patch uses a dedicated LateUpdate driver with a high execution order ([DefaultExecutionOrder(32000)]). This ensures that our facial visemes and expressions securely override any standard animations MateEngine might be playing, allowing custom expressions (Joy, Fun, Angry, etc.) triggered by SillyTavern's LLM to render properly without being wiped out by idle animations.