Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.mascot.bot/llms.txt

Use this file to discover all available pages before exploring further.

@mascotbot/core is the framework-agnostic engine — no React, no Rive on the root entry. Use it in vanilla apps, Web Workers, or Node-adjacent environments. React apps get all of this re-exported through @mascotbot/react.

​
Initialize

import { LipsyncClient } from "@mascotbot/core";

const client = await LipsyncClient.init({
  apiKey: "mascot_pub_…",
  userId: "user_42",          // optional — stable id for MAU attribution
  // licenseEndpoint, devMode, logger, fingerprintHash also accepted
});
init exchanges your key with the edge worker and loads the WASM runtime. Configuration mirrors <MascotProvider>.

​
processAudio(samples, opts?)

One-shot inference over a recorded buffer. samples is 16 kHz mono Float32 in [-1, 1]; pass opts.sampleRate if your buffer is at another rate and let the client resample. 
const { timeline, durationMs, speechMs } = await client.processAudio(audio16kMono);
// timeline   — VisemeTimeline (the serializable artifact)
// durationMs — total audio duration
// speechMs   — non-silent ms detected
timeline is the offline artifact — persist it as JSON and replay later with zero reprocessing (Offline lip sync).

​
resample(samples, fromSampleRate, toSampleRate?)

Linear resampler to prep audio for processAudio / streaming. toSampleRate defaults to 16000. 
const at16k = client.resample(decoded.getChannelData(0), decoded.sampleRate, 16000);

​
createStreamingSession()

For live audio, push 25 ms (400-sample) windows of 16 kHz mono one at a time. 
const session = client.createStreamingSession();
const frame = await session.pushWindow(audioWindow); // Promise — must await
console.log(frame.visemeId, frame.silenceDetected, frame.frameIndex);
session.close();
pushWindow is async by design (inference is off the main thread). End-of- utterance phantom visemes are suppressed by the SDK’s internal −50 dBFS silence gate. Full streaming guide, including the React wrapper: Streaming sessions.

​
Events

client.on(event, listener) returns an unsubscribe function.
EventFires when
"ready"Init finished; inference available
"refresh"A background license refresh succeeded ({ epoch })
"refused"Authorization refused — listener gets a RefusedError
"error"A runtime error — listener gets an Error
const off = client.on("refused", (err) => showBilling(err.code, err.message));
// later: off();
client.status is the current LipsyncStatus (idle | initializing | ready | running | degraded | refused | error).

​
diagnostics()

Async — resolves to an opaque health snapshot. Useful for a debug panel. Always await it (it is computed off-thread): 
const d = await client.diagnostics();
// { pendingSpeechMs, classifierEpoch, riskBand, installId, variantId }

​
Teardown

client.stop();  // stop active work, keep the instance reusable
client.close(); // release everything; the instance is done
Call one of these when you are finished so resources and the audio graph are released.

​
Vanilla Rive playback

The Rive engine lives on the /rive subpath. Construct Rive yourself, read its inputs, and drive MascotPlayback: 
import { Rive, EventType } from "@rive-app/webgl2";
import { MascotPlayback, getRiveInputs, STATE_MACHINE_NAMES } from "@mascotbot/core/rive";

const rive = new Rive({
  src: "/mascot-fox.riv",
  canvas: document.querySelector("canvas")!,
  autoplay: true,
  stateMachines: STATE_MACHINE_NAMES[0], // "mascotStateMachine" — pass only this
});

rive.on(EventType.Load, () => {
  const riveInputs = getRiveInputs(rive);
  const playback = new MascotPlayback({ riveInputs, stream: true, enableNaturalLipSync: true });

  // Offline: replay a whole timeline (from processAudio or parseTimeline).
  playback.setTimeline(timeline);
  // Streaming alternative: playback.pushVisemes([{ offset: 0, visemeId: 7 }]);
  playback.play();
});
MascotPlayback methods: setTimeline(tl), pushVisemes(cues), play(), pause(), seek(ms), reset(), setTransitionSpeed(n). getRiveInputs(rive) returns a bundle with .has(name); hasRiveInput(rive, name) is the standalone presence check. The SDK only writes mouth / is_speaking / stress — see Rive co-existence.

​
Next

Streaming sessions

createStreamingSession in depth.

PCM stream player

Play + tap raw provider PCM.

Offline lip sync

Generate → persist → replay.