Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.mascot.bot/llms.txt

Use this file to discover all available pages before exploring further.

Every hook in the React SDK, grouped by subpath. Audio-pipeline hooks come from @mascotbot/react; Rive hooks from @mascotbot/react/rive.

​
Audio pipeline — @mascotbot/react

​
useMascot()

const { client, status, error, reload } = useMascot();
The licensed inference client from the enclosing <MascotProvider>.
FieldTypeNotes
clientLipsyncClient | nullnull until init resolves
statusLipsyncStatusidle | initializing | ready | running | degraded | refused | error
errorError | nullA typed error on refused / error
reload() => voidRe-runs init (e.g. after the user fixes a key)
Gate audio work on status === "ready".

​
useProcessAudio(audioUrl)

const { result, loading, error } = useProcessAudio("/audio/greeting.wav");
Fetches the URL, decodes, resamples to 16 kHz, and runs inference once. Pass null to skip. result is a ProcessAudioResult: 
result.timeline;   // VisemeTimeline — the serializable artifact
result.durationMs; // total audio duration
result.speechMs;   // non-silent ms detected
Hand result.timeline to useMascotPlayback().setTimeline(), or JSON.stringify it to persist and replay later with zero reprocessing — Offline lip sync.

​
Rive layer — @mascotbot/react/rive

​
useMascotRive()

const { rive, isRiveLoaded, RiveComponent, setImageAsset } = useMascotRive();
The Rive instance + canvas for the enclosing <Mascot>. rive is the raw, unmodified @rive-app/* instance — yours for data binding, custom inputs, events, and ViewModels. The SDK never wraps it. setImageAsset swaps a runtime image asset (e.g. a custom face texture).

​
useMascotInputs<T>()

const { riveInputs, custom, has } = useMascotInputs<"wave" | "reveal">();
if (has("wave")) custom.wave.fire();
FieldNotes
riveInputsSDK-driven input handles (mouth / is_speaking / stress)
customYour declared inputs, typed by T. Never undefined
has(name)Authoritative presence check — use this, never raw rive.stateMachineInputs()
This is the supported way to detect and drive non-mouth inputs. See Rive co-existence.

​
useMascotPlayback(options?)

const playback = useMascotPlayback({ stream: true, enableNaturalLipSync: true });
playback.setTimeline(result.timeline);       // offline replay
playback.pushVisemes(cues);                  // streaming append
playback.stress([{ offset: 0, stress: 1 }]); // SDK-driven emphasis cues
playback.play(); playback.pause(); playback.seek(0); playback.reset();
Wraps the framework-agnostic MascotPlayback. stress([{ offset, stress }]) schedules emphasis cues — the SDK animates the Rive stress input from them on the playback clock (see Stress emphasis). Options:
OptionTypePurpose
streambooleanStreaming mode (mic / realtime); pair with useLipsyncStream
enableNaturalLipSyncbooleanSmoother, less robotic merging
naturalLipSyncConfigPartial<NaturalLipSyncConfig>Tune merging — Natural lip sync
desktopTransitionSpeed / mobileTransitionSpeednumberMouth blend rate
setSpeakingStatebooleanAuto-drive is_speaking (default on)
manualSpeakingStateControlbooleanTake manual control of is_speaking
Pass a stable naturalLipSyncConfig reference (a module constant or useState/useMemo). A fresh object literal every render reinitializes playback and breaks lip sync after the first chunk — Troubleshooting.

​
useLipsyncStream(args)

const { error, attached, pushAudio, pushBase64PCM16, reset } = useLipsyncStream({
  client,
  playback,
  source: { kind: "mic" }, // | { kind: "mediaStream", stream } | { kind: "manual" }
  enabled: isLive,
  onFrame: (f) => {},      // optional per-window telemetry
});
The unified audio→viseme stream for live audio: microphone, a tapped MediaStream (realtime providers, played audio), or manual chunk pushing. Full guide: Streaming & microphone.

​
Low-level Rive loaders

HookReturnsUse when
useLoadRive(options?){ rive, isRiveLoaded, RiveComponent, setImageAsset }You need to load Rive yourself; pass an options object ({ stateMachineName, ... }). <Mascot> uses this internally.
useRiveAsset(riveParams?, opts?)RiveState & { setImageAsset }Lowest-level escape hatch: useRive + a runtime image-asset swapper.
useSafeStateMachineInput(rive, _, name, init?)input or DEFAULT_SM_INPUTSingle-input lookup with the mascotStateMachine / InLesson fallback shim.
useIsMobileDevice()booleanTouch + UA + screen-size heuristic (e.g. to pick a transition speed).

​
Next

Streaming & mic

useLipsyncStream in depth.

Offline lip sync

useProcessAudio → persist → replay.

Rive co-existence

Driving non-mouth inputs.