> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mascot.bot/llms.txt
> Use this file to discover all available pages before exploring further.

# LipsyncClient - Vanilla JavaScript Lip Sync Core API

> Use @mascotbot/core without React: LipsyncClient.init, processAudio, createStreamingSession, resample, diagnostics, events, and the vanilla Rive engine.

`@mascotbot/core` is the framework-agnostic engine — no React, no
Rive on the root entry. Use it in vanilla apps, Web Workers, or Node-adjacent
environments. React apps get all of this re-exported through
[`@mascotbot/react`](/libraries/react-sdk).

## Initialize

```ts theme={null}
import { LipsyncClient } from "@mascotbot/core";

const client = await LipsyncClient.init({
  apiKey: "mascot_pub_…",
  userId: "user_42",          // optional — stable id for MAU attribution
  // licenseEndpoint, devMode, logger, fingerprintHash also accepted
});
```

`init` exchanges your key with the edge worker and loads the WASM runtime.
Configuration mirrors [`<MascotProvider>`](/installation#4-configure-the-client).

## `processAudio(samples, opts?)`

One-shot inference over a recorded buffer. `samples` is **16 kHz mono Float32
in `[-1, 1]`**; pass `opts.sampleRate` if your buffer is at another rate and
let the client resample.

```ts theme={null}
const { timeline, durationMs, speechMs } = await client.processAudio(audio16kMono);
// timeline   — VisemeTimeline (the serializable artifact)
// durationMs — total audio duration
// speechMs   — non-silent ms detected
```

`timeline` is the offline artifact — persist it as JSON and replay later with
zero reprocessing ([Offline lip sync](/libraries/offline-lipsync)).

### `resample(samples, fromSampleRate, toSampleRate?)`

Linear resampler to prep audio for `processAudio` / streaming. `toSampleRate`
defaults to `16000`.

```ts theme={null}
const at16k = client.resample(decoded.getChannelData(0), decoded.sampleRate, 16000);
```

## `createStreamingSession()`

For live audio, push 25 ms (400-sample) windows of 16 kHz mono one at a time.

```ts theme={null}
const session = client.createStreamingSession();
const frame = await session.pushWindow(audioWindow); // Promise — must await
console.log(frame.visemeId, frame.silenceDetected, frame.frameIndex);
session.close();
```

`pushWindow` is async by design (inference is off the main thread). End-of-
utterance phantom visemes are suppressed by the SDK's internal −50 dBFS
silence gate. Full streaming guide, including the React wrapper:
[Streaming sessions](/core/streaming).

## Events

`client.on(event, listener)` returns an unsubscribe function.

| Event       | Fires when                                             |
| ----------- | ------------------------------------------------------ |
| `"ready"`   | Init finished; inference available                     |
| `"refresh"` | A background license refresh succeeded (`{ epoch }`)   |
| `"refused"` | Authorization refused — listener gets a `RefusedError` |
| `"error"`   | A runtime error — listener gets an `Error`             |

```ts theme={null}
const off = client.on("refused", (err) => showBilling(err.code, err.message));
// later: off();
```

`client.status` is the current `LipsyncStatus`
(`idle | initializing | ready | running | degraded | refused | error`).

## `diagnostics()`

Async — resolves to an opaque health snapshot. Useful for a debug panel.
Always `await` it (it is computed off-thread):

```ts theme={null}
const d = await client.diagnostics();
// { pendingSpeechMs, classifierEpoch, riskBand, installId, variantId }
```

## Teardown

```ts theme={null}
client.stop();  // stop active work, keep the instance reusable
client.close(); // release everything; the instance is done
```

Call one of these when you are finished so resources and the audio graph are
released.

## Vanilla Rive playback

The Rive engine lives on the `/rive` subpath. Construct Rive yourself, read
its inputs, and drive `MascotPlayback`:

```ts theme={null}
import { Rive, EventType } from "@rive-app/webgl2";
import { MascotPlayback, getRiveInputs, STATE_MACHINE_NAMES } from "@mascotbot/core/rive";

const rive = new Rive({
  src: "/mascot-fox.riv",
  canvas: document.querySelector("canvas")!,
  autoplay: true,
  stateMachines: STATE_MACHINE_NAMES[0], // "mascotStateMachine" — pass only this
});

rive.on(EventType.Load, () => {
  const riveInputs = getRiveInputs(rive);
  const playback = new MascotPlayback({ riveInputs, stream: true, enableNaturalLipSync: true });

  // Offline: replay a whole timeline (from processAudio or parseTimeline).
  playback.setTimeline(timeline);
  // Streaming alternative: playback.pushVisemes([{ offset: 0, visemeId: 7 }]);
  playback.play();
});
```

`MascotPlayback` methods: `setTimeline(tl)`, `pushVisemes(cues)`, `play()`,
`pause()`, `seek(ms)`, `reset()`, `setTransitionSpeed(n)`. `getRiveInputs(rive)`
returns a bundle with `.has(name)`; `hasRiveInput(rive, name)` is the
standalone presence check. The SDK only writes mouth / `is_speaking` /
`stress` — see [Rive co-existence](/concepts/rive-coexistence).

## Next

<Columns cols={3}>
  <Card title="Streaming sessions" icon="tower-broadcast" href="/core/streaming">
    `createStreamingSession` in depth.
  </Card>

  <Card title="PCM stream player" icon="waveform" href="/core/pcm-stream-player">
    Play + tap raw provider PCM.
  </Card>

  <Card title="Offline lip sync" icon="box-archive" href="/libraries/offline-lipsync">
    Generate → persist → replay.
  </Card>
</Columns>