Class EdgeAi

The edgeAi namespace on a BoxApplet. Talks directly to the box over HTTP. Obtained via applet.edgeAi.

Index

Methods

getImageEmbedding loadModel runPrompt runTranscribe streamPrompt streamTranscribe tokenize new

Methods

getImageEmbedding

getImageEmbedding(
    model: Model,
    image: ImageBytes,
    options?: { abortSignal?: AbortSignal },
): Promise<ImageEmbeddingRef>
Encode an image on the box for model, returning an ImageEmbeddingRef to reuse across prompts. The ref is only valid for model.
Parameters
- model: Model
- image: ImageBytes
- Optionaloptions: { abortSignal?: AbortSignal }
Returns Promise<ImageEmbeddingRef>
- Defined in src/edge-ai.ts:180

loadModel

loadModel(model: Model): Promise<void>
Load a model onto the box (starts/keeps it resident). Rejects if it fails.
Parameters
- model: Model
Returns Promise<void>
- Defined in src/edge-ai.ts:170

runPrompt

runPrompt(
    model: Model,
    prompt: PromptInput,
    options?: PromptOptions,
): Promise<string>
Run a one-shot prompt against a loaded model and return the full response.
Parameters
- model: Model
- prompt: PromptInput
- Optionaloptions: PromptOptions
Returns Promise<string>
- Defined in src/edge-ai.ts:258

runTranscribe

runTranscribe(
    model: Model,
    audio: Float32Array<ArrayBuffer>,
    options: TranscribeOptions,
): Promise<string>
Transcribe complete audio with a loaded speech model and return the whole transcript. Takes only a finished buffer: a feed has no result until it ends, so there would be nothing to return.
Parameters
- model: Model
- audio: Float32Array<ArrayBuffer>
- options: TranscribeOptions
Returns Promise<string>
- Defined in src/edge-ai.ts:308

streamPrompt

streamPrompt(
    model: Model,
    prompt: PromptInput,
    options?: Omit<PromptOptions, "abortSignal">,
): PromptStream
Run a prompt and stream the generated tokens as they arrive (SSE).
Parameters
- model: Model
- prompt: PromptInput
- Optionaloptions: Omit<PromptOptions, "abortSignal">
Returns PromptStream
- Defined in src/edge-ai.ts:269

streamTranscribe

streamTranscribe(
    model: Model,
    audio: AudioSamples,
    options: Omit<TranscribeOptions, "abortSignal">,
): TranscriptionStream
Transcribe audio with a loaded speech model, streaming results as they are decoded. Consume with for await; call cancel() to stop early.

audio may be a finished buffer or a feed that produces samples over time; a feed is sent as the request body arrives rather than being collected first.
Parameters
- model: Model
- audio: AudioSamples
- options: Omit<TranscribeOptions, "abortSignal">
Returns TranscriptionStream
- Defined in src/edge-ai.ts:330

tokenize

tokenize(
    model: Model,
    prompt: string,
    options?: { abortSignal?: AbortSignal },
): Promise<Tokenization>
Tokenise prompt on the box with model's tokenizer.
Parameters
- model: Model
- prompt: string
- Optionaloptions: { abortSignal?: AbortSignal }
Returns Promise<Tokenization>
- Defined in src/edge-ai.ts:194

`Static`new

new(
    featureEnabled: boolean,
    info: UndefinedToOptional<{ token: string; url: string }> | undefined,
    logger: Logger,
): EdgeAi
Create a EdgeAi instance.

If the feature is not enabled or the info is undefined then a stub is returned that will error.

TODO IGNT-895: Make sure the returned stub properly returns an error state
Parameters
- featureEnabled: boolean
- info: UndefinedToOptional<{ token: string; url: string }> | undefined
- logger: Logger
Returns EdgeAi
- Defined in src/edge-ai.ts:159