swift

TranscribeCpp

Swift bindings for transcribe.cpp, a C/C++ speech-to-text library built on ggml. Native code ships as a prebuilt .xcframework SwiftPM binaryTarget, with Metal embedded on supported Apple slices.

Status: in development (0.3.0). Core model, session, run, stream, cancellation, backend, and family-extension APIs are implemented and tested.

Upgrading from 0.1? See the 0.2 migration guide, including the replacement of gpuDevice with exact Device values. From 0.3, see the 0.4 migration guide (roles, DiarizeSession, throwing capabilities).

Install

Apple platforms only: macOS 13+ and iOS 16+.

For development from this repository, use the package in bindings/swift. It expects bindings/swift/build-apple/TranscribeCpp.xcframework by default, or a custom artifact path through TRANSCRIBE_XCFRAMEWORK_PATH.

The standalone SwiftPM mirror is planned but not published yet:

.package(url: "https://github.com/handy-computer/transcribe-cpp-swift.git", from: "0.3.0")

Until that mirror repo and tag exist, use the release xcframework directly when you only need the raw C module:

.binaryTarget(
    name: "CTranscribe",
    url: "https://github.com/handy-computer/transcribe.cpp/releases/download/v0.3.0/TranscribeCpp.xcframework.zip",
    checksum: "<published with the release>"
)

The direct binary target exposes import CTranscribe; the local and planned Swift packages expose the wrapper product, import TranscribeCpp.

Quickstart

import TranscribeCpp

let model = try Model(path: "/path/to/model.gguf")
let session = try model.session()

// pcm: mono float32 at 16 kHz, in [-1, 1]
let transcript = try session.run(pcm, options: RunOptions(timestamps: .segment))
print(transcript.text)

for segment in transcript.segments {
    print("[\(segment.t0Ms)-\(segment.t1Ms)ms] \(segment.text)")
}

run is blocking; try await session.run(pcm) uses the async convenience overload and hops the work off the caller's thread.

Punctuation, capitalization, and text normalization

RunOptions.pnc and RunOptions.itn default to preserving each model family's shipped behavior. Probe model.supports(.pnc) or .itn before selecting .off/.on. The same RunOptions is accepted by single runs, batches, streams, and Transcribe.transcribe.

let options = RunOptions(pnc: .off, itn: .on)
let transcript = try session.run(pcm, options: options)

Prompting

vocabulary (custom terms), prompt (context, or the instruction under .instruct) and prefix (text the model continues from) take effect where model.supports() reports .vocabulary, .contextPrompt, .instruct or .transcriptPrefix.

let transcript = try session.run(pcm, options: RunOptions(vocabulary: ["Kubernetes", "gRPC"]))

Streaming models expose committed/tentative text for UI display:

let stream = try session.stream()
for chunk in chunks {                 // 16 kHz mono float32 frames
    let update = try stream.feed(chunk)
    if update.committedChanged { print(stream.text.committed) }
}
try stream.finalize()
let transcript = stream.snapshot // language, segments, words, tokens, timings

Runnable examples live in Sources/{transcribe-file,streaming,batch,backend-select,error-handling}.

Diarization

A model's roles say what it serves (.asr, .diarize, .langId, .vad). Diarization models such as Sortformer (DIARIZE only) return speaker turns from a DiarizeSession:

let model = try Model(path: "diar_streaming_sortformer_4spk-v2.1-F32.gguf")
let turns = try model.diarizeSession().run(
    pcm, options: DiarizeOptions(family: .sortformer(.init(preset: .veryHighLatency))))
for t in turns { print(t.speakerId, t.t0Ms, t.t1Ms) }

capabilities and session() throw .unsupportedRole on a model without .asr.

Language ID

Language ID models (.langId, e.g. VoxLingua107 ECAPA-TDNN) rank the model's own label codes from a LangIdSession; match result.code against an ASR model's capabilities.languages yourself. allowed: nil scores every label, an empty array throws .invalidArgument, and clips under langIdInfo.minAudioMs throw .inputTooShort. Every allowed label comes back, ranked by p; input longer than langIdInfo.maxAudioMs is scored on its first maxAudioMs.

let model = try Model(path: "lang-id-voxlingua107-ecapa-Q8_0.gguf")
let result = try model.langIdSession().run(pcm, options: LangIdOptions(allowed: ["en", "de"]))
print(result.code ?? "-", result.allowedMass)

Voice activity detection

VAD models (.vad, e.g. Silero VAD; the whisper.cpp ggml-silero-*.bin files load too) score vadInfo.frameSamples-sample frames from a VadSession. run returns per-frame probs and speech segments in samples; streamFeed / streamFlush return the probabilities of the frames each call completed, starting at firstFrame.

let model = try Model(path: "silero-vad-v6.2-F32.gguf")
let vad = try model.vadSession()
for s in try vad.run(pcm, options: VadOptions(threshold: 0.6)).segments {
    print(s.startSample, s.endSample)
}

for chunk in micChunks {
    let r = try vad.streamFeed(chunk)
    // r.probs[i] scores stream frame r.firstFrame + i
}
try vad.streamFlush()

Backends

Backends are compiled into the xcframework per Apple slice:

Slice	Backend
macOS arm64	Metal + CPU
macOS x86_64	CPU only
iOS device arm64	Metal + CPU
iOS simulator	CPU only

Request a backend policy with ModelOptions(backend:); probe availability with Transcribe.backendAvailable(_:). For exact selection, pass an entry from Transcribe.devices() to ModelOptions(device:); exact selection never falls back to another primary device.

On the CPU and on Metal, model weights are memory-mapped from the file: they are served from the page cache, which the OS can reclaim, and do not count toward the app's memory footprint (the figure jetsam enforces on iOS). Don't truncate or rewrite a model file in place while it is loaded; write a new file and rename it over the old one. ModelOptions(noMmap: true) copies the weights instead.

Concurrency and lifetime

• Model is shareable. In 0.x, compute is serialized per model, so concurrent runs queue; load one Model per worker for true parallelism.
• Session is single-threaded. Use one session from one thread at a time.
• An active Stream holds the model's compute lease until finalize, reset, drop, or a feed that fails the stream. Other runs/streams on that model fail with TranscribeError.busy.
• Transcribe.setLogHandler is best installed at startup. Repeated calls are safe; they swap the Swift handler behind one native trampoline.
• On Metal, do not keep models in globals in short-lived programs. Scope models so ARC releases GPU resources before process exit; the examples use do { } blocks for this reason.

Cancellation

Install a token on a session and cancel it from any thread:

let token = CancellationToken()
session.setCancellationToken(token)
token.cancel()

The active run, runBatch, or stream feed throws TranscribeError.aborted with any partial transcript preserved. Async run/runBatch also bridge Swift task cancellation when no custom token is installed. Role sessions (DiarizeSession, LangIdSession, VadSession, including a VAD stream feed) take a token the same way.

C, Objective-C, and C++

The xcframework also exposes the raw C module as CTranscribe. Objective-C and C++ callers use the bundled C headers directly, for example #import <CTranscribe/transcribe/extensions.h>.