Kotlin SDK

Kotlin API for running AI models on-device on Android and iOS via Kotlin Multiplatform. Supports completion, transcription, embeddings, RAG, and tool calling.

Kotlin SDK

Run AI models on-device with a simple Kotlin API.

Building

cactus build --android

Build output: android/build/lib/libcactus.so

Integration

Android-only

1. Copy libcactus.so to app/src/main/jniLibs/arm64-v8a/
2. Copy Cactus.kt to app/src/main/java/com/cactus/

Kotlin Multiplatform

Source files:

File	Copy to
Cactus.common.kt	shared/src/commonMain/kotlin/com/cactus/
Cactus.android.kt	shared/src/androidMain/kotlin/com/cactus/
Cactus.ios.kt	shared/src/iosMain/kotlin/com/cactus/
cactus.def	shared/src/nativeInterop/cinterop/

Binary files:

Platform	Location
Android	libcactus.so → app/src/main/jniLibs/arm64-v8a/
iOS	libcactus-device.a → link via cinterop

build.gradle.kts:

kotlin {
    androidTarget()

    listOf(iosArm64(), iosSimulatorArm64()).forEach {
        it.compilations.getByName("main") {
            cinterops {
                create("cactus") {
                    defFile("src/nativeInterop/cinterop/cactus.def")
                    includeDirs("/path/to/cactus/ffi")
                }
            }
        }
        it.binaries.framework {
            linkerOpts("-L/path/to/apple", "-lcactus-device")
        }
    }

    sourceSets {
        commonMain.dependencies {
            implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.6.0")
        }
    }
}

Usage

Handles are plain Long values (C pointers). All functions are top-level.

Basic Completion

import com.cactus.*
import org.json.JSONObject

val model = cactusInit("/path/to/model", null, false)
val messages = """[{"role":"user","content":"What is the capital of France?"}]"""
val resultJson = cactusComplete(model, messages, null, null, null)
val result = JSONObject(resultJson)
println(result.getString("response"))
cactusDestroy(model)

Completion with Options and Streaming

val options = """{"max_tokens":256,"temperature":0.7}"""
val tokens = mutableListOf<String>()

val resultJson = cactusComplete(model, messages, options, null) { token, _ ->
    tokens.add(token)
    print(token)
}

Audio Transcription

// From file
val result = cactusTranscribe(model, "/path/to/audio.wav", "", null, null, null)

// From PCM data (16 kHz mono)
val pcmData: ByteArray = ...
val result = cactusTranscribe(model, null, null, null, null, pcmData)

Streaming Transcription

val stream  = cactusStreamTranscribeStart(model, null)
val partial = cactusStreamTranscribeProcess(stream, audioChunk)
val final_  = cactusStreamTranscribeStop(stream)

Embeddings

val embedding      = cactusEmbed(model, "Hello, world!", true)   // FloatArray
val imageEmbedding = cactusImageEmbed(model, "/path/to/image.jpg")
val audioEmbedding = cactusAudioEmbed(model, "/path/to/audio.wav")

Tokenization

val tokens = cactusTokenize(model, "Hello, world!")  // IntArray
val scores = cactusScoreWindow(model, tokens, 0, tokens.size, 512)

Vector Index

val index = cactusIndexInit("/path/to/index", 384)

cactusIndexAdd(
    index,
    intArrayOf(1, 2),
    arrayOf("Document 1", "Document 2"),
    arrayOf(floatArrayOf(0.1f, 0.2f), floatArrayOf(0.3f, 0.4f)),
    null
)

val resultsJson = cactusIndexQuery(index, floatArrayOf(0.1f, 0.2f), null)
// JSON: {"results":[{"id":1,"score":0.99,...},...]}

cactusIndexDelete(index, intArrayOf(2))
cactusIndexCompact(index)
cactusIndexDestroy(index)

API Reference

All functions are top-level and mirror the C FFI directly. Handles are Long values.

Init / Lifecycle

fun cactusInit(modelPath: String, corpusDir: String?, cacheIndex: Boolean): Long  // throws RuntimeException
fun cactusDestroy(model: Long)
fun cactusReset(model: Long)
fun cactusStop(model: Long)
fun cactusGetLastError(): String

Completion

fun cactusComplete(
    model: Long,
    messagesJson: String,
    optionsJson: String?,
    toolsJson: String?,
    callback: CactusTokenCallback?
): String

Transcription

fun cactusTranscribe(
    model: Long,
    audioPath: String?,
    prompt: String?,
    optionsJson: String?,
    callback: CactusTokenCallback?,
    pcmData: ByteArray?
): String

fun cactusStreamTranscribeStart(model: Long, optionsJson: String?): Long  // throws RuntimeException
fun cactusStreamTranscribeProcess(stream: Long, pcmData: ByteArray): String
fun cactusStreamTranscribeStop(stream: Long): String

Embeddings

fun cactusEmbed(model: Long, text: String, normalize: Boolean): FloatArray
fun cactusImageEmbed(model: Long, imagePath: String): FloatArray
fun cactusAudioEmbed(model: Long, audioPath: String): FloatArray

Tokenization / Scoring

fun cactusTokenize(model: Long, text: String): IntArray
fun cactusScoreWindow(model: Long, tokens: IntArray, start: Int, end: Int, context: Int): String

VAD / RAG

fun cactusVad(model: Long, audioPath: String?, optionsJson: String?, pcmData: ByteArray?): String
fun cactusRagQuery(model: Long, query: String, topK: Int): String

Vector Index

fun cactusIndexInit(indexDir: String, embeddingDim: Int): Long  // throws RuntimeException
fun cactusIndexDestroy(index: Long)
fun cactusIndexAdd(index: Long, ids: IntArray, documents: Array<String>, embeddings: Array<FloatArray>, metadatas: Array<String>?): Int
fun cactusIndexDelete(index: Long, ids: IntArray): Int
fun cactusIndexGet(index: Long, ids: IntArray): String
fun cactusIndexQuery(index: Long, embedding: FloatArray, optionsJson: String?): String
fun cactusIndexCompact(index: Long): Int

Telemetry

fun cactusSetTelemetryEnvironment(cacheDir: String)
fun cactusSetAppId(appId: String)
fun cactusTelemetryFlush()
fun cactusTelemetryShutdown()

Types

fun interface CactusTokenCallback {
    fun onToken(token: String, tokenId: Int)
}

Requirements

• Android API 24+ / arm64-v8a
• iOS 14+ / arm64 (KMP only)

See Also

C API Reference

Full C API reference underlying the Kotlin bindings

Vector Index API

Vector database API for RAG applications

Swift SDK

Swift alternative for Apple platforms

Flutter SDK

Cross-platform alternative using Dart