Tantivy for Swift

Swift bindings for the tantivy 0.26.1 full-text search engine, with a small, idiomatic API for building a schema, adding documents, and querying.

Works on macOS, iOS and iPadOS (device + simulator). The Rust engine ships as a prebuilt static-library XCFramework, downloaded from the GitHub Release and checksum-verified by SwiftPM; there is nothing to compile at app-build time.

import Tantivy

// 1. Describe the schema
let schema = SchemaBuilder()
    .addTextField("title", stored: true)
    .addTextField("body")                       // indexed, not stored
    .addU64Field("year", stored: true, fast: true)
    .build()

// 2. Open an index (on disk, or .inMemory(schema:))
let index = try Index(path: indexURL, schema: schema)

// 3. Add documents and commit
let writer = try index.writer()
try writer.addDocument([
    "title": "The Old Man and the Sea",
    "body":  "He was an old man who fished alone in a skiff…",
    "year":  1952,
])
try writer.commitAndReload()                    // commit + make searchable

// 4. Query
for hit in try index.search("sea whale", limit: 10) {
    print(hit.score, hit.string("title") ?? "", hit.int("year") ?? 0)
}

Requirements

• Swift 6.2 toolchain (the package is swift-tools-version: 6.2, Swift 6 language mode)
• macOS 15+ / iOS 18+ (iPadOS uses the iOS slices; Mac Catalyst 18+)

Installation (Swift Package Manager)

The prebuilt CTantivy.xcframework is attached to each GitHub Release and referenced from Package.swift as a checksummed remote binary target, so SwiftPM downloads and verifies it on resolve. Building an app against it needs no Rust toolchain and no Git LFS — just import Tantivy.

From a Git remote — in another package's Package.swift:

dependencies: [
    .package(url: "https://github.com/your-org/TantivySwift.git", from: "0.1.0"),
],
targets: [
    .target(
        name: "YourLibrary",
        dependencies: [.product(name: "Tantivy", package: "TantivySwift")]
    ),
]

From a local path (monorepo / side-by-side checkout):

.package(path: "../TantivySwift"),

In an Xcode app — File ▸ Add Package Dependencies ▸ enter the repo URL (or "Add Local…"), then add the Tantivy library to your target.

Then use it:

import Tantivy

let index = try Index.inMemory(schema:
    SchemaBuilder().addTextField("title", stored: true).build())

How the binary is resolved. Package.swift selects the C layer in three modes, in priority order: (1) a locally-built artifacts/CTantivy.xcframework if present (a release dry-run, or testing a fresh build); (2) else the host static library from scripts/build-host.sh (local/CI development — this is the only mode that uses unsafeFlags); (3) else, the default for consumers, the checksummed xcframework downloaded from the matching GitHub Release. The binary is no longer committed to the repo, so clones are small and there is no Git LFS dependency.

Building from source (regenerating the xcframework) requires full Xcode + a Rust toolchain — see Building the XCFramework.

API

Schema

SchemaBuilder is a fluent builder. Each field has stored (return it in results), indexed (make it searchable), and fast (columnar storage) options.

Method	Field type
addTextField(_:stored:indexed:tokenizer:indexing:fast:)	tokenized full-text
addStringField(_:stored:indexed:fast:)	single raw token (exact match: ids, tags)
addU64Field / addI64Field / addF64Field / addBoolField	numerics
addDateField(_:stored:indexed:fast:)	date/time (RFC3339; Date round-trips at second precision)

tokenizer is a typed Analyzer (default .default); the cases mirror exactly what the native layer registers (a drift-guard test enforces this):

Analyzer	Behavior
.default	lowercased, split on non-alphanumeric, unstemmed
.english	.default + English stemming (tantivy en_stem)
.raw	the whole value as one token, case-sensitive (exact match)
.lowercase	the whole value as one lowercased token (case-insensitive exact match: tags, authors, enums, ids)
.whitespace	split on whitespace only

.addTextField("title", stored: true, tokenizer: .english)

indexing controls postings detail; .position (the default) is required for phrase queries.

Index

let index = try Index(path: url, schema: schema)   // create or open on disk
let index = try Index.inMemory(schema: schema)      // not persisted

index.documentCount                                 // searchable doc count
try index.reload()                                  // observe latest commit
try index.search(_:limit:fields:)                   // -> [SearchHit]
try index.count("sea")                              // match count, no docs loaded
try index.count(.range("year", 1900...2000))        // count a structured Query
try index.get("id", equals: "abc123")               // fetch one doc by id term

Index(path:) creates the index if the directory is empty, or opens the existing one (the schema must match). On iOS, pass a writable location such as Application Support or Caches.

By default searches observe new commits only after reload(). Pass reloadPolicy: .onCommit to have a background watcher reload shortly after every commit instead — near-real-time, no reload() calls, but a search immediately after commit() may briefly see the previous generation:

let index = try Index(path: url, schema: schema, reloadPolicy: .onCommit)

Writing

let writer = try index.writer(heapSize: 0)          // 0 → 50 MB indexing budget
try writer.addDocument(["title": "Hi", "tags": ["a", "b"], "year": 2024])
try writer.addDocument(someEncodableValue)          // any Encodable
try writer.addDocument(json: #"{"title":"Hi"}"#)    // raw JSON
let opstamp = try writer.commit()                   // durable; reload to search
try writer.commitAndReload()                         // commit + index.reload()
try writer.rollback()                                // discard uncommitted ops
try writer.deleteAllDocuments()
try writer.deleteDocuments(field: "id", equals: "abc123")            // delete by term
try writer.deleteDocuments(field: "id", equalsAnyOf: ["a", "b"])     // multi-term delete
try writer.deleteDocuments(matching: .range("year", 1900...1950))    // delete by query
// or, with commit + reload in one step:
try index.delete(matching: .term("status", "draft"))

Use at most one writer per index at a time. Values may be scalars or arrays (arrays populate multi-valued fields). Documents become searchable only after a commit() followed by Index.reload() — commitAndReload() does both.

Replace / upsert. tantivy has no in-place update; replace a document by deleting its id term and re-adding it (supply the whole document). upsert does both in one commit:

try index.upsert(card, idField: "id", id: card.id)   // delete-by-term + add

The id field should be a single-token field (a string/raw or numeric/bool field) so the term matches the whole value. get is the read counterpart — a scoreless fetch of the first document with that id term:

let hit = try index.get("id", equals: card.id)        // SearchHit?
let card = try cards.get(idField: "id", id: card.id)  // typed, on a collection

Querying & results

search accepts tantivy query syntax:

sea whale            # either term, in the default fields
title:whale          # field-scoped
"old man"            # phrase (needs .position indexing)
title:sea AND body:fish
year:[1900 TO 2000]                       # numeric range
created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]   # date range (RFC3339)

fields: (default empty) chooses which fields an unqualified term searches; empty means all indexed text fields. boosts: applies per-field weights:

try index.search("dune", boosts: ["title": 2.0, "body": 0.5])

orderBy: replaces relevance ranking with a field sort — the field must be a numeric (u64/i64/f64) or date field declared fast: true in the schema. Works on both the string and structured search:

try index.search("dune", orderBy: .descending("created"))        // newest first
try index.search(.term("tag", "book"), orderBy: .ascending("year"))

Field-ordered hits carry a score of 0 (tantivy returns the sort key in place of computing BM25).

Each result is a SearchHit:

hit.score                 // Float relevance score (higher = better)
hit.string("title")       // first String value, or nil
hit.int("year")           // first Int64, or nil
hit.uint("count")         // first UInt64, or nil (full u64 range, exact)
hit.double("rating")
hit.bool("active")
hit["tags"]               // [FieldValue] – all values for a field

Only stored fields come back in hits. Integers round-trip exactly across the full i64 and u64 ranges (FieldValue has both .int(Int64) and .unsigned(UInt64) cases); the numeric accessors coerce between them.

Structured queries (Query)

A second search API builds a query tree that maps directly onto tantivy's own query types (TermQuery, PhraseQuery, RangeQuery, BooleanQuery, BoostQuery, FuzzyTermQuery, RegexQuery, ExistsQuery, MoreLikeThisQuery, AllQuery) — no string parsing, no escaping.

let q: Query =
    (.term("tag", "book") && .range("year", 1900...2000))
        .excluding(.term("status", "draft"))

for hit in try index.search(q) { … }                 // [SearchHit]
let books = try index.search(q, as: Book.self)        // typed
let scored = try collection.searchScored(q)           // [(score, Book)]

Builders: .matchAll, .parsed(query, fields:), .term(field, value) (string / Int / UInt64 / Double / Bool / date:), .phrase(field, [tokens], slop:), .phrasePrefix(field, [tokens], maxExpansions:), .fuzzy(field, value, distance:…), .prefix(field, value), .autocomplete(field, value, typoTolerance:), .regex(field, pattern), .wildcard(field, pattern), .exists(field), .moreLikeThis(field, text), .range(field, 1900...2000) / .dateRange(field, from:to:), .allOf / .anyOf(_, minimumShouldMatch:), &&, ||, .excluding(_), .boosted(by:).

term/phrase match indexed tokens exactly (as tantivy does): on a tokenized text field pass already-analyzed tokens (e.g. lowercase for the default tokenizer). For analyzed/free-text input, use the string search — or embed it with .parsed.

Mixing in user input. .parsed embeds a query string (full tantivy syntax, analyzed by the engine) as a node in a structured query — the bridge between the two APIs, and the natural shape for "what the user typed plus my filters":

let q: Query = .parsed(searchField.text) && .term("tag", "book")
try index.search(q)
try index.delete(matching: .parsed("status:draft"))   // works for deletes too

Typeahead / autocomplete

.prefix and .autocomplete are the typeahead primitives — .prefix("title", "nor") matches indexed tokens that start with nor (north, norway); .autocomplete is the same but allows up to typoTolerance edits on the prefix (default 1) so mopy still finds moby. .regex("title", "mob.*") matches tokens against a (whole-token-anchored) regular expression.

try index.search(.prefix("title", "mob"))                 // exact prefix
try index.search(.autocomplete("title", "mopy"))          // typo-tolerant prefix
try index.search(.regex("title", "m(oby|ice).*"))         // regex over tokens
try index.search(.phrasePrefix("title", ["old", "ma"]))   // multi-word typeahead

.phrasePrefix is the multi-word counterpart: every term but the last matches exactly (in order, like phrase), and the last is a prefix — ["old", "ma"] matches "old man". The field needs positions (the default) when more than one term is given.

Like term, these match indexed tokens: pass an already-analyzed prefix (e.g. lowercase for the default tokenizer).

Wildcard, exists & more-like-this

.wildcard matches a token against a pattern where * is any run of characters and everything else is literal (so -, . etc. are safe) — like .regex but without writing a regex. It matches a single token on a tokenized field, the whole value on a string field.

try index.search(.wildcard("code", "AB-*"))     // whole-value match on a string field
try index.search(.wildcard("title", "*fish"))   // token suffix on a text field

.exists matches documents that have any value in a field — and its negation finds the ones missing it. The field must be fast: true (it's evaluated over the fast column):

try index.search(.exists("price"))                          // has a price
try index.search(.matchAll.excluding(.exists("price")))     // price is missing

.moreLikeThis finds documents similar to some text — "related documents". Give it field→text(s); tantivy analyzes the text, picks the most characteristic terms, and builds a weighted query from them:

try index.search(.moreLikeThis("body", article.body), limit: 5)

// Or relative to a document already in the index (reads its stored fields and
// excludes the source itself):
try index.moreLikeThis(idField: "slug", id: article.slug, fields: ["title", "body"], limit: 5)

Tune MoreLikeThisOptions on small corpora: the defaults (minDocFrequency 5, minTermFrequency 2) require terms common enough that a tiny index can match nothing — lower them toward 1. Compared fields must be stored for the by-document form. moreLikeThis is search-only — it needs relevance scoring, so count and delete(matching:) reject it.

Facet counts & aggregations

termCounts returns the top values of a field among matching documents, with counts — the classic filter sidebar. The field must be fast: true in the schema:

let tags = try index.termCounts("tag", matching: .parsed("old man"))
// [FacetCount(value: .string("book"), count: 2),
//  FacetCount(value: .string("classic"), count: 1)]

The full tantivy aggregation engine (terms, histogram, stats, min/max/avg, nested sub-aggregations; Elasticsearch-compatible request format) is available through the raw JSON API:

let json = try index.aggregate(#"{"avg_year": {"avg": {"field": "year"}}}"#)

Maintenance & stats

tantivy writes a new segment per commit and only tombstones deleted/updated documents — their space is reclaimed when segments merge. stats() reports the current layout so you can decide when that's worth doing:

let s = try index.stats()
s.documentCount   // live, searchable documents
s.deletedCount    // tombstoned (deleted/updated), not yet reclaimed
s.maxDoc          // live + deleted
s.segmentCount    // many small segments slow searches
s.segments        // per-segment [id, documentCount, deletedCount, maxDoc]

optimize() merges all segments into one, expunging deleted documents, then reloads. It's I/O- and CPU-heavy on a large index and needs the single-writer lock, so run it off the hot path (idle time / a maintenance window), guided by stats():

if s.deletedCount > s.documentCount / 2 || s.segmentCount > 16 {
    try index.optimize()        // compact + reclaim space
}
try index.garbageCollect()      // reclaim files left by merges/deletes

On an open writer the same operations are writer.merge() / writer.garbageCollect().

Convenience helpers

Index has helpers that cut the writer/commit/reload boilerplate and close the Codable loop:

// Scoped writer — auto commit + reload; nothing is committed if the body throws.
try index.write { w in
    try w.addDocument(["title": "Dune", "year": 1965])
}

// One-shot add (dictionary or Encodable), single or batch.
try index.add(["title": "Dune", "year": 1965])
try index.add(book)                          // any Encodable
try index.add(contentsOf: [book1, book2])    // one commit

// Typed search — decode hits straight into your model.
let books = try index.search("dune", as: Book.self)   // [Book]

Typed search uses a model-driven decoder: a scalar property reads a field's first stored value, an array property reads them all (a one-element multi-valued field still decodes into an array), and optionals become nil for absent fields. You can also decode a single hit with hit.decode(Book.self).

Typed collection

SearchCollection<Model> bundles a schema + index behind a typed add/search API:

struct Book: Codable { let title: String; let year: UInt64 }

let books = try SearchCollection<Book>(path: url) { s in
    s.addTextField("title", stored: true)
    s.addU64Field("year", stored: true, fast: true)
}                                            // or .inMemory(schema:) / (index:)

try books.add(Book(title: "Dune", year: 1965))
try books.add(contentsOf: [book1, book2])

books.count                                  // searchable document count
let results = try books.search("dune")       // [Book]
let scored  = try books.searchScored("dune") // [(score: Float, model: Book)]
try books.removeAll()

The schema's field names must match the model's coding keys. books.index exposes the underlying Index for anything the façade doesn't cover.

Building the XCFramework

The committed artifacts/CTantivy.xcframework is produced from tantivy 0.26.1 (pinned via crates.io in rust/Cargo.toml). To rebuild it (e.g. after changing the FFI layer):

# Requires full Xcode + rustup.
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
  scripts/build-xcframework.sh

This cross-compiles the Rust static library for:

• macOS — arm64 + x86_64 (universal)
• iOS device — arm64
• iOS simulator — arm64 + x86_64 (universal)

and assembles them into artifacts/CTantivy.xcframework. iPadOS uses the iOS slices.

Cutting a release

scripts/release.sh <version> automates a release end to end: it builds the xcframework, zips it, computes its checksum, pins release + checksum in Package.swift to the release asset, commits, and publishes the GitHub Release with the zip attached (the tag and asset are created together, so there is never a window where the pinned URL 404s). Run it from a clean main:

# Requires full Xcode, a Swift toolchain, and an authenticated `gh`.
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
  scripts/release.sh 0.1.1

Consumers then resolve the binary straight from the release — nothing large lives in the repo.

Local development without the xcframework

If artifacts/CTantivy.xcframework is absent, Package.swift automatically falls back to linking a host-only static library, so you can iterate and run the tests on your Mac:

scripts/build-host.sh        # builds rust/target/release/libtantivy_ffi.a
swift test

(If your active developer dir is the Command Line Tools, prefix swift test with DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer so XCTest is found.)

How it works

Swift  Sources/Tantivy        — idiomatic API (Schema, Index, IndexWriter, SearchHit)
  │     import CTantivy        — C module (header + module map)
  ▼
C ABI  rust/src/lib.rs         — #[no_mangle] extern "C" shim, JSON in/out
  ▼
Rust   tantivy =0.26.1         — the search engine, pinned from crates.io

The FFI surface is intentionally small and JSON-oriented: schema is a small JSON spec, documents are added as JSON objects (TantivyDocument::parse_json), and search returns a JSON envelope of hits. All heap strings crossing the boundary are owned and freed on the Rust side via tantivy_string_free.

License

The bindings in this repository follow tantivy's MIT license. tantivy © its authors.