Core Types Reference

Unrag's type system is intentionally small. A handful of types cover the core operations—ingesting content, retrieving chunks, managing documents. Understanding these types helps you work with the engine effectively and build custom components when you need them.

IngestInput

When you call engine.ingest(), you pass an object matching this type:

type IngestInput = {
  sourceId: string;
  content: string;
  metadata?: Metadata;
  chunker?: Chunker;
  chunking?: { chunkSize?: number; chunkOverlap?: number; minChunkSize?: number };
  assets?: AssetInput[];
  assetProcessing?: DeepPartial<AssetProcessingConfig>;
};

The sourceId is the logical identifier for your document. This is how you'll reference it later—for updates, deletes, or scoped retrieval. Use consistent, meaningful identifiers like docs:getting-started or ticket:12345. If you ingest with the same sourceId again, you're updating that document; the old chunks are replaced with new ones.

The content string is the text you want to chunk and embed. This is the searchable content.

The optional metadata object stores structured data alongside the document. It appears in retrieval results and can help with filtering or display. Keep values simple and serializable—the adapter stores metadata as JSONB.

The chunker parameter lets you override the chunking algorithm for this specific ingest. Pass a chunker function to use different splitting logic without changing your engine's default configuration. This is useful when you're ingesting heterogeneous content—documentation, code, and prose—with a single engine instance.

The chunking parameter overrides chunking options (chunk size, overlap, minimum size) for this ingest while keeping your configured chunker. Use this when the same algorithm should behave differently for specific content.

The assets array contains rich media inputs like images and PDFs. Connectors like Notion and Google Drive populate this automatically. Each asset can be processed into additional chunks.

The assetProcessing parameter overrides asset handling behavior for this ingest, such as enabling or disabling PDF extraction.

AssetInput

Assets are non-text inputs attached to a document—a PDF embedded in a Notion page, an image in a knowledge base article. The engine can process these into text chunks (via extraction) or embed them directly (for images when using a multimodal embedding provider).

type AssetInput = {
  assetId: string;
  kind: "image" | "pdf" | "audio" | "video" | "file";
  data:
    | { kind: "url"; url: string; headers?: Record<string, string>; mediaType?: string; filename?: string }
    | { kind: "bytes"; bytes: Uint8Array; mediaType: string; filename?: string };
  uri?: string;
  text?: string;
  metadata?: Metadata;
};

The assetId uniquely identifies the asset within the document. The kind indicates what type of media it is. The data field provides either a URL to fetch or raw bytes. The optional text field can contain a caption or alt text, which is used for embedding when direct processing isn't available.

IngestResult

After ingestion completes, you get back information about what was stored:

type IngestResult = {
  documentId: string;
  chunkCount: number;
  embeddingModel: string;
  warnings: IngestWarning[];
  durations: { totalMs: number; chunkingMs: number; embeddingMs: number; storageMs: number };
};

The documentId is the UUID assigned to this document in the database. The chunkCount tells you how many chunks were created. The durations object breaks down where time was spent—embedding typically dominates because of API latency.

The warnings array contains structured information about anything that didn't go perfectly. If an asset was skipped because extraction wasn't enabled, or if a PDF produced no text, you'll find that information here. Treat warnings as observability signals:

const result = await engine.ingest(input);
if (result.warnings.length > 0) {
  console.warn("Ingest warnings:", result.warnings);
}

IngestWarning

When assets are skipped or processing partially fails, the engine emits structured warnings rather than throwing errors. This keeps ingestion flowing while giving you visibility into what was missed.

type IngestWarning =
  | { code: "asset_skipped_unsupported_kind"; message: string; assetId: string; assetKind: AssetKind; ... }
  | { code: "asset_skipped_extraction_disabled"; message: string; assetId: string; assetKind: AssetKind; ... }
  | { code: "asset_skipped_pdf_llm_extraction_disabled"; message: string; assetId: string; assetKind: "pdf"; ... }
  | { code: "asset_skipped_image_no_multimodal_and_no_caption"; message: string; assetId: string; ... }
  | { code: "asset_skipped_pdf_empty_extraction"; message: string; assetId: string; assetKind: "pdf"; ... }
  | { code: "asset_skipped_extraction_empty"; message: string; assetId: string; assetKind: AssetKind; ... }
  | { code: "asset_processing_error"; message: string; assetId: string; stage: "fetch" | "extract" | "embed" | "unknown"; ... };

Each warning includes the assetId so you can identify which asset had the issue, plus a human-readable message. The code field lets you programmatically categorize and handle warnings.

For processing errors, the stage field indicates where the failure occurred:

RetrieveInput

When you call engine.retrieve(), you pass a query and optional parameters:

type RetrieveInput = {
  query: string;
  topK?: number;
  scope?: { sourceId?: string };
};

The query is the search string. It gets embedded using the same model that embedded your chunks, then compared against stored embeddings to find matches.

The topK parameter controls how many results you get back. The default of 8 is usually a good starting point—enough to find relevant content without overwhelming downstream processing.

The scope parameter filters results. When you provide { sourceId: "docs:" }, only chunks whose source ID starts with "docs:" are considered. This is how you implement scoped search, tenant isolation, or collection filtering.

RetrieveResult

Retrieval returns the matching chunks with metadata:

type RetrieveResult = {
  chunks: Array<Chunk & { score: number }>;
  embeddingModel: string;
  durations: { totalMs: number; embeddingMs: number; retrievalMs: number };
};

Each chunk includes a score representing similarity to the query. With cosine distance (the default), lower scores mean higher similarity. The chunks are sorted by score ascending, so the most relevant results come first.

Chunk

The chunk type represents a piece of a document:

type Chunk = {
  id: string;
  documentId: string;
  sourceId: string;
  index: number;
  content: string;
  tokenCount: number;
  metadata: Metadata;
  embedding?: number[];
  documentContent?: string;
};

During retrieval, chunks include the score field. The embedding field is present during upsert operations but not returned in query results. The documentContent field contains the full document text during upsert and may be empty if you've disabled document content storage.

Metadata

Metadata is a flexible JSON structure:

type MetadataValue = string | number | boolean | null;
type Metadata = Record<string, MetadataValue | MetadataValue[] | undefined>;

Keep metadata simple and serializable. The adapter stores it as JSONB, so you can use it in queries, but complex nested structures are harder to work with.

EmbeddingProvider

If you need to implement a custom embedding provider, it follows this interface:

type EmbeddingInput = {
  text: string;
  metadata: Metadata;
  position: number;
  sourceId: string;
  documentId: string;
};

type EmbeddingProvider = {
  name: string;
  dimensions?: number;
  embed: (input: EmbeddingInput) => Promise<number[]>;
};

The embed function receives context about what's being embedded, though most implementations only use the text field. Return a numeric array representing the embedding vector.

VectorStore

The store adapter interface handles database operations:

type VectorStore = {
  upsert: (chunks: Chunk[]) => Promise<void>;
  query: (params: { embedding: number[]; topK: number; scope?: { sourceId?: string } }) => Promise<Array<Chunk & { score: number }>>;
  delete: (input: DeleteInput) => Promise<void>;
};

The upsert method replaces stored content for the logical document. The query method finds similar chunks. The delete method removes documents by source ID or prefix.

DeleteInput

Deletion supports exact match or prefix match:

type DeleteInput =
  | { sourceId: string }
  | { sourceIdPrefix: string };

Use exact deletion for single documents. Use prefix deletion for namespaces (e.g., deleting all documents for a tenant).

Chunker and ChunkingOptions

Custom chunkers implement this interface:

type ChunkingOptions = {
  chunkSize: number;
  chunkOverlap: number;
  minChunkSize?: number;
  separators?: string[];
};

type ChunkText = {
  index: number;
  content: string;
  tokenCount: number;
};

type Chunker = (content: string, options: ChunkingOptions) => ChunkText[] | Promise<ChunkText[]>;

Your chunker receives the document content and configuration options. Return an array of chunks with sequential indices, the chunk text, and accurate token counts.

ContextEngineConfig

When creating an engine, you provide this configuration:

type ContextEngineConfig = {
  embedding: EmbeddingProvider;
  store: VectorStore;
  extractors?: AssetExtractor[];
  assetProcessing?: DeepPartial<AssetProcessingConfig>;
  storage?: { storeChunkContent?: boolean; storeDocumentContent?: boolean };
  defaults?: Partial<ChunkingOptions>;
  chunker?: Chunker;
  idGenerator?: () => string;
};