Notopia

A conversational note-taking agent — a chat-based system that lets you manage personal notes entirely through natural language. Built with LangGraph, Gemini/Ollama, and SQLite.

Features

• Add notes — create notes with titles, bodies, and tags via natural language
• Search & list — find notes by keyword, tag, date range, lexical or semantic similarity
• Modify notes — update content, titles, or tags with confirmation before changes
• Delete notes — remove notes with explicit confirmation (interrupt-based, not prompt-based)
• Reason over notes — summarise by tag, compare notes, detect contradictions
• Handwritten note ingestion — transcribe images of handwritten notes via the chat model's vision capabilities, with draft review before saving
• Multi-turn awareness — follow-up messages like "add a deadline to that last note" work via persistent conversation history
• Hybrid search — combines SQLite FTS5 (lexical) with sqlite-vec (semantic vectors) via Reciprocal Rank Fusion

Setup

Prerequisites

• Python 3.11+
• uv package manager
• A free Gemini API key from aistudio.google.com/apikey (no credit card needed)

Install

git clone "https://github.com/nouran-19/notopia.git" && cd notopia
uv sync --extra dev
cp .env.example .env
# Edit .env and set GEMINI_API_KEY=your-key-here

Run

uv run python -m notopia

Run the evaluation harness

# Deterministic tests only (no LLM needed — 22 tests)
uv run python -m pytest evals/ -v

# Full report with pass/fail summary
uv run python -m evals.report

# Include LLM judge (needs Ollama + qwen3:8b)
uv run python -m evals.report --with-judge

Optional: Ollama for local inference

If you have hardware that can run an 8B model at interactive speed:

# Install Ollama from https://ollama.com/download
ollama pull qwen3:8b           # chat model (verify: ollama show qwen3:8b → tools in Capabilities)
ollama pull nomic-embed-text   # embedding model for semantic search

# Update .env
LLM_PROVIDER=ollama
LLM_MODEL=qwen3:8b

Environment Variables

Variable	Default	Description
LLM_PROVIDER	gemini	gemini or ollama
GEMINI_API_KEY	(empty)	Required when LLM_PROVIDER=gemini
GEMINI_MODEL	gemini-3.1-flash-lite	Gemini model name
LLM_MODEL	qwen3:8b	Ollama model name (when LLM_PROVIDER=ollama)
EMBEDDING_MODEL	nomic-embed-text	Ollama embedding model for semantic search
NOTES_DB_PATH	data/notes.db	Path to notes SQLite database
AGENT_STATE_DB_PATH	data/agent_state.db	Path to LangGraph checkpoint database
USER_ID	default	Stub user scoping (no real auth)

Changing the User

There is no real authentication system. The active user is determined by the USER_ID environment variable. All notes created and searched for will be scoped to this user in the database.

Option 1: Persistent (via .env) Add this to your .env file:

USER_ID=my_custom_user

Option 2: Per-session (via Terminal) Set it before running the app (PowerShell example):

$env:USER_ID="my_custom_user"
uv run python -m notopia

Project Structure

notopia/
├── src/notopia/
│   ├── config.py        # Settings from env vars (pydantic-settings)
│   ├── storage.py       # SQLite CRUD + FTS5 full-text search
│   ├── embeddings.py    # sqlite-vec vector operations + RRF merging
│   ├── schemas.py       # Pydantic input models for all tools
│   ├── tools.py         # LangChain @tool functions with interrupt()
│   ├── llm.py           # LLM provider factory (Gemini / Ollama)
│   ├── ocr.py           # Handwriting transcription via multimodal LLM
│   ├── graph.py         # LangGraph StateGraph definition
│   └── cli.py           # Interactive CLI entry point
├── evals/
│   ├── test_scenarios.py  # 19 deterministic tests
│   ├── test_judged.py     # 3 judged scenarios with rubrics
│   ├── report.py          # Pass/fail report runner
│   └── conftest.py        # Pytest fixtures
├── docs/
│   └── tool_schemas.md    # Tool/function schema documentation
├── pyproject.toml
└── .env.example

Design Rationale & Trade-offs

Why LangGraph over a bare loop?

A hand-rolled while True: call_llm(); parse_tool_calls() loop works for trivial agents, but falls apart when you need:

1. Persistent state across restarts — LangGraph's SqliteSaver checkpointer serializes the full conversation state to SQLite. Resume a conversation after a process restart without replaying messages.
2. Structural human-in-the-loop — interrupt() physically pauses the graph and can only resume when the user responds via Command(resume=value). This is fundamentally safer than relying on the LLM to "remember" to ask for confirmation, because LLMs can skip safety checks. The confirmation logic lives in deterministic Python, not in a prompt.
3. Explicit, debuggable flow — the graph is three nodes (agent → tools → agent). You can draw it on a whiteboard. When something breaks, the checkpoint contains the exact state at the failure point.

A bare loop would require reimplementing all of this, creating a hand-rolled state serialization, a custom interrupt mechanism, and manual message-list management — more code, more bugs, less battle-tested.

Why interrupt-based confirmation?

The core safety property: destructive actions (delete, update) cannot execute without explicit user approval, and this guarantee does not depend on the LLM behaving correctly.

When delete_note calls interrupt(), the LangGraph runtime physically pauses the graph. The tool function is suspended mid-execution. No amount of prompt injection or LLM hallucination can bypass this — the code after interrupt() literally does not run until Command(resume=value) is called by the CLI.

This is the difference between "the prompt tells the LLM to ask first" (hope-based safety) and "the runtime prevents execution until approval" (structural safety).

Why hybrid search (FTS5 + sqlite-vec + RRF)?

Pure keyword search misses semantically related notes (e.g., "find notes about project deadlines" won't match a note that says "deliverable due Friday"). Pure vector search can miss exact keyword matches. Hybrid search combines both:

• FTS5 (SQLite's built-in full-text search): fast, exact keyword matching with BM25 ranking
• sqlite-vec: cosine-similarity search over 768-dim embeddings from nomic-embed-text
• Reciprocal Rank Fusion (RRF): merges the two ranked lists with the formula score = Σ 1/(k + rank). No tuning needed, works well when score distributions differ.

Why sqlite-vec over Chroma? Both notes and vectors live in the same SQLite file — no extra process, no extra dependency. At this scale (personal notes, dozens-to-hundreds of entries), a separate vector DB adds operational complexity for no benefit.

Why a general VLM for OCR instead of a specialized model?

gemini-3.1-flash-lite handles both chat and image transcription, eliminating a second model dependency. A specialized fine-tuned handwriting model (e.g. Microsoft's TrOCR) would likely perform better on messy handwriting — but:

1. Adds a separate model dependency and inference pipeline
2. Requires more implementation time than available
3. The interrupt-based confirmation step acts as a safety net: the user always reviews the transcription before it's saved

This is an explicit time-budget trade-off, not a permanent architecture decision.

Provider decision: why Gemini as default?

The development machine cannot run qwen3:8b at usable interactive speed. Rather than hide this as a limitation, it's disclosed honestly: Gemini is the default because it works reliably on constrained hardware via API. Ollama remains fully implemented and switchable via a single env var for machines with better specs.

Orchestration alternatives considered (options I thought about)

• AnythingLLM: A turnkey RAG app whose own agent-mode would absorb the exact tool-design and state-management work this project is meant to demonstrate. Using it would mean the interesting engineering decisions are made by AnythingLLM's framework, not by us.
• Letta (MemGPT): Its persistent-memory model would compete with, not complement, the SqliteSaver checkpointer already in use. Two competing state-management systems in one agent is complexity without benefit.
• Ollama: Remains the inference layer underneath LangGraph regardless — it isn't in competition with the other two, since both AnythingLLM and Letta would also call out to a model runtime like Ollama themselves.

What was deferred

• MCP server exposure
• Docker/docker-compose
• Real multi-user auth

Evaluation Harness

22 test scenarios total:

Type	Count	Method	Requires LLM?
Deterministic	19	Assert tool call results + DB state	No (mocked interrupt)
Judged (data)	3	Verify correct data retrieval + print rubric	No
LLM Judge	3	qwen3:8b evaluates against rubric	Yes (Ollama)
OCR	2	Mocked transcription, assert DB state	No

Why qwen3:8b as judge instead of Gemini? Using the same model family (Gemini) for both agent and judge introduces self-preference bias. qwen3:8b is a genuinely different model family, making the judge's assessment more independent. Since the judge only runs a handful of times per report (not on every interactive turn), local hardware slowness is not a cost concern.

Inspiration

note-gen

DataCamp: Building an AI Note-taking Application