imap-dispatcher

Rule-based IMAP mail dispatcher with optional AI classification.

Features

• Three-tier matching — substring rules first, then skip rules, then optional AI classification
• AI classification via Mistral — free-text rule descriptions, with graceful degradation if the API is unreachable
• Auto-learning — successful AI classifications are remembered so future mail from the same sender is matched instantly, without another AI call
• Feedback loop — adapts learned rules automatically when you manually move a mail the dispatcher sorted
• Dry-run mode — preview every move (and every learn) without changing anything
• Environment-variable secrets — override host/username/password/API key via env vars, 12-factor style
• Single YAML config — one file, no database, no external state beyond a small state.json
• Minimal dependencies — imapclient, pyyaml, mistralai, httpx, html2text
• Python 3.10+ — runs anywhere, no Raspberry Pi requirement
• UID-based state tracking — safe alongside other IMAP clients (e.g. Thunderbird) working the same mailbox in parallel

Quick Start

python3 -m venv .venv
source .venv/bin/activate
pip install imap-dispatcher

cp config.example.yaml config.yaml
chmod 600 config.yaml
# edit config.yaml: host, username, password, rules ...

imap-dispatcher --dry-run

Install from source

git clone <repo-url>
cd imap-dispatcher

python3 -m venv .venv
source .venv/bin/activate
pip install .

cp config.example.yaml config.yaml
chmod 600 config.yaml
# edit config.yaml: host, username, password, rules ...

imap-dispatcher --dry-run

Alternative install (venv copy)

If you prefer not to install the package (e.g. for a minimal deployment), copy the module directly and run it with python -m:

cp -r imap_dispatcher/ $INSTALL_DIR/
cp requirements.txt $INSTALL_DIR/
cd $INSTALL_DIR

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

cp config.example.yaml config.yaml
chmod 600 config.yaml

.venv/bin/python -m imap_dispatcher --config config.yaml --dry-run

Both invocation styles — imap-dispatcher ... (console script) and python -m imap_dispatcher ... (module) — work identically and are used interchangeably throughout this README.

Configuration

All options live in config.yaml (template: config.example.yaml, fully commented).

Key	Description	Default
host	IMAP server, e.g. imap.example.com	— (required)
username	Mail account username/address	— (required)
password	Password in plain text (see Security below)	— (required)
inbox	Source folder	INBOX
state_file	Path to the UID-tracking state file	state.json
log_file	Optional path to a log file (rotating, 1 MB, 3 backups); empty = stdout only	""
rules	List of sorting rules (see Rule fields below)	—
skip_rules	List of skip rules that exclude senders from AI classification	—
ai	AI classification block (see below)	—
ai_language	Language of the AI classification prompt (en | de)	en
private_domains	Domains never auto-classified by AI (see below)	common freemail providers
feedback_ignore_folders	Folders the feedback scan treats as neutral (see below)	empty list

AI classification (ai: block)

ai:
  provider: mistral                 # currently supported: mistral
  api_key: ""                       # or set MISTRAL_API_KEY in the environment (env wins)
  model: "mistral-small-latest"     # model name, default is Mistral Small

• provider — only mistral is currently supported.
• api_key — your Mistral API key (create one at console.mistral.ai); can also be provided via the MISTRAL_API_KEY environment variable.
• model — the Mistral model name; defaults to mistral-small-latest.

ai_language controls the language of the AI classification prompt: en (default) or de.

Rule fields

Each rule under rules: supports:

Field	Description
name	Label used in logs
target_folder	Destination folder for matches
from_contains	List of substrings to match in the From address
from_name_contains	List of substrings to match in the From display name
to_contains	List of substrings to match in the To header
subject_contains	List of substrings to match in the Subject header
mark_seen	Optional: true to set the \Seen flag on move
ai_description	Free-text description for AI classification (optional; only applies if no substring matcher already matched)

Every rule needs at least one matcher field. The first matching rule wins. Fields within a single rule are AND-ed (all must match); entries within a field are OR-ed (any one is enough). skip is only valid inside skip_rules: — it is rejected inside rules:.

private_domains

Senders from these domains are treated as personal mail and are never auto-classified by AI. Defaults to common freemail providers (Gmail, Outlook, Yahoo, iCloud, Proton, and others); override the list in config.yaml if needed.

feedback_ignore_folders

Some folders don't carry any classification signal — for example a personal work-in-progress folder you park mail in temporarily, or Trash. Without this setting, moving mail into such a folder is indistinguishable from a genuine correction and triggers the third feedback outcome: the sender's learned rule is un-learned. That's not what you want for a WIP folder — it would silently and repeatedly un-learn a correct rule every time you park mail there.

feedback_ignore_folders lists folder names that the feedback scan treats as neutral: a tracked mail found there is simply discarded from tracking, with no change to learned.yaml at all — the same outcome as the mail not being found anywhere. This check takes precedence over all three correction outcomes, even when the ignored folder is itself a configured rule's target_folder or equals inbox.

Default is an empty list — no install changes behavior without an explicit edit. Recommended values: Trash, Junk/Spam, and any personal work-in-progress folder.

Environment variables

12-factor style overrides (e.g. via a systemd EnvironmentFile= or a shell export): IMAP_HOST, IMAP_USERNAME, IMAP_PASSWORD, and MISTRAL_API_KEY override the corresponding values in config.yaml when set. Environment values always take precedence over YAML (env wins); an empty environment variable falls through to the YAML value.

Security

This config file contains your mail password in plain text.

• Restrict permissions: chmod 600 config.yaml
• Even better: use a dedicated app password (if your provider offers one) instead of your main password
• Never commit config.yaml to a Git repo

CLI Reference

Both invocation styles are shown below and behave identically: imap-dispatcher ... (after pip install .) and python -m imap_dispatcher ... (venv-copy path).

imap-dispatcher --config config.yaml [OPTIONS]
python -m imap_dispatcher --config config.yaml [OPTIONS]

Flag	Description
--version	Print the version and exit
--config PATH	Path to config.yaml (default: config.yaml)
--dry-run	Only log moves, do not execute them
--sort-existing	Check all existing INBOX mail against the rules and sort it
--sort-existing-ai	Like --sort-existing, but including AI classification for mail without a substring match (opt-in, slower due to ~2.6s per mail)
--limit N	Process only the last N mails (for --sort-existing / --sort-existing-ai)
--review-learned	Show learned sender rules (audit)

--review-learned reads learned.yaml from the config file's parent directory and prints an audit report, then exits — it does not connect to the mailbox.

Advanced Topics

Three-Tier Matching

Every incoming mail runs through up to three tiers until a target folder is found:

1. Tier 1 — substring rules — first matching rule wins. Fast, deterministic, no network call.
2. Tier 2 — skip rules — prevent the LLM call for known senders (e.g. known advertising without a matching rule). The mail stays in the INBOX.
3. Tier 3 — AI classification — only if Mistral was reachable at startup. On outage the dispatcher keeps running with substring rules only (graceful degradation).

incoming mail
     │
     ▼
Tier 1: substring rules ──match──► move + learn sender
     │ no match
     ▼
Tier 2: skip rules ──match──► stays in INBOX (no AI call)
     │ no match
     ▼
Tier 3: AI classification ──match──► move + learn sender
     │ no match / AI unreachable
     ▼
stays in INBOX

(later) feedback scan detects manual corrections to AI-classified moves

UID Tracking & State

• First run — the dispatcher reads the current highest UID in the mailbox and stores it in state.json. No existing mail is sorted; only a baseline is set. From the next run on, only mail with a higher UID (newly arrived) is processed.
• UIDVALIDITY reset — if the IMAP server reports a new UIDVALIDITY (e.g. after a mailbox migration), the dispatcher discards the old state and sets a new baseline. Existing mail is again not sorted — only new mail.
• Sorting existing mail — --sort-existing checks all INBOX mail against the rules and moves matches. Combinable with --dry-run for a preview. Does not modify state.json.
• Resetting state — deleting state.json has the same effect as a first run. state_file is a config key (default state.json, resolved relative to the working directory).

Auto-Learning

Every successful AI classification automatically learns the sender's full email address as a from_contains entry in a separate file learned.yaml (next to config.yaml, which is itself never modified). On the next run, tier 1 (substring matching) matches this sender directly — no more LLM call needed.

learned.yaml can be edited by hand: deleting a line removes the learned association again ("un-learn"). Use --review-learned to review all learned entries at a glance; output format is one line per entry:

from_addr -> rule_name (target_folder) [date]

Skip entries are shown as INBOX (Skip).

Guards: mails without a sender address are never learned (poisoning protection); already-learned addresses are never written twice; --dry-run suppresses every write.

Feedback Loop

When you manually move a mail that the dispatcher had sorted, it notices and adapts. Moves are tracked for 7 days; on each run the dispatcher checks whether tracked mails are still where it put them. If you moved one to a different rule's folder, the learned sender is re-learned for that rule; moving it back to INBOX means the sender is skipped from now on; moving it to any other (unmapped) folder un-learns the sender — unless that folder is listed in feedback_ignore_folders, in which case no learned-rule change is made at all.

Technical details

• Each AI-classified move is recorded as a TrackedMove (message ID, rule name, sender address, target folder, moved-at timestamp) in feedback_tracking.json, stored next to config.yaml and written atomically with mode 0600.
• Tracking entries expire after 7 days; corrections made later than that are silently ignored.
• Folders listed in feedback_ignore_folders are checked before the three outcomes below and produce no learned.yaml change — the tracking entry is simply discarded.
• Three correction outcomes:
  • Re-learn — the mail is found in another rule-mapped folder: the learned entry's rule is rewritten to the new rule.
  • Skip — the mail is moved back to INBOX: the learned entry becomes a skip entry, and the sender stays in INBOX from now on.
  • Un-learn — the mail is found in an unmapped folder: the learned entry is removed entirely.
• Corrections are logged with the [CORRECTION] tag.
• Guards: mails without a Message-ID are never tracked; a tracked mail found nowhere is silently discarded without touching learned.yaml; --dry-run logs the would-be correction but never writes.

Scheduling

Two ways to run imap-dispatcher on a schedule, Linux only.

systemd timer (recommended)

# 1. Create a dedicated, non-root system user
sudo useradd --system --no-create-home --shell /usr/sbin/nologin imap-dispatcher
sudo chown -R imap-dispatcher:imap-dispatcher $INSTALL_DIR

# 2. Copy the unit files
sudo cp imap-dispatcher.service /etc/systemd/system/
sudo cp imap-dispatcher.timer /etc/systemd/system/

The shipped imap-dispatcher.service hardcodes an install prefix (/opt/imap-dispatcher) in three lines — WorkingDirectory=, ExecStart=, and ReadWritePaths=. Edit those three paths to your $INSTALL_DIR before enabling the unit, e.g.:

sudo sed -i 's|/opt/imap-dispatcher|'"$INSTALL_DIR"'|g' /etc/systemd/system/imap-dispatcher.service

The unit also has a commented EnvironmentFile= hook for providing secrets via environment variables instead of config.yaml (see Environment variables above) — chmod 600 the referenced env file if you use it.

sudo systemctl daemon-reload
sudo systemctl enable --now imap-dispatcher.timer

# Check status and logs
systemctl status imap-dispatcher.timer
journalctl -u imap-dispatcher.service

The timer settings, quoted verbatim from imap-dispatcher.timer:

OnBootSec=2min
OnUnitActiveSec=1min
Persistent=true

OnBootSec=2min gives the network time to come up after boot. Because the service is Type=oneshot triggered by a timer, overlapping runs are impossible — the timer will not start a new run while the previous one is still active.

The unit ships a hardening block (NoNewPrivileges, ProtectSystem=strict, ReadWritePaths, and more). Check the applied score with:

systemd-analyze security imap-dispatcher.service

TimeoutStartSec=600 covers the worst-case retry runtime (IMAP retries at 5/15/45s, plus buffer).

cron (alternative)

A shorter alternative if you don't want to manage systemd units. Cron does not prevent overlapping runs by itself, so wrap the command in flock -n:

* * * * * /usr/bin/flock -n /var/lock/imap-dispatcher.lock $INSTALL_DIR/.venv/bin/imap-dispatcher --config $INSTALL_DIR/config.yaml

• flock -n skips a run while the previous one still holds the lock. systemd's oneshot+timer combination gives this for free; cron does not — and since the worst-case retry runtime (~10 minutes) can exceed the 1-minute interval, the lock matters. The kernel releases the lock automatically on process exit; no stale-lockfile cleanup needed.
• Use absolute paths — cron's PATH is minimal.
• Set log_file in config.yaml, since cron output has no journald integration.
• Env-var secrets can go in the crontab itself or a sourced env file.

Troubleshooting

• Config validation failures print to stderr with the prefix Config error: followed by indented - <detail> lines, and exit with code 1. Example details: 'host' missing (required), 'target_folder' missing (required), 'state_file' directory not writable, unsupported ai provider '...' (supported: mistral).
• no such mailbox — the IMAP folder-hierarchy separator depends on your server (Posteo uses ., e.g. INBOX.Newsletter). Verify the exact folder name with --dry-run or your mail client before trusting a rule's target_folder.
• AI unreachable at startup — logs a warning and the dispatcher continues running with substring rules only (graceful degradation); no crash.
• Retry behavior — IMAP run-level retries use delays of 5/15/45 seconds (4 attempts total); Mistral 429 rate limits are retried with 2/5/15 second backoff.
• File locations — learned.yaml and feedback_tracking.json are stored next to config.yaml (derived from --config's parent directory); state_file is a config key (default state.json, resolved relative to the working directory).
• learned.yaml and feedback_tracking.json are written with mode 0600 because they contain sender addresses.