Skip to content

feederv4: optional MongoDB sink for set requests - #211

Open
javogc wants to merge 1 commit into
COVESA:masterfrom
mongodb-industry-solutions:feeder-sinks
Open

javogc wants to merge 1 commit into
COVESA:masterfrom
mongodb-industry-solutions:feeder-sinks

Conversation

@javogc

@javogc javogc commented Oct 9, 2026

Copy link
Copy Markdown

What

An optional output for feederv4 that stores the in-scope Set requests the feeder receives from the server in a MongoDB collection.

MONGODB_URI=<connection string> ./feederv4 --mongosink [--mongodb <database>] [--mongocoll <collection>]

The database and collection default to vissr / feederSets. Each Set request becomes one document: dot-separated paths become nested documents, and numeric/boolean values keep their native type so they can be queried and aggregated, e.g.

{"_id": "2026-10-05T23:02:44.611Z", "ts": "2026-10-05T23:02:44.611Z", "Vehicle": {"Speed": 50}, "updatedAt": <date>}

Why

This lets a cloud-side VISSR deployment persist incoming vehicle data for history, analytics or backend applications without having to write a custom feeder.

Design notes

  • Backward compatible: the sink is off by default, and nothing changes when --mongosink is not used.
  • Grouping: the server delivers a multi-set request to the feeder as one message per data point. Data points arriving within 200 ms of each other are grouped, share the timestamp of the first one, and are upserted into the same document.
  • Non-blocking: writes go through a bounded queue on a background goroutine. A slow or unreachable database drops data points with an error log and never blocks the feeder.
  • Documented in feeder/feeder-template/feederv4/README.md.

Dependencies

Adds go.mongodb.org/mongo-driver/v2 (v2.4.0). As a result go mod tidy bumps the indirect golang.org/x/{net,sync,sys,text} modules and adds golang.org/x/crypto plus a few indirect driver dependencies. The go directive stays at 1.22.1. If you'd rather keep the module free of the driver, we're happy to put the sink behind a build tag.

Testing

  • Unit tests for the document mapping, typed values and request grouping (go test -race ./feeder/feeder-template/feederv4/).
  • go build ./..., go vet and go test -race over the module (same as the test workflow) pass locally.
  • Exercised end to end against MongoDB Atlas.

Branches

Per the README, features merged to master are also carried to v3.1.5, v3.2 and v4.0. Let us know if you'd like us to open those PRs.

Adds an optional, disabled-by-default output that stores the in-scope set requests
the feeder receives from the server in a MongoDB collection (--mongosink, with the
connection string in MONGODB_URI and --mongodb / --mongocoll for the names).

The data points of one multi-set request are grouped by arrival time and upserted
into one document with nested paths and typed values. Writes are asynchronous and
never block the feeder.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant