Skip to content

[Experiment] Render image overlay on top of built-in camera for SRT/SRTLA stream output - #322

Open
dimadesu wants to merge 1 commit into
mainfrom
experiment-add-bitmap-overlay-to-stream-output
Open

[Experiment] Render image overlay on top of built-in camera for SRT/SRTLA stream output#322
dimadesu wants to merge 1 commit into
mainfrom
experiment-add-bitmap-overlay-to-stream-output

Conversation

@dimadesu

@dimadesu dimadesu commented May 27, 2026

Copy link
Copy Markdown
Owner

This is just to see if it is possible to add any kind of image overlay.
Status: ✅ Works.

  • This can render hardcoded image on top of stream output (not preview).
  • So far tested only in emulator.

Summary of what's in place:

  • GL overlay pipeline: OpenGlRenderer draws a second blended quad after the camera quad each frame
  • Persistence: VideoInput.setOverlayBitmap() survives processor recreations
  • API: IVideoInput.setOverlayBitmap(bitmap) — call it anytime with a Bitmap or null to clear
  • Positioning: bottom-right, 40% of frame width, aspect-ratio preserved

The preview not showing it is because the preview SurfaceView likely renders directly from the camera source before the processor, while the encoder surface goes through the processor pipeline. If you want overlay on preview too in the future, that would require hooking into the preview surface output as well.


Is this a good way to do it if I ever decide to continue extending/improving this feature?

Honest assessment — it works for a prototype but has a few structural issues that would become painful to extend:

What's fine:

  • The GL rendering approach (second draw pass with blending) is correct and efficient
  • OverlayProgram as a self-contained shader program is clean

What's awkward if you extend:

  • pendingOverlayBitmap in VideoInput — VideoInput is managing rendering state it shouldn't own. If you add more overlay properties (position, size, opacity, multiple layers), VideoInput becomes a bag of rendering concerns. Better: have the processor factory/builder accept an overlay config, or make the processor itself durable across recreations.
  • Hardcoded position/size in OpenGlRenderer.setOverlayBitmap() — The renderer computes NDC coords at upload time. To support repositioning or resizing, you'd have to re-upload. Better: separate the bitmap upload from the placement (pass position/size as parameters or as a data class).
  • Single overlay only — If you ever want logo + text + status indicator, the current design doesn't generalize. A list of OverlayLayer(texture, rect, alpha) would be more extensible.
  • Processor recreation is the root design smell — The real issue is that StreamPack recreates the processor on config changes. Your workaround (re-apply bitmap via setter intercept) works but is fragile.

If you continue, I'd suggest:

  • Extract an OverlayConfig data class (bitmap, position, size, alpha)
  • Move overlay state ownership to the processor level (not VideoInput)
  • Support a list of overlays rather than a single one

But for validating the concept — streaming an image overlay over SRT — this is totally fine as-is.

Summary of what's in place:
- GL overlay pipeline: OpenGlRenderer draws a second blended quad after the camera quad each frame
- Persistence: VideoInput.setOverlayBitmap() survives processor recreations
- API: IVideoInput.setOverlayBitmap(bitmap) — call it anytime with a Bitmap or null to clear
- Positioning: bottom-right, 40% of frame width, aspect-ratio preserved

The preview not showing it is because the preview SurfaceView likely renders directly from the camera source before the processor, while the encoder surface goes through the processor pipeline. If you want overlay on preview too in the future, that would require hooking into the preview surface output as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant