Skip to content

Improve canvas text#831

Open
eXecutable wants to merge 4 commits into
lightning-js:mainfrom
eXecutable:improve-canvas-text
Open

Improve canvas text#831
eXecutable wants to merge 4 commits into
lightning-js:mainfrom
eXecutable:improve-canvas-text

Conversation

@eXecutable

Copy link
Copy Markdown
Contributor

getImageData forces the browser to copy the pixels drawn by the canvas to JS world. By using ImageBitmap instead the pixels can be drawn on the GPU and stay there.

@eXecutable eXecutable marked this pull request as draft July 7, 2026 13:44
@eXecutable eXecutable marked this pull request as ready for review July 7, 2026 14:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the canvas text rendering path to avoid getImageData pixel readback by using GPU-friendly sources (ImageBitmap via transferToImageBitmap on OffscreenCanvas, or passing an HTMLCanvasElement directly) as texture inputs.

Changes:

  • Extend texture source typing to allow HTMLCanvasElement (and broaden internal image sources in ImageTexture).
  • Update the canvas text renderer to capture output as ImageBitmap/HTMLCanvasElement instead of ImageData.
  • Update WebGL and Canvas texture upload paths to accept HTMLCanvasElement as a valid source type.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/core/textures/Texture.ts Adds HTMLCanvasElement to the core TextureData union.
src/core/textures/ImageTexture.ts Broadens src typing to include ImageBitmap/HTMLCanvasElement and unifies non-string load handling.
src/core/text-rendering/TextRenderer.ts Updates CanvasRenderInfo.imageData to `ImageBitmap
src/core/text-rendering/CanvasTextRenderer.ts Switches capture from getImageData to transferToImageBitmap/direct-canvas source.
src/core/renderers/webgl/WebGlCtxTexture.ts Allows uploading textures from HTMLCanvasElement in the WebGL path.
src/core/renderers/webgl/WebGlCtxSubTexture.ts Extends subtexture data union to include HTMLCanvasElement.
src/core/renderers/canvas/CanvasTexture.ts Allows HTMLCanvasElement as a direct image source in the Canvas renderer path.
src/core/CoreTextNode.ts Passes through the new canvas text image source type to ImageTexture.
Comments suppressed due to low confidence (1)

src/core/renderers/webgl/WebGlCtxTexture.ts:209

  • HTMLCanvasElement support was added to the WebGL upload path, but there’s no unit test to ensure this new branch is exercised (and that upload + UNPACK_PREMULTIPLY_ALPHA_WEBGL behavior matches the ImageData/HTMLImageElement path). Consider extending WebGlCtxTexture.test.ts with a stubbed HTMLCanvasElement and asserting it goes through the same upload branch.
    const memoryPadding = 1.1; // Add padding to account for GPU Padding
    const isImageBitmap =
      typeof ImageBitmap !== 'undefined' && tdata instanceof ImageBitmap;
    const isHTMLCanvas =
      typeof HTMLCanvasElement !== 'undefined' &&
      tdata instanceof HTMLCanvasElement;

    // If textureData is null, the texture is empty (0, 0) and we don't need to
    // upload any data to the GPU.
    if (
      isImageBitmap ||
      isHTMLCanvas ||
      tdata instanceof ImageData ||
      // not using typeof HTMLI mageElement due to web worker
      isHTMLImageElement(tdata) === true
    ) {

Comment on lines +180 to +186
if (isOffscreen) {
drawCanvas = canvas;
} else {
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
drawCanvas = stage!.platform.createCanvas();
context = drawCanvas.getContext('2d') as CanvasRenderingContext2D;
}
Comment on lines +385 to +389
let imageData: ImageBitmap | HTMLCanvasElement | null = null;
if (drawCanvas.width > 0 && drawCanvas.height > 0) {
if (isOffscreen) {
imageData = (drawCanvas as OffscreenCanvas).transferToImageBitmap();
} else {
Comment on lines +175 to +179
// For OffscreenCanvas we reuse the shared canvas and snapshot via
// transferToImageBitmap at the end. For HTMLCanvasElement we allocate a
// fresh canvas per text node so it can be passed directly as the texture
// source — no pixel readback or intermediate copy needed.
let drawCanvas: HTMLCanvasElement | OffscreenCanvas;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants