[PR #633] Add ImageRenderable for pixel based images #673

New issue

Open

opened 2026-03-02 23:47:37 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 23:47:37 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/anomalyco/opentui/pull/633
Author: @iamlemec
Created: 2/5/2026
Status: 🔄 Open

Base: main ← Head: image-renderable

📝 Commits (1)

7a5a40b add ImageRenderable element to render images with the kitty graphics protocol

📊 Changes

16 files changed (+483 additions, -4 deletions)

View changed files

📝 bun.lock (+16 -0)
📝 packages/core/package.json (+2 -0)
📝 packages/core/src/Renderable.ts (+19 -1)
📝 packages/core/src/buffer.ts (+14 -0)
➕ packages/core/src/examples/assets/blue.png (+0 -0)
➕ packages/core/src/examples/assets/snake.png (+0 -0)
➕ packages/core/src/examples/image.ts (+99 -0)
➕ packages/core/src/renderables/Image.ts (+85 -0)
📝 packages/core/src/renderables/composition/constructs.ts (+6 -0)
📝 packages/core/src/renderables/index.ts (+1 -0)
📝 packages/core/src/renderer.ts (+6 -1)
📝 packages/core/src/zig.ts (+37 -1)
📝 packages/core/src/zig/buffer.zig (+91 -0)
➕ packages/core/src/zig/kitty.zig (+48 -0)
📝 packages/core/src/zig/lib.zig (+12 -0)
📝 packages/core/src/zig/renderer.zig (+47 -1)

📄 Description

This adds an ImageRenderable element to render pixel images. The rendering closely mirrors the current/next buffer approach from OptimizedBuffer but using PixelBuffer, which is basically a collection of PixelPatch objects. This allows for flicker-free, non-reduntant rendering.

The main interface is a new renderPixels function on Renderables that allows any renderable (not just ImageRenderable) to display arbitrary numbers of images. Images are hashed to prevent redundant writes. The actual output is buffered with regular text output on the native side to prevent flicker.

I tried to keep the changes as minimal as possible. I did need to increase the size of the output buffer from 2MB to 8MB to accomodate images. The only additional dependency is pngjs for decoding PNG files. There is an example with animated images called image.ts. This currently only outputs kitty format, but other formats should not be difficult. The data input is just a Uint8Array of RGBA pixels, but could push file loading from example to core library.

In terms of z-index issues, so far as I can tell, images are always placed above text, even if the text is written afterwards (at least in kitty/ghostty). So doing truly interleaved text and pixels would be non-trivial, but not impossible. You'd basically have to go in and transparent blit out parts of the images if they are occluded. This doesn't do that, so the pixel images are always just an overlay on top.

See issue #92 for more discussion.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/anomalyco/opentui/pull/633 **Author:** [@iamlemec](https://github.com/iamlemec) **Created:** 2/5/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `image-renderable` --- ### 📝 Commits (1) - [`7a5a40b`](https://github.com/anomalyco/opentui/commit/7a5a40b2f7182767945e84fe65c5b2c2e1b57182) add ImageRenderable element to render images with the kitty graphics protocol ### 📊 Changes **16 files changed** (+483 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `bun.lock` (+16 -0) 📝 `packages/core/package.json` (+2 -0) 📝 `packages/core/src/Renderable.ts` (+19 -1) 📝 `packages/core/src/buffer.ts` (+14 -0) ➕ `packages/core/src/examples/assets/blue.png` (+0 -0) ➕ `packages/core/src/examples/assets/snake.png` (+0 -0) ➕ `packages/core/src/examples/image.ts` (+99 -0) ➕ `packages/core/src/renderables/Image.ts` (+85 -0) 📝 `packages/core/src/renderables/composition/constructs.ts` (+6 -0) 📝 `packages/core/src/renderables/index.ts` (+1 -0) 📝 `packages/core/src/renderer.ts` (+6 -1) 📝 `packages/core/src/zig.ts` (+37 -1) 📝 `packages/core/src/zig/buffer.zig` (+91 -0) ➕ `packages/core/src/zig/kitty.zig` (+48 -0) 📝 `packages/core/src/zig/lib.zig` (+12 -0) 📝 `packages/core/src/zig/renderer.zig` (+47 -1) </details> ### 📄 Description This adds an `ImageRenderable` element to render pixel images. The rendering closely mirrors the current/next buffer approach from `OptimizedBuffer` but using `PixelBuffer`, which is basically a collection of `PixelPatch` objects. This allows for flicker-free, non-reduntant rendering. The main interface is a new `renderPixels` function on `Renderables` that allows any renderable (not just `ImageRenderable`) to display arbitrary numbers of images. Images are hashed to prevent redundant writes. The actual output is buffered with regular text output on the native side to prevent flicker. I tried to keep the changes as minimal as possible. I did need to increase the size of the output buffer from 2MB to 8MB to accomodate images. The only additional dependency is `pngjs` for decoding PNG files. There is an example with animated images called `image.ts`. This currently only outputs kitty format, but other formats should not be difficult. The data input is just a `Uint8Array` of RGBA pixels, but could push file loading from example to core library. In terms of z-index issues, so far as I can tell, images are always placed above text, even if the text is written afterwards (at least in kitty/ghostty). So doing truly interleaved text and pixels would be non-trivial, but not impossible. You'd basically have to go in and transparent blit out parts of the images if they are occluded. This doesn't do that, so the pixel images are always just an overlay on top. See issue #92 for more discussion. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>