- Background media worker: polls media_jobs table, runs ffprobe+ffmpeg
with concurrency limit, generates video posters, emits socket events
- Non-blocking video upload: stores file + enqueues job, returns immediately
- Poster hydration on board load: GET /boards/:id injects poster/dimensions
from DB into canvas_state video objects
- SpatialGrid: fixed-cell (512px) spatial hash for O(nearby) culling instead
of O(all) item scanning, eliminates setTimeout violations
- Canvas-based video rendering: draws video frames to offscreen canvas then
uploads to GPU, completely eliminates GL_INVALID_OPERATION errors from
PixiJS VideoSource auto-update mechanism
- Server poster upgrade path: culling ticker and applyProcessedMedia() both
upgrade client-captured posters to server posters when available
- Pause restores server poster (paused video behaves like an image)
- Selection drag-end persistence: onObjectDragEnd broadcasts + saves + undo
- Live media:job:update socket handler patches scene data + VideoSprite
dimensions without broadcast fanout
Upload pipeline now extracts poster frame (JPEG) and metadata
(width, height, duration) from videos at upload time using ffmpeg.
Server:
- Add ffmpeg to Docker image (Alpine)
- video-utils.js: probeVideo() and extractPoster() using ffprobe/ffmpeg
- Upload response includes poster_asset_key and duration
Frontend:
- VideoObject gains poster and duration fields in scene format
- VideoSprite accepts posterAssetKey + TextureManager, loads poster
as a regular image texture on construction (no <video> needed)
- Server poster loaded/released via TextureManager ref counting
- addVideoFromUpload passes poster and duration through to scene data
- image-drop.ts forwards poster_asset_key from upload response
Result: videos with server posters render as images by default.
Zero <video> elements needed for thumbnails. Only explicit play
creates a media element.
Video memory:
- Lazy <video> creation: constructor makes placeholder only, initVideo()
creates element with preload='metadata' when near viewport
- Tiered budgets: 1 playing / 6 initialized / 4 poster textures max
- Aggressive teardown: offscreen videos lose <video> element, poster,
and all textures — zero memory for offscreen videos
- Poster textures dropped while playing (don't keep both resident)
- Dimension caching in scene data (nativeW/nativeH) avoids re-init
Server:
- HTTP Range support (206 Partial Content) for video seeking
- Proper end clamping and invalid range rejection (416)
Sync:
- Remove redundant broadcastSceneDebounced() from broadcastTransform()
- Remove duplicate broadcastElements() from drag-end handler
VideoControls:
- Event-driven (timeupdate/play/pause/seeked) instead of rAF polling
- Tracks actual HTMLVideoElement reference, rebinds on init/teardown
- Seek uses ref-based commit on pointerUp, not conditional onChange
Children converge ~25px toward combined center on group (gentle spring),
then reparent into a Container. Ungroup reparents back to viewport and
spreads children ~25px outward with matching animation.
Hiren Kangad