AYON integration: ticket SSO + task boards (v1.0) #1

Merged
Hermes merged 5 commits from ayon-integration into main 2026-09-04 23:43:01 +00:00
Showing only changes of commit ed0c606aae - Show all commits
+118 -293
View File
@@ -1,328 +1,153 @@
# RefBoard
# RefBoard AYON
A self-hosted, real-time collaborative reference board — like PureRef, but on the web, multiplayer, and with markdown notes, threaded review comments, and PDF support baked in.
Kollaborative Referenz-Bildwand (PureRef-on-the-web), umgebaut für den
AYON-only-Betrieb. Fork von [metalfinger/refboard](https://github.com/metalfinger/refboard)
(vollständige History erhalten).
Drop images, videos, and PDFs onto an infinite GPU canvas. Pan, zoom, group, align, annotate. Share a board with your team and watch each others' cursors in real time. Pin a comment to a thumbnail and resolve it like a code review.
**Das Besondere:** Es gibt keinen RefBoard-Login mehr. Nutzer öffnen Boards
aus dem AYON-Launcher heraus und sind automatisch mit ihrer AYON-Identität
eingeloggt. Boards gehören zu AYON-Tasks (`<Projekt>/<Task-Pfad>`), nicht zu
Personen. Alle Kollegen mit Projektzugriff arbeiten kollaborativ am selben
Board (Echtzeit, Socket.IO).
Built because we needed PureRef's painlessness, Miro's collaboration, and a code-review's threading — without paying three different SaaS subscriptions for them.
## Architektur
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
---
## Features
**Canvas**
- Infinite, GPU-accelerated canvas (Pixi.js v8) — handles thousands of items without dropping frames
- Drag & drop images, videos, and PDFs from your filesystem or clipboard
- Drop image URLs directly from the browser
- Pan / zoom / fit-all, selection, lasso, group, ungroup
- Undo / redo, locked layers, hidden layers
- Pen / draw tool, sticky notes, text labels
- Markdown cards (BlockNote-powered editor) right on the canvas
**PureRef-parity power tools**
- 40+ keyboard shortcuts mapped to PureRef defaults
- Align, distribute, normalize size / scale / width / height
- Auto-arrange in grid / row / column / by name / by z-order / random / stack
- Optimal pack, overlay compare (Ctrl+Y), flip H/V, reset transform, grayscale, lock
- Right-click context menu with everything
**Real-time collaboration**
- Live cursors with name labels
- Full-scene sync over Socket.IO with interaction-aware deferral
- Presence (online avatars), follow-user mode, share dialog with role-based access (owner / editor / viewer)
- Public, read-only sharing links via collection share token
**Review & threads**
- Pin a comment to any object on the canvas → starts a thread
- Threaded comments with status (open / resolved)
- Review mode toggles a clean overlay for walking through feedback
**Activity log**
- Per-board audit trail visible from the toolbar — uploads, board renames, threads, replies
- Live updates over Socket.IO (no refresh needed when collaborators are working)
- Time-grouped feed (Today / Yesterday / older), pagination, deactivation-safe author labels
**Media pipeline**
- Image variants (thumbnail / hires / LOD) generated on upload via Sharp
- Video poster + duration + dimensions extracted via ffmpeg
- PDF → page thumbnails + hires renders via poppler
- Background media worker so the upload feels instant
**Auth & admin**
- JWT-based email/password auth
- First user is auto-admin
- `SEED_ADMIN_*` env vars to bootstrap an admin on first boot
- `ALLOW_SELF_REGISTRATION` flag — when off, only admins can create accounts
- **Admin dashboard** at `/admin` — list users, create accounts, reset passwords, promote/demote between admin and member, deactivate / reactivate, and toggle self-registration on or off at runtime (admin-only, JWT-gated)
- Admin REST endpoints work with either a JWT belonging to an admin user or an `X-API-Key` header for bots
**Deployment**
- One `docker compose up` starts RefBoard + a bundled MinIO for object storage
- Dockerfile is multi-stage and self-contained
- SQLite (WAL mode) for metadata — zero external DB dependency
---
## Quick start (Docker, recommended)
Requires Docker + Docker Compose v2.
```bash
git clone https://github.com/metalfinger/refboard
cd refboard
bash scripts/setup.sh # macOS / Linux
# or, on Windows PowerShell:
# .\scripts\setup.ps1
```
Launcher (Task ausgewählt)
→ "RefBoard öffnen" (Server-Addon-Action)
→ POST /api/addons/refboard/<ver>/session (AYON-Auth via Bearer)
→ Addon prüft Projektzugriff, legt Single-Use-Ticket an (60 s TTL)
← { url: "https://<refboard>/b?ticket=…&project=…&task=…" }
→ Launcher öffnet Browser auf dieser URL
→ RefBoard-Frontend (Route /b) ruft POST /api/auth/ayon/exchange
→ RefBoard-Backend ⇄ Addon /exchange (server-to-server, X-API-Key)
→ Ticket wird atomar verbraucht (single-use) → Identität
→ RefBoard-Session-JWT wird gesetzt → weiter zum Task-Board
```
The script copies `.env.example``.env`, pulls the pre-built multi-arch image from GHCR, brings the stack up, waits for the backend to report healthy, and opens the URL in your browser. Re-running it is safe.
Beteiligte Komponenten:
Prefer the raw commands? They're equivalent to:
```bash
cp .env.example .env
docker compose pull
docker compose up -d
```
Open <http://localhost:8000>, create the first admin account on the screen RefBoard shows you, and you're in. No env-var editing required to get started — `JWT_SECRET` is generated and persisted on first boot, and the first user you register is auto-promoted to admin.
For production / non-localhost installs, see the [going-public checklist](#checklist-when-going-public) below — you'll want to set `JWT_SECRET` explicitly, lock down `CORS_ORIGIN`, and turn off self-registration.
To build from source instead of pulling (slower; needed only if you've modified the code):
```bash
docker compose up --build
```
The default image is `ghcr.io/metalfinger/refboard:latest`. Pin a specific version by setting `REFBOARD_IMAGE=ghcr.io/metalfinger/refboard:v0.5.0` in `.env`.
MinIO console (S3 dashboard) is at <http://localhost:9001> — login is whatever you set as `MINIO_ACCESS_KEY` / `MINIO_SECRET_KEY` in `.env` (defaults to `minioadmin` / `minioadmin`).
Persistent state lives under `./.docker-data/` (SQLite + MinIO objects). Back this up — that's also where the auto-generated `JWT_SECRET` lives, so losing it logs everyone out.
---
## Manual install (without Docker)
Requires Node.js 20+, **ffmpeg**, and **poppler-utils** on your `PATH`. The Docker image installs these automatically; for a manual install you need to bring them yourself.
```bash
# macOS
brew install ffmpeg poppler
# Debian / Ubuntu
sudo apt install ffmpeg poppler-utils
```
If `poppler-utils` is missing, image and video uploads still work, but PDF uploads will fail with a clear `501 POPPLER_MISSING` error rather than crashing.
You also need an S3-compatible object store reachable from the backend — easiest is to run MinIO standalone.
```bash
# 1. Object storage
docker run -d --name minio -p 9000:9000 -p 9001:9001 \
-e MINIO_ROOT_USER=minioadmin -e MINIO_ROOT_PASSWORD=minioadmin \
-v $(pwd)/.docker-data/minio:/data \
minio/minio server /data --console-address ":9001"
# 2. Backend
cd backend
npm install
DB_PATH=./data/refboard.db \
MINIO_ENDPOINT=localhost \
node server.js
# JWT_SECRET is auto-generated and persisted on first boot — set it explicitly
# only if you want to manage it via an external secrets manager. The first user
# you register in the browser becomes admin automatically.
# 3. Frontend (dev mode, separate terminal)
cd frontend
npm install
npm run dev
```
In dev mode the Vite server proxies `/api` and `/socket.io` to the backend on port 8000 — open the URL Vite prints.
For production, run `npm run build` in `frontend/` — the backend serves the built `frontend/dist` automatically.
---
## Configuration
All knobs live in `.env`. See [`.env.example`](.env.example) for the full annotated list. Highlights:
| Variable | Required? | What it does |
| Komponente | Repo | Aufgabe |
|---|---|---|
| `JWT_SECRET` | yes (in prod) | Signs auth tokens. Make it long and random. |
| `DB_PATH` | no | SQLite file path. Defaults to `/app/data/refboard.db` (Docker). |
| `MINIO_ENDPOINT` / `_PORT` / `_ACCESS_KEY` / `_SECRET_KEY` / `_BUCKET` | yes | S3-compatible storage. |
| `SEED_ADMIN_EMAIL` + `SEED_ADMIN_PASSWORD` | no | Idempotent first-boot admin bootstrap. |
| `ALLOW_SELF_REGISTRATION` | no | Initial seed only — sets the runtime toggle on first boot. After that, control it from the admin dashboard. Default `false`. |
| `MAX_FILE_SIZE_MB` | no | Per-file upload cap. Default 200. |
| `REFBOARD_API_KEY` | no | Enables programmatic upload via X-API-Key header. |
| RefBoard AYON (dieses Repo) | `Hermes/refboard-ayon` | Web-App: Canvas + Ticket-Auth + Task-Boards |
| AYON Server-Addon | `Hermes/refboard-addon` (ab v0.2.0) | Ticket-Ausstellung, `/exchange`, Launcher-Action |
| AYON Server (private Instanz) | — | Identitätsquelle, Token-Prüfung, Access-Groups |
---
### Warum ein Ticket statt des AYON-Tokens in der URL?
## Putting it on a public domain
Der AYON-Session-Token ist langlebig und dürfte nie in Browser-History/Logs
landen. Das Ticket ist ein 32-Byte-Zufallswert, der genau einmal eingelöst
werden kann und nach 60 s verfällt — die OAuth-"Code"-Äquivalenz. Die
Identität kommt ausschließlich vom Addon (`/exchange`), niemals aus der URL.
RefBoard is just an HTTP server on port 8000 — every reverse-proxy / tunneling option works. The only non-obvious bit is that it uses Socket.IO over WebSockets, so whatever fronts it must allow WS upgrades.
### Task-Boards
Pre-baked deployment configs for the common patterns (Cloudflare Tunnel sidecar, Caddy auto-TLS, Fly.io, Render, single-container FS-storage) live in [`examples/`](examples/README.md). The hand-rolled instructions below are still valid; the examples just spare you the YAML.
- Adresse/Name: `<project>/<task_path>` (z. B. `lumenfjord/assets/vegetation/fir/lookdev`)
- Board entsteht lazy beim ersten Öffnen in der Sammlung `AYON`
- `users`-Tabelle bleibt intern bestehen (SQLite-Fremdschlüssel), wird aber
nicht mehr als Login angezeigt; AYON-Nutzer bekommen Einträge
`<username>@ayon.local` mit unbenutzbarem Passwort-Sentinel
- Legacy-Login/Register-Routen sind noch vorhanden (für Alt-Instanz), im
AYON-Betrieb jedoch ungenutzt
### Cloudflare Tunnel (zero open ports, free TLS, recommended for home / studio servers)
## Deployment (orange)
This is what I run my own instance behind. No router config, no public IP, no Let's Encrypt — Cloudflare proxies the connection through an outbound tunnel from the box.
- Container: `refboard-ayon`, Host-Port **8003** → 8000
- Stack-Ordner: `/media/orange/RocketChat/refboard-ayon/` (NVMe)
- `repo/` — dieser Code (Branch `ayon-integration`)
- `compose.yaml` — Stack (inkl. `ayon-private_default` external network:
RefBoard erreicht AYON unter `http://ayon-private-server-1:5000`)
- `.env``JWT_SECRET`, `REFBOARD_API_KEY`, `AYON_EXCHANGE_URL`
- `data/` — SQLite + FS-Storage
- nginx: `/etc/nginx/sites-enabled/refboard.conf``refboard.niklashmotion.art`
(WebSocket-Upgrade, 250 M Upload)
- Wichtig nach `docker compose up -d`: das `ayon-private_default`-Netz ist
als `external: true` deklariert und überlebt Rebuilds dadurch
### Environment
| Variable | Bedeutung |
|---|---|
| `JWT_SECRET` | Signatur-Secret der RefBoard-Sessions |
| `REFBOARD_API_KEY` | Shared Secret für `/exchange` (muss mit dem AYON-Secret `refboard_api_key` übereinstimmen) |
| `AYON_EXCHANGE_URL` | z. B. `http://ayon-private-server-1:5000/api/addons/refboard/0.2.0/exchange` |
| `STORAGE_BACKEND` | `fs` (FS-Storage) |
| `CORS_ORIGIN` | CORS-Origin (Default `*`) |
## Setup in AYON (Schritt für Schritt)
> Ausführen, sobald `refboard-0.2.0.zip` und der A-Record existieren.
### 1. Addon installieren (im AYON-Klon, NICHT im Original)
```bash
# 1. Install cloudflared (macOS / Linux examples)
brew install cloudflared # macOS
# OR
sudo apt install cloudflared # Debian/Ubuntu (see Cloudflare docs for repo setup)
# zip in den Server-Container entpacken
ssh orange@niklashmotion.art 'docker exec -i ayon-private-server-1 python3 -c "
import zipfile,io,os,sys
z=zipfile.ZipFile(io.BytesIO(sys.stdin.buffer.read()))
b=\"/addons/refboard/0.2.0\"
os.makedirs(b,exist_ok=True)
[open(os.path.join(b,n),\"wb\").write(z.read(n)) for n in z.namelist() if not n.endswith(\"/\")]
"' < refboard-0.2.0.zip
# 2. Authenticate (opens browser to pick a Cloudflare account / zone)
cloudflared tunnel login
# 3. Create a named tunnel
cloudflared tunnel create refboard
# 4. Route a hostname to it (replace example.com with your zone)
cloudflared tunnel route dns refboard refboard.example.com
# 5. Run the tunnel pointed at the local RefBoard
cloudflared tunnel run --url http://localhost:8000 refboard
# AYON-Server neu starten (scannt /addons nur beim Start)
ssh orange@niklashmotion.art 'docker restart ayon-private-server-1'
```
For a permanent install, generate a config at `~/.cloudflared/config.yml`:
### 2. Bundle (WebUI: Studio Settings → Bundles)
```yaml
tunnel: refboard
credentials-file: /Users/you/.cloudflared/<tunnel-id>.json
Neues Bundle anlegen, das die Addons des aktuellen Produktions-Bundles
enthält, aber `refboard: 0.2.0`, und als Produktion aktivieren.
ingress:
- hostname: refboard.example.com
service: http://localhost:8000
- service: http_status:404
```
### 3. Settings prüfen (WebUI: Studio Settings → Addons → RefBoard)
Then `cloudflared service install` to make it boot at startup.
- `refboard_url`: `https://refboard.niklashmotion.art`
- `api_key`: `refboard_api_key` (Secret existiert bereits im AYON-Secret-Store
und muss denselben Wert wie `REFBOARD_API_KEY` in der RefBoard-`.env` haben)
> **Heads-up:** Cloudflare's free plan caps proxied request bodies at **100 MB**. If you regularly upload videos / large PDFs above that, set `MAX_FILE_SIZE_MB` accordingly, or pair Cloudflare Tunnel with a direct path for uploads (e.g. tunnel only the SPA, expose the upload API via something else), or upgrade your Cloudflare plan.
### 4. Benutzen
### Caddy reverse proxy (one-line TLS via Let's Encrypt)
1. AYON-Launcher starten und ein Projekt öffnen
2. Task auswählen → Kontext-/Aktionsmenü → **„RefBoard öffnen"**
3. Browser geht auf, automatisch eingeloggt als der eigene AYON-User, im
Board der Task
4. Teilen: Kollegen wählen dieselbe Task → „RefBoard öffnen" → alle landen
im selben Board und arbeiten in Echtzeit zusammen
If the box is publicly reachable (cloud VPS, port 443 open):
Hinweise:
- Der Board-Name ist `<Projekt>/<Task-Pfad>`; im Editor oben wird er als
Titel angezeigt
- Ohne Task-Auswahl gibt es keinen Action-Eintrag (Bewusst so gebaut:
Boards sind taskbezogen)
- Tickets sind einmalig und 60 s gültig — bei „Invalid or expired ticket"
einfach im Launcher erneut klicken
```caddy
refboard.example.com {
reverse_proxy localhost:8000
}
```
## AYON-REST-API des Addons (v0.2.0)
That's the whole `Caddyfile`. Caddy auto-provisions and renews the certificate, and `reverse_proxy` upgrades WebSockets transparently.
Alle Routen unter `/api/addons/refboard/0.2.0/`:
### nginx reverse proxy
| Endpoint | Methode | Auth | Zweck |
|---|---|---|---|
| `/health` | GET | AYON-User | Status + konfigurierte URL |
| `/session` | POST | AYON-User | `{project_name, task_id?}``{url}` (Ticket-URL) |
| `/exchange` | POST | `X-API-Key` | `{ticket}``{ayon_user, display_name, project, task}` (single-use) |
```nginx
server {
server_name refboard.example.com;
listen 443 ssl;
# ssl_certificate / ssl_certificate_key from certbot or your CA
client_max_body_size 250M;
location / {
proxy_pass http://localhost:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 86400;
}
}
```
### Tailscale (private-to-your-team access without a domain)
If you don't want it on the public internet at all:
## Lokale Entwicklung / Tests
```bash
tailscale serve --bg http://localhost:8000
# now reachable at https://<machine>.<tailnet>.ts.net
# Image bauen (ARM64-nativ auf orange getestet)
docker build -t refboard-ayon:latest .
# Container + Mock-Addon (Test-Double statt echtem AYON-Addon):
# /media/orange/RocketChat/refboard-ayon/mock-exchange.js (container :9000)
# /media/orange/RocketChat/refboard-ayon/e2e.sh (Container-interner E2E)
```
Anyone in your tailnet can hit it; no one else can.
Der E2E (Ticket-Ausstellung → Exchange → Board-Zugriff → Single-Use →
2. Nutzer am selben Board) ist mit dem Mock grün getestet.
### Checklist when going public
## Noch offen
- [ ] Set `JWT_SECRET` explicitly (`openssl rand -base64 64`). For production installs we recommend pinning the secret in `.env` rather than relying on the auto-generated one in the SQLite settings table — easier to rotate, easier to back up to a secrets manager.
- [ ] Set `NODE_ENV=production`.
- [ ] Set `CORS_ORIGIN=https://your.domain` (drop the wildcard).
- [ ] Confirm self-registration is **off** in the admin dashboard (defaults off; only flips on if you set `ALLOW_SELF_REGISTRATION=true` on first boot).
- [ ] Restrict the MinIO console (port 9001) to localhost — only the S3 API on 9000 needs to be reachable from the backend, and the backend already proxies media bytes through `/api/images/*`, so MinIO does **not** need to be exposed publicly.
- [ ] Keep `./.docker-data/` (or `DB_PATH` + MinIO data dir) backed up — that's all your state.
---
## Architecture
```
┌────────────────────────────────────────────────────────────┐
│ Browser │
│ React + TypeScript + Vite │
│ Pixi.js v8 canvas · BlockNote markdown · Socket.IO client │
└──────────────────┬─────────────────────────────┬───────────┘
│ HTTPS / WS │
▼ ▼
┌───────────────────────┐ ┌──────────────────────┐
│ Express + Socket.IO │ │ Static frontend │
│ /api/* REST │ │ (served by backend) │
│ /socket.io WS rooms │ └──────────────────────┘
│ Media worker (queue) │
└────┬──────────┬───────┘
│ │
▼ ▼
┌──────────┐ ┌──────────────────┐
│ SQLite │ │ MinIO (S3) │
│ (WAL) │ │ images / videos │
│ metadata │ │ pdf renders │
└──────────┘ └──────────────────┘
```
- **Frontend** — React + Vite + TypeScript. Pixi.js v8 powers the canvas (LOD-aware, viewport-culled, GPU-batched). BlockNote provides the markdown editor used for sticky cards. Socket.IO syncs scene state.
- **Backend** — Express for REST, Socket.IO for real-time, better-sqlite3 for metadata (WAL mode), Sharp + ffmpeg + poppler for media processing. A background worker handles thumbnails/hires/PDF rasterization out of the request path.
- **Storage** — MinIO (or any S3 API). The backend proxies media bytes through `/api/images/*` so URLs stay stable across deployment moves.
See [CHANGELOG.md](CHANGELOG.md) for the version history (v0.1.0 → v0.5.0).
---
## Roadmap
- [x] Admin dashboard frontend (live at `/admin` — user create / reset-password / role / deactivate)
- [x] Per-board activity log (uploads, board events, threads, comments — live via Socket.IO)
- [x] **Pre-built multi-arch image** at `ghcr.io/metalfinger/refboard` (linux/amd64 + linux/arm64) — published on every push to main
- [x] **Zero-edit first boot**`JWT_SECRET` auto-generated and persisted, first registered user is auto-admin
- [x] **FS storage adapter**`STORAGE_BACKEND=fs` drops the MinIO dependency for single-container installs
- [x] **One-click installers**`scripts/setup.sh` (macOS / Linux) and `scripts/setup.ps1` (Windows)
- [x] **PaaS deploy templates** — Fly.io, Render, Cloudflare Tunnel sidecar, Caddy auto-TLS in [`examples/`](examples/README.md)
- [ ] Mobile-friendly read-only board view
- [ ] Export board → PDF / image grid
- [ ] Optional remote storage adapters (S3 direct, R2)
---
## Contributing
Issues and PRs welcome. The codebase is single-language on the frontend (TypeScript + React) and small on the backend (a few hundred lines per route file). The hardest parts are in `frontend/src/canvas/` (the Pixi-powered scene graph and sync engine).
---
## License
MIT — see [LICENSE](LICENSE). Built and maintained by [Hiren Kangad](https://metalfinger.xyz).
- A-Record `refboard``89.57.47.213` (IONOS-Panel, einzeln anlegen)
- danach: `certbot --nginx -d refboard.niklashmotion.art`
- Addon v0.2.0 installieren + Bundle (siehe oben)