# Use Sluby with AI agents (/developers/ai-agents) The Sluby agent skill teaches a coding agent (Claude Code, Claude, Codex, Gemini CLI, GitHub Copilot and others that read Agent Skills) how to build on a Sluby server: which calls to make, which scopes they need, and the mistakes to avoid. This site also serves every page as plain markdown for agents that read docs directly. ## What the skill covers [#what-the-skill-covers] The skill is a `SKILL.md` file plus four reference files in the [Agent Skills](https://agentskills.io/specification) format. An agent loads `SKILL.md` when a task involves Sluby, and opens a reference file only when the task needs it. | File | What it teaches | | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SKILL.md` | Authenticating with an API key, the upload and wait-for-ready flow with `@sluby/sdk`, managing assets, playing videos, webhooks, the SDK's error classes, and the rules that prevent common mistakes: keys stay on the server, the smallest scopes, rate limits, signed URLs for private videos | | `references/rest-api.md` | The REST API: authentication, what each scope allows, the main endpoints, errors, listing and cursor paging, ETags with `If-Match`, rate limit headers, and where each server publishes its OpenAPI document | | `references/uploads.md` | Upload rules and limits, the SDK upload handle (progress, pause, resume, abort), uploading with any tus client or with `curl`, following processing and retrying | | `references/playback.md` | Playback URLs, `@sluby/player` in React for public and private videos, signed URLs and their renewal, the watch page in an iframe, playback ids, revoking access, streaming straight from Sia | | `references/webhooks.md` | Webhook events, registering an endpoint, verifying `X-Sluby-Signature-V1` with and without the SDK, retries and duplicate events | ## Download the skill [#download-the-skill] - [sluby.zip](/skills/sluby.zip): the whole skill, with a `sluby` folder at the top. - [SKILL.md](/skills/sluby/SKILL.md): the main file alone. - [Source on GitHub](https://github.com/justmert/Sluby/tree/main/docs/public/skills/sluby). ## Install the skill [#install-the-skill] Pick your agent. Each method installs the full skill, reference files included. Download [sluby.zip](/skills/sluby.zip), then unzip it into your personal skills folder to use it in every project: ```sh mkdir -p ~/.claude/skills && unzip sluby.zip -d ~/.claude/skills ``` Or into one project (commit the folder so everyone who works in that repository gets it): ```sh mkdir -p .claude/skills && unzip sluby.zip -d .claude/skills ``` Claude Code picks up the new skill in the current session, without a restart. If the `skills` folder did not exist when the session started, run `/reload-skills`. 1. On Free, Pro and Max plans, turn on **Code execution and file creation** under **Settings**, **Capabilities**. On Team and Enterprise plans, an organization owner turns on **Cloud code execution and file creation** and **Skills** under **Organization settings**, **Plugins & skills**, in the **Policy** tab. 2. Download [sluby.zip](/skills/sluby.zip). 3. In Claude, open **Customize**, then **Skills**. 4. Click **+**, then **+ Create skill**, then **Upload a skill**. 5. Upload `sluby.zip`. Download [sluby.zip](/skills/sluby.zip), then unzip it into your personal skills folder: ```sh mkdir -p ~/.agents/skills && unzip sluby.zip -d ~/.agents/skills ``` Or into a repository, so Codex uses it there: ```sh mkdir -p .agents/skills && unzip sluby.zip -d .agents/skills ``` ```sh gemini skills install https://github.com/justmert/Sluby.git --path docs/public/skills/sluby ``` This installs for your user. Add `--scope workspace` to install into the current project instead. With GitHub CLI's `skill` command: ```sh gh skill install justmert/Sluby sluby --scope user ``` Leave out `--scope user` to install into the current project. Without `--agent`, the command asks which agent to install for when you run it in an interactive terminal, and installs for GitHub Copilot otherwise. Pass `--agent` to pick another one ([`gh skill install`](https://cli.github.com/manual/gh_skill_install) lists the agent names). The [`skills`](https://github.com/vercel-labs/skills) command line tool installs into many agents, including Cursor, OpenCode, goose and Windsurf. It needs Node.js 22.20 or later. ```sh npx skills add https://github.com/justmert/Sluby/tree/main/docs/public/skills/sluby ``` Pass `-a ` to choose the agents (for example `-a cursor`), and `-g` to install for your user instead of the current project. ## Give the agent your server's address [#give-the-agent-your-servers-address] The skill does not know where your Sluby server runs. Tell the agent its address, for example `https://sluby.example.com`, and keep the API key in an environment variable such as `SLUBY_API_KEY` rather than in the chat. To create a key, see [Create and manage API keys](/studio/api-keys). Each Sluby server publishes its own OpenAPI document, and the skill tells the agent to read it for anything the skill does not show. Where it lives is on [API conventions](/developers/reference/api-conventions#openapi). ## Docs for agents [#docs-for-agents] This site serves its pages in forms that agents and other tools read without a browser: | Path | Contents | | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | [`/llms.txt`](/llms.txt) | An index of every page, with links and one-line descriptions | | [`/llms-full.txt`](/llms-full.txt) | Every page in one plain text file | | `/llms.mdx//content.md` | One page as markdown. For this page: [`/llms.mdx/developers/ai-agents/content.md`](/llms.mdx/developers/ai-agents/content.md) | | [`/openapi.json`](/openapi.json) | The OpenAPI document behind the [REST API reference](/developers/reference/api/assets/get-assets) | To give an agent one page, take the page's path and wrap it: `/developers/guides/webhooks` becomes `/llms.mdx/developers/guides/webhooks/content.md`. # Accounts, roles and sessions (/developers/concepts/accounts-and-roles) This page explains who can do what on a Sluby server: how one install maps to one team, what the admin and member roles allow, how the backend decides who made a request, and how sign-in, Studio sessions, the setup code, invite links and reset links work. For the Studio screens that manage members, see [Invite and manage members](/studio/members). For how an operator sets up sign-in, see [Set up sign-in](/self-hosting/sign-in). ## One install is one team [#one-install-is-one-team] A Sluby install serves one team, called the workspace. Every video, API key, webhook endpoint and upload belongs to the workspace, and every caller (a member's Studio session, an API key, or the local development account) acts as the workspace. Each person who can sign in is a row in the `users` table with a role. The role decides what a person can do, not which videos they see: every member sees the whole library. ## Roles and what each can do [#roles] There are two roles, `admin` and `member`. | Capability | Admin | Member | | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | | Upload, list, edit, retry and delete videos | Yes | Yes | | Create and revoke playback ids, mint signed URLs and share URLs | Yes | Yes | | Create, list, rotate and revoke the workspace's API keys | Yes | Yes | | Create and manage webhook endpoints, read their delivery logs | Yes | Yes | | Read library stats and daily play counts | Yes | Yes | | Read system status (`GET /api/v1/system/status`) | Full status: database, Redis, the Sia indexer's account usage and hosts, the last reconciliation run, the delivery cache | Health summary only: overall status, version, database, Redis and indexer health, the indexer's network and its host pool summary | | Read the reconciliation report (`GET /api/v1/reconciliation`) | The whole deployment | Missing objects of the workspace's videos only | | Start a reconciliation run | Yes | No | | Read process metrics as JSON (`GET /api/v1/metrics`) | Yes | No | | List members, change roles, remove and restore members, make reset links | Yes | No | | Create, list and withdraw invites | Yes | No | An endpoint reserved for admins answers a member with `403` and `Only an admin may use this endpoint`. [API conventions](/developers/reference/api-conventions#scopes) lists the scopes an API key needs for each endpoint. [Deletion and reconciliation](/developers/concepts/deletion-and-reconciliation) describes the reconciliation report. The backend reads the role on every request. A role change applies to the member's sessions and API keys on their next request. ## Who a request acts as [#callers] Every request under `/api/v1` (except the `/api/v1/auth` routes, the OpenAPI document at `/api/v1/openapi.json` and the reference page at `/api/v1/docs`) and every tus upload request goes through the same check, in this order: ```mermaid flowchart TD A[Request] --> B{AUTH_DISABLED on?} B -->|yes| C[Local development account, admin, all scopes] B -->|no| D{Live session cookie?} D -->|yes| E[That member, all scopes, admin if their role is admin] D -->|no| F{Authorization: Bearer key?} F -->|no| G[401 with a reason] F -->|yes| H{Key known, active and not expired?} H -->|no| I[401] H -->|yes| J{Key has a creator?} J -->|no| K[Admin, the key's scopes] J -->|yes| L{Creator removed?} L -->|yes| M[401: the key's creator was removed] L -->|no| N[Creator's current role, the key's scopes] ``` - **Studio session.** A valid session cookie wins over an API key. A session carries all three scopes (`upload`, `read` and `manage`), and acts as an admin when the member's role is `admin`. - **API key.** A key carries the scopes it was created with. It acts with its creator's role as it stands at that request. Removing the creator stops the key with `401` and `This API key's creator was removed from this workspace`; restoring the creator makes the key work again. A key with no creator acts as an admin: these are keys made on the server itself, from [`BOOTSTRAP_API_KEY`](/self-hosting/configuration#bootstrap_api_key) or the [seed script](/self-hosting/sign-in#create-an-admin-api-key-from-the-command-line). - **AUTH_DISABLED.** With `AUTH_DISABLED` on, every request acts as one local development account with the admin role and all scopes. A server with `NODE_ENV=production` refuses to start with it (see [Configuration](/self-hosting/configuration#auth_disabled)). ## Sign-in methods [#sign-in-methods] A member can have more than one way to sign in. Each way is a row in `user_identities`, linked to the member's `users` row. ### Email and password [#email-and-password] - The server trims and lowercases the email, and each email belongs to a single account. - A new password must have 10 to 256 characters. - The server stores passwords as scrypt hashes (N=32768, r=8, p=1, 16-byte random salt). - A wrong email and a wrong password get the same answer, `401` with `Wrong email or password`. An unknown email costs the same hashing time as a known one, so response timing does not reveal which emails exist. - A removed member's correct password gets `403` with `This account was removed from this workspace`. - Changing the sign-in email (`POST /api/v1/auth/email/change`) also needs the current password, counts against the same attempt limits, and keeps the password. Another account's email gets `409` with `reason: email_taken`. - A member changes their own name with `PATCH /api/v1/auth/me`. - Changing the password (`POST /api/v1/auth/password/change`) needs the current password, and ends every other session of the account while keeping the current one. A member who signs in with GitHub only has no password yet and gets `409` with `This account has no password yet. Ask an admin for a reset link to set one.` ### GitHub [#github] The server offers GitHub sign-in only when it has both `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET` set. `GET /api/v1/auth/config` reports it in `methods.github`. - Sluby asks GitHub for the `read:user` scope only. - Sluby identifies a GitHub account by its numeric account id, not its login, so renaming a GitHub login does not change who it signs in as. Sluby stores the login for display and refreshes it at each sign-in. The member's name stays the one they chose. - A member removes GitHub from their account with `DELETE /api/v1/auth/identities/github`. Sessions that signed in with GitHub end. The server refuses with `409` when this browser signed in with GitHub (`current_sign_in`) or when GitHub is the account's only way to sign in (`last_sign_in`). - Signing in with GitHub never creates an account. Sign-in refuses a GitHub account that belongs to no member (`not_a_member`). A person joins with GitHub through an invite link, and an existing member adds GitHub from their account settings. - The sign-in keeps its state in a cookie named `sluby_signin` in the browser that started it (path `/api/v1/auth`, HttpOnly, valid for 10 minutes). The callback accepts only the state that cookie holds, so it refuses a callback opened in another browser, or after 10 minutes, with `expired_handshake`. ## Sessions and cookies [#sessions] Each Studio sign-in creates a row in `studio_sessions` with the member, the sign-in method used, the browser's user agent (first 500 characters) and its expiry. The browser gets the `sluby_session` cookie ([its attributes](/developers/reference/api-conventions#session-cookie)). The cookie value holds the session row id, the account id and the expiry, signed with HMAC-SHA256 using `SESSION_SECRET`. On every request the backend checks the signature and the expiry, then that the session row is neither ended nor expired, then that the member is not removed. Because the backend checks the row each time, the server can end a session at any moment. - **Lifetime.** A session lasts 7 days from sign-in. Using it does not extend it. - **Last seen.** The backend updates the session's `last_seen_at` at most once every 5 minutes. The Settings page shows it in the list of signed-in browsers. - **Ending sessions.** A session ends when the member signs out (`POST /api/v1/auth/logout`), ends one session (`DELETE /api/v1/auth/sessions/{id}`), signs out everywhere (`DELETE /api/v1/auth/sessions`, which leaves API keys untouched), changes their password (every other session ends), uses a reset link (every session ends), or is removed by an admin (every session ends). - **Why a session stopped working.** A request with a cookie that no longer works gets `401` with a `reason`: `missing` (no cookie, or a cookie this server did not sign), `expired`, `revoked` (signed out here or elsewhere), or `not_allowed` (the member was removed). - **Cleanup.** Ended sessions keep their rows for 30 days, then the hourly sweep deletes them. - **Changing SESSION_SECRET.** The backend signs every cookie with `SESSION_SECRET`, so changing it signs every member out. [Security model](/developers/concepts/security#session-secret) lists what else depends on it. ## First admin and the setup code [#setup-code] A fresh install has no admin. While sign-in is on and no admin who can sign in exists, the backend writes a setup code to its log at every start, and `GET /api/v1/auth/config` reports `setup_required: true`. An admin counts as able to sign in when the account is not removed and has at least one sign-in method; the local development account of an `AUTH_DISABLED` run does not count. - The backend derives the code from `SESSION_SECRET` with HMAC-SHA256, so only someone who can read the server's logs or environment has it. It is 10 characters from the Crockford base32 alphabet, shown as `XXXXX-XXXXX`. - When checking a typed code, the server ignores case, spaces and dashes, and reads `I` and `L` as `1` and `O` as `0`. - `POST /api/v1/auth/setup` takes the code, a name, an email and a password, creates the first admin and signs them in. Once an admin who can sign in exists, setup answers `409` with `This server already has an admin. Sign in instead.`, so the code stops working the moment it is used. [Set up sign-in](/self-hosting/sign-in) describes where the operator finds the code. ## Invites and reset links [#invites-and-reset-links] Admins hand out two kinds of links: invite links, which let new people join, and reset links, which let one member set a password. Both are rows in the `invites` table and share these rules: - The token is 32 random bytes in base64url (43 characters). The server stores only its SHA-256 hash, so the table alone is not enough to join or to take over an account. - The full URL appears once, in the response that creates it. No later response shows it again. - A reset link works once, for 7 days. An invite link works for 1, 7 or 30 days (7 by default) and for as many people as the admin chose (see [Invite links](#invite-links)). - Using a link locks its row for the transaction, so two browsers using its last place at the same moment cannot both succeed. - A link that does not exist answers `404` with `This link does not exist. Ask an admin for a new one.` A used-up, expired or withdrawn link answers `410` with `This link was already used.`, `This link has expired.` or `This link was withdrawn.`, each followed by `Ask an admin for a new one.` ### Invite links [#invite-links] An admin creates an invite with a role (`POST /api/v1/invites`, [endpoint reference](/developers/reference/api/members/post-invites)). The link has the form `https://sluby.example.com/studio/join/`. Each new member joins with a name, email and password, or with GitHub. - `max_uses` sets how many people can join with the link: 1 (the default), a number up to 1000, or `null` for anyone with the link until it expires or is revoked. The row counts joins in `use_count` and sets `used_at` when the last allowed person joins. - Only a member invite can let more than one person join, because anyone can pass a link on. An admin invite with `max_uses` set to `null` or to a number from 2 to 1000 answers `400` with `An admin invite lets one person join. Invite more people as members.`, and the database refuses such a row as well. - `expires_in_days` is 1, 7 or 30, and 7 when left out. - A GitHub account that already belongs to a member cannot join a second time, and that attempt does not count as a join. Admins list the invites that still let someone join, with `max_uses` and `uses` for each, and can withdraw any of them. Withdrawing stops new joins. People who already joined stay members. ### Reset links [#reset-links] A reset link lets one member set a new password. An admin creates it for a member (`POST /api/v1/members/{id}/reset-link`, [endpoint reference](/developers/reference/api/members/post-members-id-reset-link)), and an operator who is locked out can print one on the server for an account that signs in with an email ([Make a reset link from the command line](/self-hosting/sign-in#make-a-reset-link-from-the-command-line)). The link has the form `https://sluby.example.com/studio/reset/`. - A member who already has a password sets a new one. - A member who signs in with GitHub only sets their first password and chooses the email they will sign in with. - Using the link ends every session the member had, then signs the current browser in. - The API refuses a reset link for a removed member with `403` and `This account was removed from this workspace`. ## Removing members and the last-admin protection [#removing-members] Admins remove members rather than delete them: - **Remove.** The member's row stays, so their API keys and history keep a name. They can no longer sign in, every session they have ends at once, and their API keys stop working. An admin cannot remove themselves; the API answers `409` with `You cannot remove yourself. Ask another admin.` - **Restore.** The member comes back with the role they had, and their API keys work again. The API refuses a role change or a removal that would leave the workspace with no admin who can sign in, with `409` and `This is the last admin who can sign in. Make someone else an admin first.` These changes run one at a time, so two admins demoting each other at the same moment cannot both succeed. # Architecture (/developers/concepts/architecture) A Sluby server is a small set of containers: a web proxy, one backend process, PostgreSQL and Redis. The backend stores every video on the Sia network through a Sia indexer. This page explains what each part does and how they connect. ## The parts at a glance [#the-parts-at-a-glance] ```mermaid flowchart LR browser["Browser: Studio, watch page, @sluby/player"] yourapp["Your server: @sluby/sdk or REST"] caddy["caddy (production only): TLS"] web["web: nginx
Studio files, proxy, delivery cache"] subgraph backend["backend: one Node.js process"] api["REST API and tus uploads
/api/v1"] gateway["Delivery gateway
/v1/objects, /v1/stream, /v1/share"] watch["Watch page
/watch"] workers["Queue workers
transcode, upload-segments, finalize,
deletion, reconcile, webhooks"] end pg[("PostgreSQL")] redis[("Redis")] disk[("Volumes: uploads, transcode-output")] indexer["Sia indexer"] hosts["Sia hosts"] receivers["Your webhook receivers"] browser --> caddy yourapp --> caddy caddy --> web web --> api web --> gateway web --> watch api --> pg api --> redis api --> disk api --> indexer workers --> redis workers --> pg workers --> disk workers --> indexer workers --> hosts workers --> receivers gateway --> pg gateway --> indexer gateway --> hosts watch --> pg browser -.->|"direct and hybrid playback"| indexer browser -.->|"direct and hybrid playback"| hosts ``` Without the production override there is no caddy container: browsers and your server talk to the `web` container directly, which publishes its port on `127.0.0.1` (`http://localhost:8080` by default). With `docker-compose.prod.yml`, caddy is the only container that publishes ports (80 and 443) and terminates TLS for your domain. See [Run in production](/self-hosting/production). Two Sia terms appear throughout: - A **host** is a storage provider on the Sia network. The Sia SDK uploads the encoded pieces of each file to hosts and downloads them from hosts. - An **indexer** is the Sia service that Sluby registers with as an app. It keeps the list of objects (stored files) that Sluby has pinned on its account and tells the SDK which hosts hold them. Sluby connects to `https://sia.storage` or to another indexer URL you configure. [Storage on Sia](/developers/concepts/storage-on-sia) explains objects, pinning and redundancy in detail. ## The web proxy (nginx) [#the-web-proxy-nginx] The `web` container runs nginx with the Studio build baked into the image. It does three jobs: - **Serves the Studio.** `/` redirects to `/studio`, and every route under `/studio` answers with the single-page app. - **Proxies to the backend.** `/api/`, `/v1/`, `/watch/` and `/health` go to the backend on port 3000. nginx never caches anything under `/api/`, sets no body size limit there and forwards request bodies without buffering them, so large uploads stream straight to the backend. - **Caches delivery.** Responses under `/v1/objects/`, `/v1/stream/` and `/v1/share/` pass through an on-disk cache, and per-client rate limits apply to `/v1/` and `/api/v1/auth/`. [Playback and delivery](/developers/concepts/playback) describes what the cache keeps and for how long. nginx passes the client address and the `X-Forwarded-Proto` header that caddy sends on to the backend. The backend uses the address for its per-client API rate limit and the scheme to recognize write requests that come from its own origin. nginx re-resolves the backend's address every 10 seconds, so a recreated backend container is found without restarting the proxy. ## The backend process [#the-backend-process] The backend is one Node.js process (Express). The same process serves HTTP and runs every queue worker, so one `backend` container handles both requests and background jobs. On start, the container first runs the database migrations, then starts the server. Before it accepts requests the process loads the session secret it keeps in the database when `SESSION_SECRET` is not set, seeds `BOOTSTRAP_API_KEY` when you set one, takes over the access settings of an older version once, and, with sign-in on, logs the setup code when the install has no admin yet. It then listens on `PORT` (3000 in the compose files) and, when `SIA_APP_KEY` is set, connects to the Sia indexer. ### HTTP surfaces [#http-surfaces] | Path | What it serves | | -------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `/api/v1` | The REST API used by the Studio, `@sluby/sdk` and your own code. See [API conventions](/developers/reference/api-conventions). | | `/api/v1/uploads` | Resumable uploads over the [tus protocol](https://tus.io). A request that carries the `Tus-Resumable` header, or uses `HEAD`, `PATCH` or `OPTIONS`, goes to the tus server. Any other request, such as the upload status `GET`, goes to the REST routes. | | `/v1/objects/:objectId`, `/v1/stream/:videoAssetId/master.m3u8`, `/v1/share/:id` | The delivery gateway: HLS manifests and media read from Sia, and share maps for direct playback. | | `/watch/:id` | The public watch page that other sites can embed in an iframe. | | `/health` | The health check. The Docker health checks of both the `backend` and the `web` container call it (the `web` one through nginx). | | `/metrics` | Prometheus metrics. A scraper inside the Docker network reaches it on the backend's port 3000. nginx forwards it too, but without [`METRICS_TOKEN`](/self-hosting/configuration#metrics_token) the backend answers a forwarded request `404`; with it, every request must send the token. | ### Queues and workers [#queues-and-workers] The backend uses [BullMQ](https://docs.bullmq.io) queues stored in Redis. The backend process runs a worker for every queue. | Queue | What a job does | Who adds jobs | | ----------------- | ------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | | `transcode` | Runs FFmpeg on an uploaded source to produce the HLS renditions and thumbnails. | The tus server, when an upload completes, and the retry endpoint. | | `upload-segments` | Uploads a finished transcode to Sia. | The `transcode` worker, and the retry endpoint when a finished transcode can be reused. | | `finalize` | Records the uploaded objects on the video, marks it `ready` and sends `asset.ready`. | The `upload-segments` worker. | | `deletion` | Unpins a deleted video's Sia objects and removes its rows. A repeating `gc` job re-drives deletions that never finished. | Deleting a video, cancelling or abandoning an upload, and the `gc` schedule. | | `reconcile` | Compares the objects Sluby tracks with the objects pinned on the indexer. | A repeating schedule and the manual trigger. | | `webhooks` | Sends one event to one webhook endpoint, with retries. | Every place that emits an event, and a manual resend of an earlier delivery. | Each pipeline stage hands off to the next under a fixed job id, so a stage that runs twice starts its successor once. [Upload pipeline](/developers/concepts/upload-pipeline) follows a video through the three processing queues, and [Deletion and reconciliation](/developers/concepts/deletion-and-reconciliation) covers the `deletion` and `reconcile` queues. The process also runs an hourly timer that removes expired incomplete uploads and their rows, removes finished source files, prunes ended Studio sessions and drops old webhook delivery logs. ### Local files [#local-files] Two Docker volumes hold files that live only on the backend's disk: - `uploads` (`UPLOAD_DIR`): incoming tus uploads and their source files. - `transcode-output` (`TRANSCODE_OUTPUT_DIR`): FFmpeg output waiting to be uploaded to Sia. Once processing finishes, the backend serves the video from Sia, not from these volumes. ## PostgreSQL and Redis [#postgresql-and-redis] PostgreSQL holds the durable state: videos, renditions, the map from every Sia object id to its video, accounts, sessions, API keys, webhooks and reconciliation runs. [Data model](/developers/concepts/data-model) describes the tables. Redis holds the BullMQ queues, the schedules of the repeating jobs, and short-lived leases. One lease stops a second execution of an upload job (BullMQ starts one when the job's lock expires) from uploading the same video to Sia while the first execution is still running; the others hold back a slab prune (the cleanup of stored Sia data that no object uses any more) while any upload is in flight. The compose file starts Redis with append-only persistence and the `noeviction` memory policy, so queued jobs survive a restart and are never evicted. ## Sia [#sia] The backend talks to Sia through the official `@siafoundation/sia-storage` SDK. It connects to the indexer at `SIA_INDEXER_URL` with the app key from `SIA_APP_KEY`, which the connect step prints when you register the server with the indexer (see [Connect Sia storage](/self-hosting/sia-storage)). The SDK uploads the pieces of each file to hosts and reports, for each piece, the host that stored it. Two paths read from Sia: - **The delivery gateway** looks each object up on the indexer, downloads the requested bytes from hosts, and streams them to the viewer through nginx. - **The browser** can download from Sia itself. The player fetches a share map from `/v1/share/:id`, which lists a time-limited `sia://` share URL for each object of the video, then uses the Sia SDK in the browser with a separate viewer identity to download from the indexer and hosts. The backend stays out of the byte path. [Playback and delivery](/developers/concepts/playback) explains the gateway, direct and hybrid modes, and [Stream from Sia in the browser](/self-hosting/direct-playback) shows how to turn on the browser path. ## Webhook receivers [#webhook-receivers] The `webhooks` worker sends HTTP requests from the backend to the URLs you register. See [Receive webhooks](/developers/guides/webhooks) and [Webhook events](/developers/reference/webhook-events). # Data model (/developers/concepts/data-model) Sluby keeps its durable state in PostgreSQL. Video bytes live on Sia; the database holds the records that describe each video and map it to the Sia objects it is stored as, plus accounts, keys, webhooks and reconciliation history. This page describes the tables and how they relate. `backend/src/db/schema.ts` defines the schema with Drizzle. A Sia **object** is one stored file on the Sia network, identified by an object id. Every manifest, playlist, media file and thumbnail of a video is its own object. **Pinning** an object keeps it stored under Sluby's account on the Sia indexer; unpinning it releases it. [Storage on Sia](/developers/concepts/storage-on-sia) explains objects in detail. ## Ownership [#ownership] One install is one team's workspace. Every video, API key and webhook endpoint carries the owner address `workspace` in its `creator_address` column (an upload carries it in its metadata), and every caller acts as that owner. What a caller can do depends on their role, not on which rows they own. See [Accounts, roles and sessions](/developers/concepts/accounts-and-roles). ## How the tables relate [#how-the-tables-relate] The diagram shows every foreign key in the schema. Tables without foreign keys are listed after it. ```mermaid erDiagram video_assets ||--o{ upload_sessions : "video_asset_id (set null)" video_assets ||--o{ processing_jobs : "video_asset_id (cascade)" upload_sessions ||--o{ processing_jobs : "upload_session_id (cascade)" video_assets ||--o{ renditions : "video_asset_id (cascade)" video_assets ||--o{ artifacts : "video_asset_id (cascade)" renditions ||--o{ artifacts : "rendition_id (cascade)" video_assets ||--o{ playback_ids : "video_asset_id (cascade)" video_assets ||--o{ asset_plays_daily : "video_asset_id (cascade)" users ||--o{ user_identities : "user_id (cascade)" users ||--o| password_credentials : "user_id (cascade)" users ||--o{ studio_sessions : "user_id (cascade)" user_identities ||--o{ studio_sessions : "identity_id (cascade)" users ||--o{ invites : "user_id (cascade)" users ||--o{ invites : "created_by (set null)" users ||--o{ api_keys : "created_by (set null)" api_keys ||--o{ api_keys : "replaced_by_id (set null)" api_keys ||--o{ webhook_endpoints : "api_key_id (set null)" webhook_endpoints ||--o{ webhook_deliveries : "webhook_endpoint_id (cascade)" ``` The label on each line names the foreign key column and what happens to the row when the referenced row is deleted: `cascade` deletes it, `set null` clears the column. Tables without foreign keys: `pending_objects`, `reconciliation_runs`, `indexer_objects`, `reconcile_feed`, `bootstrap_keys`, `instance_meta` and `allowlist_pins`. ## Videos [#videos] ### video_assets [#video_assets] One row per video: title, description, duration, resolution, status, access tier (`public` or `private`), the owner, and the Sia objects the video is stored as (`manifest_object_id`, `thumbnail_object_ids`, `sia_object_ids`), with `segment_count` and `total_storage_bytes`. The `status` column is `created`, `uploading`, `processing`, `ready` or `failed`. [Asset status](/developers/concepts/upload-pipeline#asset-status) lists when each one is set. Other columns worth knowing: - `signing_generation` is part of every signed URL of the video. Raising it invalidates every URL signed before. [Playback and delivery](/developers/concepts/playback) explains signed URLs and revocation. - `deleted_at` marks a deletion in progress. While it is set, every API read leaves the video out; the deletion worker removes the row once the video's Sia objects are unpinned. See [Deletion and reconciliation](/developers/concepts/deletion-and-reconciliation). - `last_played_at` records the newest counted play. It is written without touching `updated_at`, so a play never changes the video's ETag. ### upload_sessions [#upload_sessions] One row per tus upload: the file path on the backend's disk, the file size, bytes received so far, the SHA-256 of the finished file, an expiry time and a status (`uploading`, `completed`, `cancelled` or `failed`). `video_asset_id` is cleared, not deleted, when its video row is removed. ### processing_jobs [#processing_jobs] The record of a video's processing run, which the API reads for progress: `status` (`queued`, `processing`, `completed`, `failed` or `retrying`), `progress_percent`, `error_message`, `retry_count`, a `retryable` flag (false when the source is not a readable video) and `logs`, a list of timestamped messages per stage. The queue jobs themselves live in Redis; this table mirrors their state. See [Upload pipeline](/developers/concepts/upload-pipeline). ### renditions [#renditions] One row per quality level of a video: `name`, `width`, `height`, `video_bitrate_kbps`, `segment_count`, `byte_size`, and two object ids: `data_object_id` (the rendition's single media file) and `playlist_object_id` (its variant playlist). [Upload pipeline](/developers/concepts/upload-pipeline) describes the ladder of renditions. ### artifacts [#artifacts] The map from every stored Sia object id to its video, its rendition when it has one, and its role. The role is empty only for objects of videos stored before this table existed. | Role | Object | | ------------------ | ------------------------- | | `master_manifest` | The HLS master playlist. | | `variant_playlist` | A rendition's playlist. | | `rendition_data` | A rendition's media file. | | `thumbnail` | A thumbnail image. | Every object id in a video's `manifest_object_id`, `thumbnail_object_ids` and `sia_object_ids` has a row here. The delivery gateway finds the video behind a requested object id in this table to decide access, the per-video storage breakdown reads it, deletion collects the object ids to unpin from it, and reconciliation compares its object ids with the indexer's. ### pending_objects [#pending_objects] Sia objects that are pinned but not recorded in `artifacts`, and objects waiting to be unpinned. The upload stage writes a row right before it pins each object, and `finalize` moves the object into `artifacts`. `state` is `uploading` while the run that pinned the object can still record it, and `release` once nothing will and the object must be unpinned. The table has no foreign key on purpose: a row has to outlive its video, because an upload can still pin an object after the video row is gone, and the deletion `gc` sweep unpins what is left here. ### playback_ids [#playback_ids] Public handles for a video, each with a `playback_id` value of the form `pb_...`, a `policy` (`public` or `signed`) and a `name`. You can create several per video and delete one without changing the video. See [Playback and delivery](/developers/concepts/playback). ### asset_plays_daily [#asset_plays_daily] Play counts per video and UTC day, one row per video and day (primary key `video_asset_id`, `day`). ## Accounts and access [#accounts-and-access] ### users [#users] One row per member of the workspace: `display_name`, `role` (`admin` or `member`) and `disabled_at`, set when an admin removes the member. A removed member's row stays, so their keys and history still point at a name and restoring them is one update. ### user_identities [#user_identities] The ways an account signs in. Each row has a `provider`, the provider's stable `subject` for the person (for GitHub, the numeric account id) and the `username` reported at the last sign-in, kept for display. The pair `provider`, `subject` is unique. ### password_credentials [#password_credentials] The email and password hash of an account that signs in with a password. The email is lowercase and unique. ### studio_sessions [#studio_sessions] One row per Studio sign-in. The signed session cookie names its row ([Sessions and cookies](/developers/concepts/accounts-and-roles#sessions)), so the server can end a session on sign-out, from the sessions list, or when an admin removes the member. Each row records the account, the identity used to sign in, the user agent, `expires_at`, `revoked_at` and `last_seen_at`. ### invites [#invites] Links an admin hands out. `kind` is `invite` (new people join with `role`) or `reset` (the member `user_id` sets a new password). The table stores only the SHA-256 hash of the token, so nobody can sign in with what it holds. Each row has `expires_at`, `used_at` and `revoked_at`, plus `max_uses` (how many people can join: 1 by default, `null` for no limit) and `use_count` (how many times the link was used). A check constraint allows a `max_uses` other than 1 only on a member invite, and `used_at` is set when the last allowed use happens ([Invites and reset links](/developers/concepts/accounts-and-roles#invites-and-reset-links)). ### api_keys [#api_keys] API keys, stored as SHA-256 hashes. The API returns the raw key once, at creation. Each row has a `name`, `scopes`, a `rate_limit`, `is_active`, an optional `expires_at`, `prefix` (the first 12 characters of the raw key), `last_used_at`, `created_by` (the member who made the key) and `replaced_by_id` (the key that replaced it in a rotation). A key acts with its maker's current role and stops working while the maker is removed; a key with no maker, such as one seeded from the server, acts as an admin. [API conventions](/developers/reference/api-conventions) describes scopes. ### bootstrap_keys [#bootstrap_keys] The hash of every `BOOTSTRAP_API_KEY` ever seeded. A hash listed here is never seeded again, so deleting or rotating away the bootstrap key stays in effect across restarts while the variable is still set. ## Webhooks [#webhooks] ### webhook_endpoints [#webhook_endpoints] The workspace's webhook URLs: `url`, `events` (the event types the endpoint receives), its signing `secret`, `is_active`, and `api_key_id`, the key it was registered with (empty when someone registered it from the Studio). The link records who registered it, not who owns it, so revoking the key keeps the endpoint. ### webhook_deliveries [#webhook_deliveries] One row per delivery attempt: `event_type`, `payload`, the receiver's `status_code`, an `error` message written by Sluby, `delivered_at`, `retry_count` and `manual` (a resend someone asked for). Sluby never stores the receiver's response body. Retries wait in Redis, not in this table. See [Delivery log](/developers/reference/webhook-events#delivery-log). ## Reconciliation [#reconciliation] ### reconciliation_runs [#reconciliation_runs] One row per reconciliation pass: start and finish time, `status` (`ok`, `drift` or `failed`), object counts (`db_object_count`, `indexer_object_count`, `in_sync_count`, `orphan_count`, `missing_count`), the full lists `orphaned_ids` and `missing_ids`, and `error_message` for a failed run. ### indexer_objects and reconcile_feed [#indexer_objects-and-reconcile_feed] `indexer_objects` is reconciliation's copy of the indexer's object inventory: for each object id, whether its latest event was a deletion and when that event happened. `reconcile_feed` is a single row naming the indexer feed (the indexer URL and account key) that copy was built from, and a cursor for where the next run resumes reading. A run against a different feed starts the copy over. See [Deletion and reconciliation](/developers/concepts/deletion-and-reconciliation). ## Install state [#install-state] - `instance_meta` stores one-off facts about the install as key and JSON value pairs, such as the session secret the install generated when `SESSION_SECRET` is not set, and whether the access settings of an older version were already adopted. - `allowlist_pins` ties a GitHub login from the old allowlist setting to the account that held it. Startup reads it only to adopt an install that used the allowlist. ## Migrations [#migrations] Migrations live in `backend/drizzle` as SQL files. The backend container runs them on every start before it starts the server; the `__drizzle_migrations` table records the applied migrations, so the step changes nothing in a database that is up to date. The migration step takes a PostgreSQL advisory lock first, so several backend containers starting together migrate one at a time. # Deletion and reconciliation (/developers/concepts/deletion-and-reconciliation) This page explains what Sluby does, step by step, when you delete a video, how a background sweep finishes deletions that stopped part way, and how reconciliation compares the objects Sluby records in PostgreSQL with the objects the Sia indexer holds for your account. Two Sia terms appear throughout. The **indexer** is the Sia service Sluby stores data through: a self-hosted `indexd` or the hosted `sia.storage`. An **object** is one stored file (a playlist, a media segment, a thumbnail), named by a 64-character hex object id. While an object is **pinned**, the indexer keeps it for your account; **unpinning** tells the indexer to drop it. For the full storage model, see [Storage on Sia](/developers/concepts/storage-on-sia). ## What starts a deletion [#what-starts-a-deletion] Every path below calls the same deleter: - `DELETE /api/v1/assets/{id}` with the `manage` scope ([endpoint reference](/developers/reference/api/assets/delete-assets-id)). The Studio and the `@sluby/sdk` method `assets.delete()` both call this endpoint. - Cancelling an upload that is still in progress, with a `DELETE` request on the upload URL of the tus resumable upload protocol, or with `DELETE /api/v1/uploads/{id}` ([endpoint reference](/developers/reference/api/uploads/delete-uploads-id)). The deleter removes the video the upload created. - An upload that expires before it finishes. The hourly upload sweep marks the upload session `failed` and deletes the video when its status is still `created` or `uploading`. - An upload creation request that fails after Sluby created the video row. ## Step 1: the video disappears at once [#step-1-the-video-disappears-at-once] The deleter sets the video's `deleted_at` timestamp in the same request. From that moment: - Every read in the API treats the video as gone. `GET /api/v1/assets/{id}` and the playback routes answer `404`. - The response to `DELETE /api/v1/assets/{id}` is `200` with `{"success": true, "status": "deleting"}`. The `deleting` status tells you the rest of the work runs in the background. - A second `DELETE` for the same video answers `404`, because only a video that is not already being deleted matches. The deleter then queues a job named `delete` on the `deletion` queue, with the job id `delete-