The Architecture of skowt.cc

skowt.cc started in 2022 as a directory listing. An h5ai file index I'd spun up overnight at wtf.dromzeh.dev on a DigitalOcean droplet I had free through GitHub Education, so my friends and I had somewhere to keep files. No frontend, no search, just h5ai's directory listing.

Then I saw a TikTok comment. Someone wanted the character sheets a game shows on its preview pages, and every reply was pointing at the same overstuffed Google Drives. I had a file server sitting right there, so I put the sheets on it and replied with the link. Somebody else made a TikTok about the site, and that one blew up. I messaged them, said I was the creator, and asked them to pin my comment so people could actually find it, because the URL kept getting swallowed. They did. The pin sat at 10k likes and Google Analytics was showing hundreds of thousands of people picking through a raw file listing on a free droplet.

I was on holiday when this happened. My Windows install had also just died, so the first day went to reinstalling it. I'd built for the web before, a popular anonymous file-sharing service in Flask that I ran for a while and took down, but I'd never touched a proper web framework. This was before you could ask an AI for help, and Stack Overflow wasn't somewhere I wanted to be, so I taught myself SvelteKit by trying things and watching them break, and built the replacement in three days. I named it wanderer.moe: you're searching for assets, you're wandering, and .moe is what you see all over the gacha communities. This was before the Genshin character called Wanderer existed, which matters later.

wanderer.moe became skowt.cc. Same site the whole way through, two renames and a couple of rewrites later. I've only ever run one asset database. This is what it looks like now, plus enough of the old versions to explain why.

What it does hasn't changed. Contributors upload assets from popular games, a lot of them gacha games, and for now it's all images. Other people pull them back down, often a few hundred in one go. The traffic is spiky and downloads arrive in big batches, so most of the design comes down to one bias: keep the request path cheap, and never make the API do work it can hand to something else. It took me years to actually live up to that bias, which is part of this story too.

The Static Era#

That three-day SvelteKit build was held together with the worst idea I've ever shipped: every asset file lived inside the site repo itself, pulled in with import.meta.glob, deployed as a static site on Cloudflare Pages. The whole catalogue rode along in git. It worked because there wasn't much yet, and it was horrific.

The first real storage was R2 with a tiny Wrangler worker that listed the bucket out, and the upload pipeline was a GitHub repo we called the CDN repo: commit files to it, a workflow runs rclone into the bucket. Later I dropped even that and just kept the entire bucket mirrored on my PC, curated files by hand, and ran rclone sync. Every asset on the site passed through my hands for years. The moderation system of that era was an rclone exclude rule, and Windows still beat it once: a desktop.ini slipped past the filter and sat in the public bucket like it was a character sheet.

The DDoS and the Rewrite#

In early 2024 the site got DDoSed. I found out through a webhook I had logging API failures, which suddenly filled with rate-limit errors while the site crawled and everything else I ran stayed fine. The Cloudflare dashboard showed a traffic spike from many IPs on one ASN, so per-IP limiting was doing nothing. I blocked the entire ASN and reported it. The attacking box was a DigitalOcean VPS, which given where this site started is pretty funny.

The block was the mitigation. The rewrite happened because the attack made me open the codebase, and I'd been away from SvelteKit long enough that I couldn't read my own site anymore. I remember thinking I can't believe we've left it like this. I'd been writing Next.js for over a year by then, so I rebuilt the whole thing, new UI included, in one sitting somewhere between 13 and 17 hours, and shipped it. The old site stayed up the entire time and users woke up to a new one. That rebuild is also what added accounts, because saving assets to browser localStorage was wearing thin.

Killing (in) the Name (of)#

The rename came from resentment. wanderer.moe predates the character: Genshin later reworked Scaramouche into a playable character called Wanderer, the name leaked ahead of his release, and overnight everyone read "wanderer" as Genshin. From then on the site was a "Genshin asset site" no matter how many games I added, and something about the name itself had started to grate anyway. What do you do on the site? You scout for assets. Scout was taken, so I spelled it with the k and the w because it looked cool. I wanted .gg, but .gg is funny with its requirements, and honestly .cc looks better.

The stack came from somewhere else. skowt has always been my testing ground, but this time the flow reversed: I was deep in Originoid work, living in Elysia, tRPC, and Turso daily, and that had become my default stack. The rewrite under the skowt name is that stack pointed at the site I'd been running since 2022.

Layout#

The whole thing is a monorepo. Three apps and a handful of workspace packages. apps/web is a TanStack Start SPA that ships its own server bundle for SSR. apps/server is an Elysia process that mounts a tRPC adapter and a couple of health endpoints. apps/asset-redirect is a Cloudflare Worker that owns the public read path, and it gets its own section, because it exists to fix the thing I was most wrong about for the longest. Everything the apps don't own lives under packages/*: the routers, the better-auth config, the Drizzle schema, the env validation, the observability stack.

The frontend talks to the backend through one typed tRPC client and nothing else. No hand-written fetch wrapper, no REST layer, no generated client to keep in step with the server. The router types flow straight through, so a procedure signature change is a red squiggle in the browser code before it's a runtime error.

Bun and Elysia#

The server runs on Bun, not Node. Startup and raw throughput are the obvious wins, but the part I use most is the plugin model. Elysia exposes the request as it moves through named lifecycle hooks (onRequest, derive, onAfterHandle, onAfterResponse), so the whole logging and tracing layer hangs off those hooks instead of wrapping every handler in middleware. I write the procedure and the plugin underneath does the logging.

Data Layer#

Three stores, each holding the thing it's best at.

The relational data lives in Turso, which is hosted libSQL: an open-source fork of SQLite that adds a server you talk to over the network. That choice matters twice later: search can be a virtual table in the same database rather than a separate service, and it's also the reason a naive session check hurts, because every read is a network round-trip rather than a local file seek.

Files live in Cloudflare R2, behind pack.skowt.cc. Redis holds the ephemeral state: sessions, rate-limit windows, download batches, and a cache in front of the hot read paths. Logs and traces go to Better Stack.

Upload Path#

The API process never handles file bytes, on the way in or out. It moves metadata and URLs, and the bytes go straight between the browser and R2.

Uploading at all is contributor-only: people who've been around the Discord a long time and who I trust. I'm not worried about contributors going rogue. What the rest of this section defends against is Discord accounts getting hacked. Everything about the pipeline is shaped so that a compromised trusted account has the smallest blast radius I can give it.

Presigned Handshake#

An upload is three steps:

1. The browser calls uploads.requestUpload with the file's name, MIME type, size, and where it belongs in the catalogue. The server checks the MIME type against a MIME_TO_EXTENSION map, rejects anything over MAX_FILE_SIZE, inserts the asset row as pending, and mints a presigned PUT URL with a short expiry.
2. The browser uploads straight to R2 against that URL.
3. The browser calls uploads.commitUpload, and only then does the server confirm the file and flip its status.

A large upload never streams through Elysia. It doesn't sit in the process's memory or count against its concurrency, so the process stays small and its latency stays flat.

Distrusting the Upload#

A presigned PUT pins almost nothing: not the Content-Type, not the byte count. The browser can declare a 2MB PNG at requestUpload and then push 400MB of arbitrary bytes to the URL, and nothing it declared up front is binding.

So commitUpload treats the uploaded object as hostile and re-derives the truth, five things off one read. It stats the object, the only real size enforcement in the whole flow, and rejects anything over MAX_FILE_SIZE before reading a byte. It reads the header bytes and rejects anything that isn't actually an image. It hashes the content and checks the hash against every approved and pending asset, so the same bytes can't enter the catalogue twice; a duplicate commit comes back as a conflict pointing at the existing asset. It reads the dimensions. And it generates the derived images, a thumbnail and a 1024px preview, fail-closed: a commit that can't produce both variants is rejected outright rather than letting a broken original into the catalogue.

Storage keys derive from the asset id, never from the hash; the hash exists to answer "have I seen these bytes", and keying storage on it would break the moment two eras of the catalogue disagree about a file's identity. And every terminal rejection purges: the object gets deleted and so does the pending row and its tag joins. That last part exists because I found two ghost rows in production, pending assets whose objects were long gone, and decided rejections should clean up the whole trail.

Limbo Prefix#

The presigned URL targets one of two prefixes. Uploads from people whose trust flags say their work goes public without review write straight to the public asset/ prefix. Everyone else writes to limbo/, which the edge worker refuses to route at all.

That split is enforced by storage layout rather than by an access check I have to remember to write. The instant an object exists under asset/, anyone who can build the URL can fetch it, so a pending upload has to physically sit somewhere unservable until a moderator clears it. Approval is the operation that copies the object from limbo/ to asset/ and flips the row to approved. Moderators and the uploader can still see a pending original through a presigned inline URL, gated to exactly those people; nothing else has a path to it.

Two Producers, One Queue#

The site is no longer the only way in. A Discord bot exposes upload as a slash command, with autocomplete over games and categories, and it goes through the same single-shot ingest function the web path resolves to: same magic-byte check, same hash dedup, same fail-closed variant generation, same limbo-unless-trusted routing. A duplicate upload through the bot replies in Discord with an embed linking the asset that already has those bytes.

Download Path#

Here's where I have to eat my own words. For a long time my position on downloads was: the API's role here is accounting, not gatekeeping. There was no presigned-download flow, anyone who wanted to could construct a CDN URL and skip the download counter, and I'd decided the counter earned its place even when it was skippable, because gating the bytes would mean a presigned-download flow or a worker in front of R2 and I didn't think it was worth either.

I ended up building both. The principle didn't change, the bill did. The old path leaned on Cloudflare image resizing for thumbnails, and serverless image transforms are mad expensive even with caching: some months the resizing alone ran into the hundreds, way past what I could afford for a free site. I kept promising I'd change it and never did, for something like a year, until I finally sat down and moved the whole read path.

The Edge Worker#

pack.skowt.cc is now a Worker, and the public read policy is one pure function in it: given a hostname and a path, decide. The allowlist is derived variants only, the pre-generated {id}-thumb.webp and {id}-preview.webp, game icons, cover images. A request for a raw original 301s to the preview, which is also quietly migrating the ~40k Google Images entries that point at full-size originals onto the smaller variant. limbo/ has no route at all. Everything else 404s before the bucket is ever touched. Responses cache through the Workers cache, because every bucket read is a billed operation and the hot thumbnails should be paid for once.

Actual downloads are presigned now. downloads.generate verifies the asset ids, records the batch, bumps the counters, and mints a short-lived presigned GET per asset with content-disposition: attachment. Recording and delivery are one call, so the counter can't be skipped by an honest client, and originals aren't reachable any other way.

Download Batches in Redis#

A batch is short-lived, per-user, and disposable, which is what Redis is good at. recordDownloadBatch mints a uuidv7 for the batch id, so ids sort by creation time for free. It rejects any payload over 512KB, then runs a pipeline: set the batch JSON with a 90-day expiry, zadd the id into a per-user sorted set scored by the current time, and refresh that set's own expiry.

A per-user cap of 50 batches keeps it bounded; past 50, it reads the oldest ids off the front of the sorted set, deletes their data keys, and trims the set by rank.

Reads lean on the same structure. Listing a user's batches is one zrevrange for the page and a single mget for all the payloads at once. If a payload has expired out from under its sorted-set entry, that read treats the entry as corrupt and drops it, so the index heals itself instead of accumulating dead ids. Fetching one batch does a zscore ownership check first, so a user can't read a batch id that isn't theirs. None of this needs a table, a migration, or a background cleanup job.

Whole-Game Packs#

"Can I just download everything for a game" was a consistent ask for years, and I always refused, because the obvious implementation is pre-baked zips and pre-baked zips go stale: the moment a game updates you're regenerating archives or serving wrong ones. The version I finally built bakes nothing.

downloads.generatePack builds a manifest from the live catalogue at the moment you ask, then returns one URL for the edge worker signed with an HMAC that covers the game, the category, and a five-minute expiry, nothing user-specific. The worker verifies the signature at the edge and calls back to the API for the manifest, authorised by the same token, so there's no second credential to rotate. Then it builds the zip in the stream.

The archive is store-mode, no compression, and that's not laziness: game art is already-compressed webp and png, and store-mode means the total archive size is computable up front from the manifest. The response goes through a fixed-length stream so the browser gets a real content-length and an honest progress bar instead of chunked encoding's shrug. The worker keeps five bucket reads in flight ahead of the zip writer, and releases the prefetched bodies if you abort, so a cancelled 3GB pack doesn't leave connections dangling.

A pack deliberately ticks no per-asset counters and writes no batch history. It's a bulk export, not someone browsing, and counting it as 16,000 downloads would just make the numbers wrong.

Catalogue Query#

The catalogue query is the hot path. It runs on every game page, every filter change, every sort, every scroll. It has to stay cheap on a library with a long tail, so two things carry it: a cache, and keyset pagination.

Caching Public Reads#

Every result is keyed by its full parameter set and cached in Redis with an in-memory fallback if Redis is unreachable. The fallback is a bounded Map that evicts the oldest fifth when it fills, so a Redis outage degrades the cache instead of unbounding memory. TTLs are tiered by how fast the surface changes: two minutes on search and related assets, five on game landing pages and recent drops, ten on sitemaps and site stats.

What makes the cache safe is that everything under these keys is public, status = 'approved' data. The key builder truncates long parameters, which means two different queries can technically collide on one key. That's fine here precisely because a collision returns someone else's public search results, not anyone's private data. The comment in that file is a warning to my future self: cache per-user or non-approved data through this path and the same truncation becomes a cross-user leak.

Invalidation is the part I got wrong first. The catalogue is cached under six namespaces: search, related, recent drops, game landing pages, sitemaps, and site stats. Early on, an approval only cleared the search cache, so a freshly approved asset showed up in search at once but stayed missing from the drops and landing pages until their TTLs ran out. Now any content mutation clears all six through one function, and structural changes to games or categories clear the filters cache on top. The key names live in one registry so the write side and the invalidate side can't drift apart.

Keyset Over OFFSET#

The naive way to paginate is LIMIT n OFFSET m. It has two problems. The database walks and throws away every row before the offset, so deep pages get linearly slower, and any insert between two page loads shifts every row down, handing you a duplicate or a gap at the boundary.

Keyset pagination fixes both. Instead of counting rows to skip, the cursor pins an exact boundary row, and the next page is everything ordered past it. The query resumes with a tuple comparison, ordered by the sort column with the asset id as a tiebreaker so the ordering is total:

or(
  compareOp(asset.createdAt, cursorValue),
  and(eq(asset.createdAt, cursorValue), compareOp(asset.id, cursorId)),
)

It fetches one row past the limit as a has-more probe and trims it off. Name sorts run COLLATE NOCASE on both the comparison and the ORDER BY, which matters more than it reads: the library still carries slugs from the wanderer.moe era, and under binary collation those lowercase legacy names sort after the entire uppercase alphabet, splitting the catalogue into two visible blocks.

Cursor#

The cursor carries almost nothing. Only the boundary asset's id and the sort key it belongs to, base64url-encoded over a JSON payload:

export function encodeCursor<T extends Record<string, unknown>>(payload: T): string {
  return Buffer.from(JSON.stringify(payload)).toString("base64url");
}

The sort value itself never goes on the wire. Sort by downloads and the download count stays server-side. On the next request the server re-derives the boundary row's sort value from its id, with a lookup that also re-checks status = 'approved', so a hand-edited cursor can't point at a pending asset and use it as a paging anchor. Decoding is defensive: a malformed, truncated, or tampered cursor comes back as null, which means page one instead of a crash.

Search#

Name search is a filter on that same catalogue query, but it's the one filter that needed real machinery underneath it.

FTS5 Trigram Over LIKE#

Name search used to be LIKE '%term%'. That's a full scan, and it's case-sensitive in ways that surprise people. The replacement is an FTS5 virtual table with the trigram tokenizer, living inside the same libSQL database:

CREATE VIRTUAL TABLE IF NOT EXISTS asset_fts
  USING fts5(asset_id UNINDEXED, name, tokenize='trigram')

Trigram was the specific pick because it reproduces the substring feel of LIKE '%term%' while being indexed and case-insensitive. Swapping LIKE for MATCH keeps the behaviour and drops the scan. The one thing trigram can't do is tokenize a term under three characters, so the query layer keeps a LIKE fallback for two-character searches and switches to the index at three:

if (term.length >= 3) {
  return sql`${asset.id} IN (SELECT asset_id FROM asset_fts WHERE name MATCH ${escapeFtsMatch(term)})`;
}
return like(asset.name, `%${term}%`);

That subquery is non-correlated, so MATCH runs once and the planner uses the FTS index instead of materialising an id list into the outer query. Keeping the id list out of the outer query also keeps the bound-parameter count down and the SQL text in traces short.

Keeping the Index in Sync#

The index tracks the asset table through SQL triggers, not application code. Insert, delete, and update triggers mirror rows into asset_fts. The update trigger is scoped AFTER UPDATE OF name on purpose, so the constant view-count and download-count writes never touch the search index. The index stores every asset regardless of status, and the status = 'approved' filter runs at read time, so approving an asset costs nothing at the index.

None of this fits Drizzle. It can't model a virtual table or a trigger, so drizzle-kit push will never create either. The DDL is idempotent, IF NOT EXISTS on everything, and four paths apply it and agree: a hand-written migration, a bun db:fts script, the test harness, and a boot-time check that only backfills when the FTS row count falls behind the asset count, so a normal boot does nothing.

Running a separate search service like Meilisearch would mean another process to run and another copy of the data to keep consistent, when the database the catalogue already lives in can do the search itself.

Escaping the Match Grammar#

FTS5's MATCH has its own query language: AND, OR, NEAR, prefix *, column filters, - negation, quoted phrases. Passing raw user input into it would let someone inject an operator, or search for a term with a * in it and get a syntax error back as a 500. The escape wraps the whole term in double quotes and doubles any embedded quote:

function escapeFtsMatch(term: string): string {
  return '"' + term.replace(/"/g, '""') + '"';
}

That forces FTS5 to read every character as literal string content, which for the trigram tokenizer is a plain substring match. Adversarial inputs like " OR x, a*b, and a OR b all collapse to literals.

Sessions in Redis#

Turso is hosted libSQL, so the database sits across the network, and a session check that reads the session row and then the user row is two round-trips at roughly 57ms each. Paying that on every authenticated request taxes everything downstream of it.

better-auth takes a secondary storage adapter, so sessions run through Redis:

const authSecondaryStorage = {
  async get(key) { return authRedis.get(key); },
  async set(key, value, ttl) {
    if (ttl) await authRedis.set(key, value, "EX", ttl);
    else await authRedis.set(key, value);
  },
  async delete(key) { await authRedis.del(key); },
};

Two database round-trips at ~57ms collapse into one sub-millisecond Redis GET. This isn't a dumb TTL cache that can serve a stale session, either: better-auth invalidates the entry itself on sign-out and revocation, and when I change what a user is allowed to do, the grant gets patched into their live cached sessions rather than waiting for a re-login.

Two details earn their comments in the code. authRedis is a dedicated connection, kept off the application Redis client so a burst of per-request auth reads can't head-of-line block the app's own commands. And ioredis wants set(key, value, "EX", seconds), not the node-redis options object; get that wrong and the TTL silently drops, leaking session data past its expiry.

Rate Limiting in Lua#

Rate limiting is the other Redis-native piece, and it's a real sliding window rather than a fixed bucket. Each window is a sorted set where every request is a member scored by its millisecond timestamp. The whole check runs as one atomic Lua script:

redis.call('ZREMRANGEBYSCORE', key, '-inf', window_start)  -- evict entries older than the window
local count = redis.call('ZCARD', key)                     -- how many remain
if count < limit then
  redis.call('ZADD', key, now, request_id)                 -- admit this request
  redis.call('PEXPIRE', key, window_ms)                    -- self-expiring key
  return {1, limit - count - 1, 0}
else
  local oldest = redis.call('ZRANGE', key, 0, 0, 'WITHSCORES')
  return {0, 0, math.max(0, oldest[2] + window_ms - now)}   -- when the oldest entry ages out
end

It runs in Lua for atomicity. The trim, the count, and the conditional add have to be one server-side operation. Split them into three application-side Redis calls and two concurrent requests interleave between the ZCARD and the ZADD: both read a count under the limit, both get admitted, and the limit overshoots. Lua runs on the Redis server as a single unit, so that race can't open. On rejection it computes a real retry-after from the moment the oldest in-window request will age out, which the tRPC middleware turns into a TOO_MANY_REQUESTS with the header set. If Redis is unreachable, the limiter falls back to a per-instance in-memory window rather than failing the request.

Limits are per procedure tier, with data export capped separately at one an hour because it's expensive to serve:

The same limiter guards the /api/auth/* routes that better-auth handles outside the tRPC pipeline, keyed per auth path so frequent session reads don't share a bucket with sign-in attempts. Anonymous keys are derived from the client IP, with SSR requests attributing to the visitor's forwarded IP rather than the server's own egress address, so server-rendered traffic doesn't pile into one bucket.

Roles, Then Trust Flags#

Authorization started as four tiers on a numeric ladder, not a bag of permission flags:

export const ROLE_HIERARCHY = { user: 0, contributor: 1, staff: 2, developer: 3 };
export function hasMinimumRole(userRole, requiredRole) {
  return ROLE_HIERARCHY[userRole] >= ROLE_HIERARCHY[requiredRole];
}

The procedures compose as a chain. A base traced procedure opens the per-call span. protectedProcedure builds on it with the session-required check and narrows the session type to non-null. createMinimumRoleProcedure(role) builds on that with a single numeric comparison, and contributorProcedure, staffProcedure, and developerProcedure are that factory called with a different rung.

The first version of this post said there was no per-feature flag anywhere and that needing one would be a signal to split a role. That prediction didn't survive. The ladder answers "how far up the site does this person sit", and moderation kept asking a different question: "what would it cost if this specific account got phished." So three per-user trust flags now sit beside the ladder, skipsReview, canApprove, and canManageCatalog, each with its own procedure factory that ignores the rungs entirely. Staff role alone deliberately doesn't skip the review queue; skipping is a statement about how confident I am that an account won't get compromised, not about seniority. canApprove implies skipsReview, because someone trusted to clear other people's uploads can clear their own.

The same threat model is why deletion is soft. A deleted asset keeps its row and its objects for 30 days before a sweep purges them, so if an account with real power ever does get taken over and goes on a purge run, nothing it deletes is actually gone.

One Wide Event per Request#

Observability is the part I'm fondest of, because it's what makes the rest debuggable in production without much cost.

What Goes in the Event#

Every request emits exactly one structured event at response time, not a log line per step, built up in place as it moves through Elysia's hooks. It carries the standard OTel HTTP attributes, http.request.method, url.path, http.response.status_code, duration_ms, next to cheap domain identity: the user id, a debug id, which tRPC procedures ran, how many database queries the request made, how many outbound fetches.

That turns Better Stack's log explorer into one query surface. "Every request where this user hit a procedure that ran more than 50 database queries" is a single filter, no trace to open. Severity still exists as a field on the event, not as the axis you filter on first.

Counting Without a Request Reference#

The per-request counters, db.queries_count and fetch.calls_count, are the interesting piece. The traced database client and the fetch wrapper are deep in the stack, nowhere near the request object, so threading a counter down to them by hand would be miserable. Instead the plugin opens an AsyncLocalStorage frame on onRequest, and the wrappers bump their counters into whatever frame is active. AsyncLocalStorage follows the request across every await, so the count is correct without anyone passing it around.

onAfterHandle reads the totals back onto the event. Then onAfterResponse calls endRequestStats() to clear the frame, so a void doThing() that keeps running after the response can't bump counters into an event that already shipped. A db.queries_count over 50 flags an N+1, and a fetch.calls_count over 10 flags a runaway upstream loop, both without opening a trace.

Dual Drain#

The drain that ships those events composes two sinks, stdout and Better Stack, with per-sink failure isolation. When Better Stack's ingestion is down during an incident, the stdout drain still flushes and the failure logs to stderr with the sink that broke. A logging-backend outage can't take local logging down with it.

Tracing Under Bun#

Getting traces at all under Bun took some patching, because the off-the-shelf OTel auto-instrumentation for fetch and ioredis doesn't fire. Bun's fetch is a native Zig implementation, and the require-in-the-middle hooks those packages hang off aren't wired up for Bun as a runtime, so with the official approach zero spans land. So the observability package patches globalThis.fetch and Redis.prototype.sendCommand directly, made idempotent with Symbol.for markers, and every outbound HTTP call and Redis command produces a client span with the right semantic-convention attributes.

The database client gets the same treatment one level lower. tracedClient wraps the raw libSQL client rather than Drizzle, because that's the leaf async call with a real completion-and-error hook to drive a span's lifetime.

It spans COMMIT and ROLLBACK on their own, since those are round-trips to Turso, and without them an 800ms write transaction shows up as a 750ms gap nobody can explain. The recorded SQL is capped at 4KB, because an inArray over thousands of ids renders tens of kilobytes of SQL, and one uncapped attribute can push an OTLP request past Better Stack's body limit and drop the whole batch of spans on the floor.

Redacted Secrets#

Secrets travel wrapped in a Redacted newtype. The wrapped value lives in a WeakMap keyed by the wrapper instance, and the wrapper's toString, toJSON, and Node inspect hook all return the literal <redacted>. It's built with Object.create rather than a real constructor, so even instanceof won't out it; the only way back to the value is Redacted.value(self), which is a single grep-able call site. Secrets enter as plain strings at the env boundary, get wrapped the moment they're handed to a consumer, and move as Redacted from there. Log one by accident and you get the sentinel string, not the secret. As a backstop, the drain replaces any Redacted that reaches an event with the sentinel before it hits a sink.

The mechanism is small. The wrapper's prototype answers every serialization path with the sentinel, and make tucks the real value into the WeakMap, where nothing on the object can reach it:

const SENTINEL = "<redacted>";
const registry = new WeakMap<object, unknown>();
 
const proto = {
  toString() { return SENTINEL; },
  toJSON() { return SENTINEL; },
  [Symbol.for("nodejs.util.inspect.custom")]() { return SENTINEL; },
};
 
export const Redacted = {
  make<A>(value: A): Redacted<A> {
    const instance = Object.create(proto); // no property holds the value
    registry.set(instance, value); // it lives in the WeakMap, keyed by the instance
    return instance;
  },
  value<A>(self: Redacted<A>): A {
    return registry.get(self) as A; // the one intentional way back
  },
};

Every path a secret leaks through routes into one of those three methods. JSON.stringify calls toJSON, console.log and util.inspect hit the inspect hook, and a template literal or String() calls toString. All of them hand back <redacted>. The value was never a property on the object, so there's no field for a spread, a structuredClone, or a stray JSON.stringify to pull it out of. It sits in the WeakMap, and the only key that opens that map is the wrapper instance itself, passed to Redacted.value.

"A single grep-able call site" isn't a figure of speech; a helper checks it. scripts/check-secrets.ts greps the source for the eight protected env vars (BETTER_AUTH_SECRET, DATABASE_AUTH_TOKEN, DISCORD_CLIENT_SECRET, the two S3 keys, the Better Stack tokens, the IP-hash key) and fails if any of them appears outside a Redacted.make or Redacted.value call in the same statement, or outside a file on an explicit allowlist.

The allowlist in docs/secret-allowlist.txt is short, and every line carries its own justification: the env package's declaration site, plus the seed and provisioning scripts that run outside the env package and can't consume a Redacted. The grep is coarse on purpose and errs toward false positives a human reviews rather than misses a real leak.

Debug ID#

One thread ties it together. A stable per-browser id, minted into localStorage and sent as x-debug-id on every request. Someone reports that a page was slow, copies their debug id out of settings, and support searches Better Stack for that one value to pull every log and trace from that browser, signed in or not. That's what makes a pile of wide events findable by someone who only knows "it was slow for me."

The 2GB Morning#

All of that instrumentation earned its keep the morning the web service was sitting at around 2GB of memory. I was on my phone, so I asked the Railway agent in Discord what was going on, and it came back with: you have a memory leak, and your memory has climbed since this commit. Better Stack had flagged the same thing. Comparing the diff found it in minutes.

The commit it pointed at was SEO work: SSR so crawlers get real pages, a sitemap advertising around 39,000 asset URLs. That work planted the leak and sent out the invitations in the same breath. The SSR bundle was creating its QueryClient, tRPC client, and options proxy at module scope, which on the server means one shared instance per process, not per request, so crawlers walked the sitemap and every unique URL added an entry to a render cache nobody ever read again; aborted crawler streams pinned per-request closures on top. The fix builds those objects inside getRouter(), so each SSR request gets its own short-lived graph and the browser still constructs everything once. The same commit fixed a second, smaller leak the first one exposed: sonner queues toasts in module state, and on the server there's no <Toaster /> mounted to ever drain the queue.

Frontend Data Layer#

The browser side runs on the same instinct: keep the fast backend feeling fast, and never make the user wait on something the client could have hidden.

Search as You Type#

The search box is a local, responsive input. Every keystroke updates the field with no network anywhere near it. A 120ms debounce sits behind it, and only the settled value commits to the query and the URL. That number is deliberate. People type at roughly 150 to 250ms a character, and the server answers a name search in about 5ms off the FTS5 index, so a longer debounce is dead wait with nothing to hide behind. The old 300ms sat on top of every search as pure added latency. Search only fires at two characters and up, so one stray letter doesn't kick off a query.

What keeps it feeling solid instead of flickery is keepPreviousData. When the committed search or a filter changes, TanStack Query holds the old grid on screen and swaps the new results in underneath once they land. The grid never blanks to a spinner between queries.

Infinite Scroll#

Scrolling rides the same keyset cursor the backend hands out. Pages are 100 items, and getNextPageParam just returns the nextCursor off the last page. The trigger is an IntersectionObserver watching an invisible one-pixel sentinel, with a rootMargin of 3000px, so the next fetch fires three or four viewports before the sentinel is anywhere near the screen. A fast flick covers a thousand pixels before a fetch round-trip lands, so the margin has to outrun momentum scrolling rather than walking pace. Paired with 100-item pages, the loading spinner is effectively unreachable.

Behind the sentinel sits a runway of skeleton cards, so a momentum scroll carries on into ghost cards instead of stalling at the document floor, and the real page swaps in beneath you mid-flight.

Grid#

Every image box reserves its slot with an aspect-ratio computed from the asset's stored dimensions, falling back to a square when the dimensions aren't known, so a card holds its place before the image loads and nothing reflows when it arrives. object-contain letterboxes the real image inside the reserved box, so nothing crops. Images are loading="lazy", and they point at the pre-generated webp thumbnails, so the grid never decodes a full-size original.

The masonry is hand-rolled for the same reason. CSS multi-column layout re-balances every column when new items append, which would shuffle the cards you were looking at each time a scroll page loaded. Instead the grid partitions items into real column stacks with a greedy shortest-column-first pass, using each card's known aspect ratio to estimate its height.

The assignment is deterministic and prefix-stable, so appending a page can only add to the bottoms of columns, never move a card that's already on screen. Because the committed filters and search live in the URL as slugs, opening an asset and hitting back restores the same grid and scroll position instead of throwing you back to the top.

Batched Calls, Cached Reads#

The tRPC client uses httpBatchLink, so several procedure calls fired in the same tick collapse into one HTTP request instead of one round-trip each. TanStack Query holds results with a 60-second stale time, so stepping back to a page you just saw doesn't refetch it. TanStack Router preloads a route's data on intent, so the data for a page is often already in flight by the time the click lands.

Cross-Tab Selection Sync#

The download selection, the cart you fill before pulling a batch, is a Zustand store persisted to localStorage and capped at 350 assets. It stays in sync across open tabs with no websocket and no shared worker. The trick is the browser's own storage event: when one tab writes the selection key, every other tab fires a storage event, and a module-level listener re-reads localStorage into the in-memory store. Select an asset in one tab and the cart updates in the rest, no server round-trip involved. A typeof window guard keeps the whole thing inert during SSR. Only the selection syncs live this way; settings took the opposite route and sync across devices instead, pushed to the account on a debounce and applied on sign-in, because a theme should follow you to your phone but a half-built download cart shouldn't.

Graceful Shutdown#

This last one is small, and you never notice it until it breaks. The server handles SIGTERM as a real drain. A flag flips so the health endpoints start returning 503 and the orchestrator stops sending new traffic, then it stops the server, closes Redis, and flushes OTel before it exits. That flush is the point: without it, the last few seconds of spans die in the in-memory batch processor's queue instead of reaching Better Stack, and those are the traces you most want after a deploy or a crash. The flush is bounded, so a slow exporter can't hang the process past its grace period.

Where It Sits#

As of me writing this, that's 42,957 assets across 23 games, 27 million-plus downloads, 190 million-plus views, all of it run by one person plus the Discord regulars I trust not to get phished. Expanding past images is the next pressure the design has to absorb: the presigned path already doesn't care what the bytes are, the edge worker already refuses to serve anything it doesn't recognise, and MIME_TO_EXTENSION is one map away from learning new types.