Repository navigation
feat(render): add render worker role and cached OG image pipeline - #393
Open
Anshuu2004 wants to merge 2 commits into
Open
Anshuu2004 wants to merge 2 commits into
Anshuu2004 wants to merge 2 commits into
Conversation
Adds the first half of the worker-render pool: a `render` worker role that consumes the new `domain-og-render` queue, and a 1200x630 OG card for published profiles drawn with satori and resvg. - worker-runtime: new `render` role (and part of `all`), worker:render script, compose service, Dockerfile copies the font assets - RenderArtifact model: unique artifactKey, contentHash, status, TTL; small PNGs are stored inline for now - card data comes from the same section/privacy rule as the public profile DTO (visibleSections), so hidden sections never reach the image - content-hash cache: a render with unchanged content returns the stored artifact and does not call the renderer - GET /api/v1/profile/public/:username/og.png: the API only queues a job and waits up to 8s; unknown and unpublished handles get the same 404; ETag/304 after the visibility check; 503 + Retry-After if no worker answers in time - seo.ogImageUrl in the public profile API and og:image in the /u/:slug shell are set only for published profiles (shell needs PUBLIC_API_URL) - unpublishing deletes the stored card - bundles Inter (latin subset, SIL OFL 1.1) under backend/assets/fonts - adds satori and @resvg/resvg-js (both MPL-2.0) The dossier PDF worker and signed object storage URLs come in a follow-up PR. Refs VarshithReddy2006#352 Signed-off-by: Abhishek <abhishekcse2004@gmail.com>
…into feat/issue-352a-og-render-worker Signed-off-by: Abhishek <abhishekcse2004@gmail.com> # Conflicts: # DEPLOYMENT.md
Owner
|
@Anshuu2004 CI/CD checks are currently failing on This failure occurs because public profiles are now opt-in / private by default following the merge of PR #381. Please rebase/merge the latest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First half of #352 (worker-render pool). Adds the
renderworker role and the dynamic OG image pipeline:GET /api/v1/profile/public/:username/og.pngreturns a 1200x630 PNG drawn with satori + resvg, cached by content hash in a newRenderArtifactcollection. The API process never renders, it queues a job ondomain-og-renderand therenderworker does the drawing.Refs #352 (part 1 of 2). The dossier PDF worker and the signed object-storage URLs are the second PR, so this one does not close the issue.
What changed
renderworker role inworker-runtime.ts(also part ofall), consumingdomain-og-render. Addednpm run worker:render, aworker-renderservice indocker-compose.prod.yml, and docs inARCHITECTURE.md/DEPLOYMENT.md.RenderArtifactmodel: uniqueartifactKey,userId,type(og_image|pdf_dossier),contentHash,status,storageUrl,expiresAt(TTL index). OG cards are a few tens of KB so the PNG is stored inline in the row for now (content, not selected by default). One row per profile (og_image:<userId>), updated in place when the content changes.modules/profile/og/: card data, a satori element tree (plain objects, no JSX toolchain) and the satori -> resvg renderer. The card only contains what the public profile API would show: it uses the same section + privacy rule as the DTO (I moved that rule intovisibleSections()so both share it). Avatar is a monogram, not the remote avatar, because the worker should not fetch arbitrary URLs.renderOgArtifact()rebuilds the card from the DB, hashes it, and returns the stored artifact without calling the renderer when the hash matches an unexpiredreadyrow.ogImage.controller.ts): looks up the published profile, serves the cached PNG if the hash matches, otherwise queues one job (job id contains the hash so concurrent requests share it) and polls the artifact for up to 8s. No worker running ->503+Retry-After: 5. ETag / 304 is answered only after the visibility check.commonErrors.notFoundcall, so the body is identical.seo.ogImageUrlin the public profile API andog:imagein the/u/:usernameshell are set only for published profiles. The shell needs an absolute API origin, so there is a new optionalPUBLIC_API_URL(dev defaulthttp://localhost:<PORT>, unset in prod keeps the static preview image).satori^0.63.2 and@resvg/resvg-js^2.6.2 (both MPL-2.0). Lockfile diff is additive only (generated with--package-lock-only).@fontsource/inter5.3.0 package, licensed SIL Open Font License 1.1. Files and the license text are inbackend/assets/fonts/; the Dockerfile copiesassets/into the runtime image. Because it is a Latin subset, characters outside it are dropped from names (name falls back to the handle if nothing is left).How I tested
Run one file at a time from
backend/(the suite uses mongodb-memory-server and the mocked ioredis/bullmq fromvitest.setup.ts, no real Redis or Mongo):All of those pass.
og-image.test.tsuses the real satori + resvg: PNG magic bytes89504E47 0D0A1A0A, IHDR width/height 1200x630, cache hit verified with avi.fn()renderer spy (second render of the same content: renderer called once), re-render in place on changed content, expired artifact not trusted, uniqueartifactKey(E11000), unpublished -> 404 with the same body as an unknown handle and nothing queued, unpublish removes the stored card, 304 handling, 503 when no worker answers, andseo.ogImageUrl/og:imageonly for published profiles.worker-runtime.test.tsfails on main with the new role mapping cases (checked by running the updated test against the oldworker-runtime.ts), and passes with this change.tsc --noEmitshows only the 5 existing@devtrack/ats-coreerrors, nothing in these files.Not tested / limits
domain-og-renderworker wiring itself is covered by the role-mapping test only.Not in this PR (rest of #352)
dossierRender.worker.ts(Puppeteer PDF),dossier-renderqueue,objectStorage.tswith signed 15 minute URLs: next PR.docs/v2/DEPLOYMENT_AND_SECURITY_V2.mdanddocs/v2/MODULE_IMPLEMENTATION_PLAN.md, but there is nodocs/v2in the repo, so I put the notes inARCHITECTURE.mdandDEPLOYMENT.md.RenderArtifactmigration script: it is a brand new collection and the indexes are declared on the schema.