beacarbol/arrow

Shortszilla backend

Deploy free on Hugging Face Spaces

1. 1.Create a free account at https://huggingface.co
2. 2.Click New → Space. Name it, pick Docker as the SDK, visibility Public.
3. 3.Upload these three files to the Space (Files tab → Add file → Upload files): Dockerfile, app.py, requirements.txt
4. 4.The Space builds automatically (2-5 minutes the first time). Once running, your backend URL is: https://<your-username>-<space-name>.hf.space
5. 5.Paste that URL into the frontend's FREE_POOL array (in index.html). Done.

Free CPU Spaces sleep after inactivity; the first request after sleeping takes a little longer while it wakes back up. Free accounts can only run roughly 8 CPU-basic Spaces at the same time — you can create more, but only that many can be active simultaneously.

Deploy free on Google Cloud Run (auto-scales — good for traffic spikes)

Unlike a fixed pool of Hugging Face Spaces, Cloud Run automatically spins up more instances when many people use the site at once, and scales back down (to zero) when it's quiet. Same Dockerfile, no code changes needed.

1. 1.Create a free Google Cloud account (no charge within the free tier: 2M requests + 360k vCPU-seconds + 180k GiB-seconds per month, as of mid-2026).
2. 2.Install the gcloud CLI, then from this backend/ folder run:

   gcloud run deploy shortszilla-backend --source . --region us-central1 --allow-unauthenticated --memory 2Gi --cpu 2 --timeout 300

1. 1.Gcloud builds and deploys automatically and gives you a URL like https://shortszilla-backend-xxxxx.run.app.
2. 2.Add that URL to FREE_POOL (or PAID_POOL) in the frontend's index.html, same as any Hugging Face Space URL.

You can mix hosts freely within either pool — e.g. 2-3 Hugging Face Spaces for baseline capacity, plus one Cloud Run URL that absorbs bursts automatically.

What's new in this version

• —Feature suggestion box, emailed straight to you: the homepage now has a "Suggest a Feature" box. Submissions hit /api/feedback/submit on the coordinator backend, which emails them to FEEDBACK_TO_EMAIL (defaults to arslanshani152@gmail.com) via SMTP. Every submission is also appended to a local backup file (_feedback.jsonl) even if email isn't configured yet, so nothing is ever lost. To turn on emailing, set these as secrets on the coordinator Space (Space Settings → Variables and secrets):

  SMTP_HOST = smtp.gmail.com
  SMTP_PORT = 587
  SMTP_USER = your-sending-address@gmail.com
  SMTP_PASS = an app password (not your normal Gmail password)
  FEEDBACK_TO_EMAIL = arslanshani152@gmail.com

For Gmail specifically: turn on 2-Step Verification, then generate an App Password at Google Account → Security → App passwords, and use that as SMTP_PASS. Any other SMTP provider (Outlook, a custom domain's mail server, SendGrid's SMTP relay, etc.) works the same way — just change SMTP_HOST/SMTP_PORT accordingly.

• —Free/paid pool split + real queue: FREE_POOL and PAID_POOL are now separate arrays in the frontend. Free-tier requests join a real cross-pool wait queue (/api/queue/join + /api/queue/status) and show a live position instead of guessing; a valid credit key switches rendering to PAID_POOL and skips the queue entirely.
• —Credit infrastructure (no payment processor wired up yet): credits are tracked server-side and persist to disk, but are issued manually for now — there's no Stripe/checkout flow. To issue credits to someone:

  curl -X POST https://your-coordinator-space.hf.space/api/credits/issue \
    -H "Content-Type: application/json" \
    -H "Origin: https://www.shortszilla.com" \
    -d '{"key": "someones-credit-code", "amount": 20, "adminToken": "YOUR_ADMIN_TOKEN"}'

(The Origin header is needed because every endpoint now checks it - see "Can the backend URLs be hidden" further down - /api/credits/issue isn't exempted from that check since it's also separately gated by SHORTSZILLA_ADMIN_TOKEN below, so this curl command needs both.) Set SHORTSZILLA_ADMIN_TOKEN as a secret on whichever Space you're using as the coordinator (Space Settings → Variables and secrets) — without it, /api/credits/issue always rejects requests. The coordinator is whichever backend is FREE_POOL[0] in the frontend.

• —Portability note: adding a new backend on any host (Cloud Run, your own server, anywhere) is just adding its URL to FREE_POOL or PAID_POOL in index.html — existing Hugging Face backends keep working unchanged alongside it. There's nothing HF-specific to migrate away from.

What's new in this version (continued)

• —Real progress bar: the frontend tracks actual completed clip renders (not a fake timer), so the percentage shown genuinely reflects progress.
• —New endpoints power the pool: /api/render_clip (renders exactly one clip — download/trim/overlay/title/filler) and /api/assemble (stitches already-rendered clips into the final video: ordering, hook/outro, progress bar/zoom). /api/generate still exists and works standalone.
• —2-3x faster: clips concat via stream-copy instead of re-encoding the whole merged video, and every ffmpeg pass uses preset ultrafast.
• —Two-line title with highlighted keywords — set Line 1 / Line 2 and pick which words get colored, positioned with real font-metrics (not guessed).
• —Per-clip controls: number position (4 corners), number size, a fade/slide entrance animation, a synthesized sound effect (whoosh/ding, no external files needed), and a real AI voiceover (via free gTTS — no API key) mixed under the clip's original audio.
• —Optional crossfade transitions between clips (off by default, since it disables the fast stream-copy path and re-encodes instead).

What changed vs the buggy version

• —import asyncio was missing — every /api/generate call crashed with NameError: name 'asyncio' is not defined.
• —/api/generate assumed every clip was already downloaded to disk by /api/prepare. If the "Let me trim clips first" toggle was off on the frontend, no download ever happened and ffmpeg failed with No such file or directory. The backend now downloads a clip itself inside /api/generate if it isn't already cached.
• —Clips were trimmed with -c copy, which only works if the cut points land on a keyframe — this silently failed a lot. Clips are now re-encoded on trim, which is reliable; the merge step then uses fast stream-copy since every clip now shares an identical format.
• —CORS is now enabled for all origins, so the frontend (hosted on a different domain, e.g. Cloudflare Pages) can actually talk to this backend.

Latest round of fixes (filler bug, dead position/size code, downloads)

• —Filler clip/voice silently disappearing: found and fixed. When a render retried on a different backend than the one a filler file was uploaded to, the old fillerUploadIndex pointed at a file that only existed on the original backend - it silently resolved to nothing (no error, just a missing filler). The frontend now keeps the actual filler/voice File objects in memory and re-uploads them fresh to whichever backend a retry actually lands on.
• —"Number position" and "Number size" did nothing: the real overlay code (_cumulative_list_filters) never read numberPosition/numberSize at all - it always drew at a fixed top-left spot with a fixed size, even though both fields were faithfully sent all the way from the frontend. This is now fixed; position (top-left / center-left / bottom-center), number size, and keyword size are all genuinely honored, with bounds checks so the overlay can't run off the edge of the frame.
• —New global "spacing between ranks" setting (listGap in /api/generate and /api/assemble): adds even vertical spacing between every #1/#2/#3... row, no matter how many clips end up in the video.
• —Outro/hook text now wraps onto multiple centered lines instead of running off-screen as one long line, using real PIL text measurement (not a guessed character count) to decide where to break.
• —Download reliability (TikTok/Facebook/Instagram): two changes here.
• —requirements.txt now floors yt-dlp at a much newer version. TikTok/ Facebook/Instagram change their sites often, and an old yt-dlp is the single most common cause of downloads getting slower or breaking over time for exactly these three. Important: because Docker layer-caches the pip install step, just re-uploading app.py does NOT pick up a newer yt-dlp - only a change to requirements.txt (or an explicit "Factory rebuild" on Hugging Face Spaces / a fresh build with no cache elsewhere) actually reinstalls it. If downloads degrade again in the future, bumping the yt-dlp version pin and redeploying is the first thing to try.
• —Facebook/Instagram get their own tuned retry logic now: a longer per-request timeout (their webpage responses are genuinely slower than TikTok's/YouTube's, so the same tight budget that's fine for those two wasn't enough here), a fallback to the lighter m.facebook.com mobile endpoint with a mobile user-agent on the last attempt (instead of blindly repeating the exact same request 3x), while TikTok/YouTube keep their original fast settings completely unchanged. That said: Facebook and Instagram both actively rate-limit and gate non-browser requests - a fraction of private/restricted posts on those two platforms will still fail without a logged-in session, which is a platform-side restriction no downloader can fully bypass. If you consistently need those to work, the standard fix is exporting your own logged-in cookies (e.g. with a browser extension like "Get cookies.txt") and wiring a --cookies flag into _download() - ask if you want this added; it wasn't included here since it requires you to supply your own account's cookies file, which shouldn't be added without you explicitly wanting that tradeoff.

Free ratings/credits persistence using GitHub (no paid storage needed)

Ratings and credits used to live in /tmp, which every backend host wipes on restart (Space sleeping/waking, a redeploy, a crash-restart) - that's why ratings kept resetting to zero. Hugging Face's own "Persistent Storage" fixes this but costs a small monthly fee. This adds a free alternative: storing those two small JSON files in a GitHub repo instead, using GitHub's Contents API (GitHub itself never gets wiped, and this needs no paid add-on).

This is entirely optional - if you don't set these up, everything just keeps using local disk as before (resets on restart unless you do have HF persistent storage enabled).

Setup (~2 minutes):

1. 1.Create a new private GitHub repo just for this data, e.g. shortszilla-data. Tick "Add a README file" when creating it - the repo needs at least one commit for the Contents API to work cleanly against it.
2. 2.GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens → Generate new token. Under "Repository access" pick "Only select repositories" and choose the repo from step 1. Under "Permissions", set Contents: Read and write. Generate, and copy the token (starts with github_pat_...) - GitHub only shows it once.
3. 3.Set these as secrets on every backend Space (Space Settings → Variables and secrets):

   GITHUB_TOKEN  = github_pat_...                (the token from step 2)
   GITHUB_REPO   = yourusername/shortszilla-data  (from step 1)
   GITHUB_BRANCH = main                            (optional, this is the default)

1. 1.That's it - /api/rating/submit, /api/rating/summary, and the credits endpoints now read/write data/ratings.json and data/credits.json in that repo automatically, on every backend, all pointing at the same shared file.

Tradeoffs worth knowing:

• —This is meant for low-frequency writes (star ratings, manually-issued credits) - not a real database. If two people rate at the exact same instant on two different backends, one write can occasionally overwrite the other (GitHub's API rejects a conflicting write once, and this retries by re-reading and reapplying once, but doesn't fully solve simultaneous writes). For this feature's actual traffic that's a fine tradeoff; it would not be for anything high-frequency.
• —Every rating submission shows up as a commit in that repo's history - this is expected and harmless, just don't be surprised seeing dozens of small automated commits over time.

Can the backend URLs be hidden from the page's code?

Short honest answer: not fully, not with a static frontend calling these Spaces directly from the browser. Anyone who opens the browser's dev tools Network tab while using the site sees exactly which URL each request goes to - that's just how browsers work, and no amount of minifying/obfuscating the frontend JS changes it, since the browser itself has to know the real destination to send the request there at all.

What this update DOES add: every backend now only accepts requests whose Origin/Referer header matches shortszilla.com (or the mobile app) - see ALLOWED_ORIGINS near the top of app.py. This doesn't hide the URLs, but it does stop the much more common form of abuse: someone grabbing a Space URL from the Network tab and pasting it into their own curl command, script, or website to use your backend for free. Browsers send an honest Origin header that JS on the page can't forge, so this blocks that casual case - it doesn't stop someone determined enough to fake the header themselves in a non-browser client, but that's a much smaller group than "anyone who opened dev tools."

The actual fix, if you want the raw Space URLs never visible at all: put a reverse proxy in front of them - e.g. a Cloudflare Worker at api.shortszilla.com that holds the real Space URLs as secrets and forwards requests to whichever backend is chosen. Then the browser only ever talks to api.shortszilla.com; the real Space URLs live only in the Worker's config, never in anything the browser downloads. This is a bigger change (a new Cloudflare Worker to write and deploy, plus updating the frontend to call the proxy instead of the Spaces directly) - happy to build this if you want it, just say so.

Will Hugging Face throttle or ban the Spaces for high traffic?

Being straight with you here since I couldn't find a page that gives a precise number: Hugging Face doesn't publish a specific "X requests/day and you're banned" rate limit for Spaces. But their Content Policy does list things like "excessive... activity" and using a Space as "advertising products/services" alongside other examples of prohibited abuse, and their free CPU Spaces are officially positioned as a place for demos/prototypes, not as a production backend for a commercial-scale service - there have also been community reports of Spaces being flagged specifically for being "a demo of a commercial product" used to get free hosting. So: there's no guaranteed ban, but there is a real, honest risk that if traffic grows significantly, a Space could get flagged, restricted, or asked to move - this isn't unique to Hugging Face, most free tiers everywhere have a similar unwritten fair-use line. If/when real usage grows, migrating to a proper paid host (the Cloud Run option earlier in this README, a small VPS, or HF's own paid persistent Spaces with dedicated hardware) is the sustainable move rather than staying on the free pool indefinitely at production scale.

Fixes made after putting the Cloudflare Worker proxy in front of this

• —Queue could get stuck forever at "waiting...": /api/queue/status asks every backend in the pool how busy it is (/api/status) to decide how many people to let through next. If literally every one of those checks fails (a proxy hiccup, a backend waking from sleep, etc.), the old code read that as "zero capacity everywhere" and never let anyone forward - a health-check failure isn't proof every backend is genuinely full. Now, if every check fails, it assumes a reasonable default (one slot per pool URL) instead of freezing the queue indefinitely.
• —Frontend request timeouts were shorter than this backend's own worst-case processing time - unrelated to the proxy, this was already true before it, but became more visible once other retry logic started working correctly. /api/render_clip and /api/prepare can legitimately take well over 25 seconds (Facebook/Instagram downloads alone can take up to ~90s across retries, per the tuning described above; add real ffmpeg work on top of that for render_clip). The frontend's timeouts for these calls were increased substantially (60-150 seconds depending on the endpoint) so a slow-but-working request isn't wrongly aborted client-side before the backend even finishes.

Security fix: SSRF protection on server-to-server fetches

/api/ingest, /api/encode_segment, and the distributed-encoding feature (poolUrls/selfUrl on /api/assemble) all have this backend fetch a URL supplied by whoever calls the API. The original check only required the URL to contain /files/ and start with http(s):// - that doesn't actually stop someone from calling these endpoints directly (bypassing the frontend entirely) with a URL like http://169.254.169.254/files/../latest/meta-data/ and getting this server to fetch an internal/cloud-metadata address on their behalf (a classic SSRF attack).

Now _is_safe_fetch_url/_is_safe_backend_url actually resolve the hostname and reject anything that maps to a private, loopback, link-local, reserved, or multicast address - covering the realistic internal-network and cloud-metadata attack surface with zero configuration required. For extra strictness, set ALLOWED_FETCH_HOSTS (comma-separated hostnames, e.g. shortszilla-proxy.yourname.workers.dev) as a Space secret to additionally require the target match a specific known list - otherwise this defaults to the private-IP-block behavior only, which is safe on its own but doesn't restrict which public hosts can be reached.