User-agent: * Allow: / # Render-critical endpoints Googlebot must fetch to render the public pages. # vc-shell.js loads these client-side; Google renders JS, so allowing them lets # the indexed pages show real localized + data content instead of an untranslated # empty shell. The rest of /api/ stays disallowed. Each is a longest-match Allow # that overrides "Disallow: /api/" for that path ONLY — deliberately NOT a # "/api/vc//" prefix, because user-data endpoints (e.g. team-insights, # find-team) live alongside these and must stay blocked. Allow: /api/vc/content # PL + IPL data feeds (SEO audit 2026-08-06): the live tool pages render their # boards from these endpoints on initial load — without them Googlebot indexed # every board as an empty shell (the exact failure the WC Allow lines fixed in # June, never ported to the leagues that are now the product). Initial-render # endpoints only: click-driven detail/modal endpoints (player-card, trend, per- # player stats) stay blocked, as do account/billing/admin. /api/sport5/players- # season-data is deliberately NOT allowed (owner decision D1 2026-08-06 — it # carries the full 9cat price/position bridge); /api/fdr/projections is allowed # only because its anonymous response is a sanitized public DTO (server.js # strips the model's fitted internals for non-admins). Allow: /api/pl/deadline Allow: /api/pl/fdr Allow: /api/pl/goals-cs Allow: /api/pl/matches Allow: /api/pl/bootstrap Allow: /api/pl/fantasy-providers Allow: /api/pl/season-stats Allow: /api/pl/team-news Allow: /api/pl/set-piece Allow: /api/pl/face Allow: /api/pl/xpts # Missed by that same 2026-08-06 pass and added 2026-09-06. Each one is the ONLY # initial-render feed of a launched, free PL board, so without its Allow line the # board indexed as an empty shell exactly like the ones above did: /pl/suspensions # reads /api/pl/suspensions, /pl/ownership reads /api/pl/team-stats, /pl/goalscorer # reads /api/pl/goalscorer (which also covers the hub's /api/pl/goalscorer/top by # longest-match prefix), and /pl/compare reads both /api/pl/compare and its /stats # sibling. /api/pl/player-stats is deliberately still absent: the page fetches it # per-player by element id from a click handler, so it is click-driven detail, not # initial render — the same line the block above draws. Allow: /api/pl/suspensions Allow: /api/pl/team-stats Allow: /api/pl/goalscorer Allow: /api/pl/compare Allow: /api/pl/compare/stats Allow: /api/fdr/data Allow: /api/fdr/projections Allow: /api/vc/ipl/fixture-projections Allow: /api/vc/ipl/matches Allow: /api/vc/ipl/set-piece-takers Allow: /api/team-news/overview Allow: /api/sport5/season-stats # ── IPL (Ligat ha'Al) xPts + ownership — the same gap, found the same way ──── # The Aug-06 pass allowed the IPL feeds the FDR/matches/set-piece boards render # from, but not the ones behind /ipl/xpts and the ownership columns, so the # league's single biggest board was indexable only as an empty shell. Added # 2026-09-06 alongside the UCL block below, after an audit found Googlebot could # not fetch what any of these pages draw themselves from. `/api/pl/xpts` has been # allowed since August and is the precedent: an xPts board is public, with the # paid depth trimmed server-side (a top-10 teaser for free viewers), so what a # crawler receives is exactly what an anonymous reader receives. Allow: /api/sport5/xpts Allow: /api/ipl/nine/xpts Allow: /api/sport5/top100/latest # Added 2026-10-13: the initial-render feed of /ipl/ownership (and the home page's # ownership strip) — the direct twin of the already-allowed # /api/sport5/ucl/team-stats/ownership. Missing, Search Console reported it # "Indexed, though blocked by robots.txt" (12-Oct) and the board rendered as an empty # shell. Anonymous responses are the server-trimmed teaser (`?full=1` → `locked`), # and the route sends `X-Robots-Tag: noindex`, so the feed itself stays out of the # index. No child routes (`/combo` is a sibling and stays blocked). Allow: /api/sport5/team-stats/ownership # ⚠ AN `Allow:` IS A PREFIX MATCH, so the xpts/top100 lines above would silently open # every child route beneath them too. These are the children, re-blocked by name # — a longer path wins on longest-match exactly the way the Allow beats the # blanket `Disallow: /api/` below. `render`/`refresh`/`bootstrap` are producer # and graphic-render endpoints; the top100 tail carries per-manager squads and # mini-league insights, i.e. user data, which is the one thing this file has # never advertised. Keep this list in step with the routes: if a new # /api/sport5/xpts/* or /api/sport5/top100/* endpoint is added, it is OPEN until # it appears here. Disallow: /api/sport5/xpts/render Disallow: /api/sport5/xpts/refresh Disallow: /api/sport5/xpts/bootstrap Disallow: /api/sport5/top100/history Disallow: /api/sport5/top100/guardrails-status Disallow: /api/sport5/top100/team-insights Disallow: /api/sport5/top100/mini-league-insights Disallow: /api/sport5/top100/refresh Disallow: /api/sport5/top100/render-managers # UCL data feeds (added 2026-09-06). MISSED BECAUSE UCL LAUNCHED AFTER THE FIX, NOT # BECAUSE ANYONE DECIDED AGAINST IT: the block above was written on 2026-08-06 for the # two competitions that were live then, and the Champions League tools opened later and # ONE AT A TIME behind per-tool launch gates (src/utils/ucl-launch-gates.json), so there # was never a single moment that read as "UCL is live, go update robots.txt". The result # was that every launched UCL board — the estate the international push is aimed at — # was crawlable only as an empty shell. # # Same rule as the PL/IPL block: INITIAL-RENDER endpoints only, one longest-match Allow # per path, never an /api/vc/ucl/ prefix (user-data endpoints live in that namespace). # Verified against the fetches in public/vc/ucl/*.html. Allow: /api/vc/ucl/team-news Allow: /api/vc/ucl/set-piece-takers Allow: /api/vc/ucl/matches Allow: /api/vc/ucl/matches-board Allow: /api/vc/ucl/goals-cs Allow: /api/vc/ucl/suspensions/status Allow: /api/fdr/ucl/data Allow: /api/fdr/ucl/projections Allow: /api/sport5/ucl/team-stats/ownership Allow: /api/sport5/ucl/top100/latest Allow: /api/uefa-ucl/players-board # # ── A LATER GATE WAVE: xpts · team-analyzer · compare (opened 2026-09-06) ───────── # The block above was written the same day but a few hours EARLIER, against the gate # file as it then stood. `ucl-xpts`, `ucl-team-analyzer` and `ucl-compare` flipped true # in src/utils/ucl-launch-gates.json later that day (owner: "let's open the planner, # xpts, player compare for ucl"), so their initial-render feeds were still sitting under # `Disallow: /api/` — three newly public boards, crawlable only as empty shells. # # ⭐ THIS IS THE FAILURE MODE THIS WHOLE BLOCK EXISTS TO DOCUMENT, REPEATING WITHIN ONE # DAY. UCL opens ONE TOOL AT A TIME on the owner's word (the wave schedule in the gate # file is retired), so there is never a "UCL is live" moment to hang this file off — # there is only the next gate flipping. ➤ WHENEVER A KEY IN ucl-launch-gates.json GOES # TRUE, COME HERE AND ADD ITS INITIAL-RENDER FEEDS. Still held today, so still owed a # visit when they open: `ucl-goalscorer` and `ucl-mini-leagues`. # # Same rule as everything above, verified against the fetches in the three pages: # · /ucl/xpts renders from BOTH games of the one board — /api/uefa-ucl/xpts (default) # and /api/sport5/ucl/xpts (the source toggle). Anonymous responses are real public # data: the free cut is the current round's top 10 (UCL_XPTS_TEASER_TOP_N = 10, # src/web/uefaUclRoutes.js, read by both games), the same shape as the already-allowed # /api/pl/xpts. Both games, one Allow each, for the same reason they share one tool # key — which game a reader is on must never change what is crawlable. # · /ucl/team-analyzer's only public initial-render fetches are that same xpts pair # (loadPlannerXpts) plus /api/ucl/faces, the headshot index and the direct twin of the # already-allowed /api/pl/face. Everything else it touches is a solver, a draft or an # account read — see the withhold list below. # · /ucl/compare reads its pool and its /stats sibling, one route per game: # /api/uefa-ucl/compare(+/stats) and /api/vc/ucl/compare(+/stats). Listed in full # rather than left to the prefix, mirroring how /api/pl/compare + /stats are written. Allow: /api/uefa-ucl/xpts Allow: /api/sport5/ucl/xpts Allow: /api/ucl/faces Allow: /api/uefa-ucl/compare Allow: /api/uefa-ucl/compare/stats Allow: /api/vc/ucl/compare Allow: /api/vc/ucl/compare/stats # # ⚠ AND THE FOUR SUB-PATHS THAT Allow WOULD OTHERWISE HAVE OPENED. An Allow is a PREFIX # match, so `Allow: /api/sport5/ucl/xpts` reaches its children too — and that endpoint, # unlike /api/uefa-ucl/xpts, has four: the graphics renderer, the poll helper, and two # admin POSTs. Every one of them is on the withhold list below, so they are re-blocked # by name; a longer rule wins over a shorter one, so these beat the Allow above exactly # as that Allow beats `Disallow: /api/`. (Nothing else added here has children: # /api/uefa-ucl/xpts, /api/ucl/faces and the two compare pools have none beyond the # /stats siblings, which are wanted.) Disallow: /api/sport5/ucl/xpts/render Disallow: /api/sport5/ucl/xpts/ready Disallow: /api/sport5/ucl/xpts/refresh Disallow: /api/sport5/ucl/xpts/bootstrap # # STILL BLOCKED IN THE SAME NAMESPACE, and each for a reason the list above already # uses — do not "complete the set" by adding them: # • /api/vc/ucl/player-card, /api/vc/ucl/team-insights, /api/vc/ucl/mini-league, # /api/vc/ucl/global-league, /api/vc/ucl/live — click-driven detail or user data. # • every */render and */ready path (/api/sport5/ucl/lineup/render, # .../xpts/ready, .../set-piece/render …) — graphics + poll helpers, no indexable # content, and the render routes are admin-side. # • /api/sport5/ucl/find-team — a lookup over other people's teams. # • /api/sport5/ucl/players-season-data — the DIRECT TWIN of the IPL endpoint owner # decision D1 (2026-08-06) already withholds: it carries the full price/position # bridge for the whole player pool. The reasoning transfers unchanged, so the # answer does too. The /ucl/players board renders from /api/uefa-ucl/players-board # instead, which is allowed above. # • anything under /api/admin/ or /api/account/. # The nine /api/vc/wc/* + /api/fdr/wc/data Allow lines were removed 2026-08: the # World Cup is hidden (the pages they render now serve a coming-soon stub), so # inviting Googlebot to fetch its feeds pointed a crawler straight at the # competition we are hiding. They fall back under "Disallow: /api/" below. The # endpoints themselves are untouched — paste the lines back when WC returns. Disallow: /admin/ Disallow: /api/ Sitemap: https://vice-captain.com/sitemap.xml