Sibling hosts

Covers crawl.sibling-hosts.

Most indexation rules are about the site you are building. This one is about the other copies of it that nobody mentioned, and it is the only rule in this volume that came out of a real audit of a real Colibrity site rather than out of a guideline.

What happened

A third-party technical SEO audit of a live Colibrity client, July 2026, found four hostnames serving full, unprotected copies of the production site: an admin host, an ETL host, a dev host, and the agency's own project subdomain. All four returned 200. All four were indexable.

Nobody had done anything wrong on the site itself. Its canonicals were correct, its robots.txt was correct, its sitemap was correct. The defect was entirely outside the repository anyone was working in, which is exactly why it survived to an external audit.

Why it costs an index rather than a score

A second indexable copy of a site is not a small deduction. Google has to pick which copy is canonical, and it does not have to pick yours. When it picks the dev host, the production URLs stop being the ones that rank, internal links start pointing at the wrong host from search results, and the fix is slow because the wrong host has to be removed and then reprocessed.

It also leaks. A staging copy usually holds unpublished pages, test content, real customer data entered during UAT, and sometimes an admin interface that was never meant to face the internet.

Enumerate, do not assume

The list in the brief is not the list. Ask for it, and then look for yourself:

# Certificate transparency logs list hostnames somebody asked for a certificate for.
curl -s "https://crt.sh/?q=%25.example.com&output=json" | python3 -c \
  'import json,sys; print("\n".join(sorted({n for r in json.load(sys.stdin) for n in r["name_value"].split(chr(10))})))'

Then check each one:

for h in admin dev staging etl preview; do
  printf '%s: ' "$h"
  curl -sI "https://$h.example.com/" | grep -iE '^HTTP|^x-robots-tag' | tr '\n' ' '
  echo
done

Anything answering 200 with no X-Robots-Tag: noindex is a finding.

What counts as fixed

In order of preference:

1. The host does not exist. Best outcome, and often available - a dev host nobody uses is a dev host that can be turned off.
2. It refuses: HTTP Basic auth, or an IP allow list. This is what Colibrity's own checklist asks for, in its words: dev, staging and admin blocked by Basic Auth or IP restriction.
3. It answers X-Robots-Tag: noindex on every response, driven by environment.

A robots.txt with Disallow: / is not a fix. It blocks the fetch, not the indexing, so a linked URL on that host can still be indexed with no snippet. The reasoning is the same as on the front-end volume's indexability page.

Handover

The enumerated list ships with the project, dated. It is the only record that the question was asked, and the next person to add a subdomain needs somewhere to add it.