seo-domain-analyzer

SEO Domain Analyzer

Pull real SEO performance data for any domain — no Semrush or Ahrefs subscription needed. Uses Apify actors that scrape Semrush/Ahrefs public pages to get authority scores, traffic estimates, keyword rankings, backlink profiles, and competitor discovery.

Quick Start

# Basic domain analysis
python3 skills/seo-domain-analyzer/scripts/analyze_domain.py \
  --domain "example.com"
 
# With competitor comparison
python3 skills/seo-domain-analyzer/scripts/analyze_domain.py

Inputs

Cost

Process

Phase 1: Domain Overview (Semrush Data)

Use Apify actor devnaz/semrush-scraper to get:

Extracted metrics:

• Authority Score (0-100)
• Organic monthly traffic estimate
• Organic keywords count (how many keywords the domain ranks for)
• Paid traffic estimate (if any)
• Backlinks count (Semrush's count)
• Referring domains count
• Top organic keywords (keyword, position, traffic share)
• Top competitors (competing domains by keyword overlap)
• Traffic trend (month-over-month direction)

Phase 2: Backlink Profile (Ahrefs Data)

Use Apify actor radeance/ahrefs-scraper to get:

Extracted metrics:

• Domain Rating (DR) (0-100)
• URL Rating of homepage
• Referring domains count and trend
• Backlinks total count
• Top referring domains (which sites link to them)
• Anchor text distribution (branded vs keyword vs generic)
• Dofollow vs nofollow ratio

Phase 3: Keyword Rank Verification

For specific keywords (user-provided or auto-inferred from Phase 1), verify actual rankings using Google search:

For each keyword:

• Does the target domain appear in top 10?
• What position?
• What specific URL ranks?
• Who else ranks? (competitive landscape for that keyword)

Keyword sources (in priority order):

1. User-provided keywords
2. Top organic keywords from Semrush data (Phase 1)
3. Auto-inferred from domain content (WebSearch site:[domain] to see page titles)

Phase 4: Top Pages Analysis

From the Semrush data, extract the highest-traffic pages:

• URL
• Estimated monthly traffic
• Primary keyword(s) driving traffic
• Number of ranking keywords

If Semrush doesn't provide per-page data, supplement with:

• WebSearch: site:[domain] and note which pages appear first (proxy for importance)
• WebSearch: site:[domain] blog for top blog content

Phase 5: Competitor Discovery

Competitors are identified from multiple sources:

1. Semrush competitor data (Phase 1) — domains competing for same keywords
2. User-provided competitors — always included
3. Google SERP competitors — from Phase 3 keyword checks, note which domains consistently appear

For each competitor, run a lighter version of Phase 1 (domain overview only):

• Authority score
• Organic traffic estimate
• Keyword count
• Top keywords

Phase 6: Output

JSON Output

Markdown Summary (also generated)

Tips

• Semrush scraper data quality varies. The Apify actors scrape public Semrush pages, which show limited data for non-subscribers. Traffic estimates and top keywords are available, but detailed per-page breakdowns may be partial.
• Combine with site-content-catalog to get both the content inventory AND performance data — together they tell you what content exists AND which pieces actually drive traffic.
• Keyword rank verification via Google is the most reliable data point. Semrush/Ahrefs estimates can be off, but checking actual SERPs gives ground truth.
• Run competitors lighter. Full backlink analysis on 5 competitors gets expensive. Domain overview (Semrush only) is usually sufficient for comparison.
• Apify actors may break. These scrape Semrush/Ahrefs public pages, which can change. If an actor fails, fall back to the free seo-traffic-analyzer skill which uses web search probes.

Fallback: Free Mode

If APIFY_API_TOKEN is not set or Apify actors fail, the script falls back to:

1. WebSearch probes (like seo-traffic-analyzer skill)
2. site:[domain] for indexed page count
3. SimilarWeb free tier for traffic estimates
4. Manual Google SERP checks for keyword rankings

This gives less precise data but still produces a useful report.

Dependencies

• Python 3.8+
• requests library
• APIFY_API_TOKEN env var (for Apify mode; falls back to free probes without it)