Before you start
- API access on at least one provider: DataForSEO, Ahrefs API, SEMrush API, or Serpstat API
- A backend you can run cron / scheduled jobs on (Supabase edge functions, Cloudflare Workers, Vercel cron, Node server)
- A storage layer for time-series rank data (Postgres, BigQuery, or even a Google Sheet for sub-1K keywords)
- Working knowledge of REST and either Node, Python, or Deno
The playbook
8 steps
Pick the right provider for your use case
Three real options. **DataForSEO** is the most flexible and cheapest per query - pay-as-you-go from $0.0006-0.005 per SERP. **Ahrefs API** is the deepest dataset but only available on Enterprise plans starting ~$1,500/mo. **SEMrush API** is comparable to Ahrefs, also Enterprise-tier. **Serpstat** offers an API on standard plans, mid-tier price/quality. For most teams building rank tracking from scratch, DataForSEO is the right starting point.
Expected outcome
A provider chosen with pricing model that matches your query volume.
Authenticate and test a single keyword query
Every provider authenticates via API key in the header. Start with one keyword + one location to validate auth + parsing before scaling. The DataForSEO Live SERP endpoint returns a full JSON SERP including organic, paid, and SERP features - test that you can parse the position you actually want.
// DataForSEO Live SERP - Node example const res = await fetch('https://api.dataforseo.com/v3/serp/google/organic/live/advanced', { method: 'POST', headers: { 'Authorization': 'Basic ' + Buffer.from(`${LOGIN}:${PASSWORD}`).toString('base64'), 'Content-Type': 'application/json' }, body: JSON.stringify([{ keyword: 'keyword ranking api', location_code: 2840, // US language_code: 'en', device: 'desktop', depth: 100 }]) }); const data = await res.json(); const myPosition = data.tasks[0].result[0].items .find(i => i.type === 'organic' && i.domain === 'shuttergen.com')?.rank_absolute;Expected outcome
Single keyword + single domain query returning a parseable rank position.
Design your storage schema for time-series rank data
Rank tracking is time-series by nature. Minimum schema: keyword, domain, location, device, position, serp_features (jsonb array), captured_at. Index on (keyword, domain, captured_at desc) for fast 'latest rank' queries. Avoid storing one row per request - store one row per (keyword, domain, day) and dedupe.
-- Postgres schema create table keyword_rankings ( id bigserial primary key, keyword text not null, domain text not null, location_code int not null, device text not null check (device in ('desktop','mobile')), position int, -- null if not in top 100 url text, serp_features jsonb default '[]'::jsonb, captured_at timestamptz not null default now() ); create unique index on keyword_rankings (keyword, domain, location_code, device, date(captured_at)); create index on keyword_rankings (keyword, domain, captured_at desc);Expected outcome
Time-series schema ready to absorb daily rank checkpoints with idempotent inserts.
Batch requests to control cost and rate limits
Per-keyword queries get expensive fast. DataForSEO accepts up to 100 tasks per request; batch your keywords accordingly. For weekly tracking of 500 keywords, that's 5 batch requests, costing roughly $1-2.50 per refresh depending on plan. Rate limits: DataForSEO allows 2,000 requests/minute on standard accounts - don't hit it, queue with concurrency 5-10.
Expected outcome
Batched, rate-limit-aware request layer that scales to thousands of keywords without account bans.
Schedule the refresh job
Pick a refresh cadence per keyword tier. Top 20 priority keywords: daily. Next 100: weekly. Long tail: monthly. Use Vercel Cron, Supabase pg_cron, or Cloudflare Workers Cron Triggers - they all support standard cron syntax. Stagger the runs across hours so you don't burn your daily quota in 5 minutes.
// vercel.json - daily 6am UTC for priority keywords { "crons": [ { "path": "/api/cron/rank-priority", "schedule": "0 6 * * *" }, { "path": "/api/cron/rank-weekly", "schedule": "0 7 * * 1" } ] }Expected outcome
Automated refresh job hitting the API on a tier-appropriate cadence.
Parse SERP features alongside positions
Position #1 organic with no SERP features = lots of clicks. Position #1 organic with featured snippet + AI Overview = far fewer clicks. The API returns the full SERP item array - extract featured_snippet, people_also_ask, ai_overview, video, image_pack items into a serp_features jsonb column so you can correlate traffic shifts with SERP changes.
TipAI Overview presence is the highest-impact SERP feature in 2026. Track it explicitly - it's the single best predictor of click-through dilution.Expected outcome
SERP features stored per keyword per checkpoint, enabling correlation with traffic data.
Build alerts on meaningful deltas
Raw rank data is noise without alerting. Define meaningful deltas: position drop of 3+ on a priority keyword, position climb of 5+ (also worth knowing), SERP feature added/removed, competitor entering top 3 on your keyword. Fire to Slack via webhook or email via Resend. Quiet alerts mean missed problems; over-alerting trains the team to ignore them - tune the thresholds.
// Pseudocode alert rule if (previousPosition && currentPosition - previousPosition >= 3 && isPriorityKeyword) { await slack.send(`#seo-alerts`, `${keyword} dropped from #${previousPosition} to #${currentPosition}`); }Expected outcome
Slack/email alerts that surface meaningful changes without alert fatigue.
Cross-validate with Search Console
API-fetched rank is a SERP snapshot. Search Console reports your domain's actual impression-weighted position averaged over real queries. They diverge. Use the API for breadth (competitor positions, SERP features, daily volatility) and Search Console for ground truth on your own domain's performance. Pull Search Console daily via its API too and store alongside.
Expected outcome
Hybrid pipeline: API for breadth/competitor data, GSC for ground-truth own-domain data.
Shuttergen
Rank data drives content. Shuttergen drives the ads.
Once your rank tracking pipeline catches a competitor surge, the next move is matching them in paid. Shuttergen generates creative tuned to your rank-tracked keyword themes.
Pitfalls
What goes wrong
Hitting per-query pricing without batching
Naive one-keyword-one-request implementations cost 10x what batched implementations cost. Always batch to the provider's max (100 tasks for DataForSEO) and accept the slightly more complex parsing.
Ignoring location and device parameters
Rank for 'best CRM' in US-desktop is wildly different from UK-mobile. Always specify location_code and device explicitly. The default isn't your default; defaults are usually US-desktop.
Storing one row per request instead of deduping
Cron jobs retry, dev environments accidentally hit prod tables, edge cases happen. Use a unique index on (keyword, domain, location, device, date) so accidental double-pulls don't bloat your time-series.
Skipping SERP feature tracking
Position alone is incomplete. A keyword can lose 50% of its clickable traffic without your position changing because Google added an AI Overview. Track SERP features explicitly.
Building it but never reviewing it
The API pipeline produces data; humans turn data into decisions. Schedule a weekly 15-minute review of the dashboard or your pipeline becomes a write-only system that informs nothing.
Limits
When this playbook won't work
- Your keyword set is under 50 and refreshing weekly - just use Ahrefs/SEMrush UI; the API overhead isn't worth it at that scale
- You need ranking data for queries with personalization (logged-in Google, location-specific local pack) - API SERPs are clean but unlike real user SERPs
- Your dev capacity is zero - a managed rank tracker is cheaper than the engineering time to build and maintain a pipeline
- You're tracking enterprise-scale (100K+ keywords) - at that volume you need a dedicated SERP platform, not a DIY pipeline
Why teams build a rank tracking pipeline instead of buying one
The trade is control vs convenience. Off-the-shelf rank trackers (Ahrefs, SEMrush, AccuRanker) handle storage, alerting, and visualization for you - at $130-500/mo per project. Building on a SERP API gives you raw data you can join to your own data warehouse, blend with Search Console and analytics, and visualize in your own BI tool.
The break-even is usually around 1,000-5,000 keywords. Below that, managed tools are cheaper. Above that, DIY on DataForSEO ($0.001 per SERP) gets dramatically cheaper - $5-15/day for thousands of daily SERP pulls.
Build the pipeline if you have an existing data warehouse. If your analytics already lives in BigQuery/Snowflake/Postgres and you have engineers maintaining ETL, adding a SERP feed is incremental work. If you don't, the operational cost of running cron jobs and storage will eat the savings.
Rank data drives content. Shuttergen drives the ads. Once your rank tracking pipeline catches a competitor surge, the next move is matching them in paid. Shuttergen generates creative tuned to your rank-tracked keyword themes.
When to use which API
DataForSEO is the right choice for most builders. Pay-as-you-go pricing, generous rate limits, full SERP item parsing, supports all major search engines and locations. The Live SERP endpoint is the workhorse; Live Advanced returns full SERP features.
Ahrefs API ships the Ahrefs dataset - their keyword difficulty, traffic estimates, content-gap data - via API. Only on Enterprise plans. Pick this if you already pay for Ahrefs Enterprise and want to feed their data into your stack.
SEMrush API is the SEMrush equivalent. Same trade-off - their data, their pricing tier. Comparable depth to Ahrefs.
Serpstat is a middle option - API available on Standard ($69/mo) and up. Less polished than DataForSEO but bundles a UI you can use alongside.
SerpApi / Bright Data SERP API are alternatives to DataForSEO with similar pay-as-you-go pricing; comparable quality. Worth checking if you want a second source.
Internal: keyword-monitoring, keyword-analysis-in-seo.
FAQ
Frequently asked
What's a keyword ranking API?
Which keyword ranking API is best?
How much does a keyword ranking API cost?
Can I use a rank tracking API instead of buying Ahrefs?
How often should I refresh rank data?
How do I track SERP features via API?
Do I need a SERP API if I have Search Console?
Related
Keep reading
Resource
Keyword monitoring
UI-based rank tracking equivalent.
Resource
Keyword analysis in seo
Picking keywords before tracking them.
Resource
Analytics keywords report
Reporting layer on rank data.
Resource
Ad library api
Sister API playbook for ad data.
Research
Anatomy Of Good Meta Ad Library
Adjacent competitive data architecture.
Rank data drives content. Shuttergen drives the ads.
Once your rank tracking pipeline catches a competitor surge, the next move is matching them in paid. Shuttergen generates creative tuned to your rank-tracked keyword themes.