npx skills add ...
npx skills add datadog-labs/agent-skills --skill dd-audit-key-compromise
npx skills add datadog-labs/agent-skills --skill dd-audit-key-compromise
Investigate a potentially compromised Datadog API key — timeline of actions, geo/IP breakdown, endpoints called, anomaly flags, and remediation steps.
Reconstruct what a Datadog API key did, where requests originated, and which resources were affected.
You need the key ID of the suspect key (not the key value). Find it in Datadog UI under Organization Settings > API Keys, or from context showing @metadata.api_key.id.
| Signal | Why it matters |
|---|---|
| Country not in org's normal baseline | Possible exfiltration from unexpected region |
| ASN is a cloud/VPN provider (AWS, Cloudflare, NordVPN, etc.) | Proxied traffic; obscured origin |
| DELETE actions on monitors, dashboards, or log pipelines | Potential sabotage |
| Burst of activity in short window | Automated scraping or bulk exfiltration |
| Activity outside business hours | Off-hours access |
| Key used from multiple IPs simultaneously | Key shared or stolen |
Revoke in Datadog UI: Organization Settings > API Keys > Revoke.
Or via API (requires manage_api_keys scope):
pup audit-logs search --query "@metadata.api_key.id:KEY_ID" --from 90d --limit 500 -o json \
| jq '[.data[] | {
country: .attributes.attributes.network.client.geoip.country.name,
asn: .attributes.attributes.network.client.geoip.as.name,
ip: .attributes.attributes.network.client.ip
}]
| group_by(.country)
| map({
country: .[0].country,
count: length,
asns: [.[].asn] | unique,
ips: [.[].ip] | unique
})
| sort_by(-.count)'pup audit-logs search --query "@metadata.api_key.id:KEY_ID" --from 90d --limit 500 -o json \
| jq '[.data[] | {
method: .attributes.attributes.http.method,
path: .attributes.attributes.http.url_details.path
}]
| group_by(.path)
| map({path: .[0].path, methods: [.[].method] | unique, count: length})
| sort_by(-.count)'pup audit-logs search --query "@metadata.api_key.id:KEY_ID @action:deleted" --from 90d -o json \
| jq '[.data[] | {
timestamp: .attributes.timestamp,
resource_type: .attributes.attributes.asset.type,
resource_id: .attributes.attributes.asset.id,
ip: .attributes.attributes.network.client.ip,
country: .attributes.attributes.network.client.geoip.country.name
}]'pup audit-logs search --query "@asset.type:api_key @asset.id:KEY_ID @action:created" --from 90d -o json \
| jq '[.data[] | {
created_at: .attributes.timestamp,
created_by: .attributes.attributes.usr.email,
creator_ip: .attributes.attributes.network.client.ip,
creator_country: .attributes.attributes.network.client.geoip.country.name
}]'Key ID: <key_id>
Created: <timestamp> by <user_email>
Active period: <first_seen> to <last_seen>
Total events: <N>
Origins:
- <Country> (<ASN>): <N> events — [NORMAL / FLAG: first-time origin]
Endpoints called (top 5):
- <METHOD> <path>: <N> calls
Destructive actions: <N> deletions — [resource types affected]
Recommended actions:
1. Revoke the key immediately if not already done
2. Review affected resources: [list]
3. Check if any deleted resources need restoration
4. Audit who else had access to this keypup api-keys delete KEY_ID