Urlcheckr
← Back

Why is my check blocked by Cloudflare?

When Urlcheckr checks a URL protected by Cloudflare, the Cloudflare WAF may identify the request as automated traffic and return a 403 Forbidden (or 503 Service Unavailable) response instead of following the redirect chain.

What you'll see

In the result detail for the affected check you'll see:

  • A final status code of 403 or 503
  • The response header cf-mitigated: challenge in the response headers section

This means Cloudflare has actively mitigated the request and did not pass it through to the origin.

Why this happens

Cloudflare's WAF, Bot Fight Mode, and Managed Challenge rules inspect incoming requests and may block traffic that resembles automated bots. Urlcheckr sends requests with the User-Agent string Urlcheckr/3.0, which Cloudflare may flag as a bot.

When Cloudflare intercepts the request it returns a challenge response (cf-mitigated: challenge) rather than forwarding it to the origin. The result is that Urlcheckr cannot complete the check.

How to fix it

If you own or manage the Cloudflare zone for the domain being checked, add a custom rule to skip Cloudflare's security rules for Urlcheckr:

  1. Go to your domain in the Cloudflare dashboard.
  2. Open Security rules.
  3. Click Create rule → Custom rules.
  4. Configure the rule:
    • Rule name: Allow Urlcheckr
    • When incoming requests match: User Agent → equals → Urlcheckr/3.0
    • Then take action: Skip
    • Under Skip, enable: All remaining custom rules, All rate limiting rules, All Super Bot Fight Mode rules, and All managed rules
  5. Click Deploy.

Urlcheckr checks will now pass through Cloudflare without interruption.

Official Cloudflare documentation on Skip rules →

What about IP whitelisting?

Cloudflare does support IP-based rules, but this is not a practical solution for Urlcheckr.

Currently we use Trigger.dev to run checks in the background. Our Trigger.dev jobs run on infrastructure with a dynamic IP address, meaning the IP address is not guaranteed to stay the same between runs. Maintaining an IP allowlist would therefore require constant updates and would frequently break.

Still seeing the error?

If you've applied the WAF rule and the error persists, the rule may not have propagated yet (allow a few minutes) or the expression may not match exactly. Check that the User-Agent string in the rule is identical to what Urlcheckr sends — you can confirm the exact value in the Request headers section of the result detail.

For further help, reach out at support@urlcheckr.app.