Can Siteimprove Crawl Through MFA Logins?

Answer

No—Siteimprove’s crawler cannot bypass multi-factor authentication (MFA) logins by default. In some cases, access may be possible if the authentication provider offers a crawler-friendly solution, but this varies and depends on the provider, MFA method, and customer configuration.

Overview

Siteimprove’s crawler cannot currently bypass multi-factor authentication (MFA) logins by default. This means we are unable to access content behind MFA-protected areas unless specific accommodations are made.
However, some authentication providers do offer options to allow bot or crawler access—but not all. The feasibility depends entirely on the provider, the MFA method used, and whether the customer can help facilitate a solution.

What is MFA?

Multi-Factor Authentication (MFA) is a security protocol that requires users to verify their identity using multiple factors before gaining access. It’s designed to prevent unauthorized access and is commonly used across enterprise websites and applications.

Typical MFA methods include:

• Passwords + Time-based One-Time Passwords (TOTP)
• Push notifications to mobile devices
• CAPTCHA challenges
• Biometric verification

Each method introduces complexity for automated systems like Siteimprove’s crawler.

Which Authenticator Solutions Exist?

There are many MFA providers, each with their own implementation:

• Google Authenticator
• Microsoft Authenticator
• Okta
• Duo Security
• Authy
• Custom enterprise solutions

Some providers may offer bot-friendly options, while others may not. For example:

• TOTP (Time-based One-Time Passwords) might be technically feasible.
• However, CAPTCHA challenges or additional login steps often block crawler access.

Because of this variability, we cannot give a blanket answer to whether Siteimprove can crawl through MFA-protected sites.

What to Do When Asked About Crawling MFA-Protected Sites

Step-by-Step Guidance

1. Explain the current limitation:
   • Siteimprove cannot bypass MFA without specific support from the authentication provider.
2. Ask the customer to contact their MFA provider:
   • They should ask:
      “Are there any options from your authentication provider to allow a bot or crawler to access content behind MFA?”
3. Request documentation or contact:
   • If the provider offers a solution, the customer should: 
     • Forward the documentation to Siteimprove.
     • Or connect Siteimprove directly with the provider’s technical contact.
4. Gather key details:
   • Which MFA provider is used?
   • What authentication method is in place (e.g., TOTP, CAPTCHA)?
   • Is there a login challenge that blocks automation?
5. Escalate internally if needed:
   • If the customer provides a potential solution, escalate to the appropriate technical team for feasibility review.

Summary

• Can Siteimprove crawl through MFA by default?
  • No
• Do some providers allow crawler access?
  • Yes, but varies
• Is TOTP viable?
  • Possibly, but sometimes blocked by CAPTCHA
• What should customers do?
  • Contact their MFA provider and share options with Siteimprove