FAQ

1. Start URL — the audited URL is normalized (a missing scheme becomes https://, the fragment is dropped). A URL that cannot be parsed, or has no host, ends the audit with 502.

2. Plain-fetch crawl — the start page is fetched as 37AuditsBot. A block (HTTP 403, HTTP 429 or a Cloudflare challenge) ends the audit with 590; any other failure (timeout, DNS, TLS, HTTP 4xx/5xx, or a non-HTML answer) ends it with 500. From there the crawl is breadth first, following only <a href> links whose host is exactly the start host (no subdomains), skipping mailto:, tel: and javascript: links, and silently skipping later pages that fail, answer HTTP 4xx/5xx or are not HTML. It stops at 500 pages.

3. Detection — every crawled page is a FAQ page when it has a <script type="application/ld+json"> whose @type is FAQPage (also inside an array), an element with itemtype pointing at schema.org/FAQPage, or an h1–h4 heading that reads "FAQ", "FAQs", "Frequently Asked Questions", "Perguntas Frequentes" or "Dúvidas Frequentes".

4. Browser crawl — only when the plain-fetch crawl finds nothing is the same crawl run in a headless Chromium, so sites that build their pages or their navigation with JavaScript are judged on the rendered DOM. A browser that fails to run, even after one relaunch, ends the audit with 501.

5. Verdict — the first page recognized as a FAQ page is SUCCESS (200), naming the page, the detector and the crawl that found it. When both crawls are exhausted or hit the page limit, the audit is FAIL (400).

Every row maps to one checkRule* method and one CheckCode, in the order the auditor runs them. CheckCode suffixes use status ranges: 200 success, 400–499 fail, 500–599 error.

Every check carries startUrl, host, pagesCrawled, maxPages and method (jsoup for the plain-fetch crawl, playwright for the browser crawl) in its data; the success check adds faqUrl and detector (json-ld, microdata or heading), and the failure adds pagesCrawledWithJsoup and pagesCrawledWithBrowser.

This auditor emits no WARNING or INFORMATION checks.

Why a FAQ page matters

• Visitors who cannot find an answer to a pre-sale question rarely ask; they leave. A FAQ page keeps them on the site and lowers the volume of repetitive support requests.

• FAQPage structured data makes the answers machine readable, so search engines and AI assistants can show them in rich results and direct answers, which drives qualified traffic.

How the crawl behaves

• Only links on the exact start host are followed. A FAQ hosted on help.example.com or support.example.com is not found when the audited URL is www.example.com; audit the help site separately.

• Each crawl visits at most 500 pages, breadth first from the start URL. On a large site a FAQ page that sits deep in the navigation may never be reached; link it from the home page or the footer.

• The plain-fetch crawl reads the HTML the server sends. Sites whose navigation or content is built by JavaScript are covered by the browser crawl, which is slower: the audit can take several minutes on such sites.

• A heading alone is enough to pass. The auditor does not judge the questions or the answers, nor whether the structured data is valid; use a structured data validator for that.

Changes from the previous version

• An invalid start URL, or one without a host, is now ERROR (502) instead of FAIL.

• A start page that cannot be fetched, answers HTTP 4xx/5xx or is not HTML is now ERROR (500); it used to be skipped silently, so the audit ended in FAIL with zero pages crawled.

• A browser crawl that fails to run is now ERROR (501); the audit used to return the FAIL of the plain-fetch crawl instead.

• The "no FAQ page" verdict now carries a check (400) with the pages crawled by each crawl, the host and the limit; it used to be a bare FAIL response with no check.

This documentation is built and hosted on Mintlify, a developer documentation platform.