Fix a crawl or synchronization error

A crawl failure occurs before, during or after fetching

Before: invalid URL, private destination or quota. During: DNS, TLS, redirect, HTTP status, challenge or size. After: empty content, unusable format, extraction or indexing job. Identifying phase avoids rerunning entire base.

For full website, Wavize shows totals for discovered, processed and failed pages, but may not retain a detailed reason per URL. Isolate a representative page for a reproducible diagnosis.

Controlled retry
Fix cause first, retry a few priority URLs, then full site. Repeated retries may consume allowance and hide initial incident time.

Test URL as anonymous visitor

Use private window and, when possible, an external network.

1. No-login response

   Page should display content without account, password, prior cookie or temporary signed token.

2. Finite redirects

   Check final URL, same domain and no HTTP/HTTPS or www loop. Onboarding analysis limits redirects and crawl remains subject to destination controls.

3. Actual text present

   A page empty before JavaScript, video-only or canvas provides no usable content. Add HTML or text version.

Read server responses

Ask host for logs around test time if needed.

• 403 or challenge

  WAF, robots or anti-bot protection rejects server. Create targeted rule consistent with policy without disabling all security.

• 404 or 410

  Fix links and sitemap, exclude removed pages and do not replace with home redirects creating duplicate content.

• 429

  Server rate limits requests. Space crawl or adjust rule for Wavize; do not run identical sources in parallel.

• 5xx / TLS / DNS

  Fix outage, certificate chain or resolution before retry. A page working only from office may remain publicly unreachable.

Check quota, selection and job status

A partial source is not always a failure.

1. Monthly URL allowance

   Compare used pages and plan capacity. Exclude filters, parameters and duplicates before retry.

2. Queue and workers

   Persistent queued status may indicate busy queue or stopped worker. Record duration and source before ticket.

3. Concentrated failures

   Find common URL pattern: language, subdomain, filter, heavy file. Family-level exclusion may preserve useful crawl.

Validate correction in answers

Successful status should still produce useful knowledge.

1. Processed page

   Priority page is no longer failed or unintentionally excluded.

2. Factual question

   Ask a question with answer explicitly in fetched text.

3. No duplicate

   Old URL or version does not provide contradictory answer after retraining.

Information to send support

Do not send API key or private page.

• Source — Title, type and public URL.
• Timeline — Start time, duration in each status and last retry.
• Sample — Two or three representative URLs and public statuses.