W Wiretalk Docs
API Dashboard
Developer documentation

Website Content Sync

Crawl your website and auto-create knowledge base articles, bot replies, and canned responses. Sync KB & bot from Organization → Sites or the Agent API.

Dashboard: Dashboard → Organization → Sites → Sync KB & bot

Website content sync crawls your public website and automatically creates knowledge base articles, FAQ entries, bot auto-replies, and canned responses — so support content matches what is already on your site without manual copy-paste.

Requires a website domain on the site record. Sync runs as a background job with live progress in the dashboard.

What gets imported

Knowledge base articles

Pages are categorized into KB sections (FAQ, product, support). Each article gets a URL slug and meta description for SEO.

Bot reply templates

Common questions detected from page content become keyword-triggered bot replies in the widget.

Canned responses

Agents get matching shortcuts for frequent answers pulled from your site copy.

Widget settings hints

Business name, tagline, and contact cues from the crawl can pre-fill widget welcome content where appropriate.

Run a sync

1

Add your domain

Dashboard → Organization → Sites → set the website domain (e.g. example.com). The crawler only reads publicly reachable pages on that host.

2

Click Sync KB & bot

On the site row, click Sync KB & bot. Progress shows queued → running → completed (or failed with an error message).

3

Review & publish

Open Knowledge base and Bot replies to edit imported content before publishing. Re-run sync anytime your website changes — previous auto-imported items are replaced on the next run.

Import status

Each site stores a content_import object in widget settings:

  • status — queued, running, completed, or failed
  • progress / progress_label — live percentage and human-readable step
  • summary — counts of pages, KB articles, bot replies, and canned responses created
  • error — message when crawl or import fails (e.g. domain unreachable, no pages found)

Only one sync runs at a time per site; the button is disabled while import is in progress.

Best practices

  • Ensure your homepage and FAQ pages are publicly accessible (no login wall)
  • Use a sitemap or clear internal links so the crawler discovers all support pages
  • After sync, enable KB in the widget and turn on AI RAG for grounded auto-replies
  • Combine with domain restrictions so only your production site loads the widget

API

Trigger import programmatically from the Agent API (admin or owner):

POST /api/agent/sites/{site_id}/import-content
Authorization: Bearer {sanctum_token}

Response includes content_import with the current status. Poll site details via GET /api/agent/sites or list sites in the dashboard until status is completed.

See the REST API reference for authentication and the full Agent API surface.