Directory with detail pages

URL Fetcher ─▶ Markdown Converter ─▶ LLM Processor ─┬─▶ Table
(depth 2)      (everything)          (one record     └─▶ Data Source
                                      per page)
URL Fetcher ───────────────────────────────────────────▶ Data Source (images)

Ask an agent

Build a directory extraction workflow from into a table and persisted records. Extract name, website, and nullable contact details. Include both listing and detail-page URL patterns, test eight pages, and inspect actual coverage before increasing limits.

Replace the placeholders with a real site you are allowed to read. The example patterns below are illustrative; use the links your site actually exposes.

Build and refine

1. URL Fetcher. Start at the list page. depth: 2, max_pages: 8, mean_delay: 1, and url_pattern filters for both the list pages and the detail pages:

   {"deep_crawl": { "filters": [{ "type": "url_pattern",
     "patterns": ["*/list*", "*/detail?id=*"] }] }}

2. Markdown Converter. {"content_source": "cleaned_html", "content_filter": "none"}.

3. LLM Processor. Extract mode; no prompt. Say what to return on list pages (nothing).

4. Table + Data Source, both fed by the LLM node.

5. Optional: a second Data Source fed by the fetcher with save_images: true.

Check, then scale

• Open the fetcher’s output: did it read the detail pages, not just the list?
• Open the Markdown output: is there real text?
• Compare three records with the site.
• Only then raise max_pages.

Reuse the workflow

Save it in a workspace and keep the tested patterns and schema. Run it again when you need fresh data, or query stored records to enrich their websites. For repeat collection with keyed record matching, use MongoDB Store.