Build & operate
Reuse and composition
Reuse a saved graph, process each record, and merge independently collected information.
Choose a reuse mechanism
Section titled “Choose a reuse mechanism”| Need | Mechanism |
|---|---|
| Run the same collection process again | Save the workflow and execute a new run |
| Continue from unchanged outputs | Use rerun controls and inspect what is reused |
| Apply a small workflow to each record | Inline forEach graph |
| Read earlier persisted results | Query, optionally scoped by execution and source nodes |
| Merge branches describing the same entity | Combine with an explicit shared key |
Enrich a stored directory
Section titled “Enrich a stored directory”Query (records) → For Each (visit website → convert → extract) ↓ MongoDB Store → Query → TableGive the Query a small limit. Bind each website into the inline fetcher’s url, set fetch depth to zero, and test a few items. Open foreach_result to check skipped and failed items before writing records at scale.
For multiple enrichment sources, connect record-producing branches directly to Combine, choose a stable key, and set field rules. Inspect unkeyed and _branches, then persist the merged records. See For Each, Combine, and storage.
Keep the abstraction honest
Section titled “Keep the abstraction honest”The implemented composition mechanism is an inline engine graph. Nested For Each nodes are rejected. A saved workflow is independently reusable by running it again, but this does not imply that another workflow can invoke it by id as a versioned function. No published reusable-subworkflow registry or SDK is asserted here.