Skip to content
ez-crawl

Technical reference

Glossary

Terms used across these docs.

  • crawler-core: the engine that fetches, converts, extracts and stores.
  • crawler-api: the product backend that validates and runs workflows.
  • Depth: how many link hops from the start page the fetcher follows.
  • Glob: a pattern using * as wildcard.
  • MCP: the protocol AI agents use to talk to ez-crawl.
  • Markdown (fit): Markdown after a content filter. Can discard list content.
  • Run: one execution of a workflow or branch.
  • Semantic crawling: a downstream LLM prompt guiding the fetcher to judge page relevance.