Node reference
Data Transformer
Select or clean HTML, or extract JSON values, without a model.
transformer_type | Does |
|---|---|
cssSelector | Keeps the HTML matching css_selector |
htmlParser | Parses HTML and removes configured tags (default: script, style, noscript) |
jsonExtractor | Extracts values by JSONPath or regex in extract_pattern, e.g. $.offers[*].price |
custom | Applies transformer_config.operation |
Deterministic; no model call. Prefer it over the LLM when a page’s structure is stable.
Input and output
Section titled “Input and output”HTML modes consume raw_pages and replace cleaned_html and fit_html on a copied page. Original HTML and source URLs remain intact. JSON extraction emits transformed_data entries containing source_url, data, and, for JSONPath, path, plus transformer_metadata.
Example: isolate product content
Section titled “Example: isolate product content”{"transformer_type":"cssSelector","css_selector":"main .product"}Connect URL Fetcher → Data Transformer → Markdown Converter. A selector matching nothing produces empty transformed content; it does not fall back to the full page.
Defaults and errors
Section titled “Defaults and errors”transformer_type defaults to cssSelector. A missing selector or extraction pattern skips execution with a reason in metadata. Invalid CSS, JSONPath, or regex is rejected by validation. custom accepts only identity (skips work) or clean_html; arbitrary executable code is not supported. Set transformer_config.source explicitly for JSON extraction from llm_result or query_result.