URL to Markdown API - Web Pages to Clean Text for LLMs

Rocketyard’s URL to Markdown API turns any public web page into clean Markdown for LLMs, RAG pipelines and AI agents and returns one clean record per page: markdown, text, title, description, language, author and more. It runs on the Apify platform, so you can start it from the Apify Console, the REST API, an AI agent or a no-code tool.

Run it on Apify$0.75 per 1,000 pages, no subscription. You run it with your own Apify account, and Apify handles billing.

Who it’s for

Developers building RAG pipelines, chatbots and AI agents
AI agent and MCP users who need a “read this URL” tool
Data teams building knowledge bases from docs, help centers and blogs
SEO and content teams auditing page metadata

What you can do with it

Turn a list of URLs into clean Markdown for an LLM or a vector database
Crawl a documentation site into one Markdown dataset
Detect when a page changed and re-index it

How it works

The Actor fetches each page over plain HTTP without a browser, honours robots.txt and crawl-delay, finds the main content with Mozilla Readability and converts it to Markdown that keeps headings, lists, tables, code blocks and absolute links. Navigation, cookie banners, sidebars and footers are removed. Each record also carries the metadata and JSON-LD the page publishes, its heading outline, its links and images, a word count, a token estimate and a content hash for change detection. Give it a list of URLs, or a start page and let it crawl the same site.

Example output

A real record from a test run on nasa.gov on 2026-11-07. Fields the page didn’t publish are left out here; the Actor returns them as null. The same test also read en.wikipedia.org and developer.mozilla.org.

{
  "url": "https://www.nasa.gov/humans-in-space/",
  "finalUrl": "https://www.nasa.gov/humans-in-space/",
  "statusCode": 200,
  "contentType": "text/html",
  "title": "Humans In Space - NASA",
  "description": "In this original series, NASA astronauts take viewers on a journey to the International Space Station, exploring the life-changing experience of an orbital perspective.",
  "language": "en-US",
  "canonical": "https://www.nasa.gov/humans-in-space/",
  "author": "Dacia Massengill",
  "publishedAt": "2022-12-02T21:06:45.000Z",
  "modifiedAt": "2026-10-05T15:20:09.000Z",
  "siteName": "NASA",
  "mainImage": "https://www.nasa.gov/wp-content/uploads/2022/12/iss072e595426orig.jpg",
  "wordCount": 302,
  "tokenEstimate": 893,
  "markdown": "# Humans In Space - NASA\n\nBuilding on 60 years of exploration experience and more than two decades of continuous human presence on the International Space Station, NASA will expand humanity’s reach deeper into space than ever before.\n\n### NASA+ Series\n\n## Down to Earth\n\nIn this original series, N…",
  "text": "Building on 60 years of exploration experience and more than two decades of continuous human presence on the International Space Station, NASA will expand humanity’s reach deeper into space than ever before.\n\nNASA+ Series\n\nDown to Earth\n\nIn this original series, NASA astronauts take viewers on a…",
  "headings": [
    {
      "level": 1,
      "text": "Humans In Space - NASA"
    },
    {
      "level": 3,
      "text": "NASA+ Series"
    },
    {
      "level": 2,
      "text": "Down to Earth"
    },
    {
      "level": 2,
      "text": "Artemis II Lunar Flyby"
    },
    {
      "level": 3,
      "text": "Humans in Space"
    },
    {
      "level": 2,
      "text": "Human Spaceflight"
    }
  ],
  "links": [
    {
      "url": "https://www.nasa.gov/",
      "text": "",
      "internal": true
    },
    {
      "url": "https://www.nasa.gov/news/",
      "text": "News & Events",
      "internal": true
    },
    {
      "url": "https://www.nasa.gov/2026-news-releases/",
      "text": "News Releases",
      "internal": true
    }
  ],
  "images": [
    {
      "src": "https://images-assets.nasa.gov/image/iss075e0070083/iss075e0070083~large.jpg?w=1920&h=1280&fit=clip&crop=faces%2Cfocalpoint",
      "alt": "NASA astronaut and Expedition 75 flight engineer Anil Menon is attached to the Canadarm2 robotic arm’s latching end effector during a spacewalk to install a high-speed space-to-ground antenna on the International Space Station’s Z1 truss segment above the Unity module. Menon and ESA (European Space Agency) astronaut and Expedition 75 flight engineer Sophie Adenot (out of frame) spent six hours and 30 minutes replacing a failed antenna with a spare and making the precise connections needed to activate it for high-speed transmissions with mission control."
    },
    {
      "src": "https://images-assets.nasa.gov/image/jsc2026e391565/jsc2026e391565~large.jpg?w=1920&h=1280&fit=clip&crop=faces%2Cfocalpoint",
      "alt": "Four astronauts are pictured in orange Orion Crew Survival System suits. The astronauts on the far left and far right of the image are standing, holding their helmets. In between them, are the other two astronauts, who are seated and holding their gloves."
    }
  ],
  "contentHash": "sha256:b084d35db91f86024ac4f634e5a7f917c794a7d59ba24d873f6b69d6692dd797",
  "scrapedAt": "2026-11-07T07:01:38.852Z"
}

Fields

Field	Description	Filled in latest test
`markdown`	Main content as clean Markdown: headings, lists, tables, code blocks with their language, absolute links	100%
`text`	Main content as plain text	100%
`title`	Page title as published (og:title or `<title>`)	100%
`description`	Meta description, or the first paragraph when the page has none	100%
`language`	Language code, e.g. `en-US`	100%
`author`	Author or byline, when published	40%
`publishedAt`	Publication date (ISO 8601), when published	40%
`modifiedAt`	Last update (ISO 8601), when published	40%
`siteName`	Name of the website	80%
`mainImage`	Social preview image, or the first content image	80%
`canonical`	Canonical URL the page declares	80%
`headings`	Outline of the main content: `{ level, text }` for every heading	100%
`links`	Every link on the page: `{ url, text, internal }`, absolute and de-duplicated	100%
`images`	Images of the main content: `{ src, alt }`	40%
`jsonLd`	Every JSON-LD block the page publishes (schema.org structured data), parsed	20%
`wordCount`	Words in the main content	100%
`tokenEstimate`	Rough LLM token count of the Markdown (characters / 4)	100%
`contentHash`	`sha256:` hash of the main text: an unchanged hash means unchanged content	100%
`finalUrl`	URL after redirects	100%
`statusCode`	HTTP status of the page	100%
`contentType`	MIME type (`text/html`)	100%
`url`	the page the record came from	100%
`scrapedAt`	ISO timestamp of the extraction	100%

A field is filled when the source publishes it. The Actor never guesses a value: a missing field comes back as null. “Filled in latest test” is the share of records from the latest test run (nasa.gov, en.wikipedia.org and developer.mozilla.org) that had the field; other sources publish more or less.

Pricing

You pay $0.75 per 1,000 pages, plus a start fee of $0.00005 per run. A URL that fails returns a free error item saying why, instead of a charged empty record.

Pages	Cost
1,000	$0.75
10,000	$7.50
100,000	$75.00

Set maxItems or the run’s Maximum cost per run in Apify to cap spending.

API

Every Apify Actor is also a REST endpoint, so URL to Markdown API works as a URL to Markdown API, a web to Markdown API or a HTML to Markdown API. Call it from any language with the Apify REST API, or with Apify’s JavaScript and Python clients. You need an Apify API token.

URL to Markdown API request

curl -s -X POST "https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"outputFormats":["markdown"],"maxItems":10}'

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('rocketyard/web-page-to-markdown').call({"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"outputFormats":["markdown"],"maxItems":10});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

from apify_client import ApifyClient
client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("rocketyard/web-page-to-markdown").call(run_input={"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"outputFormats":["markdown"],"maxItems":10})
items = client.dataset(run["defaultDatasetId"]).list_items().items

The synchronous endpoint returns the records directly. For long crawls, start the run asynchronously and read the dataset when it finishes.

MCP

Apify’s MCP server exposes this Actor as a tool to Claude, Cursor and other MCP clients. Open this Actor’s MCP setup page for setup steps per client, or connect it with this URL:

https://mcp.apify.com?tools=rocketyard/web-page-to-markdown

Claude Code:

claude mcp add --transport http apify "https://mcp.apify.com?tools=rocketyard/web-page-to-markdown"

Cursor (.cursor/mcp.json) and other clients that read an mcpServers config:

{
  "mcpServers": {
    "apify": { "url": "https://mcp.apify.com?tools=rocketyard/web-page-to-markdown" }
  }
}

On first use your browser opens so you can sign in to Apify. Then ask the agent, for example: “Turn a list of URLs into clean Markdown for an LLM or a vector database”. The tool returns JSON records, and the runs are billed to that Apify account at $0.75 per 1,000 pages.

Automation

n8n, Make and Zapier all connect to Apify. Each example below uses Actor rocketyard/web-page-to-markdown with this input:

{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "outputFormats": [
    "markdown"
  ],
  "maxItems": 10
}

n8n

Copy this workflow and paste it onto an n8n canvas. It runs the Actor every morning through an HTTP Request node and returns the records as n8n items.

{
  "name": "URL to Markdown API daily",
  "nodes": [
    {
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "days",
              "daysInterval": 1,
              "triggerAtHour": 7
            }
          ]
        }
      },
      "id": "c1f6a2d0-4a57-4b5e-9d7a-0a1b2c3d4e01",
      "name": "Every day at 07:00",
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.2,
      "position": [
        0,
        0
      ]
    },
    {
      "parameters": {
        "method": "POST",
        "url": "https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpBearerAuth",
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "{\n  \"startUrls\": [\n    {\n      \"url\": \"https://en.wikipedia.org/wiki/Web_scraping\"\n    }\n  ],\n  \"outputFormats\": [\n    \"markdown\"\n  ],\n  \"maxItems\": 10\n}",
        "options": {
          "timeout": 300000
        }
      },
      "id": "c1f6a2d0-4a57-4b5e-9d7a-0a1b2c3d4e02",
      "name": "Run URL to Markdown API",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        260,
        0
      ],
      "credentials": {
        "httpBearerAuth": {
          "name": "Apify API token"
        }
      }
    }
  ],
  "connections": {
    "Every day at 07:00": {
      "main": [
        [
          {
            "node": "Run URL to Markdown API",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "settings": {}
}

Then open the HTTP Request node and create an HTTP Bearer Auth credential named Apify API token, with your token from Apify Console as the bearer token. The token stays in n8n’s credential store; the workflow only names the credential. Change the input in the node’s JSON body to your own.

The node waits up to 300 seconds, which is also the limit of Apify’s synchronous endpoint. An input that yields nothing comes back as an item with an error field, which you can drop with a Filter node. Apify also publishes a community node, @apify/n8n-nodes-apify, with a Run an Actor and get dataset operation.

Make

Add Apify’s Run an Actor module and connect your Apify account.
Choose rocketyard/web-page-to-markdown, paste the input above and set Run synchronously to Yes. Make limits how long a synchronous run can take, depending on your plan. For longer runs, start the scenario from Watch Actor Runs instead.
Add Get Dataset Items with the run’s default dataset ID.
Send the items wherever you need them, for example with Google Sheets Add a Row, mapping markdown, text, title and description.

Zapier

Add Apify’s Run Actor action, connect your Apify account, choose rocketyard/web-page-to-markdown and paste the input above.
Zapier stops synchronous runs after 30 seconds. For anything larger than a small run, let the run continue in the background and start a second Zap from the Finished Actor run trigger.
Add Fetch dataset items with the run’s dataset ID, then map markdown, text, title and description into your app.
Test with a small input and check that the Zap handles every record before you turn it on.

Limits

The Actor reads the HTML the server sends and runs no JavaScript. A page built only in the browser returns a free EMPTY error item.
Pages behind a login or a paywall are not accessed. Bot protection is reported as a free BLOCKED error item, never bypassed.
Paths that robots.txt disallows are skipped and reported as ROBOTS_DISALLOWED.

FAQ

How much does it cost?

$0.75 per 1,000 pages plus $0.00005 per run, billed by Apify. 10,000 pages cost $7.50. There is no subscription, and you can cap any run with Maximum cost per run.

Your Apify plan sets your discount tier (Platinum and Diamond are enterprise tiers):

Tier	Per 1,000 pages
Free	$0.75
Bronze	$0.75
Silver	$0.60
Gold	$0.49
Platinum	$0.49
Diamond	$0.49

How do I call it from the API?

One request starts a run and returns its results, using your Apify API token:

curl -X POST "https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items" -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" -d '{"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"outputFormats":["markdown"],"maxItems":10}'

What happens when a URL fails?

The dataset gets one free error item for it, with a code and the reason, and the other inputs in the run carry on. Error codes explains each code, whether a retry helps and what to do.

Which pages does it work on?

Any public HTML page whose text is in the HTML the server sends: articles, documentation, blogs, help centers, product and landing pages.

What happens when a field is missing?

The field comes back as null. The Actor reports what the source publishes and never fills in guesses.

Is it allowed to scrape this data?

The Actor reads public pages only, obeys robots.txt and crawl-delay, and doesn’t log in or bypass protections. You are responsible for how you use the content under each site’s terms and the law (for example copyright and GDPR).

Can I use it from an AI agent?

Yes. Connect Apify’s MCP server as shown in MCP, and the agent can call the Actor as a tool.

Alternatives

See how it compares with other web page scrapers on price, users and run success, from Apify Store data. Need a field or site behaviour that isn’t covered? Open an issue on the Actor page.