Markdown for AI agents

Building an AI agent, a crawler or a coding tool that reads our documentation? You don't have to parse our HTML. Most pages on codeandweb.com are also published as plain markdown, and this page describes exactly what you get, which headers come with it, and what happens when something is missing.

The short version

1. Append .md to a page URL to get it as markdown (drop a trailing slash first: /x/ becomes /x.md, not /x/.md): https://www.codeandweb.com/texturepacker/documentation/commandline.md
2. Or request the normal page URL with Accept: text/markdown. You are redirected to the .md URL.
3. Or grab a whole product's documentation in one file. /llms-full.txt lists the bundles.

The site index for agents is /llms.txt.

Examples

Fetch a page by its .md URL:

curl https://www.codeandweb.com/texturepacker/documentation/commandline.md

Fetch the same page through the normal URL. -L follows the redirect to the .md URL:

curl -L -H 'Accept: text/markdown' https://www.codeandweb.com/texturepacker/documentation/texture-settings

Fetch a whole product's documentation:

curl https://www.codeandweb.com/physicseditor/llms-full.txt

The same with Python and requests:

import requests

# By suffix
response = requests.get("https://www.codeandweb.com/texturepacker/documentation/commandline.md")
response.raise_for_status()
print(response.text)

# By Accept header. requests follows the 302 to the .md URL by default.
response = requests.get(
    "https://www.codeandweb.com/texturepacker/documentation/texture-settings",
    headers={"Accept": "text/markdown"},
)
response.raise_for_status()
print(response.url)   # https://www.codeandweb.com/texturepacker/documentation/texture-settings.md
print(response.text)

The .md URLs are meant for programmatic clients. Nothing redirects a browser to them. A markdown client stays on markdown with one exception: the .md URL of a page that has moved to a page without a twin redirects to that page's HTML URL, as described under "A moved page (301)" below.

Which pages have markdown

Documentation, tutorials, knowledgebase articles, product pages and the other pages we publish for search engines have a markdown version, which we call a markdown twin. Pages that are kept out of search engines, such as the license agreements, have none, and neither do pages that name another page as their canonical URL, such as tutorials for older framework versions. Interactive tools, such as the free online sprite sheet packer, have none either. If a page has a twin, its HTML says so in two places:

• a <link rel="alternate" type="text/markdown" href="..."> element in the page's <head>
• a Link: <...>; rel="alternate"; type="text/markdown" response header

The home page's twin is /index.md.

Two kinds of markdown

Depending on the page, you get one of two different markdown formats.

Our markdown twins. Pages with a twin are converted at build time from the page's main content. Navigation, footers and other page chrome are removed. Links to other pages that have a twin point at their .md URLs, so an agent can keep following links without switching formats. Some twins end with a ## Structured data section that contains the page's JSON-LD.

Cloudflare's conversion. If a page has no twin and you request it with Accept: text/markdown, our server answers with the normal HTML page, and Cloudflare, which sits in front of our site, converts that HTML into markdown on the fly. Cloudflare converts the whole page, so its output also contains page chrome such as the skip link and navigation labels, and some elements stay as inline HTML. Its front matter carries title and description, plus image when the page has an Open Graph image:

---
description: Short tutorial how to create SpriteSheets for Phaser.io using TexturePacker
title: Creating spritesheets for Phaser 2 with TexturePacker
image: https://www.codeandweb.com/static/48a0483642b2a2a60272b2076d8a1bb0/f3583/_opengraph-image.png
---

[Skip to content](#main-content)

That excerpt is what /texturepacker/tutorials/how-to-create-sprite-sheets-for-phaser2 returned for Accept: text/markdown in September 2026.

How to tell them apart: our twins always carry the source and canonical front matter keys. Cloudflare's output, as we have seen it, carries neither.

Front matter

Every twin starts with YAML front matter. All values are double-quoted strings. Our twins use Cloudflare's title and description keys with the same meaning, so code that reads those two from Cloudflare's output reads them from ours too. Cloudflare's image key has no counterpart in our twins.

A typical twin starts like this:

---
title: "What's new in BabelEdit 3"
description: "Learn about the new features we've added to BabelEdit 3: Custom languages, improved search and filtering, parameter support for flutter, text transformations"
canonical: "https://www.codeandweb.com/babeledit/tutorials/whats-new-in-babeledit-3"
date: "2022-04-28T00:00:00.000Z"
modified: "2022-04-28T00:00:00.000Z"
source: "html"
---

# What's new in BabelEdit 3

After the front matter, every twin opens with a level-1 heading.

Response headers

A markdown twin (200)

GET /texturepacker/documentation/commandline.md

Send If-Modified-Since with the Last-Modified value you stored, and you get 304 Not Modified while the file is unchanged.

An HTML page that has a twin (200)

GET /texturepacker/documentation/commandline, with a browser-style Accept header or none at all.

For HTML pages without a twin, our server sends neither the Link header nor Vary: Accept.

The home page (200)

GET / carries the rel="alternate" header like every page with a twin, plus two more Link headers that point agents at the machine-readable entry points. No other page sends these two.

The Accept: text/markdown redirect (302)

GET /texturepacker/documentation/commandline with Accept: text/markdown

The redirect only happens for pages that have a twin. It fires when text/markdown appears in your Accept header before text/html, or without text/html at all. It compares positions only, not q-values, so put text/markdown first if you want markdown. A request for the site root with Accept: text/markdown is redirected to /index.md.

A moved page (301)

If you request the .md URL of a page that has moved, you get a 301 Moved Permanently. The Location points at the new page's .md URL when the new page has a twin, and at its HTML URL otherwise. For example, /tutorials/texturepacker.md redirects to https://www.codeandweb.com/texturepacker/tutorials.md.

A URL with a trailing slash (301)

GET /texturepacker/documentation/

With Accept: text/markdown first, the same URL still answers the 302 to the .md URL directly, so an agent never takes two hops.

Markdown errors (404)

The bodies are described in the next section.

llms.txt and the bundles

/llms.txt, /llms-full.txt and the per-product llms-full.txt bundles are served as text/plain; charset=utf-8.

The API catalog

GET /.well-known/api-catalog

Errors

You get a markdown 404 instead of our HTML error page whenever you request a .md URL that does not exist, or request a missing page with an Accept header that lists text/markdown before text/html, or without text/html at all. There are three variants. Each body below is exactly what the server sends.

The page exists, but has no twin

You appended .md to a real page that has no markdown version. Request the HTML URL with Accept: text/markdown instead, and Cloudflare converts the page for you, as described under Two kinds of markdown above.

# No markdown twin for this page

This HTML page exists, but it has no markdown twin.

Fetch the HTML page with a markdown Accept header instead; our edge converts it on the fly:

    curl -H 'Accept: text/markdown' https://www.codeandweb.com/texturepacker/documentation/eula

The page does not exist

Nothing matches the path. The response suggests up to five pages with similar paths, so a typo usually lands you one link away from the right page. This is the response for /texturepacker/documentatio.md:

# Page not found

/texturepacker/documentatio\.md does not match a page on codeandweb.com.

Append `.md` to any page URL, or send `Accept: text/markdown`, to read it as markdown.

## Did you mean

- [TexturePacker Documentation: Complete Reference Guide](https://www.codeandweb.com/texturepacker/documentation)
- [TexturePacker3D documentation](https://www.codeandweb.com/texturepacker3d/documentation)
- [TexturePacker Documentation: Texture compression](https://www.codeandweb.com/texturepacker/documentation/texture-compression)
- [Texture Atlas Settings](https://www.codeandweb.com/texturepacker/documentation/texture-settings)
- [TexturePacker Docker and CI Installation](https://www.codeandweb.com/texturepacker/documentation/docker-ci)

## Explore the site as markdown

- [/llms.txt](https://www.codeandweb.com/llms.txt)
- [/llms-full.txt](https://www.codeandweb.com/llms-full.txt)
- [/babeledit/llms-full.txt](https://www.codeandweb.com/babeledit/llms-full.txt)
- [/physicseditor/llms-full.txt](https://www.codeandweb.com/physicseditor/llms-full.txt)
- [/spriteilluminator/llms-full.txt](https://www.codeandweb.com/spriteilluminator/llms-full.txt)
- [/texturepacker/llms-full.txt](https://www.codeandweb.com/texturepacker/llms-full.txt)
- [/texturepacker3d/llms-full.txt](https://www.codeandweb.com/texturepacker3d/llms-full.txt)

    curl -H 'Accept: text/markdown' https://www.codeandweb.com/llms.txt

When nothing is similar enough, the ## Did you mean list is replaced by the line No close match was found for this path.

The request was rejected

The path was unsafe, so the server refused it before looking anything up. That happens when the path contains a .. segment (also percent-encoded) or a backslash, or is longer than 1,024 bytes. The body names the reason. This is the response for /a/%2e%2e/b.md:

# Request rejected

This request path was rejected before it was looked up: the path contains a `..` segment.

    curl -H 'Accept: text/markdown' https://www.codeandweb.com/llms.txt

Some other malformed paths never reach this check. A path that contains a control character, for example, is refused earlier by the web server, with a plain HTML 403 or 404 instead of a markdown body.

Caching

An HTML page that has a twin answers differently depending on the Accept header: a browser gets the HTML, a markdown client gets the 302. That is why those responses carry Vary: Accept. If you put a cache in front of your agent, make sure it includes the Accept header in its cache key. Otherwise one client's redirect can be replayed to another client that wanted HTML, or the other way round.

The simpler option is to fetch the .md URLs directly. A .md URL has exactly one representation, whatever you send in Accept.

On our side, we never cache HTML pages at the edge, so the response you get is never a cached copy.

Discovery

• /llms.txt describes the site and our products, repeats the conventions above, and lists every tutorial's .md URL.
• /llms-full.txt lists the documentation bundles with their page counts and exact sizes in bytes.
• /robots.txt points to /llms.txt.
• /.well-known/api-catalog is an RFC 9727 API catalog for clients that look for one. It is a JSON linkset (application/linkset+json) that names /llms.txt as the service-desc, this page as the service-doc, and /llms-full.txt plus the five product bundles as alternate links. The home page links to it with Link: </.well-known/api-catalog>; rel="api-catalog".
• Every HTML page with a twin announces it with <link rel="alternate" type="text/markdown"> in its <head> and a Link: rel="alternate" response header.
• Pages with a twin also show a small View as Markdown link: above the content on documentation pages, above the footer everywhere else.

Documentation bundles

Each product's documentation is also available as a single markdown file. A bundle contains the pages of the product's documentation navigation, in navigation order. Each page starts with an HTML comment that names its canonical URL, <!-- source: https://www.codeandweb.com/... -->, and has no front matter of its own. Tutorials are not bundled. /llms.txt lists them individually.

The sizes change whenever the documentation does. /llms-full.txt always has the exact current values.

Stability

You can build on this format. Front matter keys are only ever added, never renamed or removed. When we retire a page, its .md URL will answer 410 Gone with a pointer to where the content went, rather than silently turning into a 404. No page has been retired yet, so you will not see a 410 today.

2026-10-01: the source value generated was renamed to html. Both mean "converted from the page's HTML". The key is unchanged. If you cached twins before that date, accept both values.