Agentic engine optimization: writing docs that AI agents can read and use
A vendor guide on dev.to describes AEO, a checklist for making documentation readable by AI agents as well as people, and it is a practice docs owners can start now.
The short version
- AEO aims to make every page an AI agent needs findable, clean, current and usable.
- Core pieces: Markdown page copies, an llms.txt index, a sitemap, accurate meta data, stable URLs and an MCP server.
- Adding .md to a published DocStar page URL returns that page as raw Markdown.
- DocStar scores published pages of at least 1,000 characters out of 100 and suggests fixes.
- The guide comes from DocStar's own blog, so the workflow is shown with its product.
Why it matters
More of your documentation, help pages and product information will be read first by an AI rather than a person. If those pages are vague, outdated or buried in clutter, the answers people receive will be wrong. The checklist is also a good test of whether your content is clear to anyone.
A new reader for your docs
As dev.to reported in a guide from the DocStar team, agentic engine optimization (AEO) is the work of preparing documentation so AI agents can locate it, read it and act on it. Classic search optimization aims to get a page ranked so that a person clicks. AEO assumes the visitor may well be software: a coding assistant checking an API reference partway through a task, a support bot answering a customer, or an AI search tool summarizing a product.
The guide also mentions close cousins, generative engine optimization and answer engine optimization. They share the same basic idea.
Software readers behave differently from people. Menus, page design and scripts are noise to them; they want the text in a form they can parse. They often scan many pages in one go, so they need a list of what exists. And they act on what they read, so a fuzzy page can lead to a wrong action.
The checklist agents care about
The guide boils AEO down to one goal: every page an agent needs should be easy to discover, clean, current and callable. It then lists the signals that serve that goal.
- llms.txt: a simple text file that lists your pages so an AI can see what is available and where.
- Markdown copies: each page without the surrounding layout. Markdown is lightweight plain-text formatting that machines read more reliably than a full web page.
- Sitemap and robots.txt: files that help search engines and AI crawlers find every published page.
- Meta data: short, accurate titles, descriptions and tags that an AI can quote.
- Stable URLs with 301 redirects: a 301 sends visitors from an old address to the new one, so links in AI answers keep working.
- An MCP server: a connection that lets an agent search your docs live and, for API docs, call your API.
The guide says the first five are about being read, while the last is about being used. An agent connected this way does not rely on whatever it picked up from an old crawl.
What DocStar builds in
The guide shows how this works in DocStar, the company's own documentation product. According to the post, publishing a collection generates the llms.txt file, a self-updating sitemap, and Markdown versions of pages. Add .md to a published page's address and you get the raw Markdown. If a page has no meta description or tags, the product drafts them with AI.
Published pages of at least 1,000 characters receive an AI SEO score out of 100 along with suggestions. Readers also get a Copy page button, with shortcuts to open the page in ChatGPT, Claude or Perplexity. Each collection gets a hosted MCP server, and only published content is visible to it. Drafts stay out.
One caveat in the guide: pages on DocStar's default link are marked noindex, which tells search engines to skip them. To appear in search and AI search, you need to connect your own domain. Agents connected through the MCP server can still search published docs either way.
Letting an agent do the audit
The automatic files are only part of it. The guide says the human-written content is where the effort goes: missing descriptions, vague summaries, long web addresses and thin pages. Its suggestion is to connect an AI assistant such as Claude Code, Cursor or Codex to the DocStar MCP server and give it one instruction to audit a collection.
In the example prompt, the agent checks each published page for a clear title, a 140-160 character description and 4-6 tags. It also checks whether the opening paragraph answers the page's question directly, whether the address is short, and whether the page is too thin. It drafts fixes, shows the list, and asks before saving. The guide says edits stay unpublished until a person publishes them.
The human role is judgment. You decide what stays private, approve titles and slugs (the readable end of a page address), and check the accuracy of AI-written drafts. The guide warns that password protection covers Markdown versions too, but page names and links can still show up in the sitemap and llms.txt, so sensitive details should stay out of titles.
Keeping docs current
The guide stresses that AEO is ongoing. Docs drift as products change, and an agent reading stale pages gives stale answers. Its proposed fix is to add a short note to a code repository's AGENTS.md or CLAUDE.md file, the instruction files coding agents read. The note tells the agent to draft a docs update whenever a feature or API changes, and to save it as a draft until asked to publish.
Because the index, sitemap, Markdown copies and MCP server all draw from published pages, a publish updates all of them at once. The guide also says everything can be done by hand in the app if you prefer.
Keep in mind that this is a vendor's own blog post, and it describes its own product. The underlying advice, though, applies to any docs: clear opening answers, accurate summaries, stable links and current content.
What to watch
- Whether llms.txt and Markdown copies become standard across documentation tools.
- Whether AI crawlers and agents reliably use these files, which the source does not measure.
Try it
Pick one important page in your docs or help center and check whether its first two sentences answer the page's question directly. Then see if a plain-text version of it is available.
Sources and links
- Reportingdev.to: the article we sourced
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.
Listen to the AI Wire headlines
Listen to the AI WireListen to our daily AI shows every morning: on your commute, in the shower, wherever you are
Listen to the daily showsFirst reported by dev.to · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
