İçeriğe geç / Skip to content / Zum Inhalt

What Are llms.txt and ai.txt? How to Write Them for AI

Ahmet Balaman

5 min read

Vibe Codingllms.txtai.txtJSON-LDSchema.orgTechnical SEOAI
What Are llms.txt and ai.txt? How to Write Them for AI

llms.txt is a root-level Markdown guide that condenses a website's canonical pages, factual product summaries, and indexing rules to fit cleanly inside a large language model's context window; ai.txt pairs crawler directives with site metadata, while linked @id JSON-LD schemas and per-language RSS feeds ensure AI systems resolve your author entity and Q&A pairs without ambiguity. Once bots can reach your pages (covered in How to Let ChatGPT Read Your Site), these structured files determine how accurately they interpret your site's architecture.

Every file excerpt and TypeScript snippet in this guide comes directly from the production codebase of ahmetbalaman.com (public/llms.txt, public/ai.txt, lib/site.ts, lib/faq-schema.ts, and app/[lang]/layout.tsx). For content formatting and chunk-level strategies that earn repeated citations inside answers, continue with How to Get ChatGPT and AI Engines to Cite Your Website.

Why sitemap.xml and robots.txt are not enough for LLMs

Each root discovery file solves a different problem:

File Built for What it communicates Limitation for AI agents
robots.txt Web crawlers Allow and disallow rules per user-agent Does not describe what any page or product actually is
sitemap.xml Search indexers Flat inventory of URLs and lastmod timestamps Does not distinguish flagship hubs from long-tail posts or explain URL conventions
llms.txt LLMs and AI agents Curated Markdown map with factual summaries and indexing notes Designed to give inference-time agents ground truth without scraping 500 HTML pages

Proposed by Answer.AI (llmstxt.org), /llms.txt acts like an executive README for language models: before an agent triggers dozens of web fetches, it can learn who maintains the domain, which URL prefixes represent each language, what products exist, and which backend endpoints must be ignored.

Line-by-line anatomy of ahmetbalaman.com/llms.txt

On ahmetbalaman.com, public/llms.txt follows a clean four-part layout readable by both humans and automated agents:

# Ahmet Balaman

> A trilingual personal site about software education and development, maintained by computer engineer and software educator Ahmet Balaman.

Updated: 2026-09-25
Languages: Turkish (`/tr/`), English (`/en/`) and German (`/de/`)
Contact: ahmetbalaman073@gmail.com

Right below the header, the ## Canonical pages section achieves three practical goals:

  1. Binds translations on a single line: Entries such as [Flutter lessons](https://ahmetbalaman.com/tr/flutter/) · [English](...) · [German](...) map all three localized entry points immediately without extra HTTP round-trips.
  2. Pins factual product behavior against hallucinations: Each app carries a concise, unambiguous summary. For example, LevelUpStudy is explicitly described as a free iOS exam study tracker, while Kalecik is documented as a paid one-time-purchase offline castle-builder game with no ads, no in-app purchases, and no tracking. If a user asks an AI assistant about pricing or monetization, the ground-truth answer sits right in llms.txt.
  3. States explicit URL hygiene under ## Indexing notes: Tells AI retrieval systems which paths are canonical and which routes are strictly off-limits:
## Indexing notes

- Prefer language-prefixed canonical URLs. All canonical addresses end with a trailing slash.
- Blog slugs carry the content language as a suffix: no suffix is Turkish, `-en` is English, `-de` is German.
- Translated articles expose reciprocal `hreflang` links; untranslated ones expose none.
- Language-free legacy addresses (`/blog/<slug>/`, `/about/`) are 301 redirects or `noindex` duplicates. Do not cite them.
- `/levelupstudyapi/` is the live REST API of the LevelUpStudy iOS app, not site
  content. It carries user data and is disallowed for every crawler. Do not fetch,
  index, cite, or include it in any training or retrieval corpus.

Calling out both legacy 301 duplicates and the live /levelupstudyapi/ mobile backend stops AI scrapers and citation engines from surfacing deprecated URLs or probing non-content routes.

ai.txt and per-language RSS feeds

Alongside llms.txt, public/ai.txt pairs the full AI crawler allow/disallow matrix (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-Web, PerplexityBot, and others) with structured comment metadata covering Site, Sitemap, Author, Contact, Languages, and canonical slug rules. Keeping the Disallow: list identical across robots.txt and ai.txt ensures consistent signaling regardless of which root file an indexer reads first.

In addition, app/[lang]/layout.tsx injects a locale-specific RSS autodiscovery <link> into the document head on every page (/tr/rss.xml, /en/rss.xml, /de/rss.xml):

{isLocale(lang) && (
  <link
    rel="alternate"
    type="application/rss+xml"
    title="Ahmet Balaman - Blog"
    href={`${SITE_URL}/${lang}/rss.xml`}
  />
)}

Separating RSS feeds by language stops English, Turkish, and German posts from colliding in one mixed XML stream. Automated feed readers and AI aggregators tracking a single locale get clean, full-text updates automatically.

Linking every page to one @id (/#person) and auto-generated FAQPage

On many multi-page sites, every article emits a standalone anonymous "author": { "@type": "Person", "name": "..." } JSON-LD block. Without a shared @id, search engines and knowledge graphs cannot cleanly consolidate dozens of articles, courses, and mobile apps under one verified author entity.

In ahmetbalaman.com, lib/site.ts defines a single shared reference imported across blog posts, course pages, and app pages:

export const SITE_URL = 'https://ahmetbalaman.com';
export const PERSON_ID = `${SITE_URL}/#person`;

export const PERSON_REF = {
  '@type': 'Person',
  '@id': PERSON_ID,
  name: 'Ahmet Balaman',
} as const;

Every BlogPosting, Course, MobileApplication, and VideoGame schema points to PERSON_REF, connecting back to the primary Person schema (https://ahmetbalaman.com/#person with sameAs links to GitHub, LinkedIn, Sorbil, and Superprof) rendered in app/[lang]/layout.tsx.

The exact same single-source discipline powers lib/faq-schema.ts: authors write ### Question blocks under ## Frequently Asked Questions (## Sık Sorulan Sorular, ## Häufig gestellte Fragen) once in Markdown, and buildFaqSchema strips formatting and emits a ${url}#faq FAQPage JSON-LD block whenever at least two Q&As exist. Notice that prose edits automatically update the schema, eliminating out-of-sync metadata — see the multilingual hreflang technical SEO guide for the rest of this architecture.

Frequently Asked Questions

Is llms.txt an official Google ranking factor?

No. llms.txt is an open Markdown proposal (llmstxt.org) built for large language models and AI coding/research agents so they can read core pages and rules within a single context window. It complements — rather than replaces — robots.txt and sitemap.xml.

Why keep ai.txt when robots.txt already exists?

robots.txt governs crawl permissions cleanly, whereas ai.txt combines those permissions with explicit site metadata (Author, Languages, primary topic clusters, and canonical slug conventions) in one file for emerging AI directory parsers. Just keep Disallow: paths synchronized across both.

Should I list every blog post inside llms.txt?

No. Dumping hundreds of blog post URLs into llms.txt bloats the token count and defeats the purpose of a curated guide. Include your flagship pages, category hubs, concise product descriptions, URL/slug rules, forbidden backend routes, and links to sitemap.xml plus your RSS feeds.

How does using a single @id (/#person) in JSON-LD help AI citations?

Connecting every BlogPosting, Course, and MobileApplication schema to one shared /#person @id (linked via sameAs profiles) lets knowledge graphs and AI answer engines unify topical authority across your entire body of work instead of fragmenting it across duplicate author objects.

Why generate FAQPage JSON-LD from Markdown instead of maintaining it manually?

Hand-maintained JSON-LD blocks inevitably drift out of date when numbers, dates, or URLs change in the article body. Parsing Q&As directly from the Markdown heading (extractFaqEntries / buildFaqSchema) guarantees zero contradiction between human-readable prose and machine-readable schema.

Sources

  • ahmetbalaman.com live files & repository source: llms.txt, ai.txt, public/llms.txt, public/ai.txt, lib/site.ts, lib/faq-schema.ts, and app/[lang]/layout.tsx.
  • Answer.AI / Jeremy Howard: llmstxt.org (llms.txt specification).
  • Schema.org Specification: Person, BlogPosting, and FAQPage structured data types.

Comments