How to Manage Site Indexes

When you index a site with Daneel, the vector data is stored in IndexedDB in your browser. This guide covers how to manage those indexes.

View indexed sites

1. Open Settings > Indexes.
2. You’ll see a list of all indexed domains with:
   • Number of pages crawled
   • Number of chunks embedded
   • Last indexed date

You can also see a summary on the Settings > Home dashboard card.

Re-index a site

To update an index with fresh content:

1. Navigate to the site.
2. Open the search overlay (magnifying glass icon).
3. Click Re-index. Daneel re-crawls using the same discovery method (sitemap or web crawl) and replaces the old chunks.

Alternatively, from Settings > Indexes, click the re-index button next to any domain.

For details on choosing between sitemap and web crawl discovery, see How to Index a Site.

Clear an index

To remove all indexed data for a site:

1. Open Settings > Indexes.
2. Click the Clear button next to the domain.

This deletes all chunks and embeddings for that domain from IndexedDB. The action is immediate and cannot be undone.

You can also clear a domain from the search overlay by clicking the clear button next to the domain stats.

Crawl settings

Adjust default crawl parameters in Settings > Indexes or per-crawl in the search overlay:

Setting	Default	Range	Description
Max pages	150	1–200	Maximum pages to crawl per site
Max depth	3	1–10	How deep to follow links from the sitemap
Max chunks per page	2,000	—	Upper limit on chunks per single page
Chunk size	512 tokens	—	Target size of each text chunk
Overlap	64 tokens	—	Overlap between consecutive chunks

Storage

Indexed data is stored in IndexedDB, partitioned by domain. Each domain’s data is independent — clearing one site doesn’t affect others.

For storage details, see Storage and Limits.

Next steps

• Index Your First Site if you haven’t tried it yet
• Learn how RAG works under the hood
• Back up your data to preserve your indexes