FAQ
Questions
Scans
- Why would AI crawlers miss part of my page?
- Many of them fetch the HTML and don't run JavaScript. Text that a page adds with scripts, such as prices, menus or product details, isn't in what they get.
- What does a scan do?
- It loads the page in a headless browser that runs its JavaScript and returns the whole page as markdown (with an account). It also fetches the plain HTML, compares the two, and lists the parts that only appear with JavaScript. And it reads the site's robots.txt for the main AI crawlers, and checks whether up to three PDFs the page links to on the same site contain text.
- Do I need an account?
- Not to scan. Anyone can scan a public page, one at a time, and see what only appears with JavaScript, what robots.txt allows and whether linked PDFs have text. The page as markdown needs an account.
- Can I scan a site that isn't mine?
- Yes, one public page at a time, as long as the site's robots.txt lets nectr's crawler in.
- How do I keep nectr off my site?
- Disallow Nectrbot in your robots.txt. A scan reads it before fetching anything and stops if the page is disallowed. For anything else, get in touch.
Your data
- Does nectr keep the pages I scan?
- No. A scan's result goes back to you only. nectr keeps a daily count of scans, with no address or result in it.
- Will nectr keep pages of my site?
- Only if you verify your domain and use Integrate. It will keep a markdown version of each page, its last 10 earlier versions, and daily counts of who reads them, with no IP address or user agent. They are deleted when you remove the site, or 30 days after it stops being verified.
- Does nectr train AI on content?
- No. nectr doesn't train models or sell content.
- Where does nectr run?
- Its servers, backups and email are in Switzerland, with Infomaniak. Payments go through Paddle.
Coming
- When will Integrate, Publish and Monitor open?
- There are no dates yet. Each one will be announced on the updates page when it opens.
- Will they need a verified domain?
- Yes. nectr will only serve markdown versions or publish data for a site whose owner has verified the domain. Analytics will also need Integrate, because nectr only sees requests for what it serves.
- What about llms.txt?
- Integrate will generate one from your pages and keep it current. Few AI crawlers request llms.txt today, so the markdown versions of the pages matter more.
Another question? Get in touch.