When web content is missing, test one exact article from the public URL through the configured collection path. The problem is usually reachability, source configuration, canonicalization, metadata, filtering, or refresh timing—not simply whether the page works in your own browser.
1. Test the final public article URL#
Open the exact canonical article URL in a private browser window. It must return 200 without a login, consent wall, geographic restriction, 401, 403, 404, or bot challenge. Follow redirects and test the final URL, not only the homepage. A web application firewall (WAF) may allow your browser while blocking a crawler.
If your site restricts crawlers, use the current allowlist and verification endpoints in Set up Nonli on your website. The machine-readable /crawler-ips.json and /crawler-ips.txt endpoints currently advertise IPv4 egress addresses only; Nonli does not currently publish an IPv6 crawler address to allowlist. That guide and those endpoints are the source of truth for the SDK snippet and crawler addresses; do not copy an old address from a support message.
2. Identify the collection path#
Different paths have different prerequisites:
- Nonli SDK: the script must load on every eligible article template and be activated for the brand. In a tag manager, confirm that consent rules, URL exclusions, preview-only environments, or trigger conditions do not skip article pages.
- RSS: the item must be present in the configured feed with a stable URL and publication date. Check the RSS setup guide.
- Direct URL refresh: the final URL must be reachable and its metadata available when refreshed.
- Social listening: the social source and account filters—not your website SDK—determine the result set.
3. Verify domain, canonical URL, and metadata#
Confirm that the website is assigned to the correct brand and that its domain appears in the Source filter. On the article page, inspect the final canonical URL and the visible metadata. A canonical pointing to another page can merge the content under a different URL.
Provide a specific title, description, and image through Open Graph or schema.org markup. Check og:title, og:description, og:image, the canonical link, and article:section or JSON-LD articleSection when categories matter. Relative, protected, expired, or extremely small image URLs may not be usable. The complete metadata precedence and examples remain in the website setup guide.
4. Check theme filters and refresh timing#
Find the article in the source or global data before diagnosing a bot. Then verify the theme's source, domain, language, date, URL, include/exclude term, tag, and category filters. An article can be collected correctly but hidden by one of these filters. Allow the configured collection or bot consultation interval to run before repeatedly refreshing.
If the article is visible in the theme but no automated post appears, continue with Why did my bot not publish this article?.
5. What to send to support#
Send the exact final article URL, the UTC time of the test, source type (SDK, RSS, direct URL, or listening), HTTP status and redirect chain, rendered canonical and metadata values, affected brand and theme, and the first stage where the article disappears. Remove private feed credentials and never send login cookies, API keys, or access tokens.