Hi @JVankat , I'm trying to prevent some archived content to show in the search results. I have them set as noindex, noarchive, nofollow, but they still show up.
I then tried adding the following in the NoIndex Xpaths in the crawler settings:
and when that didn't work, I tried with an ID insead:
But the pages keep showing in the result pages. Here is are examples of such a pages:
https://cotesaintluc.org/announcements/parkhaven-pool-and-wading-pools-june-22/ (this one seems to have been removed from the results for now)
So I'm guessing I am misunderstanding something. Can you help? Thanks!
//p[contains(@class, 'archive_notice')](otherwise we'd expect the full class name to match the
archive_noticestring). Manually re-indexing the first link did remove it from the index, meaning the xpaths seem to be set up correctly. I've started another re-index for your site, can you please double-check the index and confirm the archived pages aren't indexed anymore?
<meta name="robots" content="...">tags in the dom, and we only process the first one. https://snipboard.io/FSsTj9.jpg
@JVankat our index hasn't updated since the 12th July, and I can't seem to individually index a URL such as https://www.eastriding.gov.uk/environment/
*query to make a "all results" search, but maybe in your use-case querying the
/sites/indexedContentendpoint would make more sense? See our api docs on this https://docs.sitesearch360.com/api
var e=document.createElement("script");e.async=!0,e.src="https://cdn.sitesearch360.com/v13/sitesearch360-v13.min.js",document.getElementsByTagName("body").appendChild(e);part of the code directly in the navb.js instead of adding it as a part of the script tag inner text?
Sorry, forgot to change one thing, please try the following: https://pastebin.com/TAcZsa1K
And let me look into the rest of your questions tomorrow morning, need a fresh pair of eyes.