Prevent files to be indexed in search engines

Hi,

Is it possible to prevent files from files.onlosant to be indexed in search engines? Normally this can be done with the tag <meta name=”robots” content=”noindex”>, but even adding this setting in the application’s layout header, the files are still indexed. It is not possible to change the html document that is loaded when accessing the file via files.onlosant, which stores the reference to the file stored in AWS. I believe I should change the <meta> tag in this document, not the layout, right?

Thanks!

So do you have an experience page that is rendering links to application files, and it is the content of those files that are getting indexed despite having <meta name=”robots” content=”noindex”> in the <head> of your page / layout?

If so, I think the issue is that you have “noindex”, which tells crawlers not to index the current page, but you are not including “nofollow” as well, which tells crawlers not to follow any links it encounters on the page (independent of the indexing setting).

Try changing your tag to <meta name=”robots” content=”noindex,nofollow”> or just <meta name=”robots” content=”none”>.

I would check out this article on the “robots” meta tag and some alternative approaches, such as including the instructions in an HTTP header.

One note: It is up to the search crawler to honor the value of the “robots” meta tag; the browser won’t actually stop the crawling from happening. Therefore, even if you have these tags correctly implemented, you should bear in mind that some search engine could theoretically still pick up the content of the files.