github-mirror/discourse
github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-03-09 05:45:30 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
discourse/app/views/robots_txt/no_index.erb

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

8 lines
173 B
Plaintext
Raw Normal View History

FEATURE: let Google index pages so it can remove them Google insists on indexing pages so it can figure out if they can be removed from the index. see: https://support.google.com/webmasters/answer/6332384?hl=en This change ensures the we have special behavior for Googlebot where we allow indexing, but block the actual indexing via X-Robots-Tag
2020-05-11 12:14:21 +10:00
User-agent: googlebot
Replace `base_uri` with `base_path` (#10879) DEV: Replace instances of Discourse.base_uri with Discourse.base_path This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Allow: <%= Discourse.base_path + "/" %>
Disallow: <%= Discourse.base_path + "/uploads/*" %>
FEATURE: let Google index pages so it can remove them Google insists on indexing pages so it can figure out if they can be removed from the index. see: https://support.google.com/webmasters/answer/6332384?hl=en This change ensures the we have special behavior for Googlebot where we allow indexing, but block the actual indexing via X-Robots-Tag
2020-05-11 12:14:21 +10:00
Fix all the errors to get our tests green on Rails 5.1.
2017-08-31 12:06:56 +08:00
User-agent: *
Replace `base_uri` with `base_path` (#10879) DEV: Replace instances of Discourse.base_uri with Discourse.base_path This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Disallow: <%= Discourse.base_path + "/" %>
FEATURE: let Google index pages so it can remove them Google insists on indexing pages so it can figure out if they can be removed from the index. see: https://support.google.com/webmasters/answer/6332384?hl=en This change ensures the we have special behavior for Googlebot where we allow indexing, but block the actual indexing via X-Robots-Tag
2020-05-11 12:14:21 +10:00
Reference in New Issue Copy Permalink