github-mirror/discourse
github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-03-22 15:02:02 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

removed broken link and comments from no_index.erb (#25648)

Browse Source 
We intend to move guidance about how to configure robots.txt to the admin interface and a documentation topic on meta.
This commit is contained in:
Tobias Eigen 2024-02-13 20:09:24 -08:00 committed by GitHub
parent cd99e20fd1
commit 709bed5d1c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 0 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
app/views/robots_txt/no_index.erb
View File

@ -1,9 +1,3 @@
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
# Googlebot must be allowed to index so it can remove items from the index
# we return the X-Robots-Tag with noindex, nofollow which will ensure
# indexing is minimized and nothing shows up in Google search results
User-agent: googlebot User-agent: googlebot
Allow: <%= Discourse.base_path + "/" %> Allow: <%= Discourse.base_path + "/" %>
Disallow: <%= Discourse.base_path + "/uploads/*" %> Disallow: <%= Discourse.base_path + "/uploads/*" %>