github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-12-13 19:53:51 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Revert "FEATURE: add Noindex to robots.txt for disallowed routes"

Browse Source 
This reverts commit d84256a876.

This is not supported by Google and causes robots.txt to be flagged as
invalid

Removing Noindex
This commit is contained in:
Sam Saffron 2019-07-30 11:33:38 +10:00
parent 85cdf213e1
commit c587df7e2a
2 changed files with 0 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
app/views/robots_txt/index.erb
View File

@ -10,7 +10,6 @@ Crawl-delay: <%= agent[:delay] %>
<%- end -%> <%- end -%>
<% agent[:disallow].each do |path| %> <% agent[:disallow].each do |path| %>
Disallow: <%= path %> Disallow: <%= path %>
Noindex: <%= path %>
<% end %> <% end %>

1
spec/requests/robots_txt_controller_spec.rb
View File

@ -20,7 +20,6 @@ RSpec.describe RobotsTxtController do
Discourse.stubs(:base_uri).returns('/forum') Discourse.stubs(:base_uri).returns('/forum')
get '/robots.txt' get '/robots.txt'
expect(response.body).to include("\nDisallow: /forum/admin") expect(response.body).to include("\nDisallow: /forum/admin")
expect(response.body).to include("\nNoindex: /forum/admin")
end end
end end