Sam Saffron 5feb342914 Revert "FEATURE: add Noindex to robots.txt for disallowed routes" ...

This reverts commit d84256a876a9fa4fc7bcb4b8ac8c5865f8c10701.

This is not supported by Google and causes robots.txt to be flagged as
invalid

Removing Noindex

2019-07-30 11:33:38 +10:00

..

index.erb

Revert "FEATURE: add Noindex to robots.txt for disallowed routes"

2019-07-30 11:33:38 +10:00

no_index.erb

prefix the robots.txt rules with the directory when using subfolder

2018-04-11 22:05:02 +02:00