11 Commits

Author	SHA1	Message	Date
Robin Ward	f655936b9d	SECURITY: Prevent robots from indexing more routes ... These routes could contain sensitive material and should never be indexed for content.	2018-02-04 13:27:33 -05:00
Robin Ward	14410b71fb	Convert server side paths to use /u/	2017-03-30 10:23:24 -04:00
Vinoth Kannan	08c14dd689	new: server plugin outlet for indexable robots.txt	2017-02-13 17:31:10 +05:30
Neil Lalonde	ae671355da	FIX: add /tags routes to robots.txt	2017-02-03 11:57:00 -05:00
Sam	54645261aa	better disallow search ... this could get ugly	2015-04-02 17:08:00 +11:00
Robin Ward	e66c53a4a7	Add /badges to robots.txt for now, we don't have a crawlable view so ... it's better to exclude it.	2014-10-30 14:32:42 -04:00
Neil Lalonde	8267a451b2	Disallow /users/ in robots.txt	2014-05-23 10:28:26 -04:00
Neil Lalonde	9c4dc9a966	Block browser-update.js in robots.txt. Move noscript block above everything else in application layout.	2014-02-14 15:33:00 -05:00
Sam	7ad00f426c	FEATURE REMOVAL: persona login ... see: https://meta.discourse.org/t/pulling-persona-out-of-discourse-core/12613	2014-02-11 16:56:48 +11:00
Neil Lalonde	88d9f3a786	Disallow auth callbacks in robots.txt	2014-01-14 10:42:22 -05:00
Sam Saffron	c50a9e4d01	added support for disabling indexing by google using SiteSetting.allow_index_in_robots_txt = false	2013-02-11 11:02:57 +11:00