Sam f6fdc1ebe8 FEATURE: flexible crawler detection
You can use the crawler user agents site setting to amend what user agents
are considered crawlers based on a string match in the user agent

Also improves performance of crawler detection slightly
2017-09-29 12:31:50 +10:00
..
2017-08-31 15:55:56 -04:00
2017-07-28 10:20:09 +09:00
2017-07-28 11:59:12 -04:00
2017-08-24 10:40:14 -04:00
2017-07-28 10:20:09 +09:00
2017-08-21 22:28:43 +09:00
2017-07-28 10:20:09 +09:00
2017-07-28 10:20:09 +09:00
2017-07-28 10:20:09 +09:00