Sam 758e160862

FEATURE: explicitly ban outlier traffic sources in robots.txt (#11553) ...

Googlebot handles no-index headers very elegantly. It advises to leave as many routes as possible open and uses headers for high fidelity rules regarding indexes.

Discourse adds special `x-robot-tags` noindex headers to users, badges, groups, search and tag routes.

Following up on b52143feff8c32f2 we now have it so Googlebot gets special handling.

Rest of the crawlers get a far more aggressive disallow list to protect against excessive crawling.

2020-12-23 08:51:14 +11:00

..

components

FIX: Disallow zero-width and other non-printing characters in tags (#11546)

2020-12-22 09:27:37 -06:00

fabricators

FEATURE: Do not disturb (#11484)

2020-12-18 09:03:51 -06:00

fixtures

Onebox improved error handling and support for Instagram Access Tokens (#11253)

2020-11-18 12:55:16 -05:00

helpers

UX: Use appropriate logo on static pages (#11211)

2020-11-12 10:50:55 -08:00

import_export

FEATURE: Rake task to export groups (#9450)

2020-04-17 14:59:54 -07:00

initializers

FIX: We need to skip users with associated reviewables when auto-approving (#9080)

2020-03-02 14:33:52 -05:00

integration

PERF: we don't need to use a huge image to test thumbnails (#11025)

2020-10-27 12:39:52 +11:00

integrity

DEV: Check English locale for errors in CI

2020-06-03 21:54:58 +02:00

jobs

FIX: Don't enqueue imported users when there're multiple custom fields. (#11559)

2020-12-22 14:28:07 -03:00

lib

FIX: Only cache reports with exceptions for 1 minute (#11447)

2020-12-09 10:54:41 -06:00

mailers

FIX: Ensure group SMTP and message builder always uses from address for Reply-To when IMAP is enabled (#11037)

2020-10-28 07:01:58 +10:00

models

Revert "FIX: use Redis replica host and port (#11543)" (#11552)

2020-12-22 16:16:50 +11:00

multisite

DEV: Clean up S3 specs, stubs, and helpers

2020-09-28 12:02:25 +01:00

requests

FEATURE: explicitly ban outlier traffic sources in robots.txt (#11553)

2020-12-23 08:51:14 +11:00

script/import_scripts

FIX: improve Vanilla importing (#10478)

2020-08-24 16:19:57 -04:00

serializers

FIX: Don't raise an error if a custom emoji image was deleted

2020-12-09 15:40:40 -05:00

services

FIX: handle badge grant for deleted badge (#11558)

2020-12-22 20:24:51 +05:30

support

DEV: Upgrade oj gem (#11516)

2020-12-17 11:18:45 +11:00

tasks

DEV: Clean up S3 specs, stubs, and helpers

2020-09-28 12:02:25 +01:00

views/omniauth_callbacks

FEATURE: Use full page redirection for all external auth methods (#8092)

2019-10-08 12:10:43 +01:00

rails_helper.rb

DEV: Update DB.after_commit to be compatible with 'real' transactions (#11294)

2020-12-08 11:03:31 +11:00

swagger_helper.rb

DEV: Add rswag to aid in api documention (#9546)

2020-04-27 16:40:07 -06:00