This ensures that the hostname rails uses for various helpers always matches
the Discourse hostname
# Conflicts:
# config/application.rb
# spec/requests/application_controller_spec.rb
previously admin got a free pass and could set theme via cookie to anything
including themes that are not selectable
this refactor ensures that only "preview" gets a free pass, all the rest
goes through the same pipeline
- allow to specify 410 vs 404 in Discourse::NotFound exception
- remove unused `permalink_redirect_or_not_found` which
- handle JS side links to topics via Discourse-Xhr-Redirect mechanism
This refinement of previous fix moves the crawler blocking into
anonymous cache
This ensures we never poison the cache incorrectly when blocking crawlers