github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-22 09:42:02 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

FEATURE: Apply rate limits per user instead of IP for trusted users (#14706)

Browse Source 
Currently, Discourse rate limits all incoming requests by the IP address they
originate from regardless of the user making the request. This can be
frustrating if there are multiple users using Discourse simultaneously while
sharing the same IP address (e.g. employees in an office).

This commit implements a new feature to make Discourse apply rate limits by
user id rather than IP address for users at or higher than the configured trust
level (1 is the default).

For example, let's say a Discourse instance is configured to allow 200 requests
per minute per IP address, and we have 10 users at trust level 4 using
Discourse simultaneously from the same IP address. Before this feature, the 10
users could only make a total of 200 requests per minute before they got rate
limited. But with the new feature, each user is allowed to make 200 requests
per minute because the rate limits are applied on user id rather than the IP
address.

The minimum trust level for applying user-id-based rate limits can be
configured by the `skip_per_ip_rate_limit_trust_level` global setting. The
default is 1, but it can be changed by either adding the
`DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL` environment variable with the
desired value to your `app.yml`, or changing the setting's value in the
`discourse.conf` file.

Requests made with API keys are still rate limited by IP address and the
relevant global settings that control API keys rate limits.

Before this commit, Discourse's auth cookie (`_t`) was simply a 32 characters
string that Discourse used to lookup the current user from the database and the
cookie contained no additional information about the user. However, we had to
change the cookie content in this commit so we could identify the user from the
cookie without making a database query before the rate limits logic and avoid
introducing a bottleneck on busy sites.

Besides the 32 characters auth token, the cookie now includes the user id,
trust level and the cookie's generation date, and we encrypt/sign the cookie to
prevent tampering.

Internal ticket number: t54739.
This commit is contained in:
Osama Sayegh 2021-11-17 23:27:30 +03:00 committed by GitHub
parent 9be69b603c
commit b86127ad12
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
27 changed files with 973 additions and 293 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
app/controllers/application_controller.rb
View File

@ -188,13 +188,21 @@ class ApplicationController < ActionController::Base
rescue_from RateLimiter::LimitExceeded do |e|
retry_time_in_seconds = e&.available_in
response_headers = {
'Retry-After': retry_time_in_seconds.to_s
}
if e&.error_code
response_headers['Discourse-Rate-Limit-Error-Code'] = e.error_code
end
with_resolved_locale do
render_json_error(
e.description,
type: :rate_limit,
status: 429,
extras: { wait_seconds: retry_time_in_seconds },
headers: { 'Retry-After': retry_time_in_seconds.to_s }
headers: response_headers
)
end
end

18
app/models/user_api_key.rb
View File

@ -31,6 +31,24 @@ class UserApiKey < ActiveRecord::Base
@key.present?
end
def ensure_allowed!(env)
raise Discourse::InvalidAccess.new if !allow?(env)
end
def update_last_used(client_id)
update_args = { last_used_at: Time.zone.now }
if client_id.present? && client_id != self.client_id
# invalidate old dupe api key for client if needed
UserApiKey
.where(client_id: client_id, user_id: self.user_id)
.where('id <> ?', self.id)
.destroy_all
update_args[:client_id] = client_id
end
self.update_columns(**update_args)
end
# Scopes allowed to be requested by external services
def self.allowed_scopes
Set.new(SiteSetting.allow_user_api_key_scopes.split("|"))

3
app/models/user_auth_token.rb
View File

@ -4,7 +4,8 @@ require 'digest/sha1'
class UserAuthToken < ActiveRecord::Base
belongs_to :user
ROTATE_TIME = 10.minutes
ROTATE_TIME_MINS = 10
ROTATE_TIME = ROTATE_TIME_MINS.minutes
# used when token did not arrive at client
URGENT_ROTATE_TIME = 1.minute

3
config/discourse_defaults.conf
View File

@ -239,6 +239,9 @@ max_reqs_per_ip_mode = block
# bypass rate limiting any IP resolved as a private IP
max_reqs_rate_limit_on_private = false
# use per user rate limits vs ip rate limits for users with this trust level or more.
skip_per_ip_rate_limit_trust_level = 1
# logged in DoS protection
# protection will only trigger for requests that queue longer than this amount

6
lib/admin_constraint.rb
View File

@ -8,10 +8,8 @@ class AdminConstraint
def matches?(request)
return false if @require_master && RailsMultisite::ConnectionManagement.current_db != "default"
provider = Discourse.current_user_provider.new(request.env)
provider.current_user &&
provider.current_user.admin? &&
custom_admin_check(request)
current_user = CurrentUser.lookup_from_env(request.env)
current_user&.admin? && custom_admin_check(request)
rescue Discourse::InvalidAccess, Discourse::ReadOnly
false
end

6
lib/auth/current_user_provider.rb
View File

@ -14,12 +14,12 @@ class Auth::CurrentUserProvider
end
# log on a user and set cookies and session etc.
def log_on_user(user, session, cookies, opts = {})
def log_on_user(user, session, cookie_jar, opts = {})
raise NotImplementedError
end
# optional interface to be called to refresh cookies etc if needed
def refresh_session(user, session, cookies)
def refresh_session(user, session, cookie_jar)
end
# api has special rights return true if api was detected
@ -37,7 +37,7 @@ class Auth::CurrentUserProvider
raise NotImplementedError
end
def log_off_user(session, cookies)
def log_off_user(session, cookie_jar)
raise NotImplementedError
end
end

232
lib/auth/default_current_user_provider.rb
View File

@ -1,6 +1,27 @@
# frozen_string_literal: true
require_relative '../route_matcher'
# You may have seen references to v0 and v1 of our auth cookie in the codebase
# and you're not sure how they differ, so here is an explanation:
#
# From the very early days of Discourse, the auth cookie (_t) consisted only of
# a 32 characters random string that Discourse used to identify/lookup the
# current user. We didn't include any metadata with the cookie or encrypt/sign
# it.
#
# That was v0 of the auth cookie until Nov 2021 when we merged a change that
# required us to store additional metadata with the cookie so we could get more
# information about current user early in the request lifecycle before we
# performed database lookup. We also started encrypting and signing the cookie
# to prevent tampering and obfuscate user information that we include in the
# cookie. This is v1 of our auth cookie and we still use it to this date.
#
# We still accept v0 of the auth cookie to keep users logged in, but upon
# cookie rotation (which happen every 10 minutes) they'll be switched over to
# the v1 format.
#
# We'll drop support for v0 after Discourse 2.9 is released.
class Auth::DefaultCurrentUserProvider
CURRENT_USER_KEY ||= "_DISCOURSE_CURRENT_USER"
@ -19,6 +40,9 @@ class Auth::DefaultCurrentUserProvider
PATH_INFO ||= "PATH_INFO"
COOKIE_ATTEMPTS_PER_MIN ||= 10
BAD_TOKEN ||= "_DISCOURSE_BAD_TOKEN"
DECRYPTED_AUTH_COOKIE = "_DISCOURSE_DECRYPTED_AUTH_COOKIE"
TOKEN_SIZE = 32
PARAMETER_API_PATTERNS ||= [
RouteMatcher.new(
@ -52,6 +76,25 @@ class Auth::DefaultCurrentUserProvider
),
]
def self.find_v0_auth_cookie(request)
cookie = request.cookies[TOKEN_COOKIE].presence
if cookie && cookie.size == TOKEN_SIZE
cookie
end
end
def self.find_v1_auth_cookie(env)
return env[DECRYPTED_AUTH_COOKIE] if env.key?(DECRYPTED_AUTH_COOKIE)
env[DECRYPTED_AUTH_COOKIE] = begin
request = ActionDispatch::Request.new(env)
# don't even initialize a cookie jar if we don't have a cookie at all
if request.cookies[TOKEN_COOKIE].present?
request.cookie_jar.encrypted[TOKEN_COOKIE]
end
end
end
# do all current user initialization here
def initialize(env)
@env = env
@ -86,11 +129,10 @@ class Auth::DefaultCurrentUserProvider
api_key ||= request[API_KEY]
end
auth_token = request.cookies[TOKEN_COOKIE] unless user_api_key || api_key
auth_token = find_auth_token
current_user = nil
if auth_token && auth_token.length == 32
if auth_token
limiter = RateLimiter.new(nil, "cookie_auth_#{request.ip}", COOKIE_ATTEMPTS_PER_MIN , 60)
if limiter.can_perform?
@ -128,33 +170,42 @@ class Auth::DefaultCurrentUserProvider
# possible we have an api call, impersonate
if api_key
current_user = lookup_api_user(api_key, request)
raise Discourse::InvalidAccess.new(I18n.t('invalid_api_credentials'), nil, custom_message: "invalid_api_credentials") unless current_user
if !current_user
raise Discourse::InvalidAccess.new(
I18n.t('invalid_api_credentials'),
nil,
custom_message: "invalid_api_credentials"
)
end
raise Discourse::InvalidAccess if current_user.suspended? || !current_user.active
admin_api_key_limiter.performed! if !Rails.env.profile?
@env[API_KEY_ENV] = true
rate_limit_admin_api_requests!
end
# user api key handling
if user_api_key
@hashed_user_api_key = ApiKey.hash_key(user_api_key)
hashed_user_api_key = ApiKey.hash_key(user_api_key)
limiter_min = RateLimiter.new(nil, "user_api_min_#{hashed_user_api_key}", GlobalSetting.max_user_api_reqs_per_minute, 60)
limiter_day = RateLimiter.new(nil, "user_api_day_#{hashed_user_api_key}", GlobalSetting.max_user_api_reqs_per_day, 86400)
user_api_key_obj = UserApiKey
.active
.joins(:user)
.where(key_hash: @hashed_user_api_key)
.includes(:user, :scopes)
.first
unless limiter_day.can_perform?
limiter_day.performed!
end
raise Discourse::InvalidAccess unless user_api_key_obj
unless limiter_min.can_perform?
limiter_min.performed!
end
user_api_key_limiter_60_secs.performed!
user_api_key_limiter_1_day.performed!
current_user = lookup_user_api_user_and_update_key(user_api_key, @env[USER_API_CLIENT_ID])
raise Discourse::InvalidAccess unless current_user
user_api_key_obj.ensure_allowed!(@env)
current_user = user_api_key_obj.user
raise Discourse::InvalidAccess if current_user.suspended? || !current_user.active
limiter_min.performed!
limiter_day.performed!
if can_write?
user_api_key_obj.update_last_used(@env[USER_API_CLIENT_ID])
end
@env[USER_API_KEY_ENV] = true
end
@ -178,7 +229,7 @@ class Auth::DefaultCurrentUserProvider
@env[CURRENT_USER_KEY] = current_user
end
def refresh_session(user, session, cookies)
def refresh_session(user, session, cookie_jar)
# if user was not loaded, no point refreshing session
# it could be an anonymous path, this would add cost
return if is_api? || !@env.key?(CURRENT_USER_KEY)
@ -192,18 +243,18 @@ class Auth::DefaultCurrentUserProvider
if @user_token.rotate!(user_agent: @env['HTTP_USER_AGENT'],
client_ip: @request.ip,
path: @env['REQUEST_PATH'])
cookies[TOKEN_COOKIE] = cookie_hash(@user_token.unhashed_auth_token)
set_auth_cookie!(@user_token.unhashed_auth_token, user, cookie_jar)
DiscourseEvent.trigger(:user_session_refreshed, user)
end
end
end
if !user && cookies.key?(TOKEN_COOKIE)
cookies.delete(TOKEN_COOKIE)
if !user && cookie_jar.key?(TOKEN_COOKIE)
cookie_jar.delete(TOKEN_COOKIE)
end
end
def log_on_user(user, session, cookies, opts = {})
def log_on_user(user, session, cookie_jar, opts = {})
@user_token = UserAuthToken.generate!(
user_id: user.id,
user_agent: @env['HTTP_USER_AGENT'],
@ -212,7 +263,7 @@ class Auth::DefaultCurrentUserProvider
staff: user.staff?,
impersonate: opts[:impersonate])
cookies[TOKEN_COOKIE] = cookie_hash(@user_token.unhashed_auth_token)
set_auth_cookie!(@user_token.unhashed_auth_token, user, cookie_jar)
user.unstage!
make_developer_admin(user)
enable_bootstrap_mode(user)
@ -222,22 +273,29 @@ class Auth::DefaultCurrentUserProvider
@env[CURRENT_USER_KEY] = user
end
def cookie_hash(unhashed_auth_token)
hash = {
value: unhashed_auth_token,
httponly: true,
secure: SiteSetting.force_https
def set_auth_cookie!(unhashed_auth_token, user, cookie_jar)
data = {
token: unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: Time.zone.now.to_i
}
if SiteSetting.persistent_sessions
hash[:expires] = SiteSetting.maximum_session_age.hours.from_now
expires = SiteSetting.maximum_session_age.hours.from_now
end
if SiteSetting.same_site_cookies != "Disabled"
hash[:same_site] = SiteSetting.same_site_cookies
same_site = SiteSetting.same_site_cookies
end
hash
cookie_jar.encrypted[TOKEN_COOKIE] = {
value: data,
httponly: true,
secure: SiteSetting.force_https,
expires: expires,
same_site: same_site
}
end
def make_developer_admin(user)
@ -258,7 +316,7 @@ class Auth::DefaultCurrentUserProvider
end
end
def log_off_user(session, cookies)
def log_off_user(session, cookie_jar)
user = current_user
if SiteSetting.log_out_strict && user
@ -266,7 +324,7 @@ class Auth::DefaultCurrentUserProvider
if user.admin && defined?(Rack::MiniProfiler)
# clear the profiling cookie to keep stuff tidy
cookies.delete("__profilin")
cookie_jar.delete("__profilin")
end
user.logged_out
@ -274,8 +332,8 @@ class Auth::DefaultCurrentUserProvider
@user_token.destroy
end
cookies.delete('authentication_data')
cookies.delete(TOKEN_COOKIE)
cookie_jar.delete('authentication_data')
cookie_jar.delete(TOKEN_COOKIE)
end
# api has special rights return true if api was detected
@ -290,14 +348,13 @@ class Auth::DefaultCurrentUserProvider
end
def has_auth_cookie?
cookie = @request.cookies[TOKEN_COOKIE]
!cookie.nil? && cookie.length == 32
find_auth_token.present?
end
def should_update_last_seen?
return false unless can_write?
api = !!(@env[API_KEY_ENV]) || !!(@env[USER_API_KEY_ENV])
api = !!@env[API_KEY_ENV] || !!@env[USER_API_KEY_ENV]
if @request.xhr? || api
@env["HTTP_DISCOURSE_PRESENT"] == "true"
@ -308,31 +365,6 @@ class Auth::DefaultCurrentUserProvider
protected
def lookup_user_api_user_and_update_key(user_api_key, client_id)
if api_key = UserApiKey.active.with_key(user_api_key).includes(:user, :scopes).first
unless api_key.allow?(@env)
raise Discourse::InvalidAccess
end
if can_write?
api_key.update_columns(last_used_at: Time.zone.now)
if client_id.present? && client_id != api_key.client_id
# invalidate old dupe api key for client if needed
UserApiKey
.where(client_id: client_id, user_id: api_key.user_id)
.where('id <> ?', api_key.id)
.destroy_all
api_key.update_columns(client_id: client_id)
end
end
api_key.user
end
end
def lookup_api_user(api_key_value, request)
if api_key = ApiKey.active.with_key(api_key_value).includes(:user).first
api_username = header_api_key? ? @env[HEADER_API_USERNAME] : request[API_USERNAME]
@ -378,27 +410,61 @@ class Auth::DefaultCurrentUserProvider
!!@env[HEADER_API_KEY]
end
def rate_limit_admin_api_requests!
return if Rails.env == "profile"
limit = GlobalSetting.max_admin_api_reqs_per_minute.to_i
if GlobalSetting.respond_to?(:max_admin_api_reqs_per_key_per_minute)
Discourse.deprecate("DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE is deprecated. Please use DISCOURSE_MAX_ADMIN_API_REQS_PER_MINUTE", drop_from: '2.9.0')
limit = [ GlobalSetting.max_admin_api_reqs_per_key_per_minute.to_i, limit].max
end
global_limit = RateLimiter.new(
nil,
"admin_api_min",
limit,
60
)
global_limit.performed!
end
def can_write?
@can_write ||= !Discourse.pg_readonly_mode?
end
def admin_api_key_limiter
return @admin_api_key_limiter if @admin_api_key_limiter
limit = GlobalSetting.max_admin_api_reqs_per_minute.to_i
if GlobalSetting.respond_to?(:max_admin_api_reqs_per_key_per_minute)
Discourse.deprecate("DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE is deprecated. Please use DISCOURSE_MAX_ADMIN_API_REQS_PER_MINUTE", drop_from: '2.9.0')
limit = [
GlobalSetting.max_admin_api_reqs_per_key_per_minute.to_i,
limit
].max
end
@admin_api_key_limiter = RateLimiter.new(
nil,
"admin_api_min",
limit,
60,
error_code: "admin_api_key_rate_limit"
)
end
def user_api_key_limiter_60_secs
@user_api_key_limiter_60_secs ||= RateLimiter.new(
nil,
"user_api_min_#{@hashed_user_api_key}",
GlobalSetting.max_user_api_reqs_per_minute,
60,
error_code: "user_api_key_limiter_60_secs"
)
end
def user_api_key_limiter_1_day
@user_api_key_limiter_1_day ||= RateLimiter.new(
nil,
"user_api_day_#{@hashed_user_api_key}",
GlobalSetting.max_user_api_reqs_per_day,
86400,
error_code: "user_api_key_limiter_1_day"
)
end
def find_auth_token
return @auth_token if defined?(@auth_token)
@auth_token = begin
if v0 = self.class.find_v0_auth_cookie(@request)
v0
elsif v1 = self.class.find_v1_auth_cookie(@env)
if v1[:issued_at] >= SiteSetting.maximum_session_age.hours.ago.to_i
v1[:token]
end
end
end
end
end

11
lib/guardian.rb
View File

@ -529,9 +529,16 @@ class Guardian
end
def auth_token
if cookie = request&.cookies[Auth::DefaultCurrentUserProvider::TOKEN_COOKIE]
UserAuthToken.hash_token(cookie)
return if !request
token = Auth::DefaultCurrentUserProvider.find_v0_auth_cookie(request).presence
if !token
cookie = Auth::DefaultCurrentUserProvider.find_v1_auth_cookie(request.env)
token = cookie[:token] if cookie
end
UserAuthToken.hash_token(token) if token
end
private

4
lib/homepage_constraint.rb
View File

@ -8,8 +8,8 @@ class HomePageConstraint
def matches?(request)
return @filter == 'finish_installation' if SiteSetting.has_login_hint?
provider = Discourse.current_user_provider.new(request.env)
homepage = provider&.current_user&.user_option&.homepage || SiteSetting.anonymous_homepage
current_user = CurrentUser.lookup_from_env(request.env)
homepage = current_user&.user_option&.homepage || SiteSetting.anonymous_homepage
homepage == @filter
rescue Discourse::InvalidAccess, Discourse::ReadOnly
false

4
lib/middleware/anonymous_cache.rb
View File

@ -49,9 +49,9 @@ module Middleware
ACCEPT_ENCODING = "HTTP_ACCEPT_ENCODING"
DISCOURSE_RENDER = "HTTP_DISCOURSE_RENDER"
def initialize(env)
def initialize(env, request = nil)
@env = env
@request = Rack::Request.new(@env)
@request = request || Rack::Request.new(@env)
end
def blocked_crawler?

234
lib/middleware/request_tracker.rb
View File

@ -4,7 +4,6 @@ require 'method_profiler'
require 'middleware/anonymous_cache'
class Middleware::RequestTracker
@@detailed_request_loggers = nil
@@ip_skipper = nil
@ -56,6 +55,10 @@ class Middleware::RequestTracker
@@ip_skipper = blk
end
def self.ip_skipper
@@ip_skipper
end
def initialize(app, settings = {})
@app = app
end
@ -92,23 +95,25 @@ class Middleware::RequestTracker
end
end
def self.get_data(env, result, timing)
def self.get_data(env, result, timing, request = nil)
status, headers = result
status = status.to_i
helper = Middleware::AnonymousCache::Helper.new(env)
request = Rack::Request.new(env)
request ||= Rack::Request.new(env)
helper = Middleware::AnonymousCache::Helper.new(env, request)
env_track_view = env["HTTP_DISCOURSE_TRACK_VIEW"]
track_view = status == 200
track_view &&= env_track_view != "0" && env_track_view != "false"
track_view &&= env_track_view || (request.get? && !request.xhr? && headers["Content-Type"] =~ /text\/html/)
track_view = !!track_view
has_auth_cookie = Auth::DefaultCurrentUserProvider.find_v0_auth_cookie(request).present?
has_auth_cookie ||= Auth::DefaultCurrentUserProvider.find_v1_auth_cookie(env).present?
h = {
status: status,
is_crawler: helper.is_crawler?,
has_auth_cookie: helper.has_auth_cookie?,
has_auth_cookie: has_auth_cookie,
is_background: !!(request.path =~ /^\/message-bus\// || request.path =~ /\/topics\/timings/),
is_mobile: helper.is_mobile?,
track_view: track_view,
@ -132,9 +137,9 @@ class Middleware::RequestTracker
h
end
def log_request_info(env, result, info)
def log_request_info(env, result, info, request = nil)
# we got to skip this on error ... its just logging
data = self.class.get_data(env, result, info) rescue nil
data = self.class.get_data(env, result, info, request) rescue nil
if data
if result && (headers = result[1])
@ -165,7 +170,7 @@ class Middleware::RequestTracker
def call(env)
result = nil
log_request = true
info = nil
# doing this as early as possible so we have an
# accurate counter
@ -173,14 +178,20 @@ class Middleware::RequestTracker
request = Rack::Request.new(env)
if available_in = rate_limit(request)
return [
429,
{ "Retry-After" => available_in.to_s },
["Slow down, too many requests from this IP address"]
]
cookie = find_auth_cookie(env)
if error_details = rate_limit(request, cookie)
available_in, error_code = error_details
message = <<~TEXT
Slow down, too many requests from this IP address.
Please retry again in #{available_in} seconds.
Error code: #{error_code}.
TEXT
headers = {
"Retry-After" => available_in.to_s,
"Discourse-Rate-Limit-Error-Code" => error_code
}
return [429, headers, [message]]
end
env["discourse.request_tracker"] = self
MethodProfiler.start
@ -222,93 +233,8 @@ class Middleware::RequestTracker
end
end
end
log_request_info(env, result, info) unless !log_request || env["discourse.request_tracker.skip"]
end
def is_private_ip?(ip)
ip = IPAddr.new(ip) rescue nil
!!(ip && (ip.private? || ip.loopback?))
end
def rate_limit(request)
if (
GlobalSetting.max_reqs_per_ip_mode == "block" ||
GlobalSetting.max_reqs_per_ip_mode == "warn" ||
GlobalSetting.max_reqs_per_ip_mode == "warn+block"
)
ip = request.ip
if !GlobalSetting.max_reqs_rate_limit_on_private
return false if is_private_ip?(ip)
end
return false if @@ip_skipper&.call(ip)
return false if STATIC_IP_SKIPPER&.any? { |entry| entry.include?(ip) }
limiter10 = RateLimiter.new(
nil,
"global_ip_limit_10_#{ip}",
GlobalSetting.max_reqs_per_ip_per_10_seconds,
10,
global: true,
aggressive: true
)
limiter60 = RateLimiter.new(
nil,
"global_ip_limit_60_#{ip}",
GlobalSetting.max_reqs_per_ip_per_minute,
60,
global: true,
aggressive: true
)
limiter_assets10 = RateLimiter.new(
nil,
"global_ip_limit_10_assets_#{ip}",
GlobalSetting.max_asset_reqs_per_ip_per_10_seconds,
10,
global: true
)
request.env['DISCOURSE_RATE_LIMITERS'] = [limiter10, limiter60]
request.env['DISCOURSE_ASSET_RATE_LIMITERS'] = [limiter_assets10]
warn = GlobalSetting.max_reqs_per_ip_mode == "warn" || GlobalSetting.max_reqs_per_ip_mode == "warn+block"
if !limiter_assets10.can_perform?
if warn
Discourse.warn("Global asset IP rate limit exceeded for #{ip}: 10 second rate limit", uri: request.env["REQUEST_URI"])
end
if GlobalSetting.max_reqs_per_ip_mode != "warn"
return limiter_assets10.seconds_to_wait(Time.now.to_i)
else
return false
end
end
begin
type = 10
limiter10.performed!
type = 60
limiter60.performed!
false
rescue RateLimiter::LimitExceeded => e
if warn
Discourse.warn("Global IP rate limit exceeded for #{ip}: #{type} second rate limit", uri: request.env["REQUEST_URI"])
if GlobalSetting.max_reqs_per_ip_mode != "warn"
e.available_in
else
false
end
else
e.available_in
end
end
if !env["discourse.request_tracker.skip"]
log_request_info(env, result, info, request)
end
end
@ -319,4 +245,108 @@ class Middleware::RequestTracker
end
end
end
def find_auth_cookie(env)
min_allowed_timestamp = Time.now.to_i - (UserAuthToken::ROTATE_TIME_MINS + 1) * 60
cookie = Auth::DefaultCurrentUserProvider.find_v1_auth_cookie(env)
if cookie && cookie[:issued_at] >= min_allowed_timestamp
cookie
end
end
def is_private_ip?(ip)
ip = IPAddr.new(ip)
!!(ip && (ip.private? || ip.loopback?))
rescue IPAddr::AddressFamilyError, IPAddr::InvalidAddressError
false
end
def rate_limit(request, cookie)
warn = GlobalSetting.max_reqs_per_ip_mode == "warn" ||
GlobalSetting.max_reqs_per_ip_mode == "warn+block"
block = GlobalSetting.max_reqs_per_ip_mode == "block" ||
GlobalSetting.max_reqs_per_ip_mode == "warn+block"
return if !block && !warn
ip = request.ip
if !GlobalSetting.max_reqs_rate_limit_on_private
return if is_private_ip?(ip)
end
return if @@ip_skipper&.call(ip)
return if STATIC_IP_SKIPPER&.any? { |entry| entry.include?(ip) }
ip_or_id = ip
limit_on_id = false
if cookie && cookie[:user_id] && cookie[:trust_level] && cookie[:trust_level] >= GlobalSetting.skip_per_ip_rate_limit_trust_level
ip_or_id = cookie[:user_id]
limit_on_id = true
end
limiter10 = RateLimiter.new(
nil,
"global_ip_limit_10_#{ip_or_id}",
GlobalSetting.max_reqs_per_ip_per_10_seconds,
10,
global: !limit_on_id,
aggressive: true,
error_code: limit_on_id ? "id_10_secs_limit" : "ip_10_secs_limit"
)
limiter60 = RateLimiter.new(
nil,
"global_ip_limit_60_#{ip_or_id}",
GlobalSetting.max_reqs_per_ip_per_minute,
60,
global: !limit_on_id,
error_code: limit_on_id ? "id_60_secs_limit" : "ip_60_secs_limit",
aggressive: true
)
limiter_assets10 = RateLimiter.new(
nil,
"global_ip_limit_10_assets_#{ip_or_id}",
GlobalSetting.max_asset_reqs_per_ip_per_10_seconds,
10,
error_code: limit_on_id ? "id_assets_10_secs_limit" : "ip_assets_10_secs_limit",
global: !limit_on_id
)
request.env['DISCOURSE_RATE_LIMITERS'] = [limiter10, limiter60]
request.env['DISCOURSE_ASSET_RATE_LIMITERS'] = [limiter_assets10]
if !limiter_assets10.can_perform?
if warn
Discourse.warn("Global asset IP rate limit exceeded for #{ip}: 10 second rate limit", uri: request.env["REQUEST_URI"])
end
if block
return [
limiter_assets10.seconds_to_wait(Time.now.to_i),
limiter_assets10.error_code
]
end
end
begin
type = 10
limiter10.performed!
type = 60
limiter60.performed!
nil
rescue RateLimiter::LimitExceeded => e
if warn
Discourse.warn("Global IP rate limit exceeded for #{ip}: #{type} second rate limit", uri: request.env["REQUEST_URI"])
end
if block
[e.available_in, e.error_code]
else
nil
end
end
end
end

9
lib/rate_limiter.rb
View File

@ -3,7 +3,7 @@
# A redis backed rate limiter.
class RateLimiter
attr_reader :max, :secs, :user, :key
attr_reader :max, :secs, :user, :key, :error_code
def self.key_prefix
"l-rate-limit3:"
@ -37,7 +37,7 @@ class RateLimiter
"#{RateLimiter.key_prefix}:#{@user && @user.id}:#{type}"
end
def initialize(user, type, max, secs, global: false, aggressive: false)
def initialize(user, type, max, secs, global: false, aggressive: false, error_code: nil)
@user = user
@type = type
@key = build_key(type)
@ -45,6 +45,7 @@ class RateLimiter
@secs = secs
@global = global
@aggressive = aggressive
@error_code = error_code
end
def clear!
@ -55,7 +56,7 @@ class RateLimiter
rate_unlimited? || is_under_limit?
end
def seconds_to_wait(now)
def seconds_to_wait(now = Time.now.to_i)
@secs - age_of_oldest(now)
end
@ -116,7 +117,7 @@ class RateLimiter
now = Time.now.to_i
if ((max || 0) <= 0) || rate_limiter_allowed?(now)
raise RateLimiter::LimitExceeded.new(seconds_to_wait(now), @type) if raise_error
raise RateLimiter::LimitExceeded.new(seconds_to_wait(now), @type, @error_code) if raise_error
false
else
true

5
lib/rate_limiter/limit_exceeded.rb
View File

@ -16,11 +16,12 @@ class RateLimiter
# A rate limit has been exceeded.
class LimitExceeded < StandardError
attr_reader :type, :available_in
attr_reader :type, :available_in, :error_code
def initialize(available_in, type = nil)
def initialize(available_in, type = nil, error_code = nil)
@available_in = available_in
@type = type
@error_code = error_code
end
def time_left

6
lib/staff_constraint.rb
View File

@ -3,10 +3,8 @@
class StaffConstraint
def matches?(request)
provider = Discourse.current_user_provider.new(request.env)
provider.current_user &&
provider.current_user.staff? &&
custom_staff_check(request)
current_user = CurrentUser.lookup_from_env(request.env)
current_user&.staff? && custom_staff_check(request)
rescue Discourse::InvalidAccess, Discourse::ReadOnly
false
end

188
spec/components/auth/default_current_user_provider_spec.rb
View File

@ -13,16 +13,42 @@ describe Auth::DefaultCurrentUserProvider do
def initialize(env)
super(env)
end
def cookie_jar
@cookie_jar ||= ActionDispatch::Request.new(env).cookie_jar
end
end
def provider(url, opts = nil)
opts ||= { method: "GET" }
env = Rack::MockRequest.env_for(url, opts)
env = create_request_env(path: url).merge(opts)
TestProvider.new(env)
end
def get_cookie_info(cookie_jar, name)
headers = {}
cookie_jar.always_write_cookie = true
cookie_jar.write(headers)
header = headers["Set-Cookie"]
return if header.nil?
info = {}
line = header.split("\n").find { |l| l.start_with?("#{name}=") }
parts = line.split(";").map(&:strip)
info[:value] = parts.shift.split("=")[1]
parts.each do |p|
key, value = p.split("=")
info[key.downcase.to_sym] = value || true
end
info
end
it "can be used to pretend that a user doesn't exist" do
provider = TestProvider.new({})
provider = TestProvider.new(create_request_env(path: "/"))
expect(provider.current_user).to eq(nil)
end
@ -234,11 +260,10 @@ describe Auth::DefaultCurrentUserProvider do
end
describe "#current_user" do
let(:unhashed_token) do
let(:cookie) do
new_provider = provider('/')
cookies = {}
new_provider.log_on_user(user, {}, cookies)
cookies["_t"][:value]
new_provider.log_on_user(user, {}, new_provider.cookie_jar)
new_provider.cookie_jar["_t"]
end
before do
@ -251,7 +276,7 @@ describe Auth::DefaultCurrentUserProvider do
end
it "should not update last seen for suspended users" do
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
u = provider2.current_user
u.reload
expect(u.last_seen_at).to eq_time(Time.zone.now)
@ -264,7 +289,7 @@ describe Auth::DefaultCurrentUserProvider do
u.clear_last_seen_cache!
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
expect(provider2.current_user).to eq(nil)
u.reload
@ -281,7 +306,7 @@ describe Auth::DefaultCurrentUserProvider do
end
it "should not update User#last_seen_at" do
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
u = provider2.current_user
u.reload
expect(u.last_seen_at).to eq(nil)
@ -324,19 +349,26 @@ describe Auth::DefaultCurrentUserProvider do
SiteSetting.persistent_sessions = false
@provider = provider('/')
cookies = {}
@provider.log_on_user(user, {}, cookies)
@provider.log_on_user(user, {}, @provider.cookie_jar)
expect(cookies["_t"][:expires]).to eq(nil)
cookie_info = get_cookie_info(@provider.cookie_jar, "_t")
expect(cookie_info[:expires]).to eq(nil)
end
it "v0 of auth cookie is still acceptable" do
token = UserAuthToken.generate!(user_id: user.id).unhashed_auth_token
ip = "10.0.0.1"
env = { "HTTP_COOKIE" => "_t=#{token}", "REMOTE_ADDR" => ip }
expect(provider('/', env).current_user.id).to eq(user.id)
end
it "correctly rotates tokens" do
SiteSetting.maximum_session_age = 3
@provider = provider('/')
cookies = {}
@provider.log_on_user(user, {}, cookies)
@provider.log_on_user(user, {}, @provider.cookie_jar)
unhashed_token = cookies["_t"][:value]
cookie = @provider.cookie_jar["_t"]
unhashed_token = decrypt_auth_cookie(cookie)[:token]
token = UserAuthToken.find_by(user_id: user.id)
@ -347,15 +379,19 @@ describe Auth::DefaultCurrentUserProvider do
# at this point we are going to try to rotate token
freeze_time 20.minutes.from_now
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
provider2.current_user
token.reload
expect(token.auth_token_seen).to eq(true)
cookies = {}
provider2.refresh_session(user, {}, cookies)
expect(cookies["_t"][:value]).not_to eq(unhashed_token)
provider2.refresh_session(user, {}, provider2.cookie_jar)
expect(
decrypt_auth_cookie(provider2.cookie_jar["_t"])[:token]
).not_to eq(unhashed_token)
expect(
decrypt_auth_cookie(provider2.cookie_jar["_t"])[:token].size
).to eq(32)
token.reload
expect(token.auth_token_seen).to eq(false)
@ -366,10 +402,10 @@ describe Auth::DefaultCurrentUserProvider do
unverified_token = token.auth_token
# old token should still work
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
expect(provider2.current_user.id).to eq(user.id)
provider2.refresh_session(user, {}, cookies)
provider2.refresh_session(user, {}, provider2.cookie_jar)
token.reload
@ -394,23 +430,23 @@ describe Auth::DefaultCurrentUserProvider do
it "fires event when updating last seen" do
@provider = provider('/')
cookies = {}
@provider.log_on_user(user, {}, cookies)
unhashed_token = cookies["_t"][:value]
@provider.log_on_user(user, {}, @provider.cookie_jar)
cookie = @provider.cookie_jar["_t"]
unhashed_token = decrypt_auth_cookie(cookie)[:token]
freeze_time 20.minutes.from_now
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2.refresh_session(user, {}, {})
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
provider2.refresh_session(user, {}, provider2.cookie_jar)
expect(@refreshes).to eq(1)
end
it "does not fire an event when last seen does not update" do
@provider = provider('/')
cookies = {}
@provider.log_on_user(user, {}, cookies)
unhashed_token = cookies["_t"][:value]
@provider.log_on_user(user, {}, @provider.cookie_jar)
cookie = @provider.cookie_jar["_t"]
unhashed_token = decrypt_auth_cookie(cookie)[:token]
freeze_time 2.minutes.from_now
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{unhashed_token}")
provider2.refresh_session(user, {}, {})
provider2 = provider("/", "HTTP_COOKIE" => "_t=#{cookie}")
provider2.refresh_session(user, {}, provider2.cookie_jar)
expect(@refreshes).to eq(0)
end
end
@ -423,14 +459,28 @@ describe Auth::DefaultCurrentUserProvider do
it "can only try 10 bad cookies a minute" do
token = UserAuthToken.generate!(user_id: user.id)
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago
)
provider('/').log_on_user(user, {}, {})
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
RateLimiter.new(nil, "cookie_auth_10.0.0.1", 10, 60).clear!
RateLimiter.new(nil, "cookie_auth_10.0.0.2", 10, 60).clear!
ip = "10.0.0.1"
env = { "HTTP_COOKIE" => "_t=#{SecureRandom.hex}", "REMOTE_ADDR" => ip }
bad_cookie = create_auth_cookie(
token: SecureRandom.hex,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago,
)
env = { "HTTP_COOKIE" => "_t=#{bad_cookie}", "REMOTE_ADDR" => ip }
10.times do
provider('/', env).current_user
@ -441,7 +491,7 @@ describe Auth::DefaultCurrentUserProvider do
}.to raise_error(Discourse::InvalidAccess)
expect {
env["HTTP_COOKIE"] = "_t=#{token.unhashed_auth_token}"
env["HTTP_COOKIE"] = "_t=#{cookie}"
provider("/", env).current_user
}.to raise_error(Discourse::InvalidAccess)
@ -454,14 +504,23 @@ describe Auth::DefaultCurrentUserProvider do
end
it "correctly removes invalid cookies" do
cookies = { "_t" => SecureRandom.hex }
provider('/').refresh_session(nil, {}, cookies)
expect(cookies.key?("_t")).to eq(false)
bad_cookie = create_auth_cookie(
token: SecureRandom.hex,
user_id: 1,
trust_level: 4,
issued_at: 5.minutes.ago,
)
@provider = provider('/')
@provider.cookie_jar["_t"] = bad_cookie
@provider.refresh_session(nil, {}, @provider.cookie_jar)
expect(@provider.cookie_jar.key?("_t")).to eq(false)
end
it "logging on user always creates a new token" do
provider('/').log_on_user(user, {}, {})
provider('/').log_on_user(user, {}, {})
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
@provider2 = provider('/')
@provider2.log_on_user(user, {}, @provider2.cookie_jar)
expect(UserAuthToken.where(user_id: user.id).count).to eq(2)
end
@ -484,7 +543,8 @@ describe Auth::DefaultCurrentUserProvider do
expect(UserAuthToken.where(auth_token: (1..3).map { |i| "abc#{i}" }).count).to eq(3)
# On next login, gets fixed
provider('/').log_on_user(user, {}, {})
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
expect(UserAuthToken.where(user_id: user.id).count).to eq(UserAuthToken::MAX_SESSION_COUNT)
# Oldest sessions are 1, 2, 3. They should now be deleted
@ -495,38 +555,48 @@ describe Auth::DefaultCurrentUserProvider do
SiteSetting.force_https = false
SiteSetting.same_site_cookies = "Lax"
cookies = {}
provider('/').log_on_user(user, {}, cookies)
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
expect(cookies["_t"][:same_site]).to eq("Lax")
expect(cookies["_t"][:httponly]).to eq(true)
expect(cookies["_t"][:secure]).to eq(false)
cookie_info = get_cookie_info(@provider.cookie_jar, "_t")
expect(cookie_info[:samesite]).to eq("Lax")
expect(cookie_info[:httponly]).to eq(true)
expect(cookie_info.key?(:secure)).to eq(false)
SiteSetting.force_https = true
SiteSetting.same_site_cookies = "Disabled"
cookies = {}
provider('/').log_on_user(user, {}, cookies)
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
expect(cookies["_t"][:secure]).to eq(true)
expect(cookies["_t"].key?(:same_site)).to eq(false)
cookie_info = get_cookie_info(@provider.cookie_jar, "_t")
expect(cookie_info[:secure]).to eq(true)
expect(cookie_info.key?(:same_site)).to eq(false)
end
it "correctly expires session" do
SiteSetting.maximum_session_age = 2
token = UserAuthToken.generate!(user_id: user.id)
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago
)
provider('/').log_on_user(user, {}, {})
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
expect(provider("/", "HTTP_COOKIE" => "_t=#{token.unhashed_auth_token}").current_user.id).to eq(user.id)
expect(provider("/", "HTTP_COOKIE" => "_t=#{cookie}").current_user.id).to eq(user.id)
freeze_time 3.hours.from_now
expect(provider("/", "HTTP_COOKIE" => "_t=#{token.unhashed_auth_token}").current_user).to eq(nil)
expect(provider("/", "HTTP_COOKIE" => "_t=#{cookie}").current_user).to eq(nil)
end
it "always unstage users" do
user.update!(staged: true)
provider("/").log_on_user(user, {}, {})
@provider = provider("/")
@provider.log_on_user(user, {}, @provider.cookie_jar)
user.reload
expect(user.staged).to eq(false)
end
@ -658,4 +728,16 @@ describe Auth::DefaultCurrentUserProvider do
end
end
end
it "ignores a valid auth cookie that has been tampered with" do
@provider = provider('/')
@provider.log_on_user(user, {}, @provider.cookie_jar)
cookie = @provider.cookie_jar["_t"]
cookie = swap_2_different_characters(cookie)
ip = "10.0.0.1"
env = { "HTTP_COOKIE" => "_t=#{cookie}", "REMOTE_ADDR" => ip }
expect(provider('/', env).current_user).to eq(nil)
end
end

11
spec/components/current_user_spec.rb
View File

@ -7,7 +7,16 @@ describe CurrentUser do
user = Fabricate(:user, active: true)
token = UserAuthToken.generate!(user_id: user.id)
env = Rack::MockRequest.env_for("/test", "HTTP_COOKIE" => "_t=#{token.unhashed_auth_token};")
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago,
)
env = create_request_env(path: "/test").merge(
"HTTP_COOKIE" => "_t=#{cookie};"
)
expect(CurrentUser.lookup_from_env(env)).to eq(user)
end

19
spec/components/guardian_spec.rb
View File

@ -3846,9 +3846,24 @@ describe Guardian do
describe '#auth_token' do
it 'returns the correct auth token' do
token = UserAuthToken.generate!(user_id: user.id)
env = Rack::MockRequest.env_for("/", "HTTP_COOKIE" => "_t=#{token.unhashed_auth_token};")
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago,
)
env = create_request_env(path: "/").merge("HTTP_COOKIE" => "_t=#{cookie};")
guardian = Guardian.new(user, Rack::Request.new(env))
guardian = Guardian.new(user, ActionDispatch::Request.new(env))
expect(guardian.auth_token).to eq(token.auth_token)
end
it 'supports v0 of auth cookie' do
token = UserAuthToken.generate!(user_id: user.id)
cookie = token.unhashed_auth_token
env = create_request_env(path: "/").merge("HTTP_COOKIE" => "_t=#{cookie};")
guardian = Guardian.new(user, ActionDispatch::Request.new(env))
expect(guardian.auth_token).to eq(token.auth_token)
end
end

2
spec/components/hijack_spec.rb
View File

@ -58,7 +58,7 @@ describe Hijack do
end
end
env = {}
env = create_request_env(path: "/")
middleware = Middleware::RequestTracker.new(app)
middleware.call(env)

27
spec/components/middleware/anonymous_cache_spec.rb
View File

@ -6,7 +6,7 @@ describe Middleware::AnonymousCache do
let(:middleware) { Middleware::AnonymousCache.new(lambda { |_| [200, {}, []] }) }
def env(opts = {})
Rack::MockRequest.env_for("http://test.com/path?bla=1").merge(opts)
create_request_env(path: "http://test.com/path?bla=1").merge(opts)
end
describe Middleware::AnonymousCache::Helper do
@ -23,8 +23,15 @@ describe Middleware::AnonymousCache do
expect(new_helper("ANON_CACHE_DURATION" => 10, "REQUEST_METHOD" => "POST").cacheable?).to eq(false)
end
it "is false if it has an auth cookie" do
expect(new_helper("HTTP_COOKIE" => "jack=1; _t=#{"1" * 32}; jill=2").cacheable?).to eq(false)
it "is false if it has a valid auth cookie" do
cookie = create_auth_cookie(token: SecureRandom.hex)
expect(new_helper("HTTP_COOKIE" => "jack=1; _t=#{cookie}; jill=2").cacheable?).to eq(false)
end
it "is true if it has an invalid auth cookie" do
cookie = create_auth_cookie(token: SecureRandom.hex, issued_at: 5.minutes.ago)
cookie = swap_2_different_characters(cookie)
expect(new_helper("HTTP_COOKIE" => "jack=1; _t=#{cookie}; jill=2").cacheable?).to eq(true)
end
it "is false for srv/status routes" do
@ -142,14 +149,15 @@ describe Middleware::AnonymousCache do
global_setting :background_requests_max_queue_length, "0.5"
env = {
"HTTP_COOKIE" => "_t=#{SecureRandom.hex}",
cookie = create_auth_cookie(token: SecureRandom.hex)
env = create_request_env.merge(
"HTTP_COOKIE" => "_t=#{cookie}",
"HOST" => "site.com",
"REQUEST_METHOD" => "GET",
"REQUEST_URI" => "/somewhere/rainbow",
"REQUEST_QUEUE_SECONDS" => 2.1,
"rack.input" => StringIO.new
}
)
# non background ... long request
env["REQUEST_QUEUE_SECONDS"] = 2
@ -194,15 +202,16 @@ describe Middleware::AnonymousCache do
global_setting :force_anonymous_min_per_10_seconds, 2
global_setting :force_anonymous_min_queue_seconds, 1
env = {
"HTTP_COOKIE" => "_t=#{SecureRandom.hex}",
cookie = create_auth_cookie(token: SecureRandom.hex)
env = create_request_env.merge(
"HTTP_COOKIE" => "_t=#{cookie}",
"HTTP_DISCOURSE_LOGGED_IN" => "true",
"HOST" => "site.com",
"REQUEST_METHOD" => "GET",
"REQUEST_URI" => "/somewhere/rainbow",
"REQUEST_QUEUE_SECONDS" => 2.1,
"rack.input" => StringIO.new
}
)
is_anon = false
app.call(env.dup)

188
spec/components/middleware/request_tracker_spec.rb
View File

@ -3,16 +3,15 @@
require "rails_helper"
describe Middleware::RequestTracker do
def env(opts = {})
{
create_request_env.merge(
"HTTP_HOST" => "http://test.com",
"HTTP_USER_AGENT" => "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36",
"REQUEST_URI" => "/path?bla=1",
"REQUEST_METHOD" => "GET",
"HTTP_ACCEPT" => "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"rack.input" => ""
}.merge(opts)
"rack.input" => StringIO.new
).merge(opts)
end
before do
@ -140,9 +139,15 @@ describe Middleware::RequestTracker do
let(:logged_in_data) do
user = Fabricate(:user, active: true)
token = UserAuthToken.generate!(user_id: user.id)
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago
)
Middleware::RequestTracker.get_data(env(
"HTTP_USER_AGENT" => "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.72 Safari/537.36",
"HTTP_COOKIE" => "_t=#{token.unhashed_auth_token};"
"HTTP_COOKIE" => "_t=#{cookie};"
), ["200", { "Content-Type" => 'text/html' }], 0.1)
end
@ -195,6 +200,7 @@ describe Middleware::RequestTracker do
before do
RateLimiter.enable
RateLimiter.clear_all_global!
RateLimiter.clear_all!
@old_logger = Rails.logger
Rails.logger = TestLogger.new
@ -386,6 +392,177 @@ describe Middleware::RequestTracker do
status, _ = middleware.call(env2)
expect(status).to eq(200)
end
describe "diagnostic information" do
it "is included when the requests-per-10-seconds limit is reached" do
global_setting :max_reqs_per_ip_per_10_seconds, 1
called = 0
app = lambda do |_|
called += 1
[200, {}, ["OK"]]
end
env = env("REMOTE_ADDR" => "1.1.1.1")
middleware = Middleware::RequestTracker.new(app)
status, = middleware.call(env)
expect(status).to eq(200)
expect(called).to eq(1)
env = env("REMOTE_ADDR" => "1.1.1.1")
middleware = Middleware::RequestTracker.new(app)
status, headers, response = middleware.call(env)
expect(status).to eq(429)
expect(called).to eq(1)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq("ip_10_secs_limit")
expect(response.first).to include("Error code: ip_10_secs_limit.")
end
it "is included when the requests-per-minute limit is reached" do
global_setting :max_reqs_per_ip_per_minute, 1
called = 0
app = lambda do |_|
called += 1
[200, {}, ["OK"]]
end
env = env("REMOTE_ADDR" => "1.1.1.1")
middleware = Middleware::RequestTracker.new(app)
status, = middleware.call(env)
expect(status).to eq(200)
expect(called).to eq(1)
env = env("REMOTE_ADDR" => "1.1.1.1")
middleware = Middleware::RequestTracker.new(app)
status, headers, response = middleware.call(env)
expect(status).to eq(429)
expect(called).to eq(1)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq("ip_60_secs_limit")
expect(response.first).to include("Error code: ip_60_secs_limit.")
end
it "is included when the assets-requests-per-10-seconds limit is reached" do
global_setting :max_asset_reqs_per_ip_per_10_seconds, 1
called = 0
app = lambda do |env|
called += 1
env["DISCOURSE_IS_ASSET_PATH"] = true
[200, {}, ["OK"]]
end
env = env("REMOTE_ADDR" => "1.1.1.1")
middleware = Middleware::RequestTracker.new(app)
status, = middleware.call(env)
expect(status).to eq(200)
expect(called).to eq(1)
env = env("REMOTE_ADDR" => "1.1.1.1")
middleware = Middleware::RequestTracker.new(app)
status, headers, response = middleware.call(env)
expect(status).to eq(429)
expect(called).to eq(1)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq("ip_assets_10_secs_limit")
expect(response.first).to include("Error code: ip_assets_10_secs_limit.")
end
end
it "users with high enough trust level are not rate limited per ip" do
global_setting :max_reqs_per_ip_per_minute, 1
global_setting :skip_per_ip_rate_limit_trust_level, 3
envs = 3.times.map do |n|
user = Fabricate(:user, trust_level: 3)
token = UserAuthToken.generate!(user_id: user.id)
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago
)
env("HTTP_COOKIE" => "_t=#{cookie}", "REMOTE_ADDR" => "1.1.1.1")
end
called = 0
app = lambda do |env|
called += 1
[200, {}, ["OK"]]
end
envs.each do |env|
middleware = Middleware::RequestTracker.new(app)
status, = middleware.call(env)
expect(status).to eq(200)
end
expect(called).to eq(3)
envs.each do |env|
middleware = Middleware::RequestTracker.new(app)
status, headers, response = middleware.call(env)
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq("id_60_secs_limit")
expect(response.first).to include("Error code: id_60_secs_limit.")
end
expect(called).to eq(3)
end
it "falls back to IP rate limiting if the cookie is too old" do
unfreeze_time
global_setting :max_reqs_per_ip_per_minute, 1
global_setting :skip_per_ip_rate_limit_trust_level, 3
user = Fabricate(:user, trust_level: 3)
token = UserAuthToken.generate!(user_id: user.id)
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago
)
env = env("HTTP_COOKIE" => "_t=#{cookie}", "REMOTE_ADDR" => "1.1.1.1")
called = 0
app = lambda do |_|
called += 1
[200, {}, ["OK"]]
end
freeze_time(12.minutes.from_now) do
middleware = Middleware::RequestTracker.new(app)
status, = middleware.call(env)
expect(status).to eq(200)
middleware = Middleware::RequestTracker.new(app)
status, headers, response = middleware.call(env)
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq("ip_60_secs_limit")
expect(response.first).to include("Error code: ip_60_secs_limit.")
end
end
it "falls back to IP rate limiting if the cookie is tampered with" do
unfreeze_time
global_setting :max_reqs_per_ip_per_minute, 1
global_setting :skip_per_ip_rate_limit_trust_level, 3
user = Fabricate(:user, trust_level: 3)
token = UserAuthToken.generate!(user_id: user.id)
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: Time.zone.now
)
cookie = swap_2_different_characters(cookie)
env = env("HTTP_COOKIE" => "_t=#{cookie}", "REMOTE_ADDR" => "1.1.1.1")
called = 0
app = lambda do |_|
called += 1
[200, {}, ["OK"]]
end
middleware = Middleware::RequestTracker.new(app)
status, = middleware.call(env)
expect(status).to eq(200)
middleware = Middleware::RequestTracker.new(app)
status, headers, response = middleware.call(env)
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq("ip_60_secs_limit")
expect(response.first).to include("Error code: ip_60_secs_limit.")
end
end
context "callbacks" do
@ -480,5 +657,4 @@ describe Middleware::RequestTracker do
expect(headers["X-Runtime"].to_f).to be > 0
end
end
end

12
spec/jobs/export_user_archive_spec.rb
View File

@ -42,10 +42,12 @@ describe Jobs::ExportUserArchive do
user.user_profile.website = 'https://doe.example.com/john'
user.user_profile.save
# force a UserAuthTokenLog entry
Discourse.current_user_provider.new({
env = create_request_env.merge(
'HTTP_USER_AGENT' => 'MyWebBrowser',
'REQUEST_PATH' => '/some_path/456852',
}).log_on_user(user, {}, {})
)
cookie_jar = ActionDispatch::Request.new(env).cookie_jar
Discourse.current_user_provider.new(env).log_on_user(user, {}, cookie_jar)
# force a nonstandard post action
PostAction.new(user: user, post: post, post_action_type_id: 5).save
@ -198,10 +200,12 @@ describe Jobs::ExportUserArchive do
let(:component) { 'auth_tokens' }
before do
Discourse.current_user_provider.new({
env = create_request_env.merge(
'HTTP_USER_AGENT' => 'MyWebBrowser',
'REQUEST_PATH' => '/some_path/456852',
}).log_on_user(user, {}, {})
)
cookie_jar = ActionDispatch::Request.new(env).cookie_jar
Discourse.current_user_provider.new(env).log_on_user(user, {}, cookie_jar)
end
it 'properly includes session records' do

142
spec/multisite/request_tracker_spec.rb Normal file
View File

@ -0,0 +1,142 @@
# frozen_string_literal: true
require "rails_helper"
describe "RequestTracker in multisite", type: :multisite do
before do
global_setting :skip_per_ip_rate_limit_trust_level, 2
RateLimiter.enable
test_multisite_connection("default") do
RateLimiter.clear_all!
end
test_multisite_connection("second") do
RateLimiter.clear_all!
end
RateLimiter.clear_all_global!
end
def call(env, &block)
Middleware::RequestTracker.new(block).call(env)
end
def create_env(opts)
create_request_env.merge(opts)
end
shared_examples "ip rate limiters behavior" do |error_code, app_callback|
it "applies rate limits on an IP address across all sites" do
called = { default: 0, second: 0 }
test_multisite_connection("default") do
env = create_env("REMOTE_ADDR" => "123.10.71.4")
status, = call(env) do
called[:default] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(200)
env = create_env("REMOTE_ADDR" => "123.10.71.4")
status, headers = call(env) do
called[:default] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq(error_code)
expect(called[:default]).to eq(1)
end
test_multisite_connection("second") do
env = create_env("REMOTE_ADDR" => "123.10.71.4")
status, headers = call(env) do
called[:second] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq(error_code)
expect(called[:second]).to eq(0)
end
end
end
shared_examples "user id rate limiters behavior" do |error_code, app_callback|
it "does not leak rate limits for a user id to other sites" do
cookie = create_auth_cookie(
token: SecureRandom.hex,
user_id: 1,
trust_level: 2
)
called = { default: 0, second: 0 }
test_multisite_connection("default") do
env = create_env("REMOTE_ADDR" => "123.10.71.4", "HTTP_COOKIE" => "_t=#{cookie}")
status, = call(env) do
called[:default] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(200)
env = create_env("REMOTE_ADDR" => "123.10.71.4", "HTTP_COOKIE" => "_t=#{cookie}")
status, headers, = call(env) do
called[:default] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq(error_code)
expect(called[:default]).to eq(1)
end
test_multisite_connection("second") do
env = create_env("REMOTE_ADDR" => "123.10.71.4", "HTTP_COOKIE" => "_t=#{cookie}")
status, = call(env) do
called[:second] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(200)
env = create_env("REMOTE_ADDR" => "123.10.71.4", "HTTP_COOKIE" => "_t=#{cookie}")
status, headers, = call(env) do
called[:second] += 1
app_callback&.call(env)
[200, {}, ["OK"]]
end
expect(status).to eq(429)
expect(headers["Discourse-Rate-Limit-Error-Code"]).to eq(error_code)
expect(called[:second]).to eq(1)
end
end
end
context "10 seconds limiter" do
before do
global_setting :max_reqs_per_ip_per_10_seconds, 1
end
include_examples "ip rate limiters behavior", "ip_10_secs_limit"
include_examples "user id rate limiters behavior", "id_10_secs_limit"
end
context "60 seconds limiter" do
before do
global_setting :max_reqs_per_ip_per_minute, 1
end
include_examples "ip rate limiters behavior", "ip_60_secs_limit"
include_examples "user id rate limiters behavior", "id_60_secs_limit"
end
context "assets 10 seconds limiter" do
before do
global_setting :max_asset_reqs_per_ip_per_10_seconds, 1
end
app_callback = ->(env) { env["DISCOURSE_IS_ASSET_PATH"] = true }
include_examples "ip rate limiters behavior", "ip_assets_10_secs_limit", app_callback
include_examples "user id rate limiters behavior", "id_assets_10_secs_limit", app_callback
end
end

38
spec/rails_helper.rb
View File

@ -459,6 +459,44 @@ ensure
Rails.logger = old_logger
end
# this takes a string and returns a copy where 2 different
# characters are swapped.
# e.g.
# swap_2_different_characters("abc") => "bac"
# swap_2_different_characters("aac") => "caa"
def swap_2_different_characters(str)
swap1 = 0
swap2 = str.split("").find_index { |c| c != str[swap1] }
# if the string is made up of 1 character
return str if !swap2
str = str.dup
str[swap1], str[swap2] = str[swap2], str[swap1]
str
end
def create_request_env(path: nil)
env = Rails.application.env_config.dup
env.merge!(Rack::MockRequest.env_for(path)) if path
env
end
def create_auth_cookie(token:, user_id: nil, trust_level: nil, issued_at: Time.zone.now)
request = ActionDispatch::Request.new(create_request_env)
data = {
token: token,
user_id: user_id,
trust_level: trust_level,
issued_at: issued_at.to_i
}
cookie = request.cookie_jar.encrypted["_t"] = { value: data }
cookie[:value]
end
def decrypt_auth_cookie(cookie)
request = ActionDispatch::Request.new(create_request_env.merge("HTTP_COOKIE" => "_t=#{cookie}"))
request.cookie_jar.encrypted["_t"]
end
class SpecSecureRandom
class << self
attr_accessor :value

63
spec/requests/application_controller_spec.rb
View File

@ -851,4 +851,67 @@ RSpec.describe ApplicationController do
expect(response.headers["Vary"]).to eq(nil)
end
end
describe "Discourse-Rate-Limit-Error-Code header" do
fab!(:admin) { Fabricate(:admin) }
before do
RateLimiter.clear_all!
RateLimiter.enable
end
it "is included when API key is rate limited" do
global_setting :max_admin_api_reqs_per_minute, 1
api_key = ApiKey.create!(user_id: admin.id).key
get "/latest.json", headers: {
"Api-Key": api_key,
"Api-Username": admin.username
}
expect(response.status).to eq(200)
get "/latest.json", headers: {
"Api-Key": api_key,
"Api-Username": admin.username
}
expect(response.status).to eq(429)
expect(response.headers["Discourse-Rate-Limit-Error-Code"]).to eq("admin_api_key_rate_limit")
end
it "is included when user API key is rate limited" do
global_setting :max_user_api_reqs_per_minute, 1
user_api_key = UserApiKey.create!(
user_id: admin.id,
client_id: "",
application_name: "discourseapp"
)
user_api_key.scopes = UserApiKeyScope.all_scopes.keys.map do |name|
UserApiKeyScope.create!(name: name, user_api_key_id: user_api_key.id)
end
user_api_key.save!
get "/session/current.json", headers: {
"User-Api-Key": user_api_key.key,
}
expect(response.status).to eq(200)
get "/session/current.json", headers: {
"User-Api-Key": user_api_key.key,
}
expect(response.status).to eq(429)
expect(
response.headers["Discourse-Rate-Limit-Error-Code"]
).to eq("user_api_key_limiter_60_secs")
global_setting :max_user_api_reqs_per_minute, 100
global_setting :max_user_api_reqs_per_day, 1
get "/session/current.json", headers: {
"User-Api-Key": user_api_key.key,
}
expect(response.status).to eq(429)
expect(
response.headers["Discourse-Rate-Limit-Error-Code"]
).to eq("user_api_key_limiter_1_day")
end
end
end

9
spec/requests/session_controller_spec.rb
View File

@ -1454,7 +1454,8 @@ RSpec.describe SessionController do
expect(session[:current_user_id]).to eq(user.id)
expect(user.user_auth_tokens.count).to eq(1)
expect(UserAuthToken.hash_token(cookies[:_t])).to eq(user.user_auth_tokens.first.auth_token)
unhashed_token = decrypt_auth_cookie(cookies[:_t])[:token]
expect(UserAuthToken.hash_token(unhashed_token)).to eq(user.user_auth_tokens.first.auth_token)
end
context "when timezone param is provided" do
@ -1640,7 +1641,8 @@ RSpec.describe SessionController do
expect(session[:current_user_id]).to eq(user.id)
expect(user.user_auth_tokens.count).to eq(1)
expect(UserAuthToken.hash_token(cookies[:_t]))
unhashed_token = decrypt_auth_cookie(cookies[:_t])[:token]
expect(UserAuthToken.hash_token(unhashed_token))
.to eq(user.user_auth_tokens.first.auth_token)
end
end
@ -1658,7 +1660,8 @@ RSpec.describe SessionController do
expect(session[:current_user_id]).to eq(user.id)
expect(user.user_auth_tokens.count).to eq(1)
expect(UserAuthToken.hash_token(cookies[:_t]))
unhashed_token = decrypt_auth_cookie(cookies[:_t])[:token]
expect(UserAuthToken.hash_token(unhashed_token))
.to eq(user.user_auth_tokens.first.auth_token)
end
end

13
spec/requests/users_controller_spec.rb
View File

@ -4857,11 +4857,18 @@ describe UsersController do
it 'does not let user log out of current session' do
token = UserAuthToken.generate!(user_id: user.id)
env = Rack::MockRequest.env_for("/", "HTTP_COOKIE" => "_t=#{token.unhashed_auth_token};")
Guardian.any_instance.stubs(:request).returns(Rack::Request.new(env))
cookie = create_auth_cookie(
token: token.unhashed_auth_token,
user_id: user.id,
trust_level: user.trust_level,
issued_at: 5.minutes.ago,
)
post "/u/#{user.username}/preferences/revoke-auth-token.json", params: { token_id: token.id }
post "/u/#{user.username}/preferences/revoke-auth-token.json",
params: { token_id: token.id },
headers: { "HTTP_COOKIE" => "_t=#{cookie}" }
expect(token.reload.id).to be_present
expect(response.status).to eq(400)
end

3
spec/support/helpers.rb
View File

@ -14,8 +14,9 @@ module Helpers
end
def log_in_user(user)
cookie_jar = ActionDispatch::Request.new(request.env).cookie_jar
provider = Discourse.current_user_provider.new(request.env)
provider.log_on_user(user, session, cookies)
provider.log_on_user(user, session, cookie_jar)
provider
end