discourse/lib
Osama Sayegh b86127ad12
FEATURE: Apply rate limits per user instead of IP for trusted users (#14706)
Currently, Discourse rate limits all incoming requests by the IP address they
originate from regardless of the user making the request. This can be
frustrating if there are multiple users using Discourse simultaneously while
sharing the same IP address (e.g. employees in an office).

This commit implements a new feature to make Discourse apply rate limits by
user id rather than IP address for users at or higher than the configured trust
level (1 is the default).

For example, let's say a Discourse instance is configured to allow 200 requests
per minute per IP address, and we have 10 users at trust level 4 using
Discourse simultaneously from the same IP address. Before this feature, the 10
users could only make a total of 200 requests per minute before they got rate
limited. But with the new feature, each user is allowed to make 200 requests
per minute because the rate limits are applied on user id rather than the IP
address.

The minimum trust level for applying user-id-based rate limits can be
configured by the `skip_per_ip_rate_limit_trust_level` global setting. The
default is 1, but it can be changed by either adding the
`DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL` environment variable with the
desired value to your `app.yml`, or changing the setting's value in the
`discourse.conf` file.

Requests made with API keys are still rate limited by IP address and the
relevant global settings that control API keys rate limits.

Before this commit, Discourse's auth cookie (`_t`) was simply a 32 characters
string that Discourse used to lookup the current user from the database and the
cookie contained no additional information about the user. However, we had to
change the cookie content in this commit so we could identify the user from the
cookie without making a database query before the rate limits logic and avoid
introducing a bottleneck on busy sites.

Besides the 32 characters auth token, the cookie now includes the user id,
trust level and the cookie's generation date, and we encrypt/sign the cookie to
prevent tampering.

Internal ticket number: t54739.
2021-11-17 23:27:30 +03:00
..
auth FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
autospec DEV: Move chrome binary check into a shared lib (#13451) 2021-06-21 13:28:48 +10:00
backup_restore FEATURE: Direct S3 multipart uploads for backups (#14736) 2021-11-11 08:25:31 +10:00
common_passwords
compression
content_security_policy FIX: Set CSP base-uri to self (#13654) 2021-07-07 09:43:48 -04:00
demon DEV: IMAP debugging improvements (#11784) 2021-01-21 11:37:47 +10:00
discourse_dev DEV: only use the records that are auto populated by the task. (#14360) 2021-09-17 09:47:32 +05:30
email FIX: Remove List-Post email header (#14554) 2021-10-11 20:57:42 +03:00
emoji DEV: replaces huge generated emoji list by a simpler regex (#11053) 2021-04-22 08:43:06 +02:00
faker DEV: move discourse_dev gem to the core. (#13360) 2021-06-14 20:34:44 +05:30
file_store FEATURE: Direct S3 multipart uploads for backups (#14736) 2021-11-11 08:25:31 +10:00
freedom_patches DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
generators/rails DEV: removes plugin generator (#14101) 2021-08-20 11:29:06 +02:00
guardian FIX: Show right message when permanently deleting topic (#14717) 2021-10-26 18:31:15 +03:00
highlight_js
i18n DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
imap DEV: Use Rails query builder (#14759) 2021-10-28 15:14:23 +03:00
import
import_export FEATURE: include user custom fields in base exporter (#14690) 2021-10-22 10:02:56 -07:00
javascripts DEV: Allow transformed values to be used in all widget hbs statements (#13331) 2021-06-08 16:46:07 +01:00
middleware FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
migration DEV: Promote old post-deploy migrations to pre-deploy migrations (#13477) 2021-06-22 16:02:24 +01:00
onebox FIX: Display Instagram Oneboxes in an iframe (#14789) 2021-11-02 14:34:51 -04:00
plugin DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
pretty_text DEV: replaces huge generated emoji list by a simpler regex (#11053) 2021-04-22 08:43:06 +02:00
rate_limiter FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
reviewable DEV: APIs for plugin to add custom reviewable confirm modal (#12246) 2021-03-02 10:28:27 -06:00
scheduler
search FIX: remove superfluous spaces from CJK blurbs (#12629) 2021-04-12 12:46:42 +10:00
seed_data FIX: Support Ruby 3 keyword arguments 2021-10-05 11:25:00 -04:00
sidekiq
site_settings DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
stylesheet DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
svg_sprite DEV: Remove a few unused icons (#14696) 2021-10-22 12:03:58 -04:00
tasks DEV: I18n global is no longer supported (#14993) 2021-11-17 20:52:22 +01:00
theme_store FIX: Remove whitespace from theme git versions (#12857) 2021-04-27 17:45:54 +01:00
topic_query FIX: exclude topics from muted tag in category featured list. (#14925) 2021-11-16 12:10:50 +05:30
turbo_tests DEV: Upgrade Rails to 6.1.3.1 (#12688) 2021-04-21 12:36:32 +03:00
validators FIX: Support Ruby 3 keyword arguments 2021-10-05 11:25:00 -04:00
webauthn
wizard FEATURE: Enable auto dark mode on new instances (#14208) 2021-09-02 14:55:38 -04:00
admin_confirmation.rb DEV: Upgrade Redis to 4.2.1. 2020-06-15 10:05:22 +08:00
admin_constraint.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
admin_user_index_query.rb DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
age_words.rb
archetype.rb
auth.rb DEV: remove instagram login site settings and auth classes. (#11073) 2020-10-30 09:09:56 +05:30
backup_restore.rb DEV: Upgrade Rails to 6.1.3.1 (#12688) 2021-04-21 12:36:32 +03:00
badge_posts_view_manager.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
badge_queries.rb FIX: Don't grant sharing badges to users who don't exist (#13851) 2021-07-27 16:32:59 +10:00
base62.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
bookmark_manager.rb FEATURE: Topic-level bookmarks (#14353) 2021-09-21 08:45:47 +10:00
bookmark_query.rb FEATURE: Go to last unread for topic-level bookmark links (#14396) 2021-09-21 13:49:56 +10:00
bookmark_reminder_notification_handler.rb DEV: Ignore reminder_type for bookmarks (#14349) 2021-09-16 09:56:54 +10:00
browser_detection.rb
cache.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
canonical_url.rb FEATURE: default canonical URL (#9738) 2020-05-12 09:13:20 +10:00
category_badge.rb
chrome_installed_checker.rb DEV: Move chrome binary check into a shared lib (#13451) 2021-06-21 13:28:48 +10:00
comment_migration.rb
composer_messages_finder.rb FEATURE: Make allow_uploaded_avatars accept TL (#14091) 2021-08-24 10:46:28 +03:00
configurable_urls.rb Replace base_uri with base_path (#10879) 2020-10-09 12:51:24 +01:00
content_buffer.rb
content_security_policy.rb PERF: Eager load Theme associations in Stylesheet Manager. 2021-06-21 11:06:58 +08:00
cooked_post_processor.rb FIX: remove 'crawl_images' site setting (#14646) 2021-10-19 17:12:29 +05:30
crawler_detection.rb FEATURE: Implement browser update in crawler view (#12448) 2021-03-22 19:41:42 +02:00
csrf_token_verifier.rb
current_user.rb
custom_renderer.rb
custom_setting_providers.rb
db_helper.rb DEV: Upgrade Rails to 6.1.3.1 (#12688) 2021-04-21 12:36:32 +03:00
directory_helper.rb
discourse_cookie_store.rb
discourse_dev.rb DEV: move discourse_dev gem to the core. (#13360) 2021-06-14 20:34:44 +05:30
discourse_diff.rb Escape values of HTML attributes 2021-08-10 10:25:15 -04:00
discourse_event.rb DEV: Plugin API to add directory columns (#13440) 2021-06-22 13:00:04 -05:00
discourse_hub.rb
discourse_ip_info.rb
discourse_js_processor.rb DEV: Add support for class properties in babel (#13189) 2021-05-27 16:13:14 -04:00
discourse_logstash_logger.rb
discourse_plugin_registry.rb DEV: Plugin instance method for push_notification_filters (#14787) 2021-11-03 12:21:33 -05:00
discourse_redis.rb DEV: Pass kwargs to the redis gem when calling methods/commands that we don't wrap (#14530) 2021-10-06 17:42:04 +03:00
discourse_tagging.rb FIX: Show required tags to staff by default and override limit (#13242) 2021-06-02 12:43:34 -04:00
discourse_updates.rb FIX: Regression introduced in #14715 (#14842) 2021-11-09 17:20:09 +11:00
discourse.rb FEATURE: add Unseen view (#13977) 2021-08-10 18:30:34 +04:00
disk_space.rb FIX: correct upload statistics report for external storage 2020-02-20 15:15:53 +11:00
distributed_cache.rb PERF: Defer setting of distributed cache in more spots. 2021-06-04 09:13:18 +08:00
distributed_memoizer.rb DEV: Replace Time.new with Time.now (#9142) 2020-03-09 17:37:49 +01:00
distributed_mutex.rb
edit_rate_limiter.rb FEATURE: Increase daily edit limits proportionally to trust level (#13090) 2021-05-19 13:57:21 +04:00
email_backup_token.rb
email_cook.rb PERF: Avoid lookbehinds when replacing links in imported emails (#11931) 2021-02-02 17:34:00 +01:00
email_updater.rb FEATURE: add maximum limit for secondary emails (#12599) 2021-04-05 20:31:42 +05:30
email.rb FIX: Replace use of regular expression (#12838) 2021-04-27 08:48:51 +03:00
encodings.rb
enum_site_setting.rb
enum.rb
excerpt_parser.rb DEV: Remove dead code 2021-05-31 10:22:50 +08:00
feed_element_installer.rb
feed_item_accessor.rb
file_helper.rb DEV: Remove warnings on console (#14608) 2021-10-14 23:17:47 +02:00
filter_best_posts.rb
final_destination.rb FIX: Canonical URLs may be relative (#14825) 2021-11-05 14:20:14 -03:00
flag_query.rb DEV: Remove deprecated methods (#14885) 2021-11-11 12:21:25 -06:00
flag_settings.rb
gaps.rb
global_path.rb
guardian.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
has_errors.rb
hijack.rb DEV: Add more debugging context to onebox generation 2020-10-22 12:50:22 +08:00
homepage_constraint.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
html_prettify.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
html_to_markdown.rb FIX: Hoisting linebreaks shouldn't fail for HTML5 elements (#14364) 2021-09-17 10:41:34 +02:00
http_language_parser.rb FIX: Include resolved locale in anonymous cache key (#10289) 2020-07-22 18:00:07 +01:00
image_sizer.rb
import_export.rb FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
inline_oneboxer.rb FEATURE: check blocked_onebox_domains setting for inline oneboxes (#11944) 2021-02-03 21:45:22 +05:30
introduction_updater.rb FIX: replace default welcome topic post with new value from wizard 2020-04-01 15:42:45 -04:00
ip_addr.rb
js_locale_helper.rb DEV: move discourse_dev gem to the core. (#13360) 2021-06-14 20:34:44 +05:30
json_error.rb
letter_avatar.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
markdown_linker.rb
mem_info.rb
message_bus_diags.rb
method_profiler.rb DEV: Add output_sql_to_stderr! to MethodProfiler (#12445) 2021-03-19 17:48:30 +10:00
mini_sql_multisite_connection.rb DEV: upgrade mini_sql (#12465) 2021-03-24 08:48:04 +11:00
mobile_detection.rb
new_post_manager.rb SECURITY: Escape watched word in error message (#14434) 2021-09-24 11:55:15 +03:00
new_post_result.rb DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
notification_levels.rb
onebox.rb DEV: Absorb onebox gem into core (#12979) 2021-05-26 15:11:35 +05:30
oneboxer.rb FEATURE: Onebox can match engines based on the content_type (#13876) 2021-07-30 13:36:30 -04:00
onpdiff.rb
pbkdf2.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
permalink_constraint.rb
pinned_check.rb
plain_text_to_markdown.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
plugin_gem.rb FIX: ensure plugin's gems are in the gem path (#12727) 2021-04-16 10:21:39 +02:00
plugin_initialization_guard.rb DEV: Print backtrace of error when plugin fails to initialize. 2020-06-09 10:25:43 +08:00
post_action_creator.rb PERF: Update like count in visible posts without an extra GET per like (#14869) 2021-11-10 13:22:26 -03:00
post_action_destroyer.rb FIX: Unlike own posts on ownership transfer (#10446) 2020-08-19 09:21:02 -06:00
post_action_result.rb
post_creator.rb FIX: Don't publish PM archive events to acting user. (#14291) 2021-09-10 09:20:50 +08:00
post_destroyer.rb FIX: Skip post validations for system revisions when author deletes post (#14824) 2021-11-08 09:33:41 +08:00
post_jobs_enqueuer.rb FIX: Do not send emails to mailing_list_mode subscribers for PMs (#14159) 2021-08-26 15:16:35 +10:00
post_locker.rb
post_merger.rb FEATURE: TL4 & category moderators can merge posts (#12843) 2021-04-27 18:24:27 +02:00
post_revisor.rb FIX: Make PostRevisor more consistent (#14841) 2021-11-09 16:29:37 +02:00
presence_channel.rb DEV: Make PresenceChannel timeout configurable per-channel (#14697) 2021-10-22 16:24:55 +01:00
pretty_text.rb FIX: Do not check for duplicate links in Onebox (#13345) 2021-06-18 18:55:24 +03:00
promotion.rb FIX: check if BasicBadge is enabled for TL1 welcome message (#13983) 2021-08-11 08:39:25 +10:00
quote_comparer.rb FEATURE: Nokogumbo (#9577) 2020-05-05 13:46:57 +10:00
rake_helpers.rb
rate_limiter.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
read_only_header.rb
retrieve_title.rb FIX: increase chunk size to fetch title tag correctly (#14144) 2021-09-03 13:15:58 +05:30
route_format.rb
route_matcher.rb REFACTOR: Introduce RouteMatcher class 2020-10-19 10:40:55 +01:00
rtl.rb
s3_cors_rulesets.rb DEV: Skip logging in test environment (#14971) 2021-11-16 18:01:48 +03:00
s3_helper.rb FEATURE: Direct S3 multipart uploads for backups (#14736) 2021-11-11 08:25:31 +10:00
s3_inventory.rb FIX: increase inventory lag for s3 to 2 days (#11606) 2020-12-30 16:05:42 +11:00
score_calculator.rb
screening_model.rb
search.rb FIX: Use the same mode for chinese search when indexing and querying. (#14780) 2021-11-01 10:14:47 +08:00
secure_session.rb
shrink_uploaded_image.rb DEV: Improve script/downsize_uploads.rb (#13508) 2021-06-24 00:09:40 +02:00
single_sign_on_provider.rb FEATURE: Rename 'Discourse SSO' to DiscourseConnect (#11978) 2021-02-08 10:04:33 +00:00
single_sign_on.rb FIX: log proper error message when SSO nonce verification fails (#14077) 2021-08-18 18:44:12 +05:30
site_icon_manager.rb PERF: Defer setting of distributed cache in more spots. 2021-06-04 09:13:18 +08:00
site_setting_extension.rb DEV: Sanitize HTML admin inputs (#14681) 2021-10-27 11:33:07 -03:00
slug.rb FIX: Make category slugs lowercase (#11277) 2021-01-12 17:28:33 +02:00
socket_server.rb
spam_handler.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
sql_builder.rb DEV: Let's always give a drop_from param to deprecate (#14901) 2021-11-12 08:52:59 -06:00
staff_constraint.rb FEATURE: Apply rate limits per user instead of IP for trusted users (#14706) 2021-11-17 23:27:30 +03:00
staff_message_format.rb
suggested_topics_builder.rb
system_message.rb DEV: Add option to send system message to groups (#12256) 2021-03-02 18:51:50 +01:00
temporary_db.rb DEV: Add annotate rake tasks, and enforce via GitHub actions 2021-07-06 10:11:06 +01:00
temporary_redis.rb DEV: Introduce TemporaryRedis and unset DISCOURSE_* env vars in the themes:isolated_test rake task (#13401) 2021-06-23 07:38:43 +03:00
text_cleaner.rb FEATURE: Correctly convert topic title to uppercase and lowercase for Turkish default locale (#13115) 2021-05-24 18:13:30 +10:00
text_sentinel.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
theme_javascript_compiler.rb FEATURE: Allow theme tests to be run in production (take 2) (#12845) 2021-04-28 23:12:08 +03:00
theme_modifier_helper.rb Code review comments. 2021-06-21 11:06:58 +08:00
theme_settings_manager.rb DEV: use upload id to save in theme setting instead of URL. (#14341) 2021-09-16 07:58:53 +05:30
theme_settings_parser.rb DEV: support json_schema in theme settings (#12294) 2021-03-10 20:15:04 -05:00
theme_translation_manager.rb
theme_translation_parser.rb
timeline_lookup.rb FIX: ensures timeline_lookup includes last tuple (#11829) 2021-01-25 11:30:59 +01:00
topic_creator.rb FIX: include new tags in validation if user can create one. (#14744) 2021-10-28 11:59:46 +05:30
topic_list_responder.rb DEV: Refactor draft attributes for CategoryList and TopicList. 2020-07-24 10:11:30 +08:00
topic_publisher.rb FIX: Use destroy_all instead of delete_all for shared drafts 2020-03-05 11:13:43 -08:00
topic_query_params.rb FIX: Build correct topic list filter (#11473) 2020-12-11 14:20:48 +02:00
topic_query.rb FIX: exclude topics from muted tag in category featured list. (#14925) 2021-11-16 12:10:50 +05:30
topic_retriever.rb FEATURE: Fallback to system users when creating new TopicEmbed (#12386) 2021-03-15 11:58:53 -03:00
topic_subtype.rb
topic_upload_security_manager.rb DEV: Add security_last_changed_at and security_last_changed_reason to uploads (#11860) 2021-01-29 09:03:44 +10:00
topic_view.rb FIX: rename action_code_href to action_code_path (#14834) 2021-11-08 14:32:17 +11:00
topics_bulk_action.rb FIX: Don't publish PM archive events to acting user. (#14291) 2021-09-10 09:20:50 +08:00
trust_level.rb FIX: Don't store translated trust level names in anonymous cache (#13224) 2021-06-01 22:11:48 +02:00
turbo_tests.rb
twitter_api.rb DEV: Update rubocop-discourse from 2.3.2 to 2.4.0 (#11079) 2020-10-30 15:04:29 +01:00
unicorn_logstash_patch.rb DEV: Fix lint. 2020-07-21 15:55:03 +08:00
unread.rb FEATURE: Add last visit indication to topic view page. (#13471) 2021-07-05 14:17:31 +08:00
upload_creator.rb FEATURE: Direct S3 multipart uploads for backups (#14736) 2021-11-11 08:25:31 +10:00
upload_fixer.rb
upload_markdown.rb
upload_recovery.rb FIX: Support Ruby 3 keyword arguments 2021-10-05 11:25:00 -04:00
upload_security.rb FIX: Do not mark badge image uploads as secure (#13193) 2021-05-28 12:35:52 +10:00
url_helper.rb FEATURE: revert disallowing putting URLs in titles for TL0 users (#13970) 2021-08-06 20:07:42 +04:00
user_lookup.rb REVERT "FIX: do not show private group flair on user avatars" (#13991) 2021-08-10 17:25:11 +05:30
user_name_suggester.rb DEV: simplify username suggester (#14531) 2021-10-27 14:41:24 +04:00
vary_header.rb FIX: Include the Vary:Accept header on all Accept-based responses (#14647) 2021-10-25 12:53:50 +01:00
version.rb Version bump to v2.8.0.beta8 (#14948) 2021-11-15 11:13:34 -05:00
webauthn.rb DEV: stop freezing frozen strings 2020-04-30 16:48:53 +10:00
wizard.rb DEV: Allow plugins to add wizard steps after specific steps (#9315) 2020-04-01 08:36:50 -05:00
zeitwerk_config.rb FIX: Better and more secure validation of periods for TopicQuery 2021-07-23 14:24:44 -04:00