Commit Graph

102 Commits

Author SHA1 Message Date
Sam
70269c7c97 FEATURE: tighter limits on per cluster post rebakes
We have the periodical job that regularly will rebake old posts. This is
used to trickle in update to cooked markdown. The problem is that each rebake
can issue multiple background jobs (post process and pull hotlinked images)

Previously we had no per-cluster limit so cluster running 100s of sites could
flood the sidekiq queue with rebake related jobs.

New system introduces a hard limit of 300 rebakes per 15 minutes across a
cluster to ensure the sidekiq job is not dominated by this.

We also reduced `rebake_old_posts_count` to 80, which is a safer default.
2019-01-04 09:24:46 +11:00
Rishabh
a6c589d882 FEATURE: Add custom S3 Endpoint and DigitalOcean Spaces/Minio support for Backups (#6045)
- Add custom S3 Endpoints and DigitalOcean Spaces support
- Add Minio support using 'force_path_style' option and fix uploads to custom endpoint
2018-07-16 14:44:55 +10:00
Sam
87fabdc2f3 FIX: correct pool reaper
This removes a freedom patch and replaces with a custom reaper thread
it also captures an issue where reaper would fail when connections where
empty
2018-06-14 18:22:02 +10:00
Sam
ded84a4b58 PERF: improve performance once logged in rate limiter hits
If "logged in" is being forced anonymous on certain routes, trigger
the protection for any requests that spend 50ms queueing

This means that ...

1. You need to trip it by having 3 requests take longer than 1 second in 10 second interval
2. Once tripped, if your route is still spending 50m queueuing it will continue to be protected

This means that site will continue to function with almost no delays while it is scaling up to handle the new load
2018-04-23 11:55:25 +10:00
Sam
59cd7894d9 FEATURE: if site is under extreme load show anon view
If a particular path is being hit extremely hard by logged on users,
revert to anonymous cached view.

This will only come into effect if 3 requests queue for longer than 2 seconds
on a *single* path.

This can happen if a URL is shared with the entire forum base and everyone
is logged on
2018-04-18 16:58:57 +10:00
Blake Erickson
720dd2432e remove change from descourse_defaults.conf 2018-04-10 14:27:03 -06:00
Blake Erickson
0337a8f6d5 ensure correct '/'s for relative_url_root in route file 2018-04-10 14:24:29 -06:00
Guo Xiang Tan
a89f3160a5 Add new config to ensure backup/restore connects to PG directly.
* In `pg_dump` 10.3+ and 9.5.12+, in
  it does a `SELECT pg_catalog.set_config('search_path', '', false)`
  which changes the state of the current connection. This is known
  to be problematic with Pgbouncer which reuses connections. As such,
  we'll always try to connect directly to PG directly during
  the backup/restore process.
2018-03-09 10:28:03 +08:00
Sam
f0d5f83424 FEATURE: limit assets less that non asset paths
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
Sam
f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam
cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Sam
4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam
68d3c2c74f FEATURE: add global rate limiter for admin api 60 per minute
Also move configuration of admin and user api rate limiting into global
settings. This is not intended to be configurable per site
2017-12-11 11:07:22 +11:00
Guo Xiang Tan
6c04eb911d Fix typo. 2017-10-17 12:34:49 +08:00
Guo Xiang Tan
b54eb8f53c FIX: Set PG connect_timeout to 5 seconds.
* 30 seconds is alittle too long.
2017-10-17 12:32:41 +08:00
Sam
70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Sam
c106ca6778 FEATURE: fallback asset path for multi host setups 2017-03-20 15:59:17 -04:00
Sam
ff49f72ad9 FEATURE: per client user tokens
Revamped system for managing authentication tokens.

- Every user has 1 token per client (web browser)
- Tokens are rotated every 10 minutes

New system migrates the old tokens to "legacy" tokens,
so users still remain logged on.

Also introduces weekly job to expire old auth tokens.
2017-02-07 09:22:16 -05:00
Guo Xiang Tan
89b1998174 Add default port for redis_slave. 2016-03-11 15:07:07 +08:00
Guo Xiang Tan
c07c474575 FEATURE: Master-Slave Redis configuration with fallback and switch over. 2016-03-11 12:18:58 +08:00
Guo Xiang Tan
46589a1a0c FEATURE: AR adapter to failover to a replica DB server. 2016-02-05 08:51:10 +08:00
Sam Saffron
209b022385 PERF: cut down on memory usage allowed to redis
This limits the amount of backlog message bus channels can have.
2016-02-04 13:58:38 +11:00
Sam
dfe3ecb914 PERF: disable prepared statements
see: https://github.com/rails/rails/issues/21992
2015-10-19 14:02:22 +11:00
Sam
805120fc95 FEATURE: add connnection reaping based on maximum age
This feature ensures connections to the db are always attempted to be closed
after 600 seconds of idle time.
2015-10-17 11:29:16 +11:00
Sam
786cf6e61f format does not allow numbers 2015-07-23 15:33:38 +10:00
Sam
c056c3ba7e FEATURE: allow users to specify a second hostname if needed
(very rarely needed feature, mostly for multisite and origin pull cdns)
2015-07-23 15:22:54 +10:00
Sam
8252f4e110 FEATURE: allow use of redis sentinel via redis_sentinels
Use: DISCOURSE_REDIS_SENTINELS and DISCOURSE_REDIS_HOST to configure redis
sentinel
2015-06-25 16:51:48 +10:00
Sam
f3188ab8f2 remove unused setting 2015-04-30 12:06:32 +10:00
Sam
f5af4768eb FEATURE: add clean support for running Discourse in a subfolder
To setup set DISCOURSE_RELATIVE_URL_ROOT to the folder you wish
2015-03-09 13:14:29 +11:00
Sam
fff55e0ee3 FEATURE: allow users to specify if prepared statements are allowed
(they need to be disabled for pgpool based setups)
2015-02-18 11:16:53 +11:00
Sam
d56b71851b FEATURE: configurable connection reaping settings 2015-02-17 09:58:43 +11:00
Neil Lalonde
f88075cbba FEATURE: CORS settings per-site in a multisite env 2014-10-15 15:20:39 -04:00
Sam
2be03371ae FEATURE: allow a central redis cache for assets 2014-09-23 16:50:17 +10:00
Sam
f493eb5d12 FEATURE: allow version emails to be disabled globally 2014-08-23 10:02:32 +10:00
Zohar Arad
557af8b8bd Add RTL support to Discourse CSS pre-processing using the RTLit gem and configurable via discourse.conf 2014-08-08 09:31:31 +03:00
Sam
9468ebeb2e CHANGE: Mini Profiler only enabled for developers in prd 2014-07-17 08:34:41 +10:00
Adam Loving
3f1c5bef00 spelling in comment 2014-06-05 09:52:35 -07:00
Sam
30c86d834f * Increase pool size to allow for a few extra bg threads in sidekiq 2014-05-22 09:00:24 +10:00
Sam
810eda448f Merge branch 'environment-smtp-verify' of https://github.com/cfstras/discourse 2014-05-14 11:25:05 +10:00
Sam
dc06401479 PERF: reduce sidekiq worker count to 5 2014-05-14 10:21:11 +10:00
Claus Strasburger
b53248178b Added openssl_verify_mode parameter for action_mailer.
- parameter in `environments/production.rb`
- documentation & default value (nil) in `discourse_defaults.conf`
2014-05-08 22:21:36 +02:00
Sam
884346cbea Merge pull request #1951 from thoughtbot/bb-erb
Use ERB to pull conf files into app
2014-02-18 16:23:55 +11:00
Sam
73aca9063d FEATURE: allow overriding server static asset if needed 2014-02-17 10:44:37 +11:00
Britt Ballard
60e1a5aa69 Use ERB when importing conf files into app 2014-02-14 16:14:55 -08:00
slainer68
a89018db87 Use GlobalSetting to enable CORS at application level 2014-02-09 23:11:52 -08:00
Vikhyat Korrapati
6d50504a83 Make SMTP authentication mechanism configurable. 2014-01-06 18:19:42 +05:30
Sam
c7d99a288b try to default to sockets, so less people have issues configuring in prd 2014-01-06 12:25:29 +11:00
Sam
887ac10013 get rid of cache db, not needed 2014-01-06 12:24:11 +11:00
David Celis
0ed42707cc Allow configuration of Redis DB and cache DB
Hardcoding the Redis DB and Redis Caching DB to 0 and 2 in
`config/database.yml` makes an unsafe assumption that Discourse is the
only application using that install of redis-server. Instead of forcing
users to undergo yet another form of configuration, allow Discourse
admins a nicer way to configure the Redis databases used.

Signed-off-by: David Celis <me@davidcel.is>
2013-12-30 13:39:43 -08:00
Sam
8b33b45c2d should be commented out 2013-12-20 18:01:41 +11:00
Sam
4b04ac4c2b correct defaults 2013-12-20 16:23:01 +11:00
Sam
7b8d2547d0 globals now implemented and documented 2013-12-20 16:17:21 +11:00