Commit Graph

82 Commits

Author SHA1 Message Date
Neil Lalonde
4d12ff2e8a when writing cache, remove elements from the user agents list. also return a message and content type when blocking a crawler. 2018-03-27 13:44:14 -04:00
Neil Lalonde
a84bb81ab5 only applies to get html requests 2018-03-22 17:57:44 -04:00
Neil Lalonde
ced7e9a691 FEATURE: control which web crawlers can access using a whitelist or blacklist 2018-03-22 15:41:02 -04:00
Sam
0134e41286 FEATURE: detect when client thinks user is logged on but is not
This cleans up an error condition where UI thinks a user is logged on
but the user is not. If this happens user will be prompted to refresh.
2018-03-06 16:49:31 +11:00
Sam
f0d5f83424 FEATURE: limit assets less that non asset paths
By default assets can be requested up to 200 times per 10 seconds
from the app, this includes CSS and avatars
2018-03-06 15:20:39 +11:00
Sam
f295a18e94 FIX: stop double counting net calls in logs 2018-02-28 10:45:11 +11:00
Sam
ca1a3f37e3 FEATURE: add instrumentation for all external net calls 2018-02-21 15:20:29 +11:00
Guo Xiang Tan
3e835047da Remove "already initialized" constant warning. 2018-02-13 08:55:15 +08:00
Sam Saffron
df8e43abdd use lazy & instead of try
unregister ip skipper in test
raise if called when a skipper is in play
2018-02-06 10:38:15 +11:00
Robin Ward
eefd226611 Add extensibility point to request_tracker to skip IP addresses
This is useful if you want to run a per IP rate limiter but want to be
able to skip some IPs with custom logic.
2018-02-05 17:49:40 -05:00
Sam
2437b0d531 FIX: regression, missing 404 page 2018-01-23 09:00:28 +11:00
Sam
f26ff290c3 FEATURE: Shorten setting name to max_reqs
So it is consistent with other settings
2018-01-22 13:18:30 +11:00
Sam
8bf91b8dca correct tracking of x runtime 2018-01-19 17:51:19 +11:00
Sam
8ad43f01c2 FIX: correctly log topic timings as background 2018-01-19 10:37:43 +11:00
Sam
12872d03be PERF: run post timings in background
This means that if a very large amount of registered users hit
a single topic we will handle it gracefully, even if db gets slow.
2018-01-19 08:27:29 +11:00
Sam
442a17bfb2 PERF: bypass omniauth unless in an auth path 2018-01-15 12:44:54 +11:00
Sam
49ed382c2a FIX: return 429 when admin api key is limited on admin route
This also handles a general case where exceptions leak out prior to being handled by the application controller
2018-01-12 14:15:26 +11:00
Sam
cecd7d0d07 FEATURE: global rate limiter can bypass local IPs 2018-01-08 08:39:17 +11:00
Sam
715cb98e95 add better diagnostics for rate limits 2018-01-05 12:14:28 +11:00
Sam
bbc606988f improve message 2017-12-20 10:12:33 +11:00
Sam
4986ebcf24 FEATURE: optional default off global per ip rate limiter 2017-12-11 17:52:57 +11:00
Sam
df84e1c358 Correctly track hijacked requests 2017-11-28 16:47:20 +11:00
Sam
a4c539bade FEATURE: Allow registration of detailed request logger
Detailed request loggers can be used to gather rich timing info
from all requests (which in turn can be forwarded to monitoring solution)

Middleware::RequestTracker.detailed_request_logger(->|env, data| do
   # do stuff with env and data
end
2017-10-18 12:10:30 +11:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam
bdb848b4f3 Split the theme_key so we extract the key from seq 2017-06-15 14:09:44 -04:00
Sam
ac1f84d3e1 SECURITY: theme key should be an anon cache breaker 2017-06-15 09:36:27 -04:00
Sam
a3e8c3cd7b FEATURE: Native theme support
This feature introduces the concept of themes. Themes are an evolution
of site customizations.

Themes introduce two very big conceptual changes:

- A theme may include other "child themes", children can include grand
children and so on.

- A theme may specify a color scheme

The change does away with the idea of "enabled" color schemes.

It also adds a bunch of big niceties like

- You can source a theme from a git repo

- History for themes is much improved

- You can only have a single enabled theme. Themes can be selected by
    users, if you opt for it.

On a technical level this change comes with a whole bunch of goodies

- All CSS is now compiled using a custom pipeline that uses libsass
    see /lib/stylesheet

- There is a single pipeline for css compilation (in the past we used
    one for customizations and another one for the rest of the app

- The stylesheet pipeline is now divorced of sprockets, there is no
   reliance on sprockets for CSS bundling

- CSS is generated with source maps everywhere (including themes) this
    makes debugging much easier

- Our "live reloader" is smarter and avoid a flash of unstyled content
   we run a file watcher in "puma" in dev so you no longer need to run
   rake autospec to watch for CSS changes
2017-04-12 10:53:49 -04:00
Sam
ea9f7a41af remove gctools (no longer used) add gctracer for debugging 2016-12-20 15:07:30 +11:00
Sam
39a524aac8 FEATURE: brotli cdn bypass for assets
Allow CDNS that strip out brotli encoding to use brotli regardless
2016-12-05 13:57:09 +11:00
Sam
497ff76a67 make sure 1 is a string 2016-10-27 18:08:01 +11:00
Sam
3e7190866a make code a bit safer 2016-10-27 16:50:56 +11:00
Sam
8a477f1857 FEATURE: added X-Discourse-TrackView header
This header is set to 1 if the particular request is a tracked page view
2016-10-27 16:48:27 +11:00
Robin Ward
a9823ab59a FIX: Use a cookie to bypass the anon cache 2015-10-28 17:16:56 -04:00
Sam
ec4a1bb2c4 FIX: page tracking was not properly tracking transitions
PERF: move closure to self contained method so env is released earlier.
2015-09-17 11:06:33 +10:00
Régis Hanol
d7aa4e81d6 revert 8f435fcbf6 2015-07-31 15:22:30 +02:00
Neil Lalonde
86cd1a19cc FEATURE: page view stats for mobile view 2015-07-03 17:19:33 -04:00
Sam
1f9761e85d FEATURE: add a header to denote an anonymous req was cached
(X-Discourse-Cached)
2015-06-16 10:30:06 +10:00
Sam
90eaad336d FEATURE: allow users to pick a CDN for s3 assets 2015-05-26 11:13:12 +10:00
Régis Hanol
bb0c2813ac FEATURE: generate (avatar) thumbnails in a background task
FIX: keep the "uploading..." indicator until the server replies via the MessageBus
FIX: text was disapearing when uploading an avatar

PERF: always use a region for S3 (defaults to 'us-east-1')
FEATURE: ApplyCDN middleware when using S3
FIX: use the same pattern to store files on S3 and locally
PERF: keep a local cache of uploads when generating thumbnails
FEATURE: migrate_to_s3 rake task
2015-05-25 17:59:00 +02:00
Sam
f5af4768eb FEATURE: add clean support for running Discourse in a subfolder
To setup set DISCOURSE_RELATIVE_URL_ROOT to the folder you wish
2015-03-09 13:14:29 +11:00
Sam
cbe18eb0df FEATURE: allow view exclusion using custom header
Set Discourse-Track-View to either "0" or "false" to exclude request
2015-02-26 11:41:11 +11:00
Sam
fce9e296e7 background reqs failed or not are always counted seperately 2015-02-12 09:47:46 +11:00
Sam
3cf87b94c9 whitespace 2015-02-11 09:39:04 +11:00
Sam
0ce6524153 correct brokeness 2015-02-10 17:05:24 +11:00
Sam
acda6ebd60 FIX: view tracking needs to release data earlier
retaining data during queuing was causing huge memory spikes
2015-02-10 17:03:33 +11:00
Sam
820ce8765e refactor traffic report
split traffic report in 2, page view vs raw traffic
hide raw traffic report by default
improve flushing logic for application reqs
2015-02-06 14:39:16 +11:00
Sam
08b790b3c2 improve metrics gathered using in our traffic section
this also pulls out the middleware into its own home and inserts in front
2015-02-05 16:08:52 +11:00
Sam
8690c7c49f defer counting to avoid race condition 2015-02-05 12:19:21 +11:00
Sam
c150c55e2d FEATURE: rudimentary view tracking wired in 2015-02-04 16:15:16 +11:00
Sam
4f8dfd84b9 FIX: vary accept for cache, seems most correct 2014-09-09 10:25:49 +10:00