Commit Graph

577 Commits

Author SHA1 Message Date
Dan Ungureanu
5b84307774
FIX: Ensure that scheduled jobs are loaded. (#8183)
In development, the scheduled jobs are loaded lazily and MiniScheduler
cannot discover them (/sidekiq/scheduler does not show any jobs).
2019-10-14 12:14:16 +03:00
Daniel Waterworth
4c9ed7bd85 FIX: Fix rake db:create after zeitwerk changes
Post-zeitwerk, rails has deprecated autoloading modules during
initialization and forces all autoloaded modules to be reloaded after
initialization.

Requiring the file explicitly prevents autoloading and therefore
prevents the state on SiteSetting being trashed which was causing the
problem here.
2019-10-08 12:22:34 +01:00
Krzysztof Kotlarek
302e8f4393 FIX: Use migrations path for post_migrate (#8133)
That is a problem after upgrade to Rails 6. It was partially fixed here: 025d4ee91f
2019-10-02 15:28:38 +10:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Krzysztof Kotlarek
f64c9f37fa FIX: Remove versions from Active Record warm up (#8105) 2019-09-18 17:59:51 -04:00
Robin Ward
30bba6252d Allow CORS combined with HTTP Auth 2019-09-09 15:02:20 -04:00
Sam Saffron
098f9e8b5b PERF: Run multiple threads for regular job schedules
Under extreme load on large databases certain regular jobs can take quite
a while to run. We need to ensure we never starve a sidekiq from running
mini scheduler, cause without it we are unable to queue stuff such as
heartbeat jobs.
2019-08-29 15:34:36 +10:00
Roman Rizzi
3259ea60a6
DEV: Remove code deprecated by the new Reviewable API (#8023)
* Remove flag hooks and endpoints

* Remove #reject_bulk for users

* Remove code for quued_posts_controller
2019-08-26 10:33:26 -03:00
Sam Saffron
208c638900 FEATURE: add hook after all initializers
This hook allows plugins to amend middleware stack or any other settings
that need to be changed just after the intializers run
2019-08-26 10:49:26 +10:00
Sam Saffron
8db38de9d7 SECURITY: add rate limiting to anon JS error reporting
This adds a 1 minute rate limit to all JS error reporting per IP. Previously
we would only use the global rate limit.

This also introduces DISCOURSE_ENABLE_JS_ERROR_REPORTING, if it is set to
false then no JS error reporting will be allowed on the site.
2019-08-20 11:29:11 +10:00
Sam Saffron
10b36c6446 FIX: rack-mini-profiler not showing plugin frames
Previously the default stack suppressor in rack-mini-profiler was excluding
the plugin directory.

This made islolating issues more complicated cause you needed to defer to
pp=full-backtrace which is both slow and noisy
2019-08-19 15:47:53 +10:00
Robin Ward
b4878cde6f FEATURE: Add a webhook for user notifications
If enabled, this will fire a webhook whenever a user's notification has
been created. This could potentially be a lot of data depending on your
forum, and should be used carefully since it includes everything all users
will see in their feeds.
2019-08-15 14:47:25 -04:00
David Taylor
92f2202e4a SECURITY: Restrict message-bus access on login_required sites 2019-08-14 09:43:12 +01:00
David Taylor
1a8fee11a0 DEV: If only one auth provider is enabled allow GET request
In this case, the auth provider is acting as a SSO provider, and can be trusted to maintain its own CSRF protections.
2019-08-12 11:03:05 +01:00
Sam Saffron
1f47ed1ea3 PERF: message_bus will be deferred by server when flooded
The message_bus performs a fair amount of work prior to hijacking requests
this change ensures that if there is a situation where the server is flooded
message_bus will inform client to back off for 30 seconds + random(120 secs)

This back-off is ultra cheap and happens very early in the middleware.

It corrects a situation where a flood to message bus could cause the app
to become unresponsive

MessageBus update is here to ensure message_bus gem properly respects
Retry-After header and status 429.

Under normal state this code should never trigger, to disable raise the
value of DISCOURSE_REJECT_MESSAGE_BUS_QUEUE_SECONDS, default is to tell
message bus to go away if we are queueing for 100ms or longer
2019-08-09 17:48:01 +10:00
David Taylor
3b8c468832 SECURITY: Require POST with CSRF token for OmniAuth request phase 2019-08-08 11:58:00 +01:00
Gerhard Schlager
03d28342f8 FIX: Make initializer work on first db:migrate
Follow-up to 94607a2f
2019-07-30 10:23:24 +02:00
Gerhard Schlager
94607a2f6b FEATURE: Generate new VAPID keys when base_url changes
This is useful when a backup is restored on a staging site or in a development environment. It also deletes all existing push subscriptions because they get invalid when the keys change.
2019-07-30 10:08:24 +02:00
Gerhard Schlager
8f89254554 FIX: Recalculate settings when dependent settings change 2019-07-12 21:10:10 +02:00
Bianca Nenciu
1942ba1d42 PERF: Use Oj for serializing JSON. (#7820) 2019-06-28 12:16:00 +10:00
Bianca Nenciu
3fd7cf9038 Revert "PERF: Use Oj for serializing JSON. (#7780)"
This commit broke discourse-prometheus.

This reverts commit b4df8c5466.
2019-06-25 11:13:27 +03:00
Bianca Nenciu
b4df8c5466
PERF: Use Oj for serializing JSON. (#7780) 2019-06-24 18:32:00 +03:00
Sam Saffron
a01488ae67 DEV: improve on rake db:create
Followup on 3af00a65 which broke build
2019-06-14 15:06:07 +10:00
Sam Saffron
3ef4ae08f1 DEV: check for specifics when looking at ENABLE_LOGRAGE
prior to this change ENABLE_LOGRAGE=0 some_command would enable lograge
2019-06-13 15:59:20 +10:00
Sam
fa2a5f6f56
FEATURE: SKIP_DB_AND_REDIS env var (#7756)
Sometimes we would like to create a base image without any DB access, this
assists in creating custom base images with custom plugins that already
includes `public/assets`

Following this change set you can run:

```
SPROCKETS_CONCURRENT=1 DONT_PRECOMPILE_CSS=1 SKIP_DB_AND_REDIS=1 RAILS_ENV=production bin/rake assets:precompile
```

Then it is straight forward to create a base image without needing a DB or
Redis.
2019-06-13 12:58:27 +10:00
Robin Ward
ecebff5060 Only show deprecation warning if the webhook is active 2019-06-10 16:23:12 -04:00
Penar Musaraj
ffe51ac39c Minor formatting change in 014-track-setting-changes.rb 2019-06-06 09:39:22 -04:00
Penar Musaraj
f00275ded3 FEATURE: Support private attachments when using S3 storage (#7677)
* Support private uploads in S3
* Use localStore for local avatars
* Add job to update private upload ACL on S3
* Test multisite paths
* update ACL for private uploads in migrate_to_s3 task
2019-06-06 13:27:24 +10:00
Sam Saffron
62141b6316 FEATURE: enable_performance_http_headers for performance diagnostics
This adds support for DISCOURSE_ENABLE_PERFORMANCE_HTTP_HEADERS
when set to `true` this will turn on performance related headers

```text
X-Redis-Calls: 10     # number of redis calls
X-Redis-Time: 1.02    # redis time in seconds
X-Sql-Commands: 102   # number of SQL commands
X-Sql-Time: 1.02      # duration in SQL in seconds
X-Queue-Time: 1.01    # time the request sat in queue (depends on NGINX)
```

To get queue time NGINX must provide: HTTP_X_REQUEST_START

We do not recommend you enable this without thinking, it exposes information
about what your page is doing, usually you would only enable this if you
intend to strip off the headers further down the stream in a proxy
2019-06-05 16:08:11 +10:00
Sam Saffron
b114bcd294 DEV: switch message bus backend to memory for tests
This backend is a bit faster and well tested, this is part of a longer
term plan to have a `backend: :memory, threaded: false` type config for
message bus which we can use in test.

The threading in message bus causes all sorts of surprises in test, it will
be nice not to be beholden to them.
2019-05-29 16:34:55 +10:00
Rafael dos Santos Silva
315a38e0e3 FEATURE: Allow running message_bus in a different redis instance (#7616)
Adds `DISCOURSE_MESSAGE_BUS_REDIS_ENABLED` env var, that when set
to true, will allow Discourse to connect to a different redis
instance for MessageBus needs.

When enabled you can configure the same env vars user for redis,
but prefixed by `MESSAGE_BUS`, eg:

`DISCOURSE_MESSAGE_BUS_REDIS_HOST`
2019-05-28 15:52:43 +10:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron
88249932a0 PERF: remove SQL notifiers for every SQL execution
Benchmarking:

```
Benchmark.ips do |b|
  b.report("simple") do
    User.first
  end
end

ActiveSupport::Notifications.notifier.listeners_for("sql.active_record").clear

Benchmark.ips do |b|
  b.report("simple") do
    User.first
  end
end

```

```
sam@arch discourse % RAILS_ENV=production ruby script/micro_bench.rb

Before

Calculating -------------------------------------
              simple      3.289k (± 4.4%) i/s -     16.575k in   5.049771s
After

Calculating -------------------------------------
              simple      3.491k (± 3.6%) i/s -     17.442k in   5.002226s
````
2019-05-09 17:44:46 +10:00
Guo Xiang Tan
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
David Taylor
0e303c7f5d
FEATURE: Automatically generate optimized site metadata icons (#7372)
This change automatically resizes icons for various purposes. Admins can now upload `logo` and `logo_small`, and everything else will be auto-generated. Specific icons can still be uploaded separately if required.

## Core

- Adds an SiteIconManager module which manages automatic resizing and fallback

- Icons are looked up in the OptimizedImage table at runtime, and then cached in Redis. If the resized version is missing for some reason, then most icons will fall back to the original files. Some icons (e.g. PWA Manifest) will return `nil` (because an incorrectly sized icon is worse than a missing icon). 

- `SiteSetting.site_large_icon_url` will return the optimized version, including any fallback. `SiteSetting.large_icon` continues to return the upload object. This means that (almost) no changes are required in core/plugins to support this new system.

- Icons are resized whenever a relevant site setting is changed, and during post-deploy migrations

## Wizard

- Allows `requiresRefresh` wizard steps to reload data via AJAX instead of a full page reload

- Add placeholders to the **icons** step of the wizard, which automatically update from the "Square Logo"

- Various copy updates to support the changes

- Remove the "upload-time" resizing for `large_icon`. This is no longer required.

## Site Settings UX

- Move logo/icon settings under a new "Branding" tab

- Various copy changes to support the changes

- Adds placeholder support to the `image-uploader` component

- Automatically reloads site settings after saving. This allows setting placeholders to change based on changes to other settings

- Upload site settings will be assigned a placeholder if SiteIconManager `responds_to?` an icon of the same name

## Dashboard Warnings

- Remove PWA icon and PWA title warnings. Both are now handled automatically.

## Bonus

- Updated the sketch logos to use @awesomerobot's new high-res designs
2019-05-01 14:44:45 +01:00
Sam Saffron
3b95f34e7b DEV: Rails 6 compatability
Without forcing a reload on start internal state in the accelerator can be
off. In Rails 5 not translation is being called so this is not an issue but
in 6 it is called earlier on.
2019-04-29 17:13:30 +10:00
David Taylor
7826acc4a7
DEV: Replace site_setting_saved DiscourseEvent with site_setting_changed (#7401)
* DEV: Replace site_setting_saved DiscourseEvent with site_setting_changed

site_setting_saved is confusing for a few reasons:
- It is attached to the after_save of the ActiveRecord model. This is confusing because it only works 'properly' with the db_provider
- It passes the activerecord model as a parameter, which is confusing because you get access to the 'database' version of the setting, rather than the ruby setting. For example, booleans appear as 'y' or 'n' strings.
- When the event is called, the local process cache has not yet been updated. So if you call SiteSetting.setting_name inside the event handler, you will receive the old site setting value

I have deprecated that event, and added a new site_setting_changed event. It passes three parameters:
- Setting name (symbol)
- Old value (in ruby format)
- New value (in ruby format)

It is triggered after the setting has been persisted, and the local process cache has been updated.

This commit also includes a test case which describes the confusing behavior. This can be removed once site_setting_saved is removed.
2019-04-18 16:48:01 +01:00
David Taylor
c687e2b921 FIX: Use saved_change_to_value? in site_setting_saved event
Since Rails 5.2, the behavior of `attribute_changed?` inside `after_save` callbacks has changed, so we need to use `saved_change_to_attribute` instead. The site setting local_process_provider in test mode was covering up the issue.
2019-04-17 10:07:00 -04:00
Robin Ward
ba6d4b2a8d FIX: Better handling for toggling must_approve_users
If you turn it on now, default all users to approved since they were
previously. Also support approving a user that doesn't have a reviewable
record (it will be created first.)

This also includes a refactor to move class method calls to
`DiscourseEvent` into an initializer. Otherwise the load order of
classes makes a difference in the test environment and some settings
might be triggered and others not, randomly.
2019-04-16 15:56:35 -04:00
Robin Ward
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Sam Saffron
b418857830 DEV: document max_logster_logs in discourse_defaults.conf
This cleans up logster configuration a bit cause we no longer have to
check if we respond_to anything and keeps the logster limit properly
documented

Followup on da578e92
2019-03-22 14:11:40 +11:00
Osama Sayegh
da578e92f2
DEV: configure logster max logs via ENV (#7225) 2019-03-21 16:10:17 +03:00
Osama Sayegh
14c1af0a9e Bump logster version to 2.2.0 (#7200) 2019-03-19 09:39:16 +01:00
Osama Sayegh
f86ca5631a
Bump logster to v 2.1.2 (#7052) 2019-02-21 18:59:33 +03:00
Sam
8745f78277 DEV: disable async logging in development on broken Ruby
Ruby 2.5.3 has an upatched issue that crashes unicorn after fork:

https://bugs.ruby-lang.org/issues/14634

This will be patched in 2.5.4 however for now just warn people dev is slower
and disable async logging on the older rubies
2019-02-21 17:20:58 +11:00
Guo Xiang Tan
eb9565d8fc DEV: Rescue from readonly error in lograge. 2019-01-21 14:20:19 +08:00
Guo Xiang Tan
ee7ab3e2ec Pause MiniScheduler when Sidekiq is paused. 2019-01-18 17:50:24 +08:00
Guo Xiang Tan
234e0f35c5 Fix the build.
6edf285c89
2019-01-17 17:28:09 +08:00
Guo Xiang Tan
6edf285c89 FIX: Avoid rescuing error in login hint initializer.
On the first migration, trying to access the users table will throw an
error in PostgreSQL's log which has been confusing since users will
report it to us when rebuild fails.
2019-01-17 17:22:15 +08:00
Sam
a19170a4c2 DEV: avoid require_dependency for some libs
This avoids require dependency on method_profiler and anon cache.

It means that if there is any change to these files the reloader will not pick it up.

Previously the reloader was picking up the anon cache twice causing it to double load on boot.

This caused warnings.

Long term my plan is to give up on require dependency and instead use:

https://github.com/Shopify/autoload_reloader
2018-12-31 10:53:30 +11:00
Guo Xiang Tan
e564fe1866 FIX: Sidekiq fails to start if any of the multisite has problems. 2018-12-12 11:30:14 +08:00
Sam
a34bc92e1a DEV: update mini profiler
This provides us with instrumentation missing after rails upgrade

Latest version of rails uses exec_params internally which is no longer
routed to intercepted methods in mini profiler 1.0.0
2018-12-10 14:29:20 +11:00
Sam
1a71f98d28 DEV: only publish logs error count to admins 2018-12-05 17:03:37 +11:00
Sam
12f5889c85 DEV: stop logging warning when overriding open scope
ActiveRecord defines automatic scopes for enums, the Poll model defines
an enum for `{open: 1}` this mean Rails wants the scope `Poll.all.open`
to work which in turn means it has to override `open` which is defined
privately.

Rails feature req exists for: https://github.com/rails/rails/issues/34599
which will allow us to define enums without scopes which would resolve this
a lot more cleaner.
2018-12-03 13:52:09 +11:00
Penar Musaraj
03deda2147
Upgrade to FontAwesome 5 (take two) (#6673)
* Add missing icons to set

* Revert FA5 revert

 This reverts commit 42572ff

* use new SVG syntax in locales

* Noscript page changes (remove login button, center "powered by" footer text)

* Cast wider net for SVG icons in settings

- include any _icon setting for SVG registry (offers better support for plugin settings)

- let themes store multiple pipe-delimited icons in a setting

- also replaces broken onebox image icon with SVG reference in cooked post processor

* interpolate icons in locales

* Fix composer whisper icon alignment

* Add support for stacked icons

* SECURITY: enforce hostname to match discourse hostname

This ensures that the hostname rails uses for various helpers always matches
the Discourse hostname

* load SVG sprite with pre-initializers

* FIX: enable caching on SVG sprites

* PERF: use JSONP for SVG sprites so they are served from CDN

This avoids needing to deal with CORS for loading of the SVG

Note, added the svg- prefix to the filename so we can quickly tell in
dev tools what the file is

* Add missing SVG sprite JSONP script to CSP

* Upgrade to FA 5.5.0

* Add support for all FA4.7 icons

- adds complete frontend and backend for renamed FA4.7 icons

- improves performance of SvgSprite.bundle and SvgSprite.all_icons

* Fix group avatar flair preview

- adds an endpoint at /svg-sprites/search/:keyword

- adds frontend ajax call that pulls icon in avatar flair preview even when it is not in subset

* Remove FA 4.7 font files
2018-11-26 16:49:57 -05:00
Sam
42572ff138 Revert font awesome 5 changes
We are still pushing ahead on this 100% just need a bit longer to prepare
all plugins
2018-11-08 16:12:18 +11:00
Penar Musaraj
005e1ecb9b
FEATURE: Update Font Awesome to v5.4.1 and SVGs (#6557)
* First take on subsetting svg icons

* FontAwesome 5 svg subset WIP

* Include icons from plugins/badges into svg sprite subset

* add svg icon support to themes

* Add spec for SvgSprite

* Misc. SVG icon fixes

* Use FA5 svgs in local-dates plugin

* CSS adjustments, fix SVG icons in group flair

* Use SVG icons in poll plugin

* Add SVG icons to /wizard
2018-11-07 13:05:43 -05:00
Kyle Zhao
dca830cb73 Revert "FEATURE: [Experimental] Content Security Policy (#6504)"
This reverts commit fb8231077a.
2018-10-19 11:53:29 -04:00
Kyle Zhao
fb8231077a
FEATURE: [Experimental] Content Security Policy (#6504) 2018-10-19 10:39:22 -04:00
Davide Porrovecchio
005e1f5373 Add Cache-Control header to CORS (#6490) 2018-10-16 10:46:55 +11:00
Guo Xiang Tan
1c9b5e75e7 DEV: Support post deployment migrations for plugins. 2018-10-09 13:11:45 +08:00
Guo Xiang Tan
40fa96777d
FEATURE: Post deployment migrations. (#6406)
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.

The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.

```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```

To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
2018-10-08 15:47:38 +08:00
Vinoth Kannan
8430ea927e FIX: Generate webhook payloads before destroy events (#6325) 2018-10-05 16:53:59 +08:00
CheshireVillageSoftware
f3214889dc FEATURE: Updated CORS config to explicitly specifyhttp methods
See: https://stackoverflow.com/questions/20478312/default-value-for-access-control-allow-methods 

In particular we now explicitly allow DELETE and PUT which is inconsistently allowed depending on browser
2018-09-17 11:01:08 +10:00
Guo Xiang Tan
8dc1463ab3 Enable Lint/ShadowingOuterLocalVariable for Rubocop. 2018-09-04 10:16:42 +08:00
Guo Xiang Tan
0a14e0a256 Ensure params[:files] responds to map in Lograge. 2018-09-04 09:22:54 +08:00
Davide Porrovecchio
1826626272 FEATURE: Add Content-Type header to CORS
- add Content-Type to Access-Control-Allow-Headers
- update test accordingly
2018-08-28 11:19:38 +10:00
Guo Xiang Tan
6d588e43cc FIX: Avoid lograge error when controller doesn't respond to current_user.
https://meta.discourse.org/t/broken-third-party-plugins-can-break-logging/94256
2018-08-20 10:59:56 +08:00
Guo Xiang Tan
402e570c77 Logster::Logger#chained is now readable in new version of logster. 2018-08-14 09:57:09 +08:00
Sam
b504398a91 DEV: keep bin/unicorn clean from log noise 2018-08-14 08:49:57 +10:00
Guo Xiang Tan
dadbf2edb4 DEV: Log to STDOUT in development like how rails s used to. 2018-08-13 17:05:42 +08:00
Sam
c1e7d6e39a move patch cause it was causing a side effect 2018-08-07 17:25:28 +10:00
Sam
f3549291a3 DEV: use unicorn in development
This commit also cleans up a bunch of pointless noise each time we boot app

- narrative was loading i18n cause redefinition of consts
- discourse.rb was loaded twice as was auth
- bin/unicorn now does all the smart things and boots unicron in dev
- bin/rails s will boot unicorn with no params
- remove bin/puma which only causes confusion
2018-08-07 17:13:47 +10:00
Neil Lalonde
0f6dcb8c40 remove unnecessary require 2018-08-01 10:37:10 -04:00
Neil Lalonde
4ad7ce70ce REFACTOR: extract scheduler to the mini_scheduler gem 2018-07-31 17:12:55 -04:00
Vinoth Kannan
af5b88f8e2 Rename approval web hook event type to queued post 2018-07-26 10:29:38 +05:30
Vinoth Kannan
5059dad8f0 FEATURE: Webhook for post approval events 2018-07-25 23:43:19 +05:30
Davide Porrovecchio
dd9d815178 FIX: Add User Api Key headers to CORS
- add User-Api-Key and User-Api-Client-Id to Access-Control-Allow-Headers
- update test
2018-07-24 10:28:23 +10:00
Vinoth Kannan
84ab825e41
FEATURE: Webhook for user destroyed event (#6124) 2018-07-23 13:19:49 +05:30
Guo Xiang Tan
125126c893 Fix typo. 2018-07-20 07:57:00 +08:00
Guo Xiang Tan
9b18ee4bba Improve handling of file params in lograge. 2018-07-19 08:23:59 +08:00
Sam
665ade50c6 SECURITY: extra CORS headers should be set on correct host 2018-07-11 09:29:02 +10:00
Sam
b97b426d34 Revert "FIX: wrong site origin in message bus header"
This reverts commit 93b49a0dd5.
2018-07-11 09:26:43 +10:00
Neil Lalonde
93b49a0dd5 FIX: wrong site origin in message bus header 2018-07-10 15:01:03 -04:00
Dax74
ccc2d94ae8
Update link
See https://meta.discourse.org/t/wrong-link-on-manual-admin-creation/90849
2018-06-27 11:38:01 +02:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Guo Xiang Tan
8a2c5fbebb Remove unused lines. 2018-06-11 08:44:41 +08:00
Guo Xiang Tan
2aad91d4a2 PERF: Don't bloat the Sidekiq queue with Jobs::SendPushNotification. 2018-05-24 10:04:09 +08:00
Sam
6974b7d6a8 FIX: run deferred jobs inline in sidekiq 2018-05-23 12:05:37 +10:00
Guo Xiang Tan
467d91347a Missing specs for Group, Tag, Category and Flag web hooks. 2018-05-21 17:29:58 +08:00
Guo Xiang Tan
bf84037f79 FIX: Payload for webhooks should be current as of the time the event was triggered.
https://meta.discourse.org/t/group-category-tag-user-deleted-webhooks-not-firing/87752
2018-05-21 17:29:58 +08:00
Jeff Wong
2002a7c4ef FIX: regenerate vapid keys when the public key bytes is blank 2018-05-07 17:01:56 -07:00
Jeff Wong
91b31860a1
Feature: Push notifications for Android (#5792)
* Feature: Push notifications for Android

Notification config for desktop and mobile are merged.

Desktop notifications stay as they are for desktop views.

If mobile mode, push notifications are enabled.

Added push notification subscriptions in their own table, rather than through
custom fields.

Notification banner prompts appear for both mobile and desktop when enabled.
2018-05-04 15:31:48 -07:00
Sam
26ce930ac6 FIX: remove auth cookie if we see InvalidAccess 2018-04-20 11:21:51 +10:00
Guo Xiang Tan
1b9a38c5e2 FIX: Incorrect formatter used when logstash formatter is enabled. 2018-04-17 12:07:52 +08:00
Guo Xiang Tan
a902d5b27f Replace ActiveSupport::Logger with DiscourseLogstashLogger when enabled. 2018-04-13 12:08:27 +08:00
Arpit Jalan
a16b616861 FEATURE: webhook for flag events 2018-04-13 07:47:58 +05:30
Vinoth Kannan
434cbc649f FEATURE: Webhook for tag events 2018-04-04 17:49:20 +05:30
Guo Xiang Tan
e04d09d148 Add Process pid to lograge output as well. 2018-04-04 18:31:29 +08:00
Guo Xiang Tan
746889bbc1 Include hostname in lograge payload. 2018-04-04 18:24:30 +08:00
Vinoth Kannan
e7407d0adc FEATURE: Webhook for group and category events 2018-03-27 11:53:35 +05:30
Kyle Zhao
f7bd05e534 FEATURE: set 'Retry-After' header for 429 responses (#5659) 2018-03-13 23:12:41 +08:00
Sam
0134e41286 FEATURE: detect when client thinks user is logged on but is not
This cleans up an error condition where UI thinks a user is logged on
but the user is not. If this happens user will be prompted to refresh.
2018-03-06 16:49:31 +11:00
Guo Xiang Tan
66d620f7b1 FEATURE: Trigger topic webhook when topic status is updated. 2018-02-27 11:07:37 +08:00
scossar
9d0807224b Don't enqueue topic webhook unless a post has a topic 2018-02-22 14:34:59 +08:00
Sam
ca1a3f37e3 FEATURE: add instrumentation for all external net calls 2018-02-21 15:20:29 +11:00
Gerhard Schlager
ce060e2b86 FIX: Server didn't use default_locale as fallback locale 2018-01-25 14:57:41 +01:00
Guo Xiang Tan
54dc191a91 Update rails_multisite to 2.0.1. 2018-01-19 10:19:16 +08:00
Sam
442a17bfb2 PERF: bypass omniauth unless in an auth path 2018-01-15 12:44:54 +11:00
Sam
ea63abf0f7 bypass mini profiler for locales
bypass cdn for now
2018-01-09 11:30:59 +11:00
Arpit Jalan
b205910faa FIX: allow upto three url redirects in onebox (#5457) 2017-12-27 10:30:06 +11:00
Guo Xiang Tan
6c4ee9d5b5 FEATURE: Trigger user webhook when a user logs in/out.
https://meta.discourse.org/t/how-do-you-learn-how-to-build-these-plugins/57946/10?u=tgxworld
2017-12-20 17:47:30 +08:00
Guo Xiang Tan
7d8cd84fa6 Improve lograge custom payload error handling. 2017-12-12 17:27:40 +08:00
Sam
63bdc4056d More log suppressions for rate limit exceeded 2017-12-11 17:52:57 +11:00
Guo Xiang Tan
2ff3444568 Rescue from errors when trying to append custom payloads to lograge output. 2017-12-08 08:31:58 +08:00
Sam
0693ed10d1 remove logging code for now 2017-12-08 08:36:35 +11:00
Vinoth Kannan
58aeaef0c4 FIX: Rescue if default_current_user_provider raised an error 2017-12-08 01:42:55 +05:30
Vinoth Kannan
19247f9087 FIX: current_user should be called only if controller have that method 2017-12-07 23:12:34 +05:30
Sam
90a55d6f7c FIX: handle CORS in hijacked requests 2017-12-07 10:31:04 +11:00
Guo Xiang Tan
6cb2b68b22 current_username -> username. 2017-12-05 12:29:02 +08:00
Guo Xiang Tan
dc858ec120 Include request.remote_ip and current_username in lograge logs. 2017-12-05 11:57:41 +08:00
Guo Xiang Tan
900dec5bc1 Make logster ignore Rack::QueryParser::InvalidParameterError errors. 2017-11-30 15:29:19 +08:00
Guo Xiang Tan
46f8a6c97d Fix broken lograge logs. 2017-11-28 14:00:13 +08:00
Guo Xiang Tan
b907f7d307 Add more metrics to lograge's output if MethodProfiler is enabled. 2017-11-25 08:16:40 +08:00
Sam
2b8d4508e5 PERF: stop running background work between requests
Use a dedicated thread to run Scheduler::Defer

This avoids blocking of a worker during operations that require waiting.

In particular uploads risked blocking a unicorn.

This also add a queue "length" that discourse prometheus consumes.
2017-11-23 15:48:47 +11:00
Guo Xiang Tan
89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Guo Xiang Tan
0d8e9a88c9 FIX: Disable rails_multisite formatter if lograge is enabled. 2017-11-15 09:03:50 +08:00
Sam
86e6732f78 FEATURE: update rails multisite
always allow /srv/status through even if host does not match
2017-11-09 15:40:56 +11:00
Guo Xiang Tan
b840971b77 Convert params hash to a query string instead of a hash when logging. 2017-11-08 15:19:45 +08:00
Guo Xiang Tan
09cee4c315 Don't add time to logs when using logstash formatter. 2017-11-02 14:40:18 +08:00
Guo Xiang Tan
d888d3c54c EXPERIMENTAL: Allow Logstash formatter to be enable for lograge logs. 2017-11-01 08:38:06 +08:00
Guo Xiang Tan
d15068da70 Remove experimental logstash feature.
* We'll most likely be extracting it into a plugin.
2017-10-31 10:15:22 +08:00
Guo Xiang Tan
b8e580b0e0 Set logstash-logger file output to sync. 2017-10-30 20:57:02 +08:00
Guo Xiang Tan
9112d2da3b Add timestamp to lograge output. 2017-10-28 00:46:49 +08:00
Guo Xiang Tan
fba0c9e51e Continue to log to Rails default log file when LogStashLogger is
enabled.
2017-10-28 00:30:11 +08:00
Guo Xiang Tan
8b12a906c6 Fix incorrect URI for LogstashLogger. 2017-10-27 23:52:36 +08:00
Guo Xiang Tan
afdd31473d Add type when shipping logs to logstash. 2017-10-27 23:31:17 +08:00
Guo Xiang Tan
d487be759c EXPERIMENTAL: Allow app to Logstash directly. 2017-10-27 22:56:32 +08:00
Guo Xiang Tan
90d6677d97 EXPERIMENTAL: Allow lograge to be selected as the logging library. 2017-10-27 17:54:45 +08:00
Guo Xiang Tan
d2ac7a9476 Partially revert 4e89d3f778.
* SiteSetting uses DistributedCache which depends on MessageBus.
  Therefore, we need to initialize MessageBus first.
2017-10-12 15:15:24 +08:00
Guo Xiang Tan
23415caecd Print out backtrace as well. 2017-10-12 14:52:18 +08:00
Guo Xiang Tan
4e89d3f778 Only refresh site settings once on initialization. 2017-10-12 14:12:18 +08:00
Guo Xiang Tan
630f7bfba2 Remvoe unused file. 2017-10-12 13:45:00 +08:00
Guo Xiang Tan
69f56e8745 Remove unused file. 2017-10-11 16:05:41 +08:00
Sam
9b4fd0b26b correct multisite issues with scheduler 2017-10-11 18:46:53 +11:00
Guo Xiang Tan
475e86d59e WORKAROUND: Can't reinsert a deleted middleware. 2017-09-25 13:48:59 +08:00
Guo Xiang Tan
3840c50083 Fix logster not ignoring CSRF warnings. 2017-09-25 13:48:59 +08:00
Guo Xiang Tan
77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
David Taylor
7d350d0d75 Revert plugin js changes (#5139)
* Revert "Add disabled_plugins to preloadstore for login_required anonymous users (#5134)"

This reverts commit b840170f8d.

* Revert "Do not load javascripts for disabled plugins (#5103)"

This reverts commit a14ab48829.
2017-09-07 15:15:29 +02:00
David Taylor
a14ab48829 Do not load javascripts for disabled plugins (#5103)
* Do not load javascript for disabled plugins

* Appease rubocop
2017-09-06 10:06:47 +02:00
Guo Xiang Tan
9bc3038728 Fix Redis command errors when trying to start app with a readonly Redis. 2017-08-02 21:33:56 +09:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Robin Ward
0ba39109a0 FIX: Make omit_default_port the global default 2017-06-30 12:43:26 -04:00
Guo Xiang Tan
7ea288140d Allow multiple host when restricting Sidekiq queues. 2017-06-19 14:45:51 +09:00
Felix Freiberger
6729f032b0 Typofix in Onebox user_agent 2017-05-24 14:36:02 -04:00
Robin Ward
8bc93c0b01 FEATURE: Add custom User-Agent to Discourse Oneboxes 2017-05-24 12:19:36 -04:00
Sam
35bb4ae995 FIX: wizard was not showing up if more than 1 system user 2017-05-23 11:09:50 -04:00
Robin Ward
4c690f7089 Use FinalDestination to ensure public redirects for onebox 2017-05-22 16:42:49 -04:00
Sam
e7c2ad41ca Move discourse dev data out of tmp
Fix watch for restart so it works with puma
2017-05-18 11:36:24 -04:00
Guo Xiang Tan
f028d6cd52 Missed a spot when removing Sidekiq::Statistic. 2017-04-26 15:53:11 +08:00
Guo Xiang Tan
f499180bb3 Add ENV variable to only allow Sidekiq scheduler to run on a particular host. 2017-04-26 14:13:26 +08:00
Sam
a3e8c3cd7b FEATURE: Native theme support
This feature introduces the concept of themes. Themes are an evolution
of site customizations.

Themes introduce two very big conceptual changes:

- A theme may include other "child themes", children can include grand
children and so on.

- A theme may specify a color scheme

The change does away with the idea of "enabled" color schemes.

It also adds a bunch of big niceties like

- You can source a theme from a git repo

- History for themes is much improved

- You can only have a single enabled theme. Themes can be selected by
    users, if you opt for it.

On a technical level this change comes with a whole bunch of goodies

- All CSS is now compiled using a custom pipeline that uses libsass
    see /lib/stylesheet

- There is a single pipeline for css compilation (in the past we used
    one for customizations and another one for the rest of the app

- The stylesheet pipeline is now divorced of sprockets, there is no
   reliance on sprockets for CSS bundling

- CSS is generated with source maps everywhere (including themes) this
    makes debugging much easier

- Our "live reloader" is smarter and avoid a flash of unstyled content
   we run a file watcher in "puma" in dev so you no longer need to run
   rake autospec to watch for CSS changes
2017-04-12 10:53:49 -04:00
Guo Xiang Tan
66a7b0c30b FIX: Add web hook DiscourseEvents in initializer. 2017-04-12 11:55:49 +08:00
Guo Xiang Tan
1a7e954e09 FIX: Store custom emojis as uploads.
* Depending on a hardcoded directory was a flawed design
  which made it impossible to debug when custom emojis go
  missing.
2017-03-14 13:07:18 +08:00
Neil Lalonde
0f1495e64e FIX: also add Discourse-Visible to allowed headers for message bus 2017-03-06 16:00:37 -05:00
Neil Lalonde
f9aae7af6a FIX: add Discourse-Visible to CORS allowed headers for sites that use a proxy 2017-03-06 14:41:57 -05:00
Neil Lalonde
954d75f81c FIX: CORS middleware needs to happen earlier than AnonymousCache middleware 2017-03-06 12:25:12 -05:00
Sam
1935f624b8 FEATURE: reset active record cache in sidekiq if needed
This can happen in multisite environments after restores
2017-02-17 12:09:53 -05:00
Sam
ff49f72ad9 FEATURE: per client user tokens
Revamped system for managing authentication tokens.

- Every user has 1 token per client (web browser)
- Tokens are rotated every 10 minutes

New system migrates the old tokens to "legacy" tokens,
so users still remain logged on.

Also introduces weekly job to expire old auth tokens.
2017-02-07 09:22:16 -05:00
Sam
a898d6a02a FIX: silence and quiet logger not thread safe
This caused info/warning to be consistently missing
in dev mode when using puma
2017-01-31 15:44:46 -05:00
Guo Xiang Tan
b8441fba27 Merge pull request #4546 from tgxworld/fix_postgresql_failover
Fix postgresql failover
2016-11-30 09:36:52 +01:00
Sam
3bec128f5f DEV: add warning if force_https in dev 2016-11-25 11:35:29 +11:00
Guo Xiang Tan
759feef3f0 FIX: No loggers may have been chained. 2016-11-17 13:52:08 +08:00
Sam
9848e26190 FIX: force secure cookies on session if force https is enabled 2016-10-27 15:15:58 +11:00
Robin Ward
c03d25f170 FEATURE: Configure Admin Account
Adds a "Step 0" to the wizard if the site has no admin accounts where
the user is prompted to finish setting up their admin account from the
list of acceptable email addresses.

Once confirmed, the wizard begins.
2016-10-19 11:27:56 -04:00
Guo Xiang Tan
596fcfeb58 FIX: Set formatter for original Rails logger. 2016-09-15 23:51:22 +08:00
Sam
50379183e1 FIX: allow the Dont-Chunk header for remote message bus access 2016-08-31 12:57:19 +10:00
Jeff Atwood
bb5dc839a0 extend existing log suppression for empty JS errors 2016-08-29 00:47:11 -07:00
Jeff Atwood
6fd3ebb253 don't log any JavaScript "errors" from IE 9 2016-08-29 00:35:01 -07:00
Sam
4fe52c8cbe FEATURE: backend support for pushing notifications to clients 2016-08-26 12:47:10 +10:00
Sam
846a08d3c7 FIX: improve error results for message bus failures 2016-08-25 15:39:39 +10:00
Guo Xiang Tan
dc2dae2cc4 FIX: Logs notice was not displaying the right Date. 2016-08-02 12:40:28 +08:00
Guo Xiang Tan
0ffe8402a9
FIX: SiteSettings not avaliable in initializers on dev env. 2016-07-04 11:15:29 +08:00
Guo Xiang Tan
42ce59d257
Partially revert https://github.com/discourse/discourse/commit/09b92dd3. 2016-07-01 01:28:49 +08:00
Guo Xiang Tan
8db3ab5f2a Merge pull request #4292 from tgxworld/rename_use_https_to_force_https
Rename `SiteSetting#use_https` to `force_https`.
2016-06-29 15:17:57 +08:00
Guo Xiang Tan
20359788dc
Rename SiteSetting#use_https to force_https. 2016-06-29 15:02:43 +08:00
Guo Xiang Tan
09b92dd345
Bump minimum Ruby version to 2.3. 2016-06-10 11:40:42 +08:00
Jeff Atwood
8b5dfeb18f ignore a few more common meaningless JS errs 2016-06-09 16:38:46 -07:00
Guo Xiang Tan
256d7a00e9 Update sprockets. (#4167)
* Update sass-rails.

* FIX: Tilt dependency has been removed from Ember::Handlebars::Template.

* Update `DiscourseIIFE` to new Sprockets API.

* `Rails.application.assets` returns `nil` in production.

* Move sprockets-rails out of the assets group.

* Pin ember-rails to 0.18.5 which works with Sprockets 3.x.

* Update sprockets to 3.6.0.

* Make `DiscourseSassCompiler` work with Sprockets 3.

* Use `Sass::Rails::SassImporterGlobbing` instead of haxxing our own.

* Moneky patch so that we don't add dependencies for our custom css.

* FIX: Missing class.

* Upgrade ember-handlebars-template.

* FIX: require path needs to share the same root as the folder's path.

* Bump discourse-qunit-rails.

* Update ember-template-compiler.js to 1.12.2.

* `prepend` is private in Ruby 2.0.0.
2016-04-18 10:47:52 +08:00
Neil Lalonde
3e5f2bd1cf FIX: replace reference to Google_oauth2 with Google during signup 2016-03-29 18:08:55 -04:00
Robin Ward
e407ef9d0e FIX: Also support nil values for the site settings 2016-03-24 11:14:17 -04:00
Robin Ward
b270e0142b FIX: If site settings are missing don't do anything 2016-03-24 11:12:54 -04:00
Guo Xiang Tan
5dc5767851 FIX: Assigning wrong value to variable. 2016-03-21 15:18:31 +08:00
Guo Xiang Tan
54bdcd9b98 Update to new rate limit Redis config. 2016-03-21 14:00:50 +08:00
Régis Hanol
1c19548e30 Merge pull request #4094 from fantasticfears/omniauth-config
FIX: redirect output omniauth log to Rails logger instead of stdout
2016-03-19 18:03:16 +01:00
Erick Guan
7df33ca287 FIX: redirect output omniauth log to Rails logger instead of stdout 2016-03-19 13:17:13 +01:00
Jeff Atwood
40d6f062f7 clarify why block all mobile/tablet for miniprofiler 2016-03-15 16:54:40 -07:00
Jeff Atwood
d052f1746c add support for other Nexus devices as "mobile" 2016-03-15 16:46:46 -07:00
Guo Xiang Tan
90fde5053d FIX: Load Redis patch much earlier. 2016-03-11 17:29:00 +08:00
Guo Xiang Tan
c07c474575 FEATURE: Master-Slave Redis configuration with fallback and switch over. 2016-03-11 12:18:58 +08:00