Commit Graph

28632 Commits

Author SHA1 Message Date
Blake Erickson
aa0a236416
SECURITY: ensure topic is valid before updating category (#22547)
Co-authored-by: David Battersby <info@davidbattersby.com>
2023-07-11 15:23:48 -06:00
Blake Erickson
06ab681498
SECURITY: Don't reuse CSP nonce between requests (#22553)
Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
2023-07-11 15:23:04 -06:00
Blake Erickson
2a14338133
SECURITY: Prevent dismissal of topics that user can't see (#22090)
Co-authored-by: OsamaSayegh <asooomaasoooma90@gmail.com>
2023-06-13 11:10:10 -06:00
Blake Erickson
a891e06989
SECURITY: set max-height property for iframes (#22089)
Co-authored-by: David Battersby <info@davidbattersby.com>
2023-06-13 11:09:40 -06:00
Blake Erickson
5e3106387f
SECURITY: Use canonical url for topic embeddings (#22088)
This prevents duplicate topics from being created when using embed_urls
that only differ on query params.
2023-06-13 11:09:23 -06:00
Ted Johansson
c9ee11c675
FIX: Remove obsolete references to lounge category (#21608)
### What is this change?

The lounge category was replaced with the general category in https://github.com/discourse/discourse/pull/18097.

However, there are still a few references to the lounge category in code. In particular, `Category#seeded?` is erroring out in production looking for `SiteSetting.lounge_category_id`.
2023-05-17 16:48:31 +08:00
Ted Johansson
aaec964547
DEV: Add both safe and unsafe Discourse.store.download methods (stable) (#21499)
### Background

Several call sites use `FileStore#download` (through `Discourse.store.download`). In some cases the author seems aware that the method can raise an error if the download fails, and in some cases not. Because of this we're seeing some of these exceptions bubble all the way up and getting logged in production. Although they are not really actionable at that point. Rather each call site needs to be considered to figure out how to handle them.

### What is this change?

This change accomplishes primarily two things.

Firstly it separates the method into a safe version which will handle errors by returning `nil`, and an unsafe version which will re-package upstream errors in a new `FileStore::DownloadError` class.

Secondly it updates the call sites which have been doing error handling downstream to use the new safe version.

For backwards compatibility, there's an interim situation and a desired end state.

**Interim:**

```
FileStore#download      → Old unsafe version. Will raise any error and show a deprecation warning.
FileStore#download!     → New unsafe version. Will raise FileStore::DownloadError.
FileStore#download_safe → New safe version.   Will return nil.
```

**Desired end-state:**

```
FileStore#download  → New safe version.   Will return nil.
FileStore#download! → New unsafe version. Will raise FileStore::DownloadError.
```

### What's next?

We need to do a quick audit of the call sites that are using the old unsafe version without any error handling, as well as check for call sites in plugins other repos. Follow-up PRs incoming.
2023-05-12 11:38:08 +08:00
Ted Johansson
0c11acf6cf
SECURITY: Encode embed url (#21134)
The embed_url in "This is a companion discussion..." could be used for
XSS.

Co-authored-by: Blake Erickson <o.blakeerickson@gmail.com>
2023-04-18 15:05:33 +08:00
Ted Johansson
bbc7746cef
SECURITY: Ensure site setting being updated is a configurable site setting (#21132) 2023-04-18 14:32:21 +08:00
Krzysztof Kotlarek
bd301c3f08
FIX: improve performance of UserStat.ensure_consistency (#21044) (#21121)
Optimize `UserStatpost_read_count` calculation.

In addition, tests were updated to fail when code is not evaluated. Creation of PostTiming was updating `post_read_count`. Count it has to be reset to ensure that ensure_consitency correctly calculates result.

Extracting users seen in the last hour to separate Common Table Expression reduces the amount of processed rows.

Before
```
Update on user_stats  (cost=267492.07..270822.95 rows=2900 width=174) (actual time=12606.121..12606.127 rows=0 loops=1)
  ->  Hash Join  (cost=267492.07..270822.95 rows=2900 width=174) (actual time=12561.814..12603.689 rows=10 loops=1)
        Hash Cond: (user_stats.user_id = x.user_id)
        Join Filter: (x.c <> user_stats.posts_read_count)
        Rows Removed by Join Filter: 67
        ->  Seq Scan on user_stats  (cost=0.00..3125.34 rows=75534 width=134) (actual time=0.014..39.173 rows=75534 loops=1)
        ->  Hash  (cost=267455.80..267455.80 rows=2901 width=48) (actual time=12558.613..12558.617 rows=77 loops=1)
              Buckets: 4096  Batches: 1  Memory Usage: 39kB
              ->  Subquery Scan on x  (cost=267376.03..267455.80 rows=2901 width=48) (actual time=12168.601..12558.572 rows=77 loops=1)
                    ->  GroupAggregate  (cost=267376.03..267426.79 rows=2901 width=12) (actual time=12168.595..12558.525 rows=77 loops=1)
                          Group Key: pt.user_id
                          ->  Sort  (cost=267376.03..267383.28 rows=2901 width=4) (actual time=12100.490..12352.106 rows=2072830 loops=1)
                                Sort Key: pt.user_id
                                Sort Method: external merge  Disk: 28488kB
                                ->  Nested Loop  (cost=1.28..267209.18 rows=2901 width=4) (actual time=0.040..11528.680 rows=2072830 loops=1)
                                      ->  Nested Loop  (cost=0.86..261390.02 rows=13159 width=8) (actual time=0.030..3492.887 rows=3581648 loops=1)
                                            ->  Index Scan using index_users_on_last_seen_at on users u  (cost=0.42..89.71 rows=28 width=4) (actual time=0.010..0.201 rows=78 loops=1)
                                                  Index Cond: (last_seen_at > '2023-04-11 00:22:49.555537'::timestamp without time zone)
                                            ->  Index Scan using index_post_timings_on_user_id on post_timings pt  (cost=0.44..9287.60 rows=4455 width=8) (actual time=0.081..38.542 rows=45919 loops=78)
                                                  Index Cond: (user_id = u.id)
                                      ->  Index Scan using forum_threads_pkey on topics t  (cost=0.42..0.44 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=3581648)
                                            Index Cond: (id = pt.topic_id)
                                            Filter: ((deleted_at IS NULL) AND ((archetype)::text = 'regular'::text))
                                            Rows Removed by Filter: 0
Planning Time: 0.692 ms
Execution Time: 12612.587 ms
```
After
```
Update on user_stats  (cost=9473.60..12804.30 rows=2828 width=174) (actual time=677.724..677.729 rows=0 loops=1)
  ->  Hash Join  (cost=9473.60..12804.30 rows=2828 width=174) (actual time=672.536..677.706 rows=1 loops=1)
        Hash Cond: (user_stats.user_id = x.user_id)
        Join Filter: (x.c <> user_stats.posts_read_count)
        Rows Removed by Join Filter: 54
        ->  Seq Scan on user_stats  (cost=0.00..3125.34 rows=75534 width=134) (actual time=0.012..23.977 rows=75534 loops=1)
        ->  Hash  (cost=9438.24..9438.24 rows=2829 width=48) (actual time=647.818..647.822 rows=55 loops=1)
              Buckets: 4096  Batches: 1  Memory Usage: 37kB
              ->  Subquery Scan on x  (cost=9381.66..9438.24 rows=2829 width=48) (actual time=647.409..647.805 rows=55 loops=1)
                    ->  HashAggregate  (cost=9381.66..9409.95 rows=2829 width=12) (actual time=647.403..647.786 rows=55 loops=1)
                          Group Key: pt.user_id
                          Batches: 1  Memory Usage: 121kB
                          ->  Nested Loop  (cost=1.86..9367.51 rows=2829 width=4) (actual time=0.056..625.245 rows=120022 loops=1)
                                ->  Nested Loop  (cost=1.44..3692.96 rows=12832 width=8) (actual time=0.047..171.754 rows=217440 loops=1)
                                      ->  Nested Loop  (cost=1.00..254.63 rows=25 width=12) (actual time=0.030..1.407 rows=56 loops=1)
                                            Join Filter: (u.id = user_stats_1.user_id)
                                            ->  Nested Loop  (cost=0.71..243.08 rows=25 width=8) (actual time=0.018..1.207 rows=87 loops=1)
                                                  ->  Index Scan using index_users_on_last_seen_at on users u  (cost=0.42..86.71 rows=27 width=4) (actual time=0.009..0.156 rows=87 loops=1)
                                                        Index Cond: (last_seen_at > '2023-04-11 00:47:07.437568'::timestamp without time zone)
                                                  ->  Index Only Scan using user_stats_pkey on user_stats us  (cost=0.29..5.79 rows=1 width=4) (actual time=0.011..0.011 rows=1 loops=87)
                                                        Index Cond: (user_id = u.id)
                                                        Heap Fetches: 87
                                            ->  Index Scan using user_stats_pkey on user_stats user_stats_1  (cost=0.29..0.45 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=87)
                                                  Index Cond: (user_id = us.user_id)
                                                  Filter: (posts_read_count < 10000)
                                                  Rows Removed by Filter: 0
                                      ->  Index Scan using index_post_timings_on_user_id on post_timings pt  (cost=0.44..92.98 rows=4455 width=8) (actual time=0.036..2.492 rows=3883 loops=56)
                                            Index Cond: (user_id = user_stats_1.user_id)
                                ->  Index Scan using forum_threads_pkey on topics t  (cost=0.42..0.44 rows=1 width=4) (actual time=0.002..0.002 rows=1 loops=217440)
                                      Index Cond: (id = pt.topic_id)
                                      Filter: ((deleted_at IS NULL) AND ((archetype)::text = 'regular'::text))
                                      Rows Removed by Filter: 0
Planning Time: 1.406 ms
Execution Time: 677.817 ms
```
2023-04-18 10:04:50 +10:00
David Taylor
428b0c91ac
SECURITY: Limit URL length for theme remote (stable) (#20788) 2023-03-23 12:07:02 +00:00
Blake Erickson
3374457c44 SECURITY: Fix XSS in full name composer reply
We are using htmlSafe when rendering the name field so we need to escape
any html being passed in.
2023-03-16 16:25:48 -06:00
Loïc Guitaut
78a3efa710 SECURITY: Rate limit the creation of backups 2023-03-16 16:09:08 +01:00
Andrei Prigorshnev
ded6aed15e
FIX: avoid race condition when setting user status (#19817) (#20182)
This is a backport of 84e13e9.

We caught it in logs, race condition led to this error:

    ActiveRecord::RecordNotUnique 
    (PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "user_statuses_pkey"
    DETAIL:  Key (user_id)=(15) already exists.)


The reason the problem happened was that we were checking if a user has status and if not inserting status:

    if user_status
      ...
    else
      self.user_status = UserStatus.create!(status)
    end

The problem is that it's possible that another request will insert status just after we check if status exists and just before our request call `UserStatus.create!(status)`. Using `upsert` fixes the problem because under the hood `upsert` generates the only SQL request that uses "INSERT ... ON CONFLICT DO UPDATE". So we do everything in one SQL query, and that query takes care of resolving possible conflicts.
2023-02-06 20:32:45 +04:00
Keegan George
a409924425 FIX: Failing system spec for rate limited search (#20046) 2023-02-01 19:05:58 -08:00
Sam
1856ea83ec FEATURE: rate limit anon searches per second (#19708) 2023-02-01 19:05:58 -08:00
Alan Guo Xiang Tan
42d2cb2d4e
SECURITY: Hide PM count for tags by default (#20061) (#20090)
Currently `Topic#pm_topic_count` is a count of all personal messages tagged for a given tag. As a result, any user with access to PM tags can poll a sensitive tag to determine if a new personal message has been created using that tag even if the user does not have access to the personal message. We classify this as a minor leak in sensitive information.

With this commit, `Topic#pm_topic_count` is hidden from users by default unless the `display_personal_messages_tag_counts` site setting is enabled.
2023-02-01 06:43:58 +08:00
Jarek Radosz
df70c8bf1c FIX: Don't spam presence requests when getting 429 (#20084)
The presence service would retry `/presence/update` requests every second (or immediately in tests) in case where server returns 429 (rate limit) errors. That could lead to infinite spamming (until user refreshed tab/tabs)

Co-authored-by: David Taylor <david@taylorhq.com>
2023-01-31 14:04:21 +01:00
Jarek Radosz
78fe2656b4 DEV: Convert presence service tests to actual unit tests (#20076) 2023-01-31 14:04:21 +01:00
Bianca Nenciu
ecb9aa5dba
SECURITY: only show restricted tag lists to authorized users (#20005)
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-01-25 18:56:00 +02:00
Bianca Nenciu
87b95356f7 SECURITY: Remove bypass for base_url
The check used to be necessary because we validated the referrer too and
this bypass was a workaround a bug that is present in some browsers that
do not send the correct referrer.
2023-01-25 13:53:22 +02:00
Ted Johansson
3e0cc4a5d9 SECURITY: Limit the character count of group membership requests
When creating a group membership request, there is no character
limit on the 'reason' field. This can be potentially be used by
an attacker to create enormous amount of data in the database.
2023-01-25 13:53:07 +02:00
Loïc Guitaut
3dcd0bc544 SECURITY: Limit the length of drafts 2023-01-25 13:52:58 +02:00
Aleksey Bogdanov
971f97d03a FIX: text selection breaks opening of links in new tabs (#19867)
When a user checks "Open all external links in a new tab" preference
he expects not to be overruled by unrelated text selections.
Yet if text is selected during a link click the link is followed on
the same tab. This change corrects that.
2023-01-25 13:48:49 +02:00
Martin Brennan
cb6572e940 FIX: Do not add empty use/svg tags in ExcerptParser (#19969)
There was an issue where if hashtag-cooked HTML was sent
to the ExcerptParser without the keep_svg option, we would
end up with empty </use> and </svg> tags on the parts of the
excerpt where the hashtag was, in this case when a post
push notification was sent.

Fixed this, and also added a way to only display a plaintext
version of the hashtag for cases like this via PrettyText#excerpt.
2023-01-25 13:48:49 +02:00
Vinoth Kannan
01b903dc83 FIX: skip email if blank while syncing SSO attributes. (#19939)
Also, return email blank error in `EmailValidator`  when the email is blank.
2023-01-25 13:48:49 +02:00
Krzysztof Kotlarek
1ef2031fae FIX: TL4 user is not redirected to latest when delete topic (#19967)
Continue of https://github.com/discourse/discourse/pull/19766

When TL4 is allowed to delete topic, they should not be redirected to / after that action.
2023-01-25 13:48:49 +02:00
Jordan Vidrine
5361fa3421 FIX: Fix margin on mini-tag-chooser (#19953) 2023-01-25 13:48:49 +02:00
Osama Sayegh
20d344b5c6 FIX: Allow modals to scroll on mobile when keyboard is open (#19930)
Meta topic: https://meta.discourse.org/t/android-keyboard-overlaps-text-when-flagging-with-something-else/249687?u=osama

On Android, it's currently not possible to scroll modals that take input from the user (such as the flagging modal) when the keyboard is open which means that the keyboard can cover up part of the modal with no way for the user to see the covered part without closing the keyboard. This commit adds some CSS to make these modals scrollable when the keyboard is open.
2023-01-25 13:48:49 +02:00
Ted Johansson
a95b2f9140 FIX: Don't display staff-only options to non-staff in group member bulk menu (#19907)
In the group member bulk edit menu we are displaying staff-only options
to non-staff. The requests are blocked by the back-end, so there is no
harm other than to the user experience.

Notably the individual user edit menu is correctly filtering out
unavailable options. This change brings the bulk edit menu in line with
that.
2023-01-25 13:48:49 +02:00
Krzysztof Kotlarek
db3f569398 FIX: move min tag setting to tags section in edit category (#19789)
`Minimum number of tags required in a topic` should be in `Tags` panel instead of `Settings`
2023-01-25 13:48:49 +02:00
Krzysztof Kotlarek
023accf6a8 FIX: deleted misconfigured embeddable hosts (#19833)
When EmbeddableHost is configured for a specific category and that category is deleted, then EmbeddableHost should be deleted as well.

In addition, migration was added to fix existing data.
2023-01-25 13:48:49 +02:00
Alan Guo Xiang Tan
ef57771b08 FIX: Preload user sidebar attrs when ?enable_sidebar=1 (#19843)
This allows users to preview the sidebar even when
`SiteSetting.naviation_menu` is set to `false`.
2023-01-25 13:48:49 +02:00
Daniel Waterworth
07a1211183 FIX: Prevent concurrent updates to top_topics (#19854)
to prevent lock timeouts
2023-01-25 13:48:49 +02:00
Ted Johansson
051db5e958 FIX: Ruby 2 backward compatible plugin logout redirect (#19845)
This is a very subtle one. Setting the redirect URL is done by passing
a hash through a Discourse event. This is broken on Ruby 2 since the
support for keyword arguments in events was added.

In Ruby 2 the last argument is cast to keyword arguments if it is a
hash. The key point here is that creates a new copy of the hash, so
what the plugin is modifying is not the hash that was passed.
2023-01-25 13:48:49 +02:00
Alan Guo Xiang Tan
0e69aeb276
SECURITY: Default tags to show count of topics in unrestricted categories (#19929)
Currently, `Tag#topic_count` is a count of all regular topics regardless of whether the topic is in a read restricted category or not. As a result, any users can technically poll a sensitive tag to determine if a new topic is created in a category which the user has not excess to. We classify this as a minor leak in sensitive information.

The following changes are introduced in this commit:

1. Introduce `Tag#public_topic_count` which only count topics which have been tagged with a given tag in public categories.
2. Rename `Tag#topic_count` to `Tag#staff_topic_count` which counts the same way as `Tag#topic_count`. In other words, it counts all topics tagged with a given tag regardless of the category the topic is in. The rename is also done so that we indicate that this column contains sensitive information.
3. Change all previous spots which relied on `Topic#topic_count` to rely on `Tag.topic_column_count(guardian)` which will return the right "topic count" column to use based on the current scope.
4. Introduce `SiteSetting.include_secure_categories_in_tag_counts` site setting to allow site administrators to always display the tag topics count using `Tag#staff_topic_count` instead.
2023-01-20 11:59:37 +08:00
Alan Guo Xiang Tan
c368d35602
FIX: Regression in TopicTrackingState MessageBus message scope. (#19835) (#19837)
0403cda1d1 introduced a regression where
topics in non read-restricted categories have its TopicTrackingState
MessageBus messages published with the `group_ids: [nil]` option. This
essentially means that no one would be able to view the message.
2023-01-12 08:52:02 +08:00
Alan Guo Xiang Tan
0403cda1d1
FIX: Error when publishing TopicTrackingState updates for certain topics (#19812)
When a topic belongs to category that is read restricted but permission
has not been granted to any groups, publishing ceratin topic tracking state
updates for the topic will result in the `MessageBus::InvalidMessageTarget` error being raised
because we're passing `nil` to `group_ids` which is not support by
MessageBus.

This commit ensures that for said category above, we will publish the
updates to the admin groups.
2023-01-11 06:15:52 +08:00
Alan Guo Xiang Tan
8a7b62b126
DEV: Fix threading error when running jobs immediately in system tests (#19811)
```
class Jobs::DummyDelayedJob < Jobs::Base
  def execute(args = {})
  end
end

RSpec.describe "Jobs.run_immediately!" do
  before { Jobs.run_immediately! }

  it "explodes" do
    current_user = Fabricate(:user)
    Jobs.enqueue_in(1.seconds, :dummy_delayed_job)
    sign_in(current_user)
  end
end
```

The test above will fail with the following error if `ActiveRecord::Base.connection_handler.clear_active_connections!` is called before the configured Capybara server checks out a connection from the connection pool.

```
     ActiveRecord::ActiveRecordError:
       Cannot expire connection, it is owned by a different thread: #<Thread:0x00007f437391df58@puma srv tp 001 /home/tgxworld/.asdf/installs/ruby/3.1.3/lib/ruby/gems/3.1.0/gems/puma-6.0.2/lib/puma/thread_pool.rb:106 sleep_forever>. Current thread: #<Thread:0x00007f437d6cfc60 run>.
```

We're not exactly sure if this is an ActiveRecord bug or not but we've
invested too much time into investigating this problem. Fundamentally,
we also no longer understand why `ActiveRecord::Base.connection_handler.clear_active_connections!` is being called in an ensure block
within `Jobs::Base#perform` which was added in
ceddb6e0da 10 years ago. This
commit moves the logic for running jobs immediately out of the
`Jobs::Base#perform` method into another `Jobs::Base#perform_immediately` method such that
`ActiveRecord::Base.connection_handler.clear_active_connections!` is not
called. This change will only impact the test environment.
2023-01-10 13:41:25 +08:00
Kris
b572f69f5f
UX: add btn-primary class on email confirm page (#19803) 2023-01-09 15:26:56 -05:00
Daniel Waterworth
dfc449a32e
FIX: Continue if checking themes if updating one fails (#19665) 2023-01-09 11:56:43 -06:00
David Taylor
f05da14fb6
DEV: Do not include method definitions in serializer attributes
These accidental inclusions are mostly no-ops (because the method name is also included as an explicit symbol). The mistakes were made more obvious because syntax_tree adjusted the indentation of these methods
2023-01-09 14:15:00 +00:00
David Taylor
5a003715d3
DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
chapoi
a641ce4b62
UX: improve use of space for readability of admin email modal (#19797) 2023-01-09 14:24:58 +01:00
Ted Johansson
06bda1fc62
FIX: wrap plugin outlets on user preference page 2023-01-09 14:17:50 +01:00
Martin Brennan
c31772879b
FIX: Disable image optimization in iOS Safari (#19790)
There are various performance issues with the Canvas in iOS Safari
that are causing crashes when processing images with spikes of over 100%
CPU usage. The cause of this is unknown, but profiling points to
CanvasRenderingContext2D.getImageData() and
CanvasRenderingContext2D.drawImage().

Until Safari makes some progress with OffscreenCanvas or other
alternatives we cannot support this workflow. We will revisit in 6
months.

This is gated behind the hidden `composer_ios_media_optimisation_image_enabled`
site setting for people who really still want to try using this.
2023-01-09 12:16:02 +10:00
Alan Guo Xiang Tan
673089a6b4
FIX: Error condition in SidebarSiteSettingsBackfiller (#19787) 2023-01-09 07:01:58 +08:00
dependabot[bot]
15e81b6174
Build(deps): Bump jsdom from 20.0.3 to 21.0.0 in /app/assets/javascripts (#19786)
Bumps [jsdom](https://github.com/jsdom/jsdom) from 20.0.3 to 21.0.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/master/Changelog.md)
- [Commits](https://github.com/jsdom/jsdom/compare/20.0.3...21.0.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-08 23:41:14 +01:00
Roman Rizzi
7b5f7b4484
FIX: Don't change the default allowed_attribute when calling #sanitize_field (#19770) 2023-01-06 11:47:15 -03:00
David Taylor
5ce5ff053e
FIX: Bump messagebus to v4.3.1 (#19771)
Includes "FIX: Ensure non-long-polling requests are always spaced out": 233b248c96
2023-01-06 14:31:10 +00:00