Commit Graph

9247 Commits

Author SHA1 Message Date
Sam
0364ef5efe
FIX: MaxMindDB download failures caused a crash on rebuild (#20737)
We did not properly define the "name" variable leading to the error handling
to crash.

This quick fix ensures it can gracefully fail.
2023-03-20 18:23:54 +11:00
Daniel Waterworth
293cb7bde2
FIX: An ember build is required to run the system tests (#20725) 2023-03-17 13:20:49 -05:00
Blake Erickson
6b5743ba3c
Version bump to v3.1.0.beta3 (#20712) 2023-03-16 17:51:54 -06:00
Ted Johansson
39c2f63b35 SECURITY: Add FinalDestination::FastImage that's SSRF safe 2023-03-16 15:27:09 -06:00
Alan Guo Xiang Tan
fd16eade7f SECURITY: SSRF protection bypass with IPv4-mapped IPv6 addresses
As part of this commit, we've also expanded our list of private IP
ranges based on
https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
and https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml
2023-03-16 15:27:09 -06:00
Alan Guo Xiang Tan
52ef44f43b SECURITY: Monkey-patch web-push gem to use safer HTTP client
`FinalDestination::HTTP` is our patch of `Net::HTTP` which defend us
against SSRF and DNS rebinding attacks.
2023-03-16 15:27:09 -06:00
Loïc Guitaut
0bd64788d2 SECURITY: Rate limit the creation of backups 2023-03-16 16:09:22 +01:00
Daniel Waterworth
164b60cd07 DEV: Optionally, run system tests in docker:test 2023-03-15 16:46:48 -05:00
Daniel Waterworth
84f590ab83
DEV: Store theme sprites in the DB (#20501)
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
David Taylor
964f37476d
FIX: TopicQuery for NULL category.topic_id (#20664)
Our schema allows `category.topic_id` to be NULL. Null values shouldn't actually happen in production, but it is very common in tests because `Fabricate(:category)` skips creating the definition topic to improve performance. Before this commit, a NULL category.topic_id would cause all subcategory topics to be excluded from a TopicQuery result. This is because, in postgres, `NULL <> anything` is falsy. Instead, we can use `IS DISTINCT FROM`, which will return true when NULL is compared to a non-NULL value.
2023-03-13 19:33:26 +00:00
Rafael dos Santos Silva
0a5b078ac7
FEATURE: Hook for suggested topic customization (#20618) 2023-03-13 15:37:49 -03:00
Alan Guo Xiang Tan
e4b11e7643
FEATURE: Only list watching group messages in messages notifications panel (#20630)
Why is this change required?

Prior to this change, we would list all group messages that a user
has access to in the user menu messages notifications panel dropdown.
However, this did not respect the topic's notification level setting and
group messages which the user has set to 'normal' notification level were
being displayed

What does this commit do?

With this commit, we no longer display all group messages that a user
has access to. Instead, we only display group messages that a user is
watching in the user menu messages notifications panel dropdown.

Internal Ref: /t/94392
2023-03-13 08:09:38 +08:00
Alan Guo Xiang Tan
0d9efa938b
DEV: Avoid logging routing errors (#20622)
The logs are usually caused by the client and is of no use to us.
2023-03-10 17:17:59 +08:00
Jay Pfaffman
4350a903ec
clear security keys in disable_2fa rake task (#20586) 2023-03-09 15:38:59 -05:00
Martin Brennan
5ea89d1fcb
FIX: UploadReference order by tiebreaker for UploadSecurity (#20602)
Follow up to 4d2a95ffe6. Sometimes
due to the original UploadReference migration or other issues,
multiple UploadReference records can have the exact same
created_at date and time. To tiebreak and correct the SQL order
when this happens, we can add a secondary `id ASC` ordering
when we check for the first upload reference.
2023-03-09 11:52:26 +10:00
Martin Brennan
6feb436303
DEV: Change external upload rate limit maximums to settings (#20577)
Way back when this was introduced way back in b96c10a903
I didn't have any frame of reference for what these max rate
limit numbers should be, so 10 seemed like a reasonable limit
until a real world case where this did not make sense came
along.

The time has come.

Moving these into site settings, which are hidden since in most
cases there is no need to change these.
2023-03-08 15:27:17 +10:00
Alan Guo Xiang Tan
cf0a0945e4
Revert "DEV: Allow webmock to intercept FinalDestination::HTTP requests (#20575)" (#20576) 2023-03-08 11:26:32 +08:00
Alan Guo Xiang Tan
500d0f6daf
DEV: Allow webmock to intercept FinalDestination::HTTP requests (#20575) 2023-03-08 10:40:01 +08:00
Sam
3f5fa4eb09
DEV: avoid mocking FinalDestination (#20570) 2023-03-08 09:09:18 +08:00
Martin Brennan
360d0dde65
DEV: Change Bookmarkable registration to DiscoursePluginRegistry (#20556)
Similar spirit to e195e6f614,
this moves the Bookmarkable registration to DiscoursePluginRegistry
so plugins which are not enabled do not register additional
bookmarkable classes.
2023-03-08 10:39:12 +10:00
Osama Sayegh
3f908c047d
FIX: Use the default value correctly for theme settings of type uploads (#20541)
When a theme setting of type `upload` has a default upload, it should return the URL of the specified default upload until a custom upload is used for the setting. However, currently this isn't the case and we get null instead of the default upload URL.

The reason for this is because the `super` method of `#value` already returns the default upload URL (if there's one), so we can't pass that to `cdn_url` which expects an upload ID:

c961dcc757/lib/theme_settings_manager.rb (L212)

This commit fixes the bug by skipping the call to `cdn_url` when we fallback to the default upload for the setting value.
2023-03-06 11:41:47 +03:00
Alan Guo Xiang Tan
66c50547b4
DEV: Experimental /filter route to filter through topics (#20494)
This commit introduces an experimental `/filter` route which allows a
user to input a query string to filter through topics.

Internal Ref: /t/92833
2023-03-03 09:46:21 +08:00
Leonardo Mosquera
e204c61bd8
FIX: #present -> #presence typo in TopicRetriever (#20509) 2023-03-02 13:05:45 -03:00
chapoi
e52bbc1230
UX/DEV: Review queue redesign fixes (#20239)
* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* DEV: switch to selectKit

* UX: color approve/reject buttons in RQ

* DEV: regroup actions

* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* Join questions for flagged post with "or" with new I18n function
* Move ReviewableScores component out of context
* Add CSS classes to reviewable-item based on human type

* UX: add table header for scoring

* UX: don't display % score

* UX: prefix modifier class with dash

* UX: reviewQ flag table styling

* UX: consistent use of ignore icon

* DEV: only show context question on pending status

* UX: only show table headers on pending status

* DEV: reviewQ regroup actions for hidden posts

* UX: reviewQ > approve/reject buttons

* UX: reviewQ add fadeout

* UX: reviewQ styling

* DEV: move scores back into component

* UX: reviewQ mobile styling

* UX: score table on mobile

* UX: reviewQ > move meta info outside table

* UX: reviewQ > score layout fixes

* DEV: readd `agree_and_keep` and fix the spec tests.

* Fix the spec tests

* fix the quint test

* DEV: readd deleting replies

* UX: reviewQ copy tweaks

* DEV: readd test for ignore + delete replies

* Remove old

* FIX: Add perform_ignore back in for backwards compat

* DEV: add an action alias `ignore` for `ignore_and_do_nothing`.

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Vinoth Kannan <svkn.87@gmail.com>
2023-03-02 16:40:53 +01:00
Martin Brennan
e195e6f614
DEV: Move about_stat_groups to DiscoursePluginRegistry (#20496)
Follow up to 098ab29d41. Since
we just used a `cattr_reader` on `About` this was not safe
for multisite, since some sites could have the chat plugin
enabled and some may not. Using `DiscoursePluginRegistry` gets
around this issue, and makes it so the chat stats only show
for a site if `chat_enabled` is true.
2023-03-02 08:10:16 +10:00
Keegan George
666b4a7e6b
DEV: Define form template field inputs (#20430) 2023-03-01 11:07:13 -08:00
Loïc Guitaut
8b67a534a0 FIX: Allow floats for zoom level in Google Maps onebox
Sometimes we get Maps URL containing a zoom level as a float (17.5z and
not 17z) but this doesn’t work with our current onebox implementation.

While Google accepts those float zoom levels, it removes automatically
the floating part in the URL (thus when visiting a Maps URL containing
17.5z, the URL will be rewritten shortly after as 17z). When putting a
float zoom level in an embedded URL, this actually breaks (Maps API
returns a 400 error).

This patch addresses the issue by allowing the onebox engine to match on
a zoom level expressed as a float but we only keep the integer part thus
rendering properly maps.
2023-03-01 12:45:33 +01:00
Jarek Radosz
ddbe6cf3b5
DEV: Remove site setting "diags" (#20481)
Added almost exactly 10 years ago (in a2cca2540e), I don't think they're useful anymore 😅
2023-03-01 11:31:57 +01:00
Sam
71be74ffd3
FIX: recalculating trust levels was not working (#20492)
The recalculate code was never firing cause TrustLevel.calculate unconditionally
returned a trust level. (albeit a wrong one)

New code ensures we only bypass promotion checks for cases where trust level
is locked.

see: https://meta.discourse.org/t/user-trust-level-resets-to-zero-when-unlocked/255444
2023-03-01 15:35:21 +11:00
Constanza
c051992098
FIX: mark posted as true for post authors in the TopicUser table during ensure_consistency task (#20473)
The ensure_consistency rake task was not marking posted as true for post authors in the TopicUser table, post migration. Create another step to set posted='t'.
2023-02-28 20:45:32 -04:00
Sam
654959faa4
DEV: reduce amount of errors logged when notifying on flags (#20472)
Forcing distributed muted to raise when a notify reviewable job is running
leads to excessive errors in the logs under many conditions.

The new pattern

1. Optimises the counting of reviewables so it is a lot faster
2. Holds the distributed lock for 2 minutes (max)


The downside is the job queue can get blocked up when tons of notify
reviewables are running at the same time. However this should be very
rare in the real world, as we only notify when stuff is flagged which
is fairly infrequent.

This also give a fair bit more time for the notifications which may be
a little slow on large sites with tons of mods.
2023-03-01 08:58:32 +11:00
Bianca Nenciu
ccb345bd88
FEATURE: Update topic/comment embedding parameters (#20181)
This commit implements many changes to topic and comments embedding. It
deprecates the class_name field from EmbeddableHost and suggests using
the className parameter. discourse_username parameter has been
deprecated and it will fetch it from embedded site from the author or
discourse-username meta.

See the updated code sample from Admin > Customize > Embedding page.

* FEATURE: Add className parameter for Discourse embed

* DEV: Hide class_name from EmbeddableHost

* DEV: Deprecate class_name field of EmbeddableHost

* FEATURE: Use either author or discourse-username meta tag

* DEV: Deprecate discourse_username parameter

* DEV: Improve embed code sample
2023-02-28 14:31:59 +02:00
Jarek Radosz
116781c748
DEV: Update ember-cli-htmlbars to 6.2.0 (#20467)
* Build(deps): Bump ember-cli-htmlbars in /app/assets/javascripts

Bumps [ember-cli-htmlbars](https://github.com/ember-cli/ember-cli-htmlbars) from 6.1.1 to 6.2.0.
- [Release notes](https://github.com/ember-cli/ember-cli-htmlbars/releases)
- [Changelog](https://github.com/ember-cli/ember-cli-htmlbars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ember-cli/ember-cli-htmlbars/compare/v6.1.1...v6.2.0)

---
updated-dependencies:
- dependency-name: ember-cli-htmlbars
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Load additional files

* Update code

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-27 18:43:28 +01:00
Jarek Radosz
f63d18d6ca
DEV: Update syntax_tree to 6.0.1 (#20466) 2023-02-27 17:20:00 +01:00
Osama Sayegh
a509441148
DEV: Include unread topics in New topic lists and link to it in sidebar (#20432)
This commit introduces a few experimental changes to the New topics list and "Everything" link in the sidebar:

1. Make the New topics list include unread topics
2. Make the Everything section in the sidebar link to the New topics list (`/new`)
3. Remove "unread" or "new" text next to the count and keep the count
4. The count is a sum of new and unread topics counts

All of these of changes are behind an off-by-default feature flag. I've not written extensive tests for these changes because they're highly experimental.

Internal topic: t/77234.
2023-02-27 15:11:01 +03:00
Ted Johansson
b50b63808c
DEV: Make Guardian#can_see? default to false for unwatched objects (#20412)
When invoking e.g. `can_see?(Foo.new)`, the guardian checks if there's a method `#can_see_foo?` defined and if so uses that to determine whether the user can see it or not.

When such a method is not defined, the guardian currently returns `true`, but it is probably a better call (pun intended) to make it "safe by default" and return `false` instead. I.e. if you can't explicitly see it, you can't see it at all.

This change makes the change to `Guardian#can_see?` to fall back to `false` if no visibility check method is defined.

For `#can_see_user?` and `#can_see_tag?` we don't have any particular logic that prevents viewing. We previously relied on the implicit `true` value, but since that's now change to `false`, I have explicitly implemented these two methods in `UserGuardian` and `TagGuardian` modules. If in the future we want to add some logic for it, this would be the place.

To be clear, **the behaviour remains the same**, but the `true` value is now explicit rather than implicit.
2023-02-24 15:57:01 +08:00
Penar Musaraj
c052df412a
DEV: Add support for aliases in HighlightJS languages (#20380)
We were only supporting the main name of each HighlightJS language. So, by default, you could not use `js` or `jsx` to highlight Javascript, given they are aliases for `javascript`.

This PR adds a list of aliases as a constant to core (built via a rake task), and then checks against the `highlighted_languages` site settings plus the list of aliases when processing a code block.
2023-02-23 15:06:06 -05:00
Loïc Guitaut
a9f2c6db64 SECURITY: Show only visible tags in metadata
Currently, the topic metadata show both public and private
tags whereas only visible ones should be exposed.
2023-02-23 17:22:20 +01:00
Martin Brennan
4ab1f76499
DEV: Fix bin/turbo_rspec runtime recording (#20407)
This commit 57caf08e13 broke
`bin/turbo_rspec` timing recording via `TurboTests::Runner`,
because we changed to using all `spec/*` folders except
`spec/system` as default for the runner, rather than
the old `['spec']` array, which is what `TurboTests::Runner`
was relying on to determine whether to record test run
time with `ParallelTests::RSpec::RuntimeLogger`.

Instead, we can just pass a new `use_runtime_info` boolean to the
runner class and use it when running against the default set of
spec files using `bin/turbo_rspec` and the turbo rspec rake task.
2023-02-23 07:47:11 +10:00
Krzysztof Kotlarek
b9d037770c
DEV: configurable public sidebar sections (#20303)
Extension of https://github.com/discourse/discourse/pull/20057

Admin can create a public session visible to everyone. An additional checkbox is displayed for staff members.
2023-02-22 08:55:44 +11:00
Loïc Guitaut
f7c57fbc19 DEV: Enable unless cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Leonardo Mosquera
509fee0f5a
FIX: allow changing default DNS query timeout of 2s via GlobalSetting (#20383)
The current default timeout is hardcoded to 2 seconds which is proving
too low for certain cases, and resulting in sporadic timeouts due to slow DNS queries.
2023-02-21 09:54:29 +11:00
Osama Sayegh
f91631b625
DEV: Update javascript:update_constants rake task following template colocation (#20365)
As of ba3f62f576, handlebars templates are colocated with js files so the path to hbs templates referenced by this rake task is no longer valid. This commit fixes the path to hbs templates and updates a couple of files that are generated by the rake task.
2023-02-20 06:20:47 +03:00
Martin Brennan
7ad92b7ab9
FIX: Do not enqueue update_post_uploads_secure_status unnecessarily (#20366)
We call `post.update_uploads_secure_status` in both
`PostCreator` and `PostRevisor`. Only the former was checking
if `SiteSetting.secure_uploads?` was enabled, but the latter
was not. There is no need to enqueue the job
`UpdatePostUploadsSecureStatus` if secure_uploads is not
enabled for the site.
2023-02-20 11:21:54 +10:00
Sam
cd247d5322
FEATURE: Roll out new search optimisations (#20364)
- Reduce duplication of terms in post index from unlimited to 6. This will
result in reduced index size and reduced weighting for posts containing
a huge amount of duplicate terms. (Eg: a post containing "sam sam sam sam
sam sam sam sam", will index as "sam sam sam sam sam sam", only including
the word up to 6 times.) This corrects a flaw where title weighting could
be ignored.

- Prioritize exact matches of words in titles. Our search always performs
a prefix match. However we want to give special weight to exact title matches
meaning that a search for "sum" will find topics such as "the sum of us" vs
"summer in spring".

- Pick up fixes to our search algorithm which are missing from old indexes.
Specifically pick up the fix that indexes URLs properly. (`https://happy.com`
was stemmed to `happi` in keywords and then was not searchable)

see also:

https://meta.discourse.org/t/refinements-to-search-being-tested-on-meta/254158

Indexing will take a while and work in batches, in the background.
2023-02-20 11:53:35 +11:00
Leonardo Mosquera
6fa9a8990d
Add username template variables to Welcome User email template (#20352)
* Add username and name_or_username variables to SystemMessage defaults

* Allow username and name variables on welcome_user email template overrides

* Satisfy linting

* Add test
2023-02-17 18:38:21 -03:00
Osama Sayegh
75ae70c27c
DEV: rtlcss_wrapper renamed to rtlcss (#20331)
The `rtlcss_wrapper` gem has been renamed to `rtlcss` per bd89847a39.
2023-02-16 18:15:56 +03:00
Osama Sayegh
5423e7c5b7
DEV: Add backend support for unread and new topics list (#20293)
This commit adds backend support for a new topics list that combines both the current unread and new topics lists. We're going to experiment with this new list (name TBD) internally and decide if this feature is something that we want to fully build.

Internal topic: t/77234.
2023-02-16 16:02:09 +03:00
Selase Krakani
5a6e5a4012
FIX: Unsubscribing via key associated with deleted topic (#20275)
Currently, clicking on the unsubscribe link with a `key` associated with a
deleted topic results in an HTTP 500 response.

This change fixes that by skipping any attempt to run topic related flow if topic
isn't present.
2023-02-16 10:47:01 +00:00
natiz
9c30ae93b3
DEV: Add after_email_send app event (#20178) 2023-02-15 11:23:45 -05:00