Commit Graph

9714 Commits

Author SHA1 Message Date
Martin Brennan
7afb5fc481
DEV: Use Discourse::SYSTEM_USER_ID in fixtures/009_users (#24743)
I couldn't find where we created the system user and
this is why -- everywhere else in the app we reference
SYSTEM_USER_ID but here.
2023-12-07 09:04:45 +10:00
Jarek Radosz
694b5f108b
DEV: Fix various rubocop lints (#24749)
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
David Taylor
48ec946702
UX: Restore category badge colours on 404 page (#24754) 2023-12-06 17:49:19 +00:00
Jarek Radosz
6a66dc1cfb
DEV: Fix Lint/BooleanSymbol (#24747) 2023-12-06 13:19:09 +01:00
Jarek Radosz
138bf486d3
DEV: Fix Lint/DuplicateMethods (#24746) 2023-12-06 13:18:34 +01:00
Jarek Radosz
4280c01153
DEV: Fix Lint/ShadowedArgument (#24733) 2023-12-06 13:16:10 +01:00
David Battersby
8b46dc8bb5
FEATURE: Add thumbnails for chat image uploads (#24328)
Introduces the concept of image thumbnails in chat, prior to this we uploaded and used full size chat images within channels and direct messages.

The following changes are covered:
- Post processing of image uploads to create the thumbnail within Chat::MessageProcessor
- Extract responsive image ratios into CookedProcessorMixin (used for creating upload variations)
- Add thumbnail to upload serializer from plugin.rb
- Convert chat upload template to glimmer component using .gjs format
- Use thumbnail image within chat upload component (stores full size img in orig-src data attribute)
- Old uploads which don't have thumbnails will fallback to full size images in channels/DMs
- Update Magnific lightbox to use full size image when clicked
- Update Glimmer lightbox to use full size image (enables zooming for chat images)
2023-12-06 14:59:18 +08:00
Martin Brennan
30d5e752d7
DEV: Revert guardian changes (#24742)
I took the wrong approach here, need to rethink.

* Revert "FIX: Use Guardian.basic_user instead of new (anon) (#24705)"

This reverts commit 9057272ee2.

* Revert "DEV: Remove unnecessary method_missing from GuardianUser (#24735)"

This reverts commit a5d4bf6dd2.

* Revert "DEV: Improve Guardian devex (#24706)"

This reverts commit 77b6a038ba.

* Revert "FIX: Introduce Guardian::BasicUser for oneboxing checks (#24681)"

This reverts commit de983796e1.
2023-12-06 16:37:32 +10:00
Martin Brennan
9057272ee2
FIX: Use Guardian.basic_user instead of new (anon) (#24705)
c.f. de983796e1

There will soon be additional login_required checks
for Guardian, and the intent of many checks by automated
systems is better fulfilled by using BasicUser, which
simulates a logged in TL0 forum user, rather than an
anon user.

In some cases the use of anon still makes sense (e.g.
anonymous_cache), and in that case the more explicit
`Guardian.anon_user` is used
2023-12-06 11:56:21 +10:00
Martin Brennan
a5d4bf6dd2
DEV: Remove unnecessary method_missing from GuardianUser (#24735)
Followup to 77b6a038ba, this
was a mistake and should have been removed before merge.
2023-12-06 11:24:19 +10:00
Martin Brennan
77b6a038ba
DEV: Improve Guardian devex (#24706)
It's quite confusing for blank? to be overridden
on AnonymousUser and BasicUser to represent
whether the fake user is authenticated or not;
we can achieve the same thing more clearly with
a wrapper GuardianUser class around these
user classes. Also fixes an issue where
`def user` would be returning nil.
2023-12-06 10:57:04 +10:00
Martin Brennan
de983796e1
FIX: Introduce Guardian::BasicUser for oneboxing checks (#24681)
Through internal discussion, it has become clear that
we need a conceptual Guardian user that bridges the
gap between anon users and a logged in forum user with
an absolute baseline level of access to public topics,
which can be used in cases where:

1. Automated systems are running which shouldn't see any
   private data
1. A baseline level of user access is needed

In this case we are fixing the latter; when oneboxing a local
topic, and we are linking to a topic in another category from
the current one, we need to operate off a baseline level of
access, since not all users have access to the same categories,
and we don't want e.g. editing a post with an internal link to
expose sensitive internal information.
2023-12-05 09:25:23 +10:00
David Taylor
3aeff56faf
DEV: Run assets:precompile ember build with CI=1 (#24696)
This will improve the output to print the current step (rather than the existing behavior which just says 'building...')
2023-12-04 15:49:50 +00:00
Jarek Radosz
7196613e2e
DEV: Fix various spec linting issues (#24672)
Duplicated specs, incorrect descriptions, incorrect assertions, incorrect filenames, old todo
2023-12-04 13:45:19 +01:00
David Taylor
c2887d3f8c
DEV: Remove unused Ember::Handlebars freedom patch (#24688)
We no longer compile Ember templates in ruby, so this is unused
2023-12-04 12:40:08 +00:00
Jarek Radosz
74011232e9
FIX: Request html when fetching inline onebox data (#24674)
We do expect to receive html
2023-12-04 11:36:42 +10:00
David Taylor
ecf7a4f0c6
FIX: Ensure app-cdn CORS is not overridden by cors_origin setting (#24661)
We add `Access-Control-Allow-Origin: *` to all asset requests which are requested via a configured CDN. This is particularly important now that we're using browser-native `import()` to load the highlightjs bundle. Unfortunately, user-configurable 'cors_origins' site setting was overriding the wldcard value on CDN assets and causing CORS errors.

This commit updates the logic to give the `*` value precedence, and adds a spec for the situation. It also invalidates the cache of hljs assets (because CDNs will have cached the bad Access-Control-Allow-Origin header).

The rack-cors middleware is also slightly tweaked so that it is always inserted. This makes things easier to test and more consistent.
2023-12-01 12:57:11 +00:00
Ted Johansson
54e813e964
FIX: Don't error out when trying to retrieve title and URL won't encode (#24660) 2023-12-01 15:03:06 +08:00
Martin Brennan
c58cd697d2
FIX: Further improvements for plugin list (#24622)
Followup e37fb3042d

* Automatically remove the prefix `Discourse ` from all the plugin titles to avoid repetition
* Remove the :discourse_dev: icon from the author. Consider a "By Discourse" with no labels as official
* We add a `label` metadata to plugin.rb
  * Only plugins made by us in `discourse` and `discourse-org` GitHub organizations will show these in the list
* Make the plugin author font size a little smaller
* Make the commit sha look like a link so it's more obvious it goes to the code

Also I added some validation and truncation for plugin metadata
parsing since currently you can put absolutely anything in there
and it will show on the plugin list.
2023-11-30 10:53:17 +10:00
Daniel Waterworth
eef93ac926
DEV: Allow setting max_length for field types using the plugin API (#24635) 2023-11-29 14:17:12 -06:00
David Taylor
265a8cd2b2
DEV: Fixup plugin initialization guard (#24628)
In development, I sometimes get `nil` `location.absolute_path` values. It looks like this is sometimes expected (https://bugs.ruby-lang.org/issues/10561) so we should fallback to `.path` and add a nil check.
2023-11-29 11:44:09 +00:00
Blake Erickson
21d614215b
DEV: Use staged user check instead (#24578)
This change refactors the check `user.groups.any?` and instead uses
`user.staged?` to check if the user is staged or not.

Also fixes several tests to ensure the users have their auto trust level
groups created.

Follow up to:

- 8a45f84277
- 447d9b2105
- c89edd9e86
2023-11-28 07:34:02 -07:00
David Taylor
5783f231f8
DEV: Introduce DISCOURSE_ASSET_URL_SALT (#24596)
This value is included when generating static asset URLs. Updating the value will allow site operators to invalidate all asset urls to recover from configuration issues which may have been cached by CDNs/browsers.
2023-11-28 11:28:40 +00:00
David Taylor
16b6e86932 DEV: Introduce feature-flag for Ember 5 upgrade
This commit introduces the scaffolding for us to easily switch between Ember 3.28 and Ember 5 on the `main` branch of Discourse. Unfortunately, there is no built-in system to apply this kind of flagging within yarn / ember-cli. There are projects like `ember-try` which are designed for running against multiple version of a dependency, but they do not allow us to 'lock' dependency/sub-dependency versions, and are therefore unsuitable for our use in production.

Instead, we will be maintaining two root `package.json` files, and two `yarn.lock` files. For ember-3, they remain as-is. For ember5, we use a yarn 'resolution' to override the version for ember-source across the entire yarn workspace.

To allow for easy switching with minimal diff against the repository, `package.json` and `yarn.lock` are symlinks which point to `package-ember3.json` and `yarn-ember3.lock` by default. To switch to Ember 5, we can run `script/switch ember version 5` to update the symlinks to point to `package-ember5.json` and `package-ember3.json` respectively. In production, and when using `bin/ember-cli` for development, the ember version can also be upgraded using the `EMBER_VERSION=5` environment variable.

When making changes to dependencies, these should be made against the default `ember3` versions, and then `script/regen_ember_5_lockfile` should be used to regenerate `yarn-ember5.lock` accordingly. A new 'Ember Version Lockfiles' GitHub workflow will automate this process on Dependabot PRs.

When running a local environment against Ember 5, the two symlink changes will show up as git diffs. To avoid us accidentally committing/pushing that change, another GitHub workflow is introduced which checks the default Ember version and raises an error if it is greater than v3.

Supporting two ember versions simultaneously obviously carries significant overhead, so our aim will be to get themes/plugins updated as quickly as possible, and then drop this flag.
2023-11-27 16:40:22 +00:00
Martin Brennan
09c446c1ae
FIX: Handle missing git repo details in plugin list (#24539)
Followup to e37fb3042d,
in some cases we cannot get git information for the
plugin folder (e.g. permission issues), so we need
to only try and get information about it if
commit_hash is present.
2023-11-24 12:08:10 +10:00
Sam
c2fd090d7d
DEV: revert missing license for maxmind changes (#24538)
Reverts
 
 - DEV: maxmind license checking failing tests #24534 
 - UX: Show if MaxMind key is missing on IP lookup #18993

These changes are leading to surprising results, our logs are now filling up with warnings on dev environments 

We need the change to be redone
2023-11-24 11:31:11 +11:00
Sam
85d2b5fa48
DEV: maxmind license checking failing tests (#24534)
This improves the implementation of #18993

1. Error message displayed to user is clearer
2. open_db will also be called, even if license key is blank, as it was previously
3. This in turn means no need to keep stubbing 'maxmind_license_key'
2023-11-24 09:38:46 +11:00
MichaIng
c58a41cb3e
UX: Show on IP lookup if MaxMind key is missing (#18993)
as discussed in https://meta.discourse.org/t/maxminddb-not-found-error/148512/7.
 
shows a warning to the admin if no license for maxmind is found
2023-11-24 08:02:05 +11:00
Alan Guo Xiang Tan
aaadce0652
Revert "FIX: plugin:install_all_gems Rake task not installing plugin gem (#24522)" (#24524)
This breaks the `plugin:install_all_gems` Rake task when used before
Redis is running. Need to go back to the drawing board.

This reverts commit 189aa5fa4e.
2023-11-23 13:01:54 +08:00
Blake Erickson
c89edd9e86
DEV: Convert email_in_min_trust to groups (#24515)
This change converts the `email_in_min_trust` site setting to
`email_in_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change

After a couple of months we will remove the
`email_in_min_trust` setting entirely.

Internal ref: /t/115696
2023-11-22 18:03:28 -07:00
Alan Guo Xiang Tan
189aa5fa4e
FIX: plugin:install_all_gems Rake task not installing plugin gem (#24522)
Why this change?

This regressed in dec68d780c where the
commit assumes that plugin gems are always installed when the
`plugin:install_all_gems` Rake task is ran as it would run the our Rails
initializers which activates plugins and install the gems. However, this
assumption only holds true when the `LOAD_PLUGINS` is present and set to
`1`.

What does this change do?

This commit changes the `plugin:install_all_gems` to load the Rails
environment with `LOAD_PLUGINS` set to `1` such that the plugin gems
will be installed as part of our initialization process for the app.

The commit also removes the `plugin:install_gems` Rake task which is
currently a noop and does not seem to be used anywhere..
2023-11-23 08:29:51 +08:00
Gerhard Schlager
5b97f79569
DEV: Replace starts_with? with start_with? in HtmlToMarkdown (#24521)
This allows us to use that class without loading Rails, e.g. in imports (converters).
2023-11-23 00:57:24 +01:00
Gerhard Schlager
9f3399ca8f
FIX: Remove multiple consecutive dashes from tag names (#24520) 2023-11-23 00:57:12 +01:00
Alan Guo Xiang Tan
1e290eed7b
PERF: Switch plugins:update_all to git pull concurrently (#24513)
Why this change?

Similar to d0117ff6e3, `plugins:update_all` spends most of its time waiting
on the network. On my local machine, this takes up to 2 mins when I have
all the official plugins installed. On a 32 cores machine, the total
time is cut down to 4 seconds.

What does this change do?

1. Move the logic in the `plugin:update` Rake task into a method.
2. Updates the `plugin:update` and `plugin:update_all` to rely on the
   new method.
3. Wraps the method call to update a plugin in `plugin:update_all` in a
   `Concurrent::Promise`

This change also adds the `--quiet` option to the `git pull` option
since the `git pull` output is just noise for 99% of the time.
2023-11-23 07:08:32 +08:00
Martin Brennan
e395e5e002
FIX: Use plugin category name for plugin list (#24477)
Followup to e37fb3042d

Some plugins like discourse-ai and discourse-saml do not
nicely change from kebab-case to Title Case (e.g. Ai, Saml),
and anyway this method of getting the plugin name is not
translated either.

Better to use the plugin setting category if it exists,
since that is written by a human and is translated.
2023-11-23 08:40:55 +10:00
Blake Erickson
8a45f84277
DEV: Convert approve_new_topics_unless_trust_level to groups (#24504)
* DEV: Convert approve_new_topics_unless_trust_level to groups

This change converts the `approve_new_topics_unless_trust_level` site
setting to `approve_new_topics_unless_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Hides the old setting
- Adds the new site setting
- Add a deprecation warning
- Updates to use the new setting
- Adds a migration to fill in the new setting if the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates tests to account for the new change

After a couple of months we will remove the
`approve_new_topics_unless_trust_level` setting entirely.

Internal ref: /t/115696

* add missing translation

* Add keyword entry

* Add migration
2023-11-22 10:44:59 -07:00
Alan Guo Xiang Tan
d0117ff6e3
PERF: Switch plugin:install_all_official to clone plugins concurrently (#24511)
Why this change?

`plugin:install_all_official` is quite slow at the moment taking roughly
1 minute and 51 seconds on my machine. Since most of the time is spent
waiting on the network, we can actually speed up the Rake task
significantly by executing the cloning concurrently. With a 8 cores
machine, cloning all plugins will only take 15 seconds.

What does this change do?

This change wraps the `git clone` operation in the
`plugin:install_all_official` Rake task in a `Concurrent::Promise` which
basically runs the `git clone` operation in a Thread. The `--quiet`
option has also been added to `git clone` since running stuff
concurrently messes up the output. That could be fixed but it has been
determined to be not worth it since the output from `git clone` is
meaningless to us.
2023-11-22 21:43:42 +08:00
Alan Guo Xiang Tan
48f3c638cc
DEV: Support customizing docker:test:setup with envs (#24508)
Why this change?

There are instances where we would like to customize what the
`docker:test:setup` Rake task does.

What does this change do?

Adds a bunch of env variables that could be set to customize what the
`docker:test:setup` Rake test does.
2023-11-22 15:12:02 +08:00
Martin Brennan
4e7929abb6
FIX: Wrong argument error being thrown in UrlHelper (#24506)
We were throwing ArgumentError in UrlHelper.normalised_encode,
but it was incorrect -- we were passing ArgumentError.new
2 arguments which is not supported. Fix this and have a hint
of which URL is causing the issue for debugging.
2023-11-22 15:19:40 +10:00
Alan Guo Xiang Tan
86da47f58d
FIX: docker:test Rake task did not run system tests in parallel (#24507)
Why this change?

We support a `USE_TURBO` environment variable which tells the
`docker:test` rake task to run rspec tests in parallel. However, this
currently does not apply to system tests.

What does this change do?

This commit runs system specs for both core and plugins using
`./bin/turbo_rspec` when the `USE_TURBO` environment is present. Note
that when running system specs, we will only spawn X number of test
processes where X is half the number of available CPU cores. This is
done because we have to leave CPU resources for the chrome processes
that will be created.
2023-11-22 11:51:43 +08:00
Blake Erickson
447d9b2105
DEV: Convert approve_unless_trust_level to groups (#24357)
This change converts the `approve_unless_trust_level` site setting to
`approve_unless_allowed_groups`.

See: https://meta.discourse.org/t/283408

- Adds the new site setting
- Adds a deprecation warning
- Updates core to use the new settings.
- Adds a migration to fill in the new setting of the old setting was
  changed
- Adds an entry to the site_setting.keywords section
- Updates many tests to account for the new change

After a couple of months we will remove the `approve_unless_trust_level`
setting entirely.

Internal ref: /t/115696
2023-11-21 11:31:42 -07:00
Roman Rizzi
75e2c6b506
DEV: Pass the user who requested the summary to the strategy. (#24489)
This change allows the `discourse-ai` plugin to log the user who requested the summary in the `AiApiAuditLog`.
2023-11-21 13:27:27 -03:00
Martin Brennan
e37fb3042d
FEATURE: Admin plugin list redesign (#24363)
* Remove checkmark for official plugins
* Add author for plugin, which is By Discourse for all discourse
  and discourse-org github plugins
* Link to meta topic instead of github repo
* Add experimental flag for plugin metadata and show this as a
  badge on the plugin list if present

---------

Co-authored-by: chapoi <101828855+chapoi@users.noreply.github.com>
2023-11-21 09:37:11 +10:00
Martin Brennan
146da75fd7
FEATURE: Add setting & preference for search sort default order (#24428)
This commit adds a new `search_default_sort_order` site setting,
set to "relevance" by default, that controls the default sort order
for the full page /search route.

If the user changes the order in the dropdown on that page, we remember
their preference automatically, and it takes precedence over the site
setting as a default from then on. This way people who prefer e.g.
Latest Post as their default can make it so.
2023-11-20 10:43:58 +10:00
Alan Guo Xiang Tan
e0ef88abca
DEV: Run QUnit tests for official Discourse themes (#24405)
Why this change?

As the number of themes which the Discourse team supports officially
grows, we want to ensure that changes made to Discourse core do not
break the plugins. As such, we are adding a step to our Github actions
test job to run the QUnit tests for all official themes.

What does this change do?

This change adds a new job to our tests Github actions workflow to run the QUnit
tests for all official plugins. This is achieved with the following
changes:

1. Update `testem.js` to rely on the `THEME_TEST_PAGES` env variable to set the
   `test_page` option when running theme QUnit tests with testem. The
   `test_page` option [allows an array to be specified](https://github.com/testem/testem#multiple-test-pages) such that tests for
   multiple pages can be run at the same time. We are relying on a ENV variable
   because  the `testem` CLI does not support passing a list of pages
   to the `--test_page` option.

2. Support a `/testem-theme-qunit/:testem_id/theme-qunit` Rails route in the development environment. This
   is done because testem prefixes the path with a unique ID to the configured `test_page` URL.
   This is problematic for us because we proxy all testem requests to the
   Rails server and testem's proxy configuration option does not allow us
   to easily rewrite the URL to remove the prefix. Therefore, we configure a proxy in testem to prefix `theme-qunit` requests with
  `/testem-theme-qunit` which can then be easily identified by the Rails server and routed accordingly. 

3. Update `qunit:test` to support a `THEME_IDS` environment variable
   which will allow it to run QUnit tests for multiple themes at the
   same time.

4. Support `bin/rake themes:qunit[ids,"<theme_id>|<theme_id>"]` to run
   the QUnit tests for multiple themes at the same time.

5. Adds a `themes:qunit_all_official` Rake task which runs the QUnit
   tests for all the official themes.
2023-11-17 07:17:32 +08:00
David Taylor
849002e90b
DEV: Silence successful db:migrate output in docker.rake (#24417)
Followup to 9449a0e0ed
2023-11-16 16:07:36 +00:00
Alan Guo Xiang Tan
416cef9ed1
DEV: Respect SKIP_TEST_DATABASE when running rake db:create (#24407)
Why this change?

By default the `db:create` Rake task in activerecord creates the
databases for both the development and test environment. This while
seemingly odd is by design from Rails. In order to avoid creating the
test database, Rails supports the `SKIP_TEST_DATABASE` environment
variable which we should respect when creating the multisite test
database.
2023-11-16 20:01:12 +08:00
Martin Brennan
68a912952c
FIX: min/max not passed to NumberField for site settings (#24402)
When we started using NumberField for integer site settings
in e113eff663, we did not end up
passing down a min/max value for the integer to the field, which
meant that for some fields where negative numbers were allowed
we were not accepting that as valid input.

This commit passes down the min/max options from the server for
integer settings then in turn passes them down to NumberField.

c.f. https://meta.discourse.org/t/delete-user-self-max-post-count-not-accepting-1-to-disable/285162
2023-11-16 12:37:05 +10:00
Alan Guo Xiang Tan
6ce55e5347
DEV: Run system tests for official themes (#24378)
Why this change?

As the number of themes which the Discourse team supports officially
grows, we want to ensure that changes made to Discourse core do not
break the plugins. As such, we are adding a step to our Github actions
test job to run the system tests for all official themes.

What does this change do?

This change adds a step to our Github actions test job to run the system
tests for all official plugins. This is achieved by the introduction of
the `themes:install_all_official` Rake task which installs all the
themes that are officially supported by the Discourse team.
2023-11-16 07:11:35 +08:00
Kris
797da5870b
FEATURE: remove category badge style options, set bullet style as default (#24198) 2023-11-13 10:46:15 -05:00