Commit Graph

629 Commits

Author SHA1 Message Date
Juan David Martínez Cubillos
08440b0035
DEV: Add tl3_custom_promotions plugin modifier to tl3_promotions.rb ()
* DEV: Add tl3_custom_promotions plugin modifier to tl3_promotions.rb

* added tests

* added tests for demotions

* changed argument order in test
2024-11-22 15:28:43 -05:00
Angus McLeod
ec7de0fd68
Require permitted scopes when registering a client () 2024-11-19 15:28:04 -05:00
Keegan George
eb2992a628
REVERT: Check for features sooner () 2024-11-13 11:30:34 -08:00
Keegan George
8dc474952b
DEV: Check for features sooner ()
This PR updates the Job for checking new features from the features feed from every day to every hour. This allows for easier rollout of features.
2024-11-13 10:08:13 -08:00
Alan Guo Xiang Tan
322a3be2db
DEV: Remove logical OR assignment of constants ()
Constants should always be only assigned once. The logical OR assignment
of a constant is a relic of the past before we used zeitwerk for
autoloading and had bugs where a file could be loaded twice resulting in
constant redefinition warnings.
2024-10-16 10:09:07 +08:00
Ted Johansson
e60876ce49
FIX: Appropriately handle uninstalled problem checks ()
When running checks, we look to the existing problem check trackers and try to grab their ProblemCheck classes.

In some cases this is no longer in the problem check repository, e.g. when the check was part of a plugin that has been uninstalled.

In the case where the check was scheduled, this would lead to an error in one of the jobs
2024-09-18 10:11:52 +08:00
Ted Johansson
776b4ec8e2
DEV: Remove old problem check system - Part 1 ()
We're now using the new, database-backed problem check system. This PR removes parts of the old, Redis-backed system that is now defunct.
2024-09-06 17:00:25 +08:00
锦心
319075e4dd
FIX: Ensure JsLocaleHelper to not output deprecated translations ()
* FIX: Ensure JsLocaleHelper to obly outputs up-to-date translations

The old implementation forgot to filter out deprecated
translations, causing these translations to incorrectly override the new
locale in the frontend.

This commit fills in the forgotten where clause, filtering only the
up-to-date part.

Related meta topic: https://meta.discourse.org/t/outdated-translation-replacement-causing-missing-translation/314352
2024-07-29 15:21:25 +08:00
Guhyoun Nam
784c04ea81
FEATURE: Add Mechanism to redeliver all failed webhook events ()
Background:
In order to redrive failed webhook events, an operator has to go through and click on each. This PR is adding a mechanism to retry all failed events to help resolve issues quickly once the underlying failure has been resolved.

What is the change?:
Previously, we had to redeliver each webhook event. This merge is adding a 'Redeliver Failed' button next to the webhook event filter to redeliver all failed events. If there is no failed webhook events to redeliver, 'Redeliver Failed' gets disabled. If you click it, a window pops up to confirm the operator. Failed webhook events will be added to the queue and webhook event list will show the redelivering progress. Every minute, a job will be ran to go through 20 events to redeliver. Every hour, a job will cleanup the redelivering events which have been stored more than 8 hours.
2024-07-08 15:43:16 -05:00
Gabriel Grubba
f3a89620a1
FEATURE: Add WebHookEventsDailyAggregate ()
* FEATURE: Add WebHookEventsDailyAggregate

Add WebHookEventsDailyAggregate model to store daily aggregates of web hook events.
Add AggregateWebHooksEvents job to aggregate web hook events daily.
Add spec for WebHookEventsDailyAggregate model.

* DEV: Update annotations for web_hook_events_daily_aggregate.rb

* DEV: Update app/jobs/scheduled/aggregate_web_hooks_events.rb

Co-authored-by: Martin Brennan <martin@discourse.org>

* DEV: Address review feedback

Solves:
- https://github.com/discourse/discourse/pull/27542#discussion_r1646961101
- https://github.com/discourse/discourse/pull/27542#discussion_r1646958890
- https://github.com/discourse/discourse/pull/27542#discussion_r1646976808
- https://github.com/discourse/discourse/pull/27542#discussion_r1646979846
- https://github.com/discourse/discourse/pull/27542#discussion_r1646981036

* A11Y: Add translation to retain_web_hook_events_aggregate_days key

* FEATURE: Purge old web hook events daily aggregate

Solves: https://github.com/discourse/discourse/pull/27542#discussion_r1646961101

* DEV:  Update tests for web_hook_events_daily_aggregate

Update WebHookEventsDailyAggregate to not use save! at the end
Solves: https://github.com/discourse/discourse/pull/27542#discussion_r1646984601

* PERF: Change job query to use WebHook table instead of WebHookEvent table

* DEV: Update tests to use `fab!`

* DEV: Address code review feedback.

Add idempotency to job
Add has_many to WebHook

* DEV: add test case for job and change job query

* DEV: Change AggregateWebHooksEvents job test name

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
2024-06-25 13:56:47 -03:00
Alan Guo Xiang Tan
adc824a9bc
FIX: Jobs::EnsureS3UploadsExistence broken for multisite ()
This is a follow-up to 8cf4ed5f88.
2024-06-10 16:26:39 +08:00
Alan Guo Xiang Tan
8cf4ed5f88
DEV: Introduce hidden s3_inventory_bucket site setting ()
This commit introduces a hidden `s3_inventory_bucket` site setting which
replaces the `enable_s3_inventory` and `s3_configure_inventory_policy`
site setting.

The reason `enable_s3_inventory` and `s3_configure_inventory_policy`
site settings are removed is because this feature has technically been
broken since it was introduced. When the `enable_s3_inventory` feature
is turned on, the app will because configure a daily inventory policy for the
`s3_upload_bucket` bucket and store the inventories under a prefix in
the bucket. The problem here is that once the inventories are created,
there is nothing cleaning up all these inventories so whoever that has
enabled this feature would have been paying the cost of storing a whole
bunch of inventory files which are never used. Given that we have not
received any complains about inventory files inflating S3 storage costs,
we think that it is very likely that this feature is no longer being
used and we are looking to drop support for this feature in the not too
distance future.

For now, we will still support a hidden `s3_inventory_bucket` site
setting which site administrators can configure via the
`DISCOURSE_S3_INVENTORY_BUCKET` env.
2024-06-10 13:16:00 +08:00
Alan Guo Xiang Tan
df16ab0758
FIX: S3Inventory to ignore files older than last backup restore date ()
This commit updates `S3Inventory#files` to ignore S3 inventory files
which have a `last_modified` timestamp which are not at least 2 days
older than `BackupMetadata.last_restore_date` timestamp.

This check was previously only in `Jobs::EnsureS3UploadsExistence` but
`S3Inventory` can also be used via Rake tasks so this protection needs
to be in `S3Inventory` and not in the scheduled job.
2024-05-24 10:54:06 +08:00
Jeff Wong
3a3ee5e04a
DEV: replace .each with .find_each for paginated queries ()
Large batches of reviewables may require paginated queries.
2024-05-23 15:42:21 -07:00
Ted Johansson
3137e60653
DEV: Database backed admin notices ()
This PR introduces a basic AdminNotice model to store these notices. Admin notices are categorized by their source/type (currently only notices from problem check.) They also have a priority.
2024-05-23 09:29:08 +08:00
Régis Hanol
958437e7dd
FIX: send activity summaries based on "last seen" ()
instead of "last emailed" so that people getting email notifications (from a watched topic for example) also get the activity summaries.

Context - https://meta.discourse.org/t/activity-summary-not-sent-if-other-emails-are-sent/293040

Internal Ref - t/125582

Improvement over 95885645d9
2024-05-22 10:23:03 +02:00
Natalie Tay
777b8f6d51
Revert "FIX: send activity summaries based on "last seen"" ()
This reverts commit 95885645d9.
2024-05-15 14:09:29 +08:00
Régis Hanol
95885645d9 FIX: send activity summaries based on "last seen"
instead of "last emailed" so that people getting email notifications (from a watched topic for example) also get the activity summaries.

Context - https://meta.discourse.org/t/activity-summary-not-sent-if-other-emails-are-sent/293040

Internal Ref - t//125582
2024-05-06 15:22:52 +02:00
Vinoth Kannan
9d88f80f26
UX: make first admin a moderator to review user approvals. ()
Previously, when the new site was created and after the first admin login, no one will receive notifications to review the user approval queue since only the moderators would receive the PMs about it. Also, this PR will change the "pending_users_reminder_delay_minutes" site setting to 5 minutes while the site is in bootstrap mode.
2024-04-10 20:59:03 +05:30
Ted Johansson
6e95c152ed
DEV: Rename problem check jobs to avoid namespace clashes ()
Doing the following renames:

Jobs::ProblemChecks → Jobs::RunProblemChecks
Jobs::ProblemCheck → Jobs::RunProblemCheck

This is to disambiguate the ProblemCheck class name, ease fuzzy finding, and avoid needing to use :: in a bunch of places.
2024-03-07 12:26:58 +08:00
Ted Johansson
1bcb521fbf
DEV: Add DB backed problem checks to support perform_every config ()
As part of problem checks refactoring, we're moving some data to be DB backed. In this PR it's the tracking of problem check execution. When was it last run, when was the last problem, when should it run next, how many consecutive checks had problems, etc.

This allows us to implement the perform_every feature in scheduled problem checks for checks that don't need to be run every 10 minutes.
2024-02-27 11:17:39 +08:00
Vinoth Kannan
b3238bfc34
FEATURE: call hub API to update Discourse discover enrollment. ()
Now forums can enroll their sites to be showcased in the Discourse [Discover](https://discourse.org/discover) directory. Once they enable the site setting `include_in_discourse_discover` to enroll their forum the `CallDiscourseHub` job will ping the `api.discourse.org/api/discover/enroll` endpoint. Then the Discourse Hub will fetch the basic details from the forum and add it to the review queue. If the site is approved then the forum details will be displayed in the `/discover` page.
2024-02-23 11:42:28 +05:30
Sam
207cb2052f
FIX: muted tags breaking hot page when filtered to tags ()
Also, remove experimental setting and simply use top_menu for feature detection

This means that when people eventually enable the hot top menu, there will
be topics in it


Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
2024-02-23 17:11:39 +11:00
Ted Johansson
a72dc2f420
DEV: Introduce a problem checks API ()
Previously, problem checks were all added as either class methods or blocks in AdminDashboardData. Another set of class methods were used to add and run problem checks.

As of this PR, problem checks are promoted to first-class citizens. Each problem check receives their own class. This class of course contains the implementation for running the check, but also configuration items like retry strategies (for scheduled checks.)

In addition, the parent class ProblemCheck also serves as a registry for checks. For example we can get a list of all existing check classes through ProblemCheck.checks, or just the ones running on a schedule through ProblemCheck.scheduled.

After this refactor, the task of adding a new check is significantly simplified. You add a class that inherits ProblemCheck, you implement it, add a test, and you're good to go.
2024-02-23 11:20:32 +08:00
Sam
ebd3971533
FEATURE: experiment with hot sort order ()
This introduces a new experimental hot sort ordering. 

It attempts to float top conversations by first prioritizing a  topics with lots of recent activity (likes and users responding) 

The schedule that updates hot topics is disabled unless the hidden site setting: `experimental_hot_topics` is enabled. 

You can control "decay" with `hot_topic_gravity` and `recency` with `hot_topics_recent_days` 

Data is stored in the new `topic_hot_scores` table and you can check it out on the `/hot` route once 
enabled. 
---------

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2024-01-17 13:01:04 +11:00
marstall
3837657449
FIX: refactor calling of timed backup deletion
refactor calling of timed backup deletion so it runs regardless of SiteSetting.automatic_backups_enabled value
2024-01-08 15:41:00 -05:00
Jarek Radosz
694b5f108b
DEV: Fix various rubocop lints ()
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
Jarek Radosz
4280c01153
DEV: Fix Lint/ShadowedArgument () 2023-12-06 13:16:10 +01:00
Sam
b09422428d
DEV: update syntax tree to latest ()
update format to latest syntax tree
2023-11-29 16:38:07 +11:00
Martin Brennan
186e415e38
DEV: Housekeeping for CleanUpUploads job ()
Followup to 9db8f00b3d, we
don't need this dead code any more. Also made some minor
improvements and comments.
2023-11-20 09:50:09 +10:00
Ted Johansson
47e58357b6
DEV: Parallel scheduled admin checks ()
This PR does some preparatory refactoring of scheduled admin checks in order for us to be able to do custom retry strategies for some of them.

Instead of running all checks in sequence inside a single, scheduled job, the scheduled job spawns one new job per check.

In order to be concurrency-safe, we need to change the existing Redis data structure from a string (of serialized JSON) to a list of strings (of serialized JSON).
2023-11-03 09:05:29 +08:00
Krzysztof Kotlarek
8c355d9e99
FIX: log for CleanUpTags job ()
In previous [PR](https://github.com/discourse/discourse/pull/23864) we introduced setting to automatically delete unused tags. This action should be logged.
2023-10-18 03:24:14 +00:00
Krzysztof Kotlarek
cb8190d32f
FEATURE: option to automatically delete unused tags ()
Introduced a new site setting that enables the automatic and daily removal of unused tags.
2023-10-11 23:58:56 +00:00
Renato Atilio
d93c2cb3d2
FEATURE: site settings to revoke api keys older than a number of days ()
* FEATURE: site settings to revoke api keys older than a number of days
2023-09-15 16:31:29 -03:00
Ted Johansson
7a53fb65da
FIX: Don't show admin warnings about deleted translation overrides ()
We recently introduced this advice to admins when some translation overrides are outdated or using unknown interpolation keys:

However we missed the case where the original translation key has been renamed or altogether removed. When this happens they are no longer visible in the admin interface, leading to the confusing situation where we say there are outdated translations, but none are shown.

Because we don't explicitly handle this case, some deleted translations were incorrectly marked as having unknown interpolation keys. (This is because I18n.t will return a string like "Translation missing: foo", which obviously has no interpolation keys inside.)

This change adds an additional status, deprecated for TranslationOverride, and the job that checks them will check for this status first, taking precedence over invalid_interpolation_keys. Since the advice only checks for the outdated and invalid_interpolation_keys statuses, this fixes the problem.
2023-07-14 16:52:39 +08:00
Ted Johansson
9915236e42
FEATURE: Warn about outdated translation overrides in admin dashboard ()
This PR adds a feature to help admins stay up-to-date with their translations. We already have protections preventing admins from problems when they update their overrides. This change adds some protection in the other direction (where translations change in core due to an upgrade) by creating a notice for admins when defaults have changed.

Terms:

- In the case where Discourse core changes the default translation, the translation override is considered "outdated".
- In the case above where interpolation keys were changed from the ones the override is using, it is considered "invalid".
- If none of the above applies, the override is considered "up to date".

How does it work?

There are a few pieces that makes this work:

- When an admin creates or updates a translation override, we store the original translation at the time of write. (This is used to detect changes later on.)
- There is a background job that runs once every day and checks for outdated and invalid overrides, and marks them as such.
- When there are any outdated or invalid overrides, a notice is shown in admin dashboard with a link to the text customization page.

Known limitations

The link from the dashboard links to the default locale text customization page. Given there might be invalid overrides in multiple languages, I'm not sure what we could do here. Consideration for future improvement.
2023-07-10 10:06:40 +08:00
Alessio Cosenza
56718504ac
FEATURE: Add hooks for email poller plugins ()
While we are unable to support OAUTH2 with pop3 (due to upstream dependency ), we are adding the support for mail pollers plugin. Doing so, it would be possible to write a plugin which then uses other ways (microsoft graph sdk for example) to poll emails from a mailbox.

The idea is that a plugin would define a class which inherits from Email::Poller and defines a poll_mailbox static method which returns an array of strings. Then the plugin could call register_mail_poller(<class_name>) to have it registered. All the configuration (oauth2 tokens, email, etc) could be managed by sitesettings defined in the plugin.
2023-06-26 13:16:03 +08:00
Faizaan Gagan
d1334a7aaf
FIX: consider users.created_at for inactive cleanup () 2023-05-23 13:41:23 +05:30
Faizaan Gagan
efdfddf7fc
FIX: consider users with trashed topics/posts for inactive cleanup ()
* FIX: consider users with trashed topics/posts for inactive cleanup

* defer checking for missing associations
2023-05-23 00:26:24 +05:30
Bianca Nenciu
cc18a99105
FEATURE: Add new notification for admin problems ()
Add new notification for admin problems to replace old PM-based flow.
2023-05-03 19:35:22 +03:00
Krzysztof Kotlarek
a8e28060d1
FIX: rename notify_about_flags_after to notify_about_reviewable_item_after ()
Change name and description for SiteSetting to make it easier to understand.
2023-05-02 08:08:22 +10:00
Bianca Nenciu
142d2ab65e
FEATURE: Move bootstrap mode indicator to header ()
The information about bootstrap mode has been moved from the notice to
the README topic for admins.
2023-03-24 15:59:03 +02:00
chapoi
e52bbc1230
UX/DEV: Review queue redesign fixes ()
* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* DEV: switch to selectKit

* UX: color approve/reject buttons in RQ

* DEV: regroup actions

* UX: add type tag and design update

* UX: clarify status copy in reviewQ

* Join questions for flagged post with "or" with new I18n function
* Move ReviewableScores component out of context
* Add CSS classes to reviewable-item based on human type

* UX: add table header for scoring

* UX: don't display % score

* UX: prefix modifier class with dash

* UX: reviewQ flag table styling

* UX: consistent use of ignore icon

* DEV: only show context question on pending status

* UX: only show table headers on pending status

* DEV: reviewQ regroup actions for hidden posts

* UX: reviewQ > approve/reject buttons

* UX: reviewQ add fadeout

* UX: reviewQ styling

* DEV: move scores back into component

* UX: reviewQ mobile styling

* UX: score table on mobile

* UX: reviewQ > move meta info outside table

* UX: reviewQ > score layout fixes

* DEV: readd `agree_and_keep` and fix the spec tests.

* Fix the spec tests

* fix the quint test

* DEV: readd deleting replies

* UX: reviewQ copy tweaks

* DEV: readd test for ignore + delete replies

* Remove old

* FIX: Add perform_ignore back in for backwards compat

* DEV: add an action alias `ignore` for `ignore_and_do_nothing`.

---------

Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Vinoth Kannan <svkn.87@gmail.com>
2023-03-02 16:40:53 +01:00
Loïc Guitaut
f7c57fbc19 DEV: Enable unless cops
We discussed the use of `unless` internally and decided to enforce
available rules from rubocop to restrict its most problematic uses.
2023-02-21 10:30:48 +01:00
Régis Hanol
b704e338ef
DEV: extract anniversary badge query ()
So it can easily be overwritten in a plugin for example.

### Added more tests to provide better coverage

We previously only had `u.silenced_till IS NULL` but I made it consistent with pretty much every other places where we check for "active" users.

These two new lines do change the query a tiny bit though. 

**Before** 

- You could not get the badge if you were currently silenced (no matter what period is being checked)
- You could get the badge if you were suspended 😬 

**After**

- You can't get the badge if you were silenced during the past year
- You can't get the badge if you were suspended during the past year


### Improved the performance of the query by using `NOT EXISTS` instead of `LEFT JOIN / COUNT() = 0`

There is no difference in behaviour between 

```sql
LEFT JOIN user_badges AS ub ON ub.user_id = u.id AND ...
[...]
HAVING COUNT(ub.*) = 0
```

and

```sql
NOT EXISTS (SELECT 1 FROM user_badges AS ub WHERE ub.user_id = u.id AND ...)
```

The only difference is performance-wise. The `NOT EXISTS` is 10-30% faster on very large databases (aka. posts and users in X millions). I checked on 3 of the largest datasets I could find.
2023-01-16 11:55:00 +01:00
Daniel Waterworth
dfc449a32e
FIX: Continue if checking themes if updating one fails () 2023-01-09 11:56:43 -06:00
David Taylor
5a003715d3
DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
Osama Sayegh
1c03d6f9b9
FEATURE: Send notifications to admins when new features are released ()
This commit adds a new notification that gets sent to admins when the site gets new features after an upgrade/deploy. Clicking on the notification takes the admin to the admin dashboard at `/admin` where they can see the new features under the "New Features" section.

Internal topic: t/87166.
2022-12-15 20:12:53 +03:00
Daniel Waterworth
167181f4b7
DEV: Quote values when constructing SQL ()
All of these cases should already be safe, but still good to quote for
"defense in depth".
2022-11-01 14:05:13 -05:00
David Taylor
3115f38de2
PERF: Move dominant color calculation to separate job ()
This will ensure that any potential problems with this process do not affect the performance or reliability of the PeriodicalUpdates job.
2022-10-06 13:26:08 +01:00