Commit Graph

437 Commits

Author SHA1 Message Date
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Robin Ward
5af7c90bab FEATURE: Hide Reviewable scores, change score filter to Priority
We found score hard to understand. It is still there behind the scenes
for sorting purposes, but it is no longer shown.

You can now filter by minimum priority (low, med, high) instead of
score.
2019-05-07 14:05:23 -04:00
Guo Xiang Tan
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
Sam Saffron
9be70a22cd DEV: introduce new API to look up dynamic site setting
This removes all uses of both `send` and `public_send` from consumers of
SiteSetting and instead introduces a `get` helper for dynamic lookup

This leads to much cleaner and safer code long term as we are always explicit
to test that a site setting is really there before sending an arbitrary
string to the class

It also removes a couple of risky stubs from the auth provider test
2019-05-07 11:00:30 +10:00
Sam Saffron
f8eddd40ad PERF: remove avg_time calculations and regular jobs from posts and topics
After careful analysis of large data-sets it became apparent that avg_time
had no impact whatsoever on "best of" topic scoring. Calculating avg_time
was a very costly operation especially on large databases.

We have some longer term plans of introducing other weighting that is read
time based into our scoring for "best of" and "top" topics, but in the
interim to stop a large amount of work that is not achieving any value we
are removing the jobs.

Column removal will follow once we decide on a new replacement metric.
2019-05-06 15:59:01 +10:00
Guo Xiang Tan
24347ace10 FIX: Properly associate user_profiles background urls via upload id.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
Sam Saffron
8fc2b0124f PERF: destroy users in batches of 50 users
This speeds up the process as we no longer need to commit a transaction
per user deleted
2019-04-26 22:51:33 +10:00
Gerhard Schlager
6d77156a94 FIX: Don't try to delete inactive admins 2019-04-25 11:59:50 +02:00
Guo Xiang Tan
e351b2ee26 Revert "FEATURE: stop running migrate to new scheme once done"
This reverts commit 2e346e6ea2.

Super seeded by 573ddcbbd0.
2019-04-24 14:03:31 +08:00
Sam Saffron
2e346e6ea2 FEATURE: stop running migrate to new scheme once done
This setting will automatically disable itself when there are no longer any
problem uploads
2019-04-24 14:32:39 +10:00
Guo Xiang Tan
149411ec90 PERF: Speed up Upload.migrate_to_new_scheme by limiting remap scope.
Doing a `LIKE` on `Post#raw` and `Post#cooked` takes forever on large
sites.
2019-04-24 11:56:48 +08:00
Robin Ward
6f56fba016 UX: Update post actions to "Approve Post" and "Reject Post"
This should be more clear.
2019-04-23 12:19:11 -04:00
Robin Ward
14f9d40e48 FEATURE: Clarify Reviewable User Actions
"Approve" is now "Approve User" and "Delete" is a dropdown with a choice
that allows you to block.
2019-04-17 11:44:17 -04:00
Dan Ungureanu
e23c87c442
FIX: Prevent exception in rescue block.
> Job exception: undefined method `id' for nil:NilClass
> clean_up_inactive_users.rb:25:in `rescue in block in execute'
2019-04-16 01:27:39 +03:00
Robin Ward
f7ebfb1acc FIX: Only consider pending queued posts for cleaning up uploads 2019-04-12 14:39:32 -04:00
Régis Hanol
ab79060468 FIX: don't send activation reminder to staged users 2019-04-10 17:29:08 +02:00
Bianca Nenciu
3d545d66df FEATURE: Send user activation reminders. (#7280) 2019-04-10 16:53:52 +02:00
Tarek Khalil
442fb2facb FEATURE: Remove ignore feature SiteSetting and enable ignore by default (#7349) 2019-04-10 12:54:59 +02:00
Sam Saffron
ec1c3559da PERF: correct clean up inactive so it does not clog scheduler
also add a hard limit of 1000 users per job run so we do not clog the
scheduler

destroyer.destroy has a transaction and this can have some serious complications
with the open record set find_each has going
2019-04-09 22:25:05 +10:00
Guo Xiang Tan
6d0e8821f8 FIX: Optimize query and avoid bloating memory in Jobs::MigrateUploadScheme. 2019-04-09 15:45:30 +08:00
Vinoth Kannan
914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Guo Xiang Tan
108c231d1c FIX: Clean up topic_search_data of trashed topics.
This keeps the index and table smaller.
2019-04-08 16:53:39 +08:00
Guo Xiang Tan
c4997ce85f FIX: Don't try and reindex posts that have been trashed. 2019-04-08 16:38:43 +08:00
Robin Ward
c9ddc6ce30 FIX: Make sure the site setting works and fix build 2019-04-04 17:44:10 -04:00
Vinoth Kannan
2c7a50860f DEV: disable missing post uploads check by default. 2019-04-05 02:29:10 +05:30
Vinoth Kannan
a385f6c48a REFACTOR: create custom field even when missing post uploads not found 2019-04-04 21:49:44 +05:30
Vinoth Kannan
44e87372da FEATURE: Add scheduled job to ensure s3 uploads existence 2019-04-04 20:37:44 +05:30
Robin Ward
588f61f717 FIX: Linting 2019-04-03 18:08:07 -04:00
Vinoth Kannan
3b581de3b8 remove unrelated code
already added in a separate job
2019-04-04 02:26:28 +05:30
Vinoth Kannan
b3fb0a7039 FEATURE: ensure consistency of post uploads in cooked content 2019-04-04 02:23:28 +05:30
Robin Ward
c1ea63bdc1 FIX: Reviewables should not be created for users until they are active
Conversely, if a user is deactivated the reviewable should automatically
be rejected.

Before this fix, if a user was not active they'd still show in the
review queue but without an "Approve" button which was confusing.
2019-04-03 15:25:00 -04:00
Guo Xiang Tan
d151425353
PERF: Delete search data of posts from trashed topics periodically. (#7302)
This keeps both the index and table smaller.
2019-04-03 10:10:41 +08:00
Guo Xiang Tan
feb731bffd FIX: Regenerate optimized images instead of migrating from old scheme.
`OptimizedImage.migrate_to_new_scheme` was optimizing optimized images
which we don't need to do. Regnerating the optimized image is way easier.
2019-04-03 09:45:02 +08:00
Guo Xiang Tan
d8704c11ca PERF: Better use of index when queueing a topci for search reindex.
Also move `Search::INDEX_VERSION` to `SearchIndexer` which is where the
version is actually being used.
2019-04-02 09:53:37 +08:00
Guo Xiang Tan
aa2311a7b0 FIX: Don't reindex posts belonging to a deleted topic for search.
Posts belonging to a deleted topic can't be index for search so we need
to avoid loading those post ids.
2019-04-02 07:36:53 +08:00
Guo Xiang Tan
3fc5dbb045 FIX: Don't attempt to reindex posts that have an empty raw.
If the post ids keep loading, we might end up in a situations where
we're always loading the same post ids over and over again without
indexing anything new.

Follow up to daeda80ada.
2019-04-02 07:13:33 +08:00
Robin Ward
76669bb5a6 FIX: Don't refer to pending review items as flags
They could be queued posts or users, and the notice should reflect that
properly.
2019-04-01 14:46:56 -04:00
Guo Xiang Tan
daeda80ada
FIX: Don't index posts with empty Post#raw for search. (#7263)
* DEV: Remove unnecessary join in `Jobs::ReindexSearch`.

* FIX: Don't index posts with empty `Post#raw` for search.
2019-04-01 10:06:27 +08:00
Tarek Khalil
b1cb95fc23
FEATURE: Introduce ignore duration selection (#7266)
* FEATURE: Introducing new UI for tracking User's ignored or muted states
2019-03-29 10:14:53 +00:00
Robin Ward
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
Guo Xiang Tan
b58c965aad FIX: Destroy optimized image if attempting to migrate to new scheme fails. 2019-03-26 14:37:34 +08:00
Tarek Khalil
fed2dd9148 FEATURE: Add scheduled job to purge expired ignored users (#7211) 2019-03-20 11:01:43 +01:00
Guo Xiang Tan
4020c87680 DEV: Refactor tests for Jobs::CleanUpInactiveUsers.
* Remove use of 0 in favor of `TrustLevel.levels[:newuser]`.
* Consolidate two tests into a single one.
* Test that disabling the feature works.
* Avoid loading full ActiveRecord object in test when we only need to
know the existence of the record.
2019-03-19 09:57:21 +08:00
Bianca Nenciu
2347661a74 FEATURE: Clean up inactive users. (#7172) 2019-03-18 16:25:15 +01:00
Tarek Khalil
bd6d31c9ec
FEATURE: Add IgnoredUsersSummary daily job (#7144)
* FEATURE: Add `IgnoredUsersSummary` daily job

## Why?

This is part of the [Ability to ignore a user feature](https://meta.discourse.org/t/ability-to-ignore-a-user/110254/8).

We want to:

1. Send an automatic group PM that goes out to moderators
2. When {x} users have Ignored the same user, threshold defined by a site setting, default of 5
3. Only send this message every X days which is defined by another site setting
2019-03-14 22:51:43 +00:00
Guo Xiang Tan
b0c8fdd7da FIX: Properly support defaults for upload site settings. 2019-03-13 16:36:57 +08:00
Dan Ungureanu
32bae48fd3 DEV: Use User#human? User#bot? (#7140) 2019-03-12 07:58:14 +08:00
Gerhard Schlager
1121514799 UX: Localize date format in "new user of the month" message 2019-03-06 21:58:25 +01:00
Robin Ward
78ddc82952 FIX: Respect min_flags_staff_visibility for new flags too
There was a situation where if:

* There were new flags to review that met the visibility threshold

AND

* There were old flags that *didn't* meet the threshold

THEN

a pending flags notification would be sent out. This fixes that case.
Staff should not be notified of flags if they do not meet the threshold
and are old.
2019-01-25 11:27:43 -05:00
Robin Ward
96b2585a91 REFACTOR: Remove unncessary stubs from pending flags reminder
They seem to be calculated fine by the application, and stubbing
makes the tests more brittle and prone to regression.
2019-01-24 13:45:58 -05:00