Commit Graph

469 Commits

Author SHA1 Message Date
Sam Saffron
3d85cc1e69 PERF: run expensive clean up uploads less frequently
Previously every hour we would run a full scan of the entire DB searching
for expired uploads that need to be moved to the tombstone folder.

This commit amends it so we only run the job 2 times per clean_orpha_uploads_grace_period_hours

There is a upper bound of 7 days so even if the grace period is set really
high it will still run at least once a week.

By default we have a 48 grace period so this amends it to run this cleanup
daily instead of hourly. This eliminates 23 times we run this ultra expensive
query.
2019-10-28 11:14:52 +11:00
Dan Ungureanu
96b8710b39
DEV: Fix heisentest (ensure that user ID really does not exist). 2019-10-14 12:25:43 +03:00
Sam Saffron
78c5adc093 DEV: stop hardcoding ids in specs
hardcoded ids are unreliable for running specs and lead to erratic failures
2019-10-08 09:37:07 +11:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Robin Ward
d251f12c9c Tweak calculation for reviewable sensitivities/priorities
Previously, calculating thresholds for reviewables was done based on the
50th and 85th percentile across all reviewables. However, many forum
owners provided feedback that these thresholds were too easy to hit, in
particular when it came to auto hiding content.

The calculation has been adjusted to base the priorities on reviewables
that have a minimum of 2 scores (flags). This should push the amount of
flags required to hide something higher then before.
2019-09-19 14:07:56 -04:00
Robin Ward
d5b52abf2f FIX: Require a min amount of reviewables before calculating thresholds
On forums with very few flags you don't want to calculate averages
because they won't be very useful. Stick with the defaults until we hit
15 reviewables at least.
2019-09-19 13:42:50 -04:00
Régis Hanol
aa511c5b59 FIX: support <img> in code blocks when inlining uploads
Simpler code is better :)

Also added moar specs to ensure <img> tag inside code blocks are properly ignored.
2019-09-12 21:25:14 +02:00
David Taylor
67a98946b8 FIX: Do not log 'pull_hotlinked_images' edits in the staff action log 2019-09-12 15:55:45 +01:00
Joffrey JAFFEUX
a25869969a
DEV: adds event hook when add/remove user to group (#8038) 2019-09-10 11:58:08 -05:00
David Taylor
e2449f9f23 Revert "Revert "Revert "FIX: Heartbeat check per sidekiq process (#7873)"""
This reverts commit c3497559be.
2019-08-30 11:26:16 +01:00
Sam Saffron
c3497559be Revert "Revert "FIX: Heartbeat check per sidekiq process (#7873)""
This reverts commit e805d44965.
We now have mechanisms in place to ensure heartbeat will always
be scheduled even if the scheduler is overloaded per: 098f938b
2019-08-30 10:12:10 +10:00
OsamaSayegh
e805d44965 Revert "FIX: Heartbeat check per sidekiq process (#7873)"
This reverts commit 340855da55.
2019-08-27 11:56:23 +00:00
Osama Sayegh
340855da55
FIX: Heartbeat check per sidekiq process (#7873)
* FIX: Heartbeat check per sidekiq process

* Rename method

* Remove heartbeat queues of previous bootups

* Regis feedback

* Refactor before_start

* Update lib/demon/sidekiq.rb

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Update lib/demon/sidekiq.rb

Co-Authored-By: Régis Hanol <regis@hanol.fr>

* Expire redis keys after 3600 seconds

* Don't use redis to store the list of queues
2019-08-26 09:33:49 +03:00
Gerhard Schlager
00b75b4f4e FIX: Don't try to delete staged, unused admins and mods 2019-08-21 15:29:51 +02:00
Arpit Jalan
0603636cea FIX: include default label when exporting reports 2019-07-26 12:57:13 +05:30
Arpit Jalan
eb9155f3fe
FEATURE: send max 200 emails every minute for bulk invites (#7875)
DEV: deprecate `invite.via_email` in favor of `invite.emailed_status`

This commit adds a new column `emailed_status` in `invites` table for
 tracking email sending status.
 0 - not required
 1 - pending
 2 - bulk pending
 3 - sending
 4 - sent

For normal email invites, invite record is created with emailed_status
 set to 'pending'.

When bulk invites are sent invite record is created with emailed_status
 set to 'bulk pending'.

For invites that generates link, invite record is created with
 emailed_status set to 'not required'.

When invite email is in queue emailed_status is updated to 'sending'

Once the email is sent via `InviteEmail` job the invite emailed_status
 is updated to 'sent'.
2019-07-19 11:29:12 +05:30
Gerhard Schlager
26b843e5e8 Correct typo in spec name 2019-07-04 11:37:37 +02:00
Gerhard Schlager
d513c28e3b FIX: Don't send notification email when user isn't allowed to see topic 2019-07-01 14:03:03 +02:00
Bianca Nenciu
b2eb0f4ad6 FEATURE: Export any type of report supporting table mode. (#7662) 2019-06-28 08:50:31 +02:00
Guo Xiang Tan
8deaef3872 FIX: Don't replace img tags within anchor tags with markdown format.
Follow up to 9a25b0d614.
2019-06-21 12:32:02 +08:00
Guo Xiang Tan
9a25b0d614 FIX: Edge case with anchor tag in InlineUploads. 2019-06-21 09:55:56 +08:00
Vinoth Kannan
9f0574dcfd SPEC: Update webhook event attributes even when an error raised
1e3cb7575d
2019-06-21 00:45:35 +05:30
Guo Xiang Tan
c9db897777 FIX: Remove onebox src from Jobs::PullHotlinkedImages.
The test that was added is incorrect because the post was not cooked.
2019-06-14 09:21:25 +08:00
Guo Xiang Tan
40e67971f9 DEV: Add spec for Email::Sender for upload links in plain text emails. 2019-06-11 16:02:24 +08:00
Guo Xiang Tan
fb0a655e8a FEATURE: Update pull hotlinked images to use Upload#short_url. 2019-06-11 15:17:29 +08:00
Guo Xiang Tan
9d0fba64c0 FIX: Use attachment format in user export system post take 2. 2019-06-11 12:15:11 +08:00
David Taylor
54afa314fb FIX: Do not download emojis in pull_hotlinked_images 2019-06-07 13:00:52 +01:00
Guo Xiang Tan
3b4432f258 DEV: Refactor specs for Jobs::PullHotlinkedImages. 2019-06-06 14:56:21 +08:00
Arpit Jalan
e7fe7010b8
FIX: use hijack for processing bulk invites (#7679)
FIX: do not store bulk invite CSV file on server
2019-06-04 20:19:46 +05:30
Sam Saffron
74c4f926fc FIX: drop deleted posts from search index
This does two things

1. Our "index grace period" has been wound down to 1 day, there is no point
keeping a bloated index for a week, usually when people delete stuff they
mean for it to be removed

2. We were never dropping deleted posts from the index, only posts from
deleted topics

These changes speed up search a tiny bit and reduce background work.
2019-06-04 17:19:59 +10:00
Arpit Jalan
028121b95b
FIX: delete system generated message when user_export record is deleted (#7595)
FIX: system generated message for user export should be closed by default
2019-05-28 16:38:41 +05:30
Robin Ward
b85fa62882 FIX: Don't send replies to flags that are ignored by expiry
The `AutoQueueHandler` will ignore really old flags. In that case, don't
notify the user that the moderator is looking into it. They probably
never saw it because it didn't meet the reviewable minimum priority.
2019-05-27 17:24:36 -04:00
Robin Ward
e74cd54fc6 REFACTOR: Replace score bonuses with low/med/high priorities
We removed score from the UX so it makes more sense to have sites set
priorities instead of score bonuses.
2019-05-23 11:54:45 -04:00
Guo Xiang Tan
df1e6eed5a FIX: Pull hotlinked images for lightbox links as well. 2019-05-23 15:44:37 +08:00
Robin Ward
225593d445 FIX: Small tweak to percentiles based on feedback 2019-05-16 13:45:36 -04:00
Gerhard Schlager
bbab60fa9b FIX: Don’t try to delete inactive moderators
follow-up to 6d77156a
2019-05-13 21:44:03 +02:00
Guo Xiang Tan
8165ceb320 Make rubocop happy. 2019-05-13 09:55:44 +08:00
Daniel Waterworth
54cfc48975 Use prefabricated users in bulk_invite_spec.rb 2019-05-10 08:34:04 -04:00
Robin Ward
5af7c90bab FEATURE: Hide Reviewable scores, change score filter to Priority
We found score hard to understand. It is still there behind the scenes
for sorting purposes, but it is no longer shown.

You can now filter by minimum priority (low, med, high) instead of
score.
2019-05-07 14:05:23 -04:00
Daniel Waterworth
e219588142 DEV: Prefabrication (test optimization) (#7414)
* Introduced fab!, a helper that creates database state for a group

It's almost identical to let_it_be, except:

 1. It creates a new object for each test by default,
 2. You can disable it using PREFABRICATION=0
2019-05-07 13:12:20 +10:00
Guo Xiang Tan
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
Sam Saffron
9be70a22cd DEV: introduce new API to look up dynamic site setting
This removes all uses of both `send` and `public_send` from consumers of
SiteSetting and instead introduces a `get` helper for dynamic lookup

This leads to much cleaner and safer code long term as we are always explicit
to test that a site setting is really there before sending an arbitrary
string to the class

It also removes a couple of risky stubs from the auth provider test
2019-05-07 11:00:30 +10:00
Guo Xiang Tan
24347ace10 FIX: Properly associate user_profiles background urls via upload id.
`Upload#url` is more likely and can change from time to time. When it
does changes, we don't want to have to look through multiple tables to
ensure that the URLs are all up to date. Instead, we simply associate
uploads properly to `UserProfile` so that it does not have to replicate
the URLs in the table.
2019-05-02 14:58:24 +08:00
Robin Ward
404b35bd04 FEATURE: Category Reviewable by Group
Allow a group to review content in a particular category.
2019-04-30 15:23:06 -04:00
Sam Saffron
4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Sam Saffron
45285f1477 DEV: remove update_attributes which is deprecated in Rails 6
See: https://github.com/rails/rails/pull/31998

update_attributes is a relic of the past, it should no longer be used.
2019-04-29 17:32:25 +10:00
Daniel Waterworth
ad44243a57 Removed unused let blocks (#7446)
The bodies of these blocks were never evaluated.
2019-04-29 15:08:56 +08:00
Gerhard Schlager
6d77156a94 FIX: Don't try to delete inactive admins 2019-04-25 11:59:50 +02:00
Robin Ward
70097966ed FIX: Don't add the reviewable score twice 2019-04-23 15:46:18 -04:00
Guo Xiang Tan
d07605d885 DEV: Fix failing spec. 2019-04-23 18:25:28 +08:00