Commit Graph

33151 Commits

Author SHA1 Message Date
Sam Saffron
a9af77d650 PERF: ensure priority for rake rebake tasks is ultra low
Flooding the default queue with lots of jobs is never a good thing. This
reduces the risk of sidekiq going into a bad state post rebake
2019-05-22 10:31:49 +10:00
Sam Saffron
d9413a61d2 PERF: move crawl_topic_links to the low queue
Crawling topic links can be somewhat delayed no need to run it in the default
queue.
2019-05-22 10:18:49 +10:00
Sam Saffron
d6daa60d0a FEATURE: multisite support for posts:rebake_uncooked_posts 2019-05-22 10:04:54 +10:00
Sam Saffron
a2428bd862 FEATURE: warn about sidekiq overload prior to migrating
Also makes pre-flight check ensure there is no giant backlog of posts that
need to be cooked
2019-05-22 10:04:33 +10:00
Sam Saffron
5fdc7b7ca2 Correct 59012fc0
Logic was flipped here by mistake, oops
2019-05-22 09:48:03 +10:00
Sam Saffron
73f178a634 FEATURE: posts:rebake_uncooked_posts to look at mismatching baked_version
also amends flagging onebox updates to set baked_version to nil
2019-05-22 09:43:31 +10:00
Sam Saffron
4f296608da FEATURE: add uploads:s3_migration_status for looking at current status
Also a few minor cleanups and better progress reporting
2019-05-22 09:00:32 +10:00
Sam Saffron
59012fc0f7 PERF: mark posts for rebake vs forcing a rebake inline when migrating to s3
Rebaking posts can be expensive instead of blocking here simply mark posts
for rebake.

We can then work through them faster in other jobs, plus this should not
hold of a datacenter migration.
2019-05-22 08:39:25 +10:00
Joffrey JAFFEUX
0ed1cfd8d9
FIX: ensures we have an elementFromPoint before using it (#7583) 2019-05-21 21:26:54 +02:00
Gerhard Schlager
e8de30939d DEV: Fix the smoke test 2019-05-21 19:02:19 +02:00
Vinoth Kannan
338b927f95 DEV: include local uploads while finding missing uploads
in rake task 'posts:missing_uploads'
2019-05-21 22:07:08 +05:30
Vinoth Kannan
7b82850f66 FIX: migrate_to_s3 task should remap attachment links too. 2019-05-21 21:58:11 +05:30
Daniel Waterworth
f46d2ad086 DEV: Update test-prof (#7572)
* Updated test-prof

* Made rails_helper.rb use new test-prof APIs

Instead of the previous temporary hacks.

* Added environment option to disable prefabrication

It was removed mistakenly
2019-05-21 22:07:40 +10:00
Guo Xiang Tan
92a6f71fd3 DEV: Update smoke test to test editing of a post. 2019-05-21 17:44:46 +08:00
AhmadFCheema
32e7f1efb4 Fix typo for cookies_error in client.en.yml (#7580) 2019-05-21 10:21:04 +02:00
Vinoth Kannan
793915fe6a DEV: should ignore missing post uploads when a user export destroyed 2019-05-21 13:32:36 +05:30
Joffrey JAFFEUX
6c6a6771af
FIX: adds groupBy to lodash available functions list (#7579) 2019-05-21 08:58:57 +02:00
Joffrey JAFFEUX
236adc9ccb
FIX: shows parent category name in title anchor (#7576) 2019-05-21 08:33:22 +02:00
Sam Saffron
8360415453 FEATURE: big warning for uploads missing which can be very very slow on S3 2019-05-21 16:11:56 +10:00
Sam Saffron
cb86d8279a FEATURE: add toggle for uploads:missing so it can skip external
Validating s3 uploads in uploads:missing can be very expensive, allow to
bypass.
2019-05-21 16:11:56 +10:00
Osama Sayegh
c86d2666c4
Pull translations from Transifex (#7578) 2019-05-21 07:18:15 +03:00
Osama Sayegh
0b5924d61e
Refactor keyboard shortcuts modal to fix RTL locales (#7545)
* Refactor keyboard shortcuts modal to fix RTL locales

* Feedback

* Remove lowercase modifier keys
2019-05-21 06:39:32 +03:00
Sam Saffron
307c526840 DEV: correct test that assumed group 123 did not exist
This is an incorrect assumption leading to a flaky test, cause for all we
know, since sequences do not reset every test, group 123 could exist.
2019-05-21 12:57:14 +10:00
Sam Saffron
f4d4f7871e FEATURE: make posts:missing_uploads multisite friendly
Previously this rake job would only run on a single site which is a bit
misleading

This also adds `VERBOSE=1 rake posts:missing_uploads` that will provide a
full report of missing uploads
2019-05-21 12:45:51 +10:00
Sam Saffron
0f76e3090c FIX: use tar -xzvf to extract maxmind vs Ruby wrappers
The tar/gz extractor in Ruby is somewhat buggy, instead rely on battle
tested `tar` and `gzip` Linux commands
2019-05-21 10:48:18 +10:00
Gerhard Schlager
edc6ac0c66 Update translations 2019-05-21 00:04:37 +02:00
Gerhard Schlager
0afcad148a DEV: Always use %{count} in pluralized strings 2019-05-20 23:26:22 +02:00
Gerhard Schlager
ca2ba22a48 DEV: Prettify *.en_US.yml files 2019-05-20 23:21:43 +02:00
Vinoth Kannan
2941c77abc FIX: skip upload recovery if file not found in s3 2019-05-21 00:06:36 +05:30
Jeff Atwood
555305325a copyedit on likes
since these never appear side by side on the same user page tab, I feel their meaning is clear on each individual tab without being quite so verbose
2019-05-20 09:56:06 -07:00
David Taylor
0fbff66d97 DEV: Correct rake task syntax error 2019-05-20 17:43:30 +01:00
David Taylor
31902159af DEV: Allow skipping failed migrations when running S3 migration
Use the SKIP_FAILED environment variable to skip failed sites. Use with caution - make sure you go back and re-run the failed migrations afterwards.
2019-05-20 17:25:56 +01:00
David Taylor
a15cca9a0f DEV: Improve error message for posts:missing_uploads during S3 migration 2019-05-20 16:09:22 +01:00
Joffrey JAFFEUX
3e4e3fc7d7
DEV: adds memory/event listeners reporting (#7575) 2019-05-20 16:35:23 +02:00
Vinoth Kannan
2bfc0cf145 FIX: skip old scheme upload URLs while validating s3 uploads remap 2019-05-20 19:13:41 +05:30
Vinoth Kannan
9f43852609 FIX: remove the unwanted code. 2019-05-20 19:10:21 +05:30
David Taylor
41bc90dd3e DEV: Add progress indicator for post rebake during s3 migration
Now that we run sidekiq jobs synchronously, this is important
2019-05-20 14:19:58 +01:00
David Taylor
77a06209cb DEV: Skip S3 migration if all uploads are already migrated
This makes the task resumable in a multisite context
2019-05-20 14:17:37 +01:00
Joffrey JAFFEUX
bc8c77723e
FIX: puts focus-event at application level (#7568)
Also attempts to use simpler and newer APIs
2019-05-20 13:48:03 +02:00
Gerhard Schlager
b788948985 FEATURE: English locale with international date formats
Makes en_US the new default locale
2019-05-20 13:47:20 +02:00
Gerhard Schlager
a58aa9b4bf Update translations 2019-05-20 13:42:05 +02:00
Joffrey JAFFEUX
9bbf421125
FIX: better node check for clickoutside in widgets (#7573) 2019-05-20 12:45:33 +02:00
Vinoth Kannan
bc0c4b7253 FIX: should not migrate the system uploads to s3 2019-05-20 14:27:34 +05:30
Vinoth Kannan
be3fb85a04 DEV: add post migration checks and raise error if failed. 2019-05-20 14:18:28 +05:30
Joffrey JAFFEUX
0ddfeb9c64
FIX: ensures mousetrap is reset with the application (#7569) 2019-05-20 10:45:31 +02:00
Sam Saffron
3c1fa3910f DEV: track the correct channel in specs
These tests are erratically failing due to distributed cache messages that
can land between tests

Ensure we are only looking for the correct messages
2019-05-20 18:27:09 +10:00
Osama Sayegh
14bae6d52d Make email_excerpt method take an optional post param (#7570)
The spoiler alert plugin replaces spoiler text found in email excerpts with posts URL, which means it needs to have a reference to the post it's processing.

This change makes `email_excerpt` accepts an optional post param, which calls `PrettyText.format_for_email` which then triggers the `reduce_cooked` event that the plugin subscribes to.
2019-05-20 10:04:23 +02:00
Sam Saffron
50e6ee55a9 DEV: add "WAIT_FOR_URL" env to smoke test
This allows you to wait up to N seconds for the smoke test url to come up
in some cases you want to kick off the smoke test prior to having the smoke
test env ready to accept connections
2019-05-20 17:23:09 +10:00
Arpit Jalan
ce89f19250 Bump onebox version.
- use Vimeo engine for private links only
- if og:video_url is missing, make one using Vimeo ID
2019-05-20 12:24:43 +05:30
Sam Saffron
08371db0cc FIX: ensure we don't queue any jobs during s3 migration
Previously we could flood sidekiq with jobs which is not ideal.

This ensures we are 100% done when the job is done.
2019-05-20 16:28:50 +10:00