discourse/spec/jobs
Martin Brennan e3d495850d
FEATURE: Overhaul email threading (#17996)
See https://meta.discourse.org/t/discourse-email-messages-are-incorrectly-threaded/233499
for thorough reasoning.

This commit changes how we generate Message-IDs and do email
threading for emails sent from Discourse. The main changes are
as follows:

* Introduce an outbound_message_id column on Post that
  is either a) filled with a Discourse-generated Message-ID
  the first time that post is used for an outbound email
  or b) filled with an original Message-ID from an external
  mail client or service if the post was created from an
  incoming email.
* Change Discourse-generated Message-IDs to be more consistent
  and static, in the format `discourse/post/:post_id@:host`
* Do not send References or In-Reply-To headers for emails sent
  for the OP of topics.
* Make sure that In-Reply-To is filled with either a) the OP's
  Message-ID if the post is not a direct reply or b) the parent
  post's Message-ID
* Make sure that In-Reply-To has all referenced post's Message-IDs
* Make sure that References is filled with a chain of Message-IDs
  from the OP down to the parent post of the new post.

We also are keeping X-Discourse-Post-Id and X-Discourse-Topic-Id,
headers that we previously removed, for easier visual debugging
of outbound emails.

Finally, we backfill the `outbound_message_id` for posts that have
a linked `IncomingEmail` record, using the `message_id` of that record.
We do not need to do that for posts that don't have an incoming email
since they are backfilled at runtime if `outbound_message_id` is missing.
2022-09-26 09:14:24 +10:00
..
regular FEATURE: Overhaul email threading (#17996) 2022-09-26 09:14:24 +10:00
about_stats_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
activation_reminder_emails_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
auto_expire_user_api_keys_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
auto_queue_handler_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
automatic_group_membership_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
bookmark_reminder_notifications_spec.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
bulk_grant_trust_level_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
bulk_invite_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
bump_topic_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_dismissed_topic_users_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_associated_accounts_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_crawler_stats_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_email_change_requests_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_email_logs_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_inactive_users_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
clean_up_post_reply_keys_spec.rb DEV: Don’t use change { … }.by(0) in specs 2022-07-26 10:34:15 +02:00
clean_up_unused_staged_users_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
clean_up_uploads_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
clean_up_user_export_topics_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
cleanup_imap_sync_log_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
close_topic_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
correct_missing_dualstack_urls_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
crawl_topic_link_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
create_linked_topic_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
create_missing_avatars_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
create_recent_post_search_indexes_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
create_user_reviewable_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
dashboard_stats_spec.rb FIX: Don't delete previous messages when we're inside the sent_recently window. (#18239) 2022-09-13 12:43:24 -03:00
delete_replies_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
delete_topic_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
disable_bootstrap_mode_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
download_avatar_from_url_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
download_backup_email_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
download_profile_background_from_url_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
emit_web_hook_event_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
enable_bootstrap_mode_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
enqueue_digest_emails_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
enqueue_suspect_users_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
ensure_s3_uploads_existence_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
export_csv_file_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
export_user_archive_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
feature_topic_users_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
fix_out_of_sync_user_uploaded_avatar_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
fix_primary_emails_for_staged_users_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
fix_s3_etags_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
fix_user_usernames_and_groups_names_clash_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
grant_anniversary_badges_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
grant_new_user_of_the_month_badges_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
heartbeat_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
ignored_users_summary_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
invalidate_inactive_admins_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
invite_email_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
jobs_base_spec.rb FIX: properly log all internal job failures (#17805) 2022-08-05 17:40:22 +10:00
jobs_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
mass_award_badge_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
migrate_badge_image_to_uploads_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
notify_category_change_spec.rb FIX: Don't notify editor when category or tag change (#17833) 2022-08-10 18:55:29 +03:00
notify_mailing_list_subscribers_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
notify_moved_posts_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
notify_reviewable_spec.rb DEV: Enable new user menu when experimental sidebar hamburger is enabled (#18133) 2022-08-31 21:15:01 +03:00
notify_tag_change_spec.rb FIX: Don't notify editor when category or tag change (#17833) 2022-08-10 18:55:29 +03:00
old_keys_reminder_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
open_topic_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
pending_queued_posts_reminder_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
pending_reviewables_reminder_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
pending_users_reminder_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
periodical_updates_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
poll_mailbox_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
post_update_topic_tracking_state_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
post_uploads_recovery_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
problem_checks_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
process_bulk_invite_emails_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
process_email_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
process_post_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
process_shelved_notifications_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
publish_topic_to_category_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
pull_hotlinked_images_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
pull_user_profile_hotlinked_images_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
purge_expired_ignored_users_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
rebake_custom_emoji_posts_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
reindex_search_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
remove_banner_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
reviewable_priorities_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
send_system_message_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
suspicious_login_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
sync_acls_for_uploads_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
sync_topic_user_bookmarked_spec.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
tl3_promotions_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
toggle_topic_closed_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
topic_timer_enqueuer_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
truncate_user_flag_stats_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
unsilence_users_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
update_animated_uploads_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
update_gravatar_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
update_s3_inventory_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
update_username_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
user_email_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00