Commit Graph

483 Commits

Author SHA1 Message Date
Régis Hanol
f01169d6ff FIX: don't send email when the post was deleted 2018-08-22 13:13:58 +02:00
Guo Xiang Tan
791ddb8630 PERF: Memoize EmailReceiver#sent_to_mailinglist_mirror?. 2018-08-21 15:44:47 +08:00
Guo Xiang Tan
7dcc69aef4 DEV: Refactor test to not call private method. 2018-08-21 14:29:58 +08:00
Sam
ca5a6f0a9d FIX: handle concurrently creating post reply keys
In some very rare conditions this would be called concurrently and fail
2018-08-21 10:59:18 +10:00
Régis Hanol
de92913bf4 FIX: store the topic links using the cooked upload url 2018-08-14 12:23:32 +02:00
Régis Hanol
ac2513b0f2 FEATURE: automatic PM when a user's email is revoked 2018-08-03 16:39:22 +02:00
OsamaSayegh
a157dfd418 UX: better rejection message when reply via email is too short 2018-08-02 22:43:53 +03:00
Guo Xiang Tan
fad9c2b971 PERF: Move EmailLog#reply_key into new post_reply_keys table. 2018-07-24 13:51:53 +08:00
Guo Xiang Tan
ae8b0a517f PERF: Split skipped email logs into a seperate table. 2018-07-24 13:14:37 +08:00
Guo Xiang Tan
3874d40910 Prepare to drop EmailLog#topic_id. 2018-07-18 10:22:24 +08:00
Neil Lalonde
7b3ef4d13f FIX: use email color settings consistently in notification emails 2018-07-16 12:30:42 -04:00
Maja Komel
0942e2c795 allow adding tags as a custom subject format for emails (#5846)
allow adding tags as a custom subject format for emails
2018-07-11 12:24:07 +10:00
Patrick Gansterer
28dd7fb562 FEATURE: Create hidden posts for received spam emails (#6010)
* Add possibility to add hidden posts with PostCreator

* FEATURE: Create hidden posts for received spam emails

Spamchecker usually have 3 results: HAM, SPAM and PROBABLY_SPAM
SPAM gets usually directly rejected and needs no further handling.
HAM is good message and usually gets passed unmodified.
PROBABLY_SPAM gets an additional header to allow further processing.
This change addes processing capabilities for such headers and marks
new posts created as hidden when received via email.
2018-07-05 11:07:46 +02:00
Régis Hanol
8a53941fe0 FIX: less aggressive gmail eliding 2018-07-04 20:04:46 +02:00
Jeff Wong
4599cc8435 FIX: PM participants listed inline 2018-06-11 18:14:25 -07:00
Arpit Jalan
f9ab3848ed FEATURE: support disabling emails for non-staff users 2018-06-07 18:31:08 +05:30
Sam
89ad2b5900 DEV: Rails 5.2 upgrade and global gem upgrade
This updates tests to use latest rails 5 practice
and updates ALL dependencies that could be updated

Performance testing shows that performance has not regressed
if anything it is marginally faster now.
2018-06-07 14:21:33 +10:00
Ryan Mulligan
fac4bf2f85 ignore emails that are from the reply by email addresses (#5843) 2018-05-23 10:04:45 +02:00
Régis Hanol
86eb3528ec FEATURE: clearer error message when receiving a reply to an old notification 2018-05-09 18:51:01 +02:00
Régis Hanol
6b1ff0edd3 FIX: always update bounce score (instead of doing it once per day) 2018-05-09 16:40:52 +02:00
Régis Hanol
bae7203f42 FIX: proper category > subcategory order in List-Id description header 2018-05-05 01:51:53 +02:00
Jeff Wong
62a8904729
Feature: Include participants at the bottom of PM emails (#5797)
* Feature: Include participants at the bottom of PM emails

... as undecorated links.

https://meta.discourse.org/t/email-notification-recipients-unclear-when-pm-is-sent-to-multiple-users/26934/13?u=featheredtoast

Fix: missing translation for PM mentions

* display membership count as `group (count)`
2018-05-03 15:50:06 -07:00
Régis Hanol
7d223c0e55
FEATURE: add description in 'List-Id' mail headers
https://meta.discourse.org/t/minor-feature-request-descriptions-in-list-id-headers/86681
2018-05-03 16:39:25 +02:00
Régis Hanol
2d561a0422 FIX: don't extract divs with a 'gmail_default' class 2018-05-03 12:29:21 +02:00
Régis Hanol
2b0e505121 Extract signatures from emails sent with Newton 2018-04-19 12:39:55 +02:00
Régis Hanol
fe32733a57 extract signatures from emails sent using Zimbra 2018-04-13 19:04:27 +02:00
Gerhard Schlager
f2d00e5eff FEATURE: Use Message-ID for detecting email replies to group
Ignores the site setting "find_related_post_with_key" and always tries to honor the `In-Reply-To` and `References` header for emails sent to a group.

The senders email address must be included in the `To` or `CC` header of a previous email sent to the group and the `Message-ID` of that email must be included in the current email's `In-Reply-To` or `References` header.
2018-04-05 11:00:38 +02:00
Guo Xiang Tan
142571bba0 Remove use of rescue nil.
* `rescue nil` is a really bad pattern to use in our code base.
  We should rescue errors that we expect the code to throw and
  not rescue everything because we're unsure of what errors the
  code would throw. This would reduce the amount of pain we face
  when debugging why something isn't working as expexted. I've
  been bitten countless of times by errors being swallowed as a
  result during debugging sessions.
2018-04-02 13:52:51 +08:00
Régis Hanol
a0386655a6 Add support to elide content from ProtonMail emails 2018-03-30 10:41:32 +02:00
Gerhard Schlager
fcd352e089 FIX: Try fixing unparsable email addresses
The mail gem returns `UnstructuredField` when it fails to parse email addresses, but the `Receiver` always expects an `AddressList`.
2018-03-27 18:28:54 +02:00
Régis Hanol
20ba54d536
FIX: extracting mail content from exchange emails 2018-03-14 22:02:43 +01:00
Guo Xiang Tan
a9713ca20b REFACTOR: Always prefer to raise an error if record fails to save. 2018-03-13 12:01:01 +08:00
Gerhard Schlager
832f0a9c4d FIX: Typo prevented extraction of email signatures 2018-03-06 11:34:47 +01:00
Régis Hanol
482c615ef8 FEATURE: extract signatures from most popular email services/software 2018-03-02 01:51:15 +01:00
Régis Hanol
3c430a3949 FEATURE: begone gmail signatures! 2018-02-27 15:19:34 +01:00
Régis Hanol
73ee62f55f FEATURE: automatically elide forwarded emails and signature from outlook 2018-02-27 15:00:50 +01:00
Régis Hanol
fd33090646 FEATURE: automatically elides gmail quotes 2018-02-26 23:54:02 +01:00
Felix Wolfsteller
c302c28a7d Switch ids in References-Header field of mails. (#5567)
This change allows email-clients to show threaded views of mails as
expected.  Apparently most algorithms expect the message ids of mails
in the Reference-header-field to be sorted such that they build a
traversal through the thread, so the oldest (original) message being
first, then its child, grandchild and so on until it arrives at the
message id that the "new" mail (that is to be sent) is the reply to.

MSGA [1]
+- Re: MSGA [1-1]
|  +- Re: Re: MSGA [1-2-1]
|  +- Re: Re: MSGA [1-2-2]
+- Re: MSGA [1-1]

If the stuff in brackets would be the message ID, the References-Header
field of a message that is a reply to [1-2-1] should look like:

References: 1, 1-1, 1-2-1

Discussion took place in:
https://meta.discourse.org/t/e-mail-threading-in-ml-mode-does-not-work-in-thunderbird

Main information taken from:
https://www.jwz.org/doc/threading.html
2018-02-22 10:48:23 +01:00
Leo McArdle
5d9d0fcb4f FEATURE: add setting which adds group name to PM email subject (#5475) 2018-02-19 10:20:17 +01:00
Régis Hanol
61930e092a FIX: support incoming emails with just an attachment 2018-02-16 18:14:56 +01:00
Régis Hanol
361030d8ad Also try popular Windows-1252 encoding when decoding incoming email 2018-01-30 23:45:04 +01:00
Gerhard Schlager
bb54eb1192 Improvements to mbox importer
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
2018-01-17 12:04:57 +01:00
Gerhard Schlager
e0d73a957d FEATURE: Allow posting via email to read-only mailing list mirror category 2018-01-05 11:21:53 +01:00
Gerhard Schlager
d7cd7e4dc7 FIX: Never mark emails sent to mailing list mirror as auto-generated 2018-01-05 11:21:53 +01:00
Gerhard Schlager
ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Gerhard Schlager
16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
Neil Lalonde
85a59c632d allow header_instructions to be passed in to message builder 2017-12-05 15:12:24 -05:00
Arpit Jalan
a5e58e7afa UX: add onebox favicon styling in email 2017-12-05 14:08:30 +05:30
Gerhard Schlager
1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager
aea161fabd suppress rejection email when email was sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager
9207dee69a FEATURE: escape HTML when cooking plaintext emails 2017-11-15 20:22:11 +01:00
Gerhard Schlager
d3baae5365 removes whitespaces and uses scope 2017-11-13 15:23:24 +01:00
Gerhard Schlager
5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager
4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward
1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Régis Hanol
be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Gerhard Schlager
880d154381 FIX: deleting staged user of rejected email shouldn't delete incoming email 2017-10-31 15:13:23 +01:00
Gerhard Schlager
c0bb97b5cb FIX: delete staged users when the incoming email is rejected 2017-10-11 16:17:01 +02:00
Régis Hanol
3bdd8f57c1 FIX: invited staged users would sometimes not get notified of replies 2017-10-06 16:37:28 +02:00
Gerhard Schlager
1477a0e910 Adds a rake task for refreshing posts received via email
This is useful when the email_reply_trimmer gem was updated and you want to apply those changes to existing posts.
2017-10-06 14:29:11 +02:00
Gerhard Schlager
9ff1c23a38 fix typo 2017-10-04 00:01:33 +02:00
Gerhard Schlager
7f50380221 FIX: respect email domain whitelist/blacklist when creating staged users 2017-10-03 16:36:08 +02:00
Gerhard Schlager
76706f9144 FIX: don't create staged users when incoming email is rejected
FIX: don't send subscription mail to new users
2017-10-03 16:36:08 +02:00
Gerhard Schlager
6e04f05173 that shouldn't have been there... 2017-09-15 17:33:31 +02:00
Gerhard Schlager
d51eee4dbc FIX: don't try to send a rejection message when the sender was not detected 2017-09-15 17:30:02 +02:00
Gerhard Schlager
a2187b0acd FIX: better error handling for incoming emails 2017-09-15 17:30:02 +02:00
Gerhard Schlager
31ecb4fecf FIX: Handle incoming emails without email address in From header (#5177) 2017-09-12 22:35:24 +02:00
Leo McArdle
99527af38a FIX: show rejected emails with unrecognized errors (#5026)
Although 407a23663d will send rejection
messages for unrecognized errors, sometimes processing the email will
raise an error which has a blank message.

This commit:

1. Shows rejected emails which have already been processed and contain
   a blank error in /admin/email/rejected

2. Replaces new blank error messages with the error type
2017-08-04 16:20:44 +02:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Leo McArdle
407a23663d FEATURE: send rejection email for unrecognized errors 2017-07-21 18:26:52 +01:00
Leo McArdle
d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
David Taylor
9294826f31 Allow inclusion of elided text from emails when sending to a category… (#4844)
* Allow inclusion of elided text from emails when sending to a category address
2017-06-29 13:03:14 +09:00
Régis Hanol
66920cfafd Merge pull request #4938 from LeoMcA/ses-fix
FEATURE: setting to only use the key when finding the related post of…
2017-06-22 16:15:57 +02:00
Arpit Jalan
6796de8bf2 FIX: replace site_name with email_prefix in subject when use_site_subject is true 2017-06-21 21:15:49 +05:30
Leo McArdle
5e0efb3410 FEATURE: setting to only use the key when finding the related post of an email reply
this fixes email-in threading problems when using a SMTP server which modifies the message_id
header, like Amazon SES
2017-06-19 12:22:44 +01:00
Neil Lalonde
eb9f306e64 FIX: fail to send email notification if post has img tag with no src attribute 2017-06-15 10:36:19 -04:00
Régis Hanol
54e8fb0d89 FEATURE: new 'allow_staff_to_upload_any_file_in_pm' site setting 2017-06-12 22:41:29 +02:00
Robin Ward
602e4d0da4 FIX: Return nil when the reply_key is missing 2017-06-08 14:28:48 -04:00
Gerhard Schlager
8299e7e8c3
Add new, experimental version of mbox importer 2017-05-29 20:59:18 +02:00
Régis Hanol
9dddb81cf6 FIX: remove memoization on class method used in a job 2017-05-22 23:35:41 +02:00
Régis Hanol
4fb335f1f0 FIX: don't process the same incoming email more than once 2017-05-18 16:43:07 +02:00
Régis Hanol
80435cf2af Add mutex around incoming email processing 2017-05-18 01:09:51 +02:00
Leo McArdle
59922ce0a4 FEATURE: remove table wrapping posts in notification emails 2017-05-16 10:37:53 -04:00
Régis Hanol
9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Neil Lalonde
5be7a2dad4 FIX: invalid html in notification emails when template has been customized 2017-05-10 14:01:26 -04:00
Régis Hanol
bf322281e3 Improve handling of inlined images in incoming emails 2017-05-03 23:02:02 +02:00
Guo Xiang Tan
90cd35c496 PERF: Reduce allocated memory when generating mailing list email.
Before: 10590714 bytes
After: 5303394 bytes
2017-05-03 16:59:44 +08:00
Claus Strasburger
e9bb9a167b Fix reply-by-mail for 8-bit transfer encodings
The mail class seems to handle mails sent with Content-Transfer-Encoding: 8bit
somewhat weirdly: It decodes them (to utf-8), changes the raw source to base64,
and does not modify the Content-Type:charset= header.

This leads to Discourse trying the message encoding (in my example ISO-8859-1)
first, and if that does not contain any unparseable characters, it uses that.
Sadly, in ISO-8859-1, every byte sequence is valid.

Fix this by always trying to decode as UTF-8 first. The probability of someone
using another encoding that cleanly (but wrongly) decodes as UTF-8 should be
fairly low.
2017-04-30 23:30:40 +02:00
Régis Hanol
aba76bace6 add support to keep img tags when converting to html 2017-04-28 22:14:46 +02:00
Régis Hanol
0ec15af970 restore the 'incoming_email_prefer_html' site setting 2017-04-27 14:31:11 +02:00
Robin Ward
bf9c4a7828 FEATURE: secure_email site setting to prevent data going out in email 2017-04-26 13:05:56 -04:00
Régis Hanol
b76674f640 FEATURE: convert incoming emails in HTML to markdown
- remove incoming_email_prefer_html site setting
- remove HtmlCleaner class
2017-04-26 16:49:06 +02:00
Guo Xiang Tan
423f2ab228 FIX: Processing incoming email should be done in a background job. 2017-04-24 13:57:28 +08:00
Ryan C. Gordon
a51c191a66 Make Email::Receiver.check_address() into a class method. 2017-04-05 23:10:36 -04:00
Arpit Jalan
9f930125f5 FIX: replace site_name in email subject with site title 2017-03-22 23:38:46 +05:30
Arpit Jalan
1853a4852c FIX: use email prefix only in subject 2017-03-21 20:29:57 +05:30
Arpit Jalan
521c88fe58 FIX: enqueue activation email for invited user that has password set 2017-03-20 17:13:21 +05:30
Régis Hanol
ee9d621d9c FIX: surround the FROM alias with " in order to support the @ character 2017-03-07 23:37:21 +01:00
Neil Lalonde
e634b37f9a FIX: from field of emails should be including email_site_title or site title settings 2017-02-27 14:23:07 -05:00
Régis Hanol
d88562e72b Revert "use RFC-complient signature separator dash-dash-space" 2017-02-14 14:58:08 +01:00