Commit Graph

337 Commits

Author SHA1 Message Date
jbrw
a9b6f4d829
FIX - use ImageMagick to determine size of svg images (#12230)
SVG files can have dimensions expressed in inches, centimeters, etc., which may lead to the dimensions being misinterpreted (e.g. “8in” ends up as 8 pixels).

If the file type is `svg`, ask ImageMagick to work out what size the SVG file should be rendered on screen.

NOTE: The `pencil.svg` file was obtained from https://freesvg.org/1534028868, which has placed the file in to the public domain.
2021-03-01 11:44:00 -05:00
jbrw
fff8a24f2b
FIX: Don’t display error if only error is a missing image (#12216)
`Onebox.preview` can return 0-to-n errors, where the errors are missing OpenGraph attributes (e.g. title, description, image, etc.). If any of these attributes are missing, we construct an error message and attach it to the Oneboxer preview HTML. The error message is something like:

 “Sorry, we were unable to generate a preview for this web page, because the following oEmbed / OpenGraph tags could not be found: description, image”

However, if the only missing tag is `image` we don’t need to display the error, as we have enough other data (title, description, etc.) to construct a useful/complete Onebox.
2021-02-25 14:30:40 -05:00
Bianca Nenciu
3246c3cc92
DEV: Update mail and use fork (#10639)
Version 2.8 brings some changes to how address fields are handled and
this commits updates that and should also include a fix which handles
encoded attachment filenames.

The fork contains a bugfix to correctly decode mail attachments.
2021-02-18 20:15:02 +02:00
jbrw
70050a8ba3
FIX: should_alter_quality should respect png_to_jpg_quality (#12055)
`convert_to_jpeg!` is only called if `convert_png_to_jpeg?` and/or `should_alter_quality?` is true.

`convert_png_to_jpeg?` can be disabled by setting `SiteSetting.png_to_jpg_quality` to 100.

However, `should_alter_quality?` could be true if `SiteSetting.recompress_original_jpg_quality` was lower than the quality of the uploaded file, regardless of file type.

This commits changes `should_alter_quality?` so that uploaded png files will use the `SiteSetting.png_to_jpg_quality` value, rather than ``SiteSetting.recompress_original_jpg_quality` value.
2021-02-12 13:37:35 -05:00
Bianca Nenciu
74b95c88ac
FIX: Use ImageMagick to detect animated images (#11702)
This is a fallback when FastImage cannot be used (animated WEBP images).
2021-01-13 19:01:30 +02:00
jbrw
331236d6d7
Onebox improved error handling and support for Instagram Access Tokens (#11253)
* FEATURE: display error if Oneboxing fails due to HTTP error

- display warning if onebox URL is unresolvable
- display warning if attributes are missing

* FEATURE: Use new Instagram oEmbed endpoint if access token is configured

Instagram requires an Access Token to access their oEmbed endpoint. The requirements (from https://developers.facebook.com/docs/instagram/oembed/) are as follows:

- a Facebook Developer account, which you can create at developers.facebook.com
- a registered Facebook app
- the oEmbed Product added to the app
- an Access Token
- The Facebook app must be in Live Mode

The generated Access Token, once added to SiteSetting.facebook_app_access_token, will be passed to onebox. Onebox can then use this token to access the oEmbed endpoint to generate a onebox for Instagram.

* DEV: update user agent string

* DEV: don’t do HEAD requests against news.yahoo.com

* DEV: Bump onebox version from 2.1.5 to 2.1.6

* DEV: Avoid re-reading templates

* DEV: Tweaks to onebox mustache templates

* DEV: simplified error message for missing onebox data

* Apply suggestions from code review
Co-authored-by: Gerhard Schlager <mail@gerhard-schlager.at>
2020-11-18 12:55:16 -05:00
Rafael dos Santos Silva
cdb7ddc258
DEV: Update MomentJS and highlightJS
We updated version of moment and moment-timezone as our current versions are outdated making Discourse Dates broken on places where timezone had updates, like here in Brazil.

This also update highlightJS to the latest version and corrected a test that relied on a no longer supported locale in
moment.
2020-11-11 12:34:26 +11:00
jbrw
aeb24bd4b5
FIX - don't attempt to optimized animated images (#11031)
* FIX - don't attempt to optimized animated images

* ensure_safe_paths before calling ImageMagick
2020-10-26 15:10:19 -04:00
Krzysztof Kotlarek
333ddd4011 SECURITY: return error on oversized images 2020-09-14 10:45:11 +10:00
Rachel Carvalho
812e0d6b5e
FIX: improve Vanilla importing (#10478)
* ensure emails don't have spaces
* import banned users as suspended for 1k yrs
* upgrade users to TL2 if they have comments
* topic: import views, closed and pinned info
* import messages
* encode vanilla usernames for permalinks. Vanilla usernames can contain spaces and special characters.
* parse Vanilla's new rich body format
2020-08-24 16:19:57 -04:00
Gerhard Schlager
f51ccea028
FIX: Backups should use relative paths for local uploads
This also ensures that restoring a backup works when it was created with the wrong upload paths in the time between ab4c0a4970 (shortly after v2.6.0.beta1) and this fix.
2020-08-21 15:22:28 +02:00
Penar Musaraj
87e2c9de24
DEV: Plugins can extend color definitions (#10383) 2020-08-06 09:46:17 -04:00
David Taylor
16c65a94f7
PERF: Preload S3 inventory data for multisite clusters 2020-07-29 10:31:55 +01:00
Krzysztof Kotlarek
e0d9232259
FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
Penar Musaraj
7559758e10
FEATURE: Support converting HEIF images to JPEG (#10079) 2020-07-22 21:40:09 -04:00
Bianca Nenciu
bd842cd2b0
FEATURE: Parse images in email signatures (#10137)
* FEATURE: Parse images in email signatures

* DEV: Fix tests

* Code review
2020-07-08 15:50:30 +10:00
Martin Brennan
e5da2d24e5
FIX: Add attachment content-disposition for all non-image files (#10058)
This will make it so the original filename is used when downloading all non-image files, bringing S3Store into line with the to_s3 migration and local storage. Video and audio files will still stream correctly in HTML players as well.

See https://meta.discourse.org/t/cannot-download-non-image-media-files-original-filenames-lost-when-uploaded-to-s3/152797 for a lot of extra context.
2020-06-17 11:16:37 +10:00
Gerhard Schlager
859d9b75a7 FIX: Restoring backup from PG12 could fail on PG10
The `EXECUTE FUNCTION` syntax for `CREATE TRIGGER` statements was introduced in PostgreSQL 11. We need to replace `EXECUTE FUNCTION` with `EXECUTE PROCEDURE` in order to be able to restore backups created with PG12 on PG10.
2020-06-16 16:04:14 +02:00
Guo Xiang Tan
9ab5801a1b
FIX: Allow post migrations using #change to carry out unsafe migration 2020-05-15 14:23:27 +08:00
tshenry
0d394e6b76
FEATURE: Include category position when exporting categories (#9658) 2020-05-07 12:17:15 +10:00
Martin Brennan
867bc3b48e
FIX: Change base importer to create new Bookmark records (#9603)
Also add a spec and fixture with a mock importer that we can use to test the create_X methods of the base importer
2020-05-01 11:34:55 +10:00
tshenry
e41f1e6ec2
FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
Vinoth Kannan
884eea7a83 FEATURE: add support for upload format in theme settings. 2020-04-15 18:34:02 +05:30
romanrizzi
4673f31c75 FIX: Bulk badge awards should work even if the CSV has nil values 2020-02-27 11:08:24 -03:00
Martin Brennan
3af2670bd5
FIX: Consider webp a supported image format for upload (#9015)
* Also fixes an issue where if webp was a downloaded hotlinked
  image and then secure + sent in an email, it was not being
  redacted because webp was not a supported media format in
  FileHelper
* Webp originally removed as an image format in
  https://github.com/discourse/discourse/pull/6377
  and there was a spec to make sure a .bin webp
  file did not get renamed from its type to webp.

  However we want to support webp images now to make
  sure they are properly redacted if secure media is
  on, so change the example in the spec to use tiff,
  another banned format, instead
2020-02-21 13:08:01 +10:00
Roman Rizzi
9441362c72
FEATURE: Support uploading a csv with either user emails or usernames (#8971) 2020-02-18 10:53:12 -03:00
Gerhard Schlager
453bec9394 FEATURE: Add small action post to indicate forwarded email
This happens only when the sender of the email didn't write anything in their email.
2020-02-12 16:23:17 +01:00
David Taylor
5919618a87
DEV: Drop legacy OpenID 2.0 support (#8894)
This is not used in core or official plugins, and has been printing a deprecation notice since v2.3.0beta4. All OpenID 2.0 code and dependencies have been dropped. The user_open_ids table remains for now, in case anyone has missed the deprecation notice, and needs to migrate their data.

Context at https://meta.discourse.org/t/-/113249
2020-02-07 17:32:35 +00:00
Gerhard Schlager
e84d88ddea FIX: Email attachments with a size of 0 bytes caused error 2020-02-04 12:30:43 +01:00
Gerhard Schlager
e474cda321 REFACTOR: Restoring of backups and migration of uploads to S3 2020-01-14 11:41:35 +01:00
Roman Rizzi
d69c5eebcf
Feature: Mass award badge (#8694)
* UI: Mass grant a badge from the admin ui

* Send the uploaded CSV and badge ID to the backend

* Read the CSV and grant badge in batches

* UX: Communicate the result to the user

* Don't award if badge is disabled

* Create a 'send_notification' method to remove duplicated code, slightly shrink badge image. Replace router transition with href.

* Dynamically discover current route
2020-01-13 11:20:26 -03:00
Vinoth Kannan
cfd1549abe FIX: allow underscore in file extension while downloading the uploads. 2020-01-03 10:09:07 +05:30
Gerhard Schlager
c88797bf0e FIX: Ignore DMARC for emails sent to mailing list mirror 2019-12-06 13:29:39 +01:00
Leo McArdle
2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00
Vinoth Kannan
3bb7ad4be1
FEATURE: remove support for 'suppress_from_latest' category setting. (#8308) 2019-11-18 12:28:35 +05:30
Leo McArdle
e7ff6809a3 FEATURE: add SES spam header to recognised spam headers (#8254) 2019-10-28 12:46:53 -04:00
Krzysztof Kotlarek
99086edf85
FIX: Allow themes to upload and serve js files (#8188)
If you set `config.public_file_server.enabled = false` when you try to get uploaded js file you will get an error:
`Security warning: an embedded <script> tag on another site requested protected JavaScript. If you know what you're doing, go ahead and disable forgery protection on this action to permit cross-origin JavaScript embedding.`

The reason is that content type is `application/javascript` and in Rails 5 guard looked like that:
https://github.com/rails/rails/blob/5-2-stable/actionpack/lib/action_controller/metal/request_forgery_protection.rb#L278-L280
However, in Rails 6 `application` was added to regex:
https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/request_forgery_protection.rb#L282-L284

This pull request is related to https://meta.discourse.org/t/uploaded-js-file-for-theme-causes-a-rejection/129753/8
2019-10-14 15:40:33 +11:00
Roman Rizzi
5357ab3324
SECURITY: Safely decompress backups when restoring. (#8166)
* SECURITY: Safely decompress backups when restoring.

* Fix tests and update theme_controller_spec to work with zip files instead of .tar.gz
2019-10-09 11:41:16 -03:00
Jarek Radosz
20514f2e44
DEV: Update markdown-it from 8.4.1 to 10.0.0 (#8164) 2019-10-08 13:00:22 +02:00
Gerhard Schlager
7e0eeed292 FEATURE: Add attachments to outgoing emails
This feature is off by default and can can be configured with the `email_total_attachment_size_limit_kb` site setting.

Co-authored-by: Maja Komel <maja.komel@gmail.com>
2019-07-25 15:57:45 +02:00
Vinoth Kannan
ad04ce9f43 FIX: remove post upload record creation inside 'find_missing_uploads' method. 2019-07-19 01:44:08 +05:30
Gerhard Schlager
a77d8bee68 FIX: Use correct locale when translating without cache
follow-up to 6d8eb9c1
2019-06-05 14:19:56 +02:00
Arpit Jalan
e7fe7010b8
FIX: use hijack for processing bulk invites (#7679)
FIX: do not store bulk invite CSV file on server
2019-06-04 20:19:46 +05:30
Gerhard Schlager
4a1755b780 FIX: Don't create staged users when processing forwarded email fails 2019-05-09 23:47:47 +02:00
Sam Saffron
9be70a22cd DEV: introduce new API to look up dynamic site setting
This removes all uses of both `send` and `public_send` from consumers of
SiteSetting and instead introduces a `get` helper for dynamic lookup

This leads to much cleaner and safer code long term as we are always explicit
to test that a site setting is really there before sending an arbitrary
string to the class

It also removes a couple of risky stubs from the auth provider test
2019-05-07 11:00:30 +10:00
David Taylor
0644c10bfe FIX: Prioritize VERP key over final_recipient header in bounced email
Per RFC3464 2.3.2, the final_recipient header may not match the address we originally sent the email to.
2019-05-03 12:12:44 +01:00
David Taylor
0e303c7f5d
FEATURE: Automatically generate optimized site metadata icons (#7372)
This change automatically resizes icons for various purposes. Admins can now upload `logo` and `logo_small`, and everything else will be auto-generated. Specific icons can still be uploaded separately if required.

## Core

- Adds an SiteIconManager module which manages automatic resizing and fallback

- Icons are looked up in the OptimizedImage table at runtime, and then cached in Redis. If the resized version is missing for some reason, then most icons will fall back to the original files. Some icons (e.g. PWA Manifest) will return `nil` (because an incorrectly sized icon is worse than a missing icon). 

- `SiteSetting.site_large_icon_url` will return the optimized version, including any fallback. `SiteSetting.large_icon` continues to return the upload object. This means that (almost) no changes are required in core/plugins to support this new system.

- Icons are resized whenever a relevant site setting is changed, and during post-deploy migrations

## Wizard

- Allows `requiresRefresh` wizard steps to reload data via AJAX instead of a full page reload

- Add placeholders to the **icons** step of the wizard, which automatically update from the "Square Logo"

- Various copy updates to support the changes

- Remove the "upload-time" resizing for `large_icon`. This is no longer required.

## Site Settings UX

- Move logo/icon settings under a new "Branding" tab

- Various copy changes to support the changes

- Adds placeholder support to the `image-uploader` component

- Automatically reloads site settings after saving. This allows setting placeholders to change based on changes to other settings

- Upload site settings will be assigned a placeholder if SiteIconManager `responds_to?` an icon of the same name

## Dashboard Warnings

- Remove PWA icon and PWA title warnings. Both are now handled automatically.

## Bonus

- Updated the sketch logos to use @awesomerobot's new high-res designs
2019-05-01 14:44:45 +01:00
Sam Saffron
4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Guo Xiang Tan
ce4c8e957b PERF: Avoid looking up the same group multiple times during bulk invite.
Also cache the guardian check because it does a query to check if the
user is an owner of the group.
2019-04-16 09:42:25 +08:00
Guo Xiang Tan
6e6cdfaf80 FIX: Bulk invite should not add users to automatic groups.
* Also checks that the user creating the bulk invite has permission.
2019-04-16 09:42:25 +08:00
David Taylor
6a05f190c6
PERF: Do not create staged users for most rejected incoming emails (#7301)
Previously we would create users, then destroy them at the end of the job if the post was rejected. Now we do not create users unless required.
2019-04-08 10:36:39 +01:00
Neil Lalonde
1812a38f0a FIX: upload watched words should use UTF-8 2019-03-21 13:46:16 -04:00
Penar Musaraj
dafba62931 DEV: Ignore SVG sprite symbols missing ids 2019-03-18 22:07:19 -04:00
Penar Musaraj
d6d4a5ba4a FEATURE: support custom icons in themes (#7155)
* First take

* Add support for sprites in themes

Automatically register any custom icons added via themes or plugins

* Fix theme sprite caching

* Simplify test

* Update lib/svg_sprite/svg_sprite.rb

Co-Authored-By: pmusaraj <pmusaraj@gmail.com>

* Fix /svg-sprite/search request
2019-03-15 17:16:15 +11:00
Sam
b2187301fd FEATURE: allow users to easily track/watch/mute topics via email
If you reply to an email with the word "mute" a topic will be muted
If you reply to an email with the word "track" a topic will be tracked
If you reply to an email with the word "watch" a topic will be watched

These ninja command can help advanced mailing list ex-users, saves a trip
to the website
2019-03-06 18:38:49 +11:00
Penar Musaraj
9bf11a7c02
FEATURE: Add localized timezone names (#7004)
Uses github.com/discourse/moment-timezone-names-translations to translate timezone names.
Plugins can also provide their own timezone name translations.
2019-02-25 14:40:02 -05:00
Joffrey JAFFEUX
900e187627
DEV: removing blank files (#7057) 2019-02-25 09:52:44 +01:00
Gerhard Schlager
4799cf2811 FIX: moment_js locale files provided by plugins were ignored 2019-02-19 15:28:47 +01:00
Régis Hanol
c256121833 FIX: add support for style element in SVGs 2019-02-07 12:09:06 +01:00
David Taylor
a3b47c1dd1 FEATURE: Allow string theme settings to display with multiple lines
To use, add `textarea: true` to the theme settings.yml. For example:

```
my_setting:
  default: "some string"
  textarea: true
```
2019-02-05 14:14:53 +00:00
Vinoth Kannan
b4f713ca52
FEATURE: Use amazon s3 inventory to manage upload stats (#6867) 2019-02-01 10:10:48 +05:30
Régis Hanol
db76e5a368 FIX: properly handle attachments in received emails
This regressed in 1ac3e547 when we added support for eml attachments.
2019-01-25 19:13:34 +01:00
David Taylor
afd449089f
FEATURE: Import and export themes in a .tar.gz format (#6916) 2019-01-23 14:40:21 +00:00
Penar Musaraj
e11c6ffa89 FEATURE: allow extending CSP base-uri and object-src
Plus, ensure :none is stripped, it cannot be combined with other sources
2019-01-09 15:34:14 -05:00
Rafael dos Santos Silva
f73fe36772 FEATURE: PWA compatibility checks in the Dashboard (#6850) 2019-01-09 08:46:11 +08:00
Sam
766e67ce57 FEATURE: introduce lossy color optimization on resized pngs
This feature ensures optimized images run via pngquant, this results extreme amounts of savings for resized images. Effectively the only impact is that the color palette on small resized images is reduced to 256.

To ensure safety we only apply this optimisation to images smaller than 500k.

This commit also makes a bunch of image specs less fragile.
2019-01-02 17:20:02 +11:00
Kyle Zhao
488fba3c5f
FEATURE: allow plugins and themes to extend the default CSP (#6704)
* FEATURE: allow plugins and themes to extend the default CSP

For plugins:

```
extend_content_security_policy(
  script_src: ['https://domain.com/script.js', 'https://your-cdn.com/'],
  style_src: ['https://domain.com/style.css']
)
```

For themes and components:

```
extend_content_security_policy:
  type: list
  default: "script_src:https://domain.com/|style_src:https://domain.com"
```

* clear CSP base url before each test

we have a test that stubs `Rails.env.development?` to true

* Only allow extending directives that core includes, for now
2018-11-30 09:51:45 -05:00
Vinoth Kannan
cedd2118c4
FEATURE: If PM email bounced for staged user then alert in whisper reply (#6648) 2018-11-27 00:29:37 +05:30
Sam
86255faa08 FEATURE: do not switch to JPEG unless you meet 75k byte savings
This also adjusts the algorithm to expect

- 30% saving for JPEG conversion

AND

- Minimum of 75K bytes saved

The reasoning for increase of saving requirements is cause PNG may have been
uploaded unoptimized, 30% saving on PNG is very possible
2018-11-21 11:01:08 +11:00
Régis Hanol
4459665dee
REFACTOR: use tables instead of custom fields for polls (#6359)
Co-authored-by: Guo Xiang Tan <tgx_world@hotmail.com>
2018-11-19 14:50:00 +01:00
Guo Xiang Tan
44391ee8ab
FEATURE: Upload Site Settings. (#6573) 2018-11-14 15:03:02 +08:00
Maja Komel
1ac3e5473a FIX: don't strip eml attachments from received emails 2018-11-05 09:35:22 +01:00
Bianca Nenciu
effbef7d0b UX: Use user locale for locations. (#6527)
* UX: Use user locale for locations.

* DEV: Added MaxMindDB test data and fixed test.
2018-10-25 10:54:01 +00:00
David Taylor
7ac08f936e
FEATURE: Upload tags from CSV (#6484) 2018-10-15 09:12:54 +01:00
Gerhard Schlager
7a41a783a4 FIX: Don't reply to Unsubscribe email sent to mailing list mirror 2018-10-11 16:09:22 +02:00
Guo Xiang Tan
5039a6c3f1 FIX: Strip null bytes in mail subjects. 2018-10-11 09:46:32 +08:00
Guo Xiang Tan
40fa96777d
FEATURE: Post deployment migrations. (#6406)
This moves us away from the delayed drops pattern which
was problematic on two counts. First, it uses a hardcoded "delay for"
duration which may be too short for certain deployment strategies.
Second, delayed drop doesn't ensure that it only runs after
the latest application code has been deployed. If the migration runs
and the application code fails to deploy, running the migration after
"delay for" has been met will cause the application to blow up.

The new strategy allows post deployment migrations to be skipped if the
env `SKIP_POST_DEPLOYMENT_MIGRATIONS` is provided.

```
SKIP_POST_DEPLOYMENT_MIGRATIONS=1 rake db:migrate
-> deploy app servers
SKIP_POST_DEPLOYMENT_MIGRATIONS=0 rake db:migrate
```

To aid with the generation of a post deployment migration, a generator
has been added. Simply run `rails generate post_migration`.
2018-10-08 15:47:38 +08:00
Penar Musaraj
da9eee5262 FIX: Force enable user PM emails option when user posts to a group by email. 2018-10-02 12:38:10 +08:00
Gerhard Schlager
e2770bc1c4 FIX: async reload of locales could result in missing translations 2018-10-01 17:14:36 +02:00
Bianca Nenciu
5407036ef9 DEV: Run prettier. (#6420) 2018-09-21 11:02:23 +00:00
Guo Xiang Tan
4a92c5b2d6 UploadRecovery should recover attachments too. 2018-09-19 10:44:36 +08:00
Guo Xiang Tan
ce6a0a5e9e FIX: Moving upload to tombstone should update modification time.
A upload created a long time ago will be nuked from the tombstone
immediately if it gets deleted.
2018-09-18 10:48:29 +08:00
Guo Xiang Tan
71caf7521d Drop tif, tiff, webp and bmp from supported images.
https://meta.discourse.org/t/cr2-raw-files-are-being-treated-as-tiff-files/96775/3?u=tgxworld
2018-09-12 09:29:54 +08:00
Kris
faf09bb8c8 Replacing default brown category color 2018-08-24 14:18:14 -04:00
Guo Xiang Tan
36a7028f19 FEATURE: Clean up PostReplyKey records.
* Default retention of 90 days.
2018-08-23 10:40:02 +08:00
Sam
d7b1919ead correct specs 2018-08-20 12:46:14 +10:00
Sam
8b5e42ea16 FIX: always test and coerce to image on upload
In the past the filename of the origin was used as the source
for the extension of the file when optimizing on upload.

We now use the actual calculated extension based on upload data.
2018-08-20 12:18:59 +10:00
Guo Xiang Tan
2b57239389 FIX: Upload's content is the only source of truth for the file type. 2018-08-07 13:15:00 +08:00
David Taylor
812add18bd REFACTOR: Serve auth provider information in the site serializer.
At the moment core providers are hard-coded in Javascript, and plugin providers get added to the JS payload at compile time. This refactor means that we only ship enabled providers to the client.
2018-08-06 09:25:48 +01:00
Joffrey JAFFEUX
066010db7d
FEATURE: introduces list/compact_list components 2018-08-03 16:41:37 -04:00
Gerhard Schlager
5d421fb946 FIX: Try respecting charset in HTTP header of RSS feed 2018-08-01 10:41:20 +02:00
Gerhard Schlager
ff942ed2f3 FIX: Try detecting encoding of RSS feed 2018-08-01 10:41:20 +02:00
Guo Xiang Tan
30242972d3 Add spec for 4bf3bf6786. 2018-07-26 09:16:14 +08:00
David Taylor
eda1462b3b
FEATURE: List, revoke and reconnect associated accounts. Phase 1 (#6099)
Listing connections is supported for all built-in auth providers. Revoke and reconnect is currently only implemented for Facebook.
2018-07-23 16:51:57 +01:00
Guo Xiang Tan
1d74ccaaf8 Add compatibility for ImageMagick7. 2018-07-17 15:50:58 +08:00
Patrick Gansterer
28dd7fb562 FEATURE: Create hidden posts for received spam emails (#6010)
* Add possibility to add hidden posts with PostCreator

* FEATURE: Create hidden posts for received spam emails

Spamchecker usually have 3 results: HAM, SPAM and PROBABLY_SPAM
SPAM gets usually directly rejected and needs no further handling.
HAM is good message and usually gets passed unmodified.
PROBABLY_SPAM gets an additional header to allow further processing.
This change addes processing capabilities for such headers and marks
new posts created as hidden when received via email.
2018-07-05 11:07:46 +02:00
Régis Hanol
8a53941fe0 FIX: less aggressive gmail eliding 2018-07-04 20:04:46 +02:00
OsamaSayegh
e58ed247f2 REFACTOR: uploads controller specs to requests (#5907) 2018-06-04 11:13:52 +08:00
Ryan Mulligan
fac4bf2f85 ignore emails that are from the reply by email addresses (#5843) 2018-05-23 10:04:45 +02:00
Régis Hanol
86eb3528ec FEATURE: clearer error message when receiving a reply to an old notification 2018-05-09 18:51:01 +02:00
Régis Hanol
ddb092f397 FIX: update mail gem to fix UTF-8 parsing issue 2018-04-25 21:53:37 +02:00
Gerhard Schlager
fcd352e089 FIX: Try fixing unparsable email addresses
The mail gem returns `UnstructuredField` when it fails to parse email addresses, but the `Receiver` always expects an `AddressList`.
2018-03-27 18:28:54 +02:00
Gerhard Schlager
19c5afc69d Protect against accidental table renames 2018-03-27 13:18:13 +11:00
Sam
6a3c8fe69c FEATURE: protect against accidental column or table drops
Often we need to amend our schema, it is tempting to use
drop_table, rename_column and drop_column to amned schema
trouble though is that existing code that is running in production
can depend on the existance of previous schema leading to application
breaking until new code base is deployed.

The commit enforces new rules to ensure we can never drop tables or
columns in migrations and instead use Migration::ColumnDropper and
Migration::TableDropper to defer drop the db objects
2018-03-21 15:43:32 +11:00
Sam
b57d4c87a4 FEATURE: update CommonMark to version 0.28 2018-03-08 12:19:53 +11:00
OsamaSayegh
282f53f0cd FEATURE: Theme settings (2) (#5611)
Allows theme authors to specify custom theme settings for the theme. 

Centralizes the theme/site settings into a single construct
2018-03-04 19:04:23 -05:00
Régis Hanol
fd33090646 FEATURE: automatically elides gmail quotes 2018-02-26 23:54:02 +01:00
Sam
720e1965e3 FEATURE: add category suppress from latest
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
2018-02-22 09:56:35 +11:00
Guo Xiang Tan
c7c8f38eac Use proper encoding for email fixtures. 2018-02-21 17:06:35 +08:00
Régis Hanol
61930e092a FIX: support incoming emails with just an attachment 2018-02-16 18:14:56 +01:00
Sam
e456ae1b6c Revert "Upgrade mail gem to drop dependency on mime-types."
This reverts commit 498cc15aa9.

Noticed a certain amount of extra allocations due to this upgrade,
reverting to confirm if this is the case
2018-02-16 07:04:22 +11:00
Guo Xiang Tan
498cc15aa9 Upgrade mail gem to drop dependency on mime-types.
Boot Memory Usage

```
Total allocated: 225695379 bytes (1990663 objects)
Total retained:  32795116 bytes (291607 objects)
```

```
Total allocated: 218342892 bytes (1897350 objects)
Total retained:  30123426 bytes (261198 objects)
```
2018-02-15 16:24:26 +08:00
Gerhard Schlager
eb52c5469e FEATURE: Allow plugins to register a new locale 2018-01-25 14:57:41 +01:00
Gerhard Schlager
9f7ae908d8 Add specs to check email domain whitelist/blacklist for To and Cc 2018-01-10 16:57:26 +01:00
Arpit Jalan
6c1ebbb95c add test case for csv BOM handling 2018-01-09 15:49:41 +05:30
Gerhard Schlager
f086d28b30 FIX: Do not validate messages sent to mailing list mirror 2018-01-05 11:21:53 +01:00
Gerhard Schlager
ceb7590bcb FIX: bounced email can contain multiple status codes 2018-01-03 17:59:20 +01:00
Kyle Zhao
5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Gerhard Schlager
1a3ab7c02e ignore some site settings for emails sent to mailinglist mirror category 2017-11-17 15:29:14 +01:00
Gerhard Schlager
5210e3e744 FEATURE: accept incoming email with reply_key mismatch when original email was forwarded 2017-11-12 23:44:22 +01:00
Gerhard Schlager
4dc4bc70c8 FIX: ignore_by_title should match case-insensitive 2017-11-12 01:43:18 +01:00
Robin Ward
4270c93666 FIX: Missing yml file 2017-11-10 14:29:36 -05:00
Robin Ward
1f14350220 Rename "Blocked" to "Silenced" 2017-11-10 14:10:27 -05:00
Régis Hanol
be0c7609f1 FIX: validates attachments against current authorized extensions 2017-11-07 19:17:33 +01:00
Vinoth Kannan
59529ea54a generalize spec fixture 2017-11-04 11:47:26 +05:30
Vinoth Kannan
a00af4d85a FEATURE: Rake task to export and import category structure 2017-11-01 17:17:05 +05:30
Gerhard Schlager
c0bb97b5cb FIX: delete staged users when the incoming email is rejected 2017-10-11 16:17:01 +02:00
Gerhard Schlager
7f50380221 FIX: respect email domain whitelist/blacklist when creating staged users 2017-10-03 16:36:08 +02:00
Gerhard Schlager
76706f9144 FIX: don't create staged users when incoming email is rejected
FIX: don't send subscription mail to new users
2017-10-03 16:36:08 +02:00
Kyle Zhao
15cd3b78ae integration test for PollFeed job 2017-10-02 01:16:11 -04:00
Gerhard Schlager
31ecb4fecf FIX: Handle incoming emails without email address in From header (#5177) 2017-09-12 22:35:24 +02:00
Erick Guan
1646bc0031 FIX: fails loud if default setting is not set
Noted:
- `push_api_secret_key` is set in initializer. Shimed with ''
- `default_theme_key` is set in seeding. Shimed with ''
2017-08-15 12:07:25 +02:00
Guo Xiang Tan
2c39743d5d Introduce multisite tests for better coverage. 2017-08-08 12:58:22 +09:00
Guo Xiang Tan
3f24ed2b3e Can't revert due to incompatibility of new site setting types.
Revert "Revert "FEATURE: Site settings defaults per locale""

This reverts commit 439fe8ba24.
2017-08-07 10:43:09 +09:00
Guo Xiang Tan
439fe8ba24 Revert "FEATURE: Site settings defaults per locale"
This reverts commit 468a8fcd20.
2017-08-07 10:31:50 +09:00
Erick Guan
468a8fcd20 FEATURE: Site settings defaults per locale
This change-set allows setting different defaults for different locales. 

It also:

- Adds extensive testing around site setting validation

- raises deprecation error if site setting has the default property based on env

- relocated site settings for dev and tests in the initializer

- deprecated client_setting in the site setting's loading process

- ensure it raises when a enum site setting being set

- default_locale is promoted to `required` category.

- fixes incorrect default setting and validation

- fixes ensure type check for site settings

- creates a benchmark for site setting

- sets reasonable defaults for Chinese
2017-08-02 12:24:19 -04:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan
0ab0c82f11 Reenable skipped Email::Receiver spec. 2017-07-24 22:27:24 +09:00
Sam
fc90dd3592 Work in progress, CommonMark spec 2017-07-20 17:47:16 -04:00
David Taylor
9294826f31 Allow inclusion of elided text from emails when sending to a category… (#4844)
* Allow inclusion of elided text from emails when sending to a category address
2017-06-29 13:03:14 +09:00
Sam
6a1f579c6e FIX: don't search for plugins in nested subdirectories 2017-05-16 17:28:56 -04:00
Sam
bc0b9af576 FEATURE: support uploads for themes
This allows themes to bundle various assets
2017-05-10 15:47:11 -04:00
Arpit Jalan
e89d0a6b20 FIX: importing a theme via file was broken 2017-05-08 12:03:24 +05:30
Régis Hanol
bf322281e3 Improve handling of inlined images in incoming emails 2017-05-03 23:02:02 +02:00
Claus Strasburger
e9bb9a167b Fix reply-by-mail for 8-bit transfer encodings
The mail class seems to handle mails sent with Content-Transfer-Encoding: 8bit
somewhat weirdly: It decodes them (to utf-8), changes the raw source to base64,
and does not modify the Content-Type:charset= header.

This leads to Discourse trying the message encoding (in my example ISO-8859-1)
first, and if that does not contain any unparseable characters, it uses that.
Sadly, in ISO-8859-1, every byte sequence is valid.

Fix this by always trying to decode as UTF-8 first. The probability of someone
using another encoding that cleanly (but wrongly) decodes as UTF-8 should be
fairly low.
2017-04-30 23:30:40 +02:00
Régis Hanol
887e9af84f FEATURE: new 'max_image_megapixels' site setting 2017-01-11 23:37:12 +01:00
Régis Hanol
185dcb2ca1 handle emails with localized headers 😠 2017-01-09 22:59:30 +01:00
Régis Hanol
98c62bccb5 FIX: mark forwarded email as read by the forwarder
FIX: 'Re:' prefix is mostly used for replies and not forwarded emails
2017-01-06 15:33:55 +01:00
Régis Hanol
62763f025c FIX: wasn't able to parse FROM email in the embedded email 2016-12-01 18:34:47 +01:00
Régis Hanol
214f5bff5c don't send more than 1 reply per day to auto-generated emails 2016-06-28 16:42:05 +02:00