Commit Graph

283 Commits

Author SHA1 Message Date
Joffrey JAFFEUX
0d3d2c43a0
DEV: s/\$redis/Discourse\.redis (#8431)
This commit also adds a rubocop rule to prevent global variables.
2019-12-03 10:05:53 +01:00
Penar Musaraj
102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Sam Saffron
e7cf4579a8 DEV: improve usability of subfolder specs
Previously people were not consistent about mocking which left internals in
a fragile state when running subfolder specs.

This introduces a simple helper `set_subfolder` which you can use to set
the subfolder for the spec. It takes care of proper configuration of subfolder
and teardown.

```
# usage
set_subfolder "/my_amazing_subfolder"
```

You should no longer stub base_uri or global_settings
2019-11-15 16:48:24 +11:00
Vinoth Kannan
129e308dac fix the incorrect specs
Previously the 'local_cdn_url' method didn't returned the correct cdn url. So we written few incorrect spec tests too.\n\nf92a6f7ac5228342177bf089d269e2f69a69e2f5
2019-10-14 12:49:11 +05:30
Sam Saffron
e9cf0de588 FEATURE: do not replace ↔ with an emoji
↔ technically can be an emoji, but usually will not be used in that way
2019-08-30 15:06:23 +10:00
Joffrey JAFFEUX
5d72561f4f
FIX: allows replacement of digits and symbols emojis (#7978) 2019-08-07 11:38:58 +02:00
David Taylor
39e0442de9 FIX: Various watched words improvements
- Client-side censoring fixed for non-chrome browsers. (Regular expression rewritten to avoid lookback)
- Regex generation is now done on the server, to reduce repeated logic, and make it easier to extend in plugins
- Censor tests are moved to ruby, to ensure everything works end-to-end
- If "watched words regular expressions" is enabled, warn the admin when the generated regex is invalid
2019-08-02 15:29:12 +01:00
Gerhard Schlager
9a11a8b33b FEATURE: Site setting for typographic quotation marks
Adds locale defaults for German and French
2019-07-11 23:19:28 +02:00
Guo Xiang Tan
06d974d55c FEATURE: Add base62 sha1 to cooked data attribute
* FEATURE: Add base62 sha1 to data attribute in `Post#cooked`.

* FIX: Use `Upload#short_url` when quoting an image.
2019-06-11 11:15:45 +10:00
Guo Xiang Tan
799bd62803 DEV: Improve PrettyText spec to test for markdown image title attr. 2019-06-10 11:00:23 +08:00
Guo Xiang Tan
f0620e7118 FEATURE: Support [description|attachment](upload://<short-sha>) in MD take 2.
Previous attempt was missing `post_uploads` records.
2019-05-29 09:26:32 +08:00
Penar Musaraj
7c9fb95c15 Temporarily revert "FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603)"
This reverts commit b1d3c678ca.

We need to make sure post_upload records are correctly stored.
2019-05-28 16:37:01 -04:00
Guo Xiang Tan
b1d3c678ca FEATURE: Support [description|attachment](upload://<short-sha>) in MD. (#7603) 2019-05-28 11:18:21 -04:00
Gerhard Schlager
e224100023 FIX: Don't convert :) into Emoji when emojis or emoji shurtcuts are disabled 2019-05-22 11:44:39 +02:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Arpit Jalan
9758857a78 FIX: escape Vimeo iframe URL 2019-05-09 23:01:50 +05:30
Daniel Waterworth
e219588142 DEV: Prefabrication (test optimization) (#7414)
* Introduced fab!, a helper that creates database state for a group

It's almost identical to let_it_be, except:

 1. It creates a new object for each test by default,
 2. You can disable it using PREFABRICATION=0
2019-05-07 13:12:20 +10:00
Sam Saffron
4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Daniel Waterworth
ad44243a57 Removed unused let blocks (#7446)
The bodies of these blocks were never evaluated.
2019-04-29 15:08:56 +08:00
Joffrey JAFFEUX
7334362b77
FEATURE: generic theme component bbcode wrapper (#7400)
Usage:

```
[wrap=name foo=bar]
hello world
[/wrap]
```
2019-04-24 10:37:34 +02:00
Gerhard Schlager
a7bc1ecbae FEATURE: Add support for Unicode usernames and group names
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2019-04-23 13:00:27 +02:00
Régis Hanol
664e90bd17 FIX: ensure local images use local CDN when uploads are stored on S3
When the S3 store was enabled, we were only applying the S3 CDN.
So all images stored locally, like the emojis, were never put on the local CDN.

Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls'
in order to get final URLs.

I also removed the unnecessary PrettyText.add_s3_cdn method since this is already
handled in the CookedPostProcessor.
2019-02-20 19:24:38 +01:00
Penar Musaraj
2a007bafa2 FEAT: Allow image resize by width or height
`|150x` resizes to 150px wide + auto-height. `x150` resizes to 150px tall and auto-width.

Resize value can be from 1 to 999 (incl. for percentages).
2019-02-07 15:04:39 -05:00
Penar Musaraj
421d47da1e FIX: user and group mentions in subfolder installs 2019-01-29 21:54:29 -05:00
Sam Saffron
69aa8f18c2 FEATURE: allow for custom excerpt BBCODE
This allows fidelity in controlling excerpt (text that shows up when you pin a topic or link to it externally):

```
I am some text

[excerpt]
This is some **custom** markdown that should be the excerpt
[/excerpt]

More text
```

Previous solution relied on DIVs, unfortunately DIVs do not play well,
by design with mixing markdown unless you have a preceding newline eg:

```
<div class='hello'>

this will be treated properly as markdown

</div>
```

This extra newline is not desirable.

I am also considering adding

```
[div class=excerpt]
[/div]
```

This would offer lots of flexibility to themes and plugins that do not want the extra annoying newline.
2018-12-25 17:02:28 +02:00
David Taylor
9248ad1905 DEV: Enable Style/SingleLineMethods and Style/Semicolon in Rubocop (#6717) 2018-12-04 11:48:13 +08:00
Maja Komel
1073634271 FIX: show generic title when quoting off-topic secure category posts 2018-12-03 09:42:32 +11:00
David Taylor
afcf149c34 FIX: Fix mentions for mixed case group names 2018-11-26 15:34:56 +00:00
Guo Xiang Tan
57e2f4990d
PERF: Move processing of inline onebox out of V8 context. (#6658) 2018-11-26 09:21:38 +08:00
Guo Xiang Tan
28a6cf8228 FIX: Mention lookup should be case insensitive. 2018-11-22 16:32:56 +08:00
Guo Xiang Tan
3f636b2d19 FIX: Check whether group is mentionable by user when cooking post. 2018-11-22 16:16:33 +08:00
Guo Xiang Tan
672e95bcb4 FIX: Staged users should not be mentionable. 2018-11-22 15:00:46 +08:00
Guo Xiang Tan
c5a70eca6e
PERF: Move mention lookups out of the V8 context. (#6640)
We were looking up each mention one by one without any form of caching and that results
in a problem somewhat similar to an N+1. When we have to do alot of DB
lookups, it also increased the time spent in the V8 context which may
eventually lead to a timeout. The change here makes it such that mention lookups only does a single
DB query per post that happens outside of the V8 context.
2018-11-22 14:28:48 +08:00
Guo Xiang Tan
45f299dfdd PERF: Try to match users before groups.
User mentions are more common than group mentions so
this will allow us to avoid an extra query.
2018-11-16 16:41:20 +08:00
Penar Musaraj
26956bbe1a FIX: Safari issue with some emojis (#6456)
https://meta.discourse.org/t/emojis-selected-on-ios-displaying-additional-rectangles/86132/8
2018-10-08 10:32:25 +08:00
Guo Xiang Tan
f3aef2cc83 FIX: Incorrect/missing extension in short_url fails to map to upload.
`Hash#invert` causes us to lose keys if the hash contains similar
values.
2018-09-05 21:48:58 +08:00
Vinoth Kannan
24a14af15a FIX: Respect invalidate_oneboxes option for inline oneboxes 2018-09-03 22:33:43 +05:30
Joffrey JAFFEUX
10a3499d68
uses emoji versions for specs (#6276) 2018-08-16 13:45:30 +02:00
Sam
280c318c49 FEATURE: allow ruby tags in Markdown 2018-08-03 11:47:36 +10:00
Sam
0b3d51a8bc FEATURE: whitelist lang attribute 2018-08-02 16:53:08 +10:00
Andrew Schleifer
dba22bbde2 rollback changes
This reverts:
* 1baba84c438e "fix s3 subfolders harder"
* ea5e57938edf "fix test for absolute_base_url change"
2018-07-06 17:16:40 -05:00
Andrew Schleifer
f8b90226cb fix test for absolute_base_url change 2018-07-06 17:08:18 -05:00
Sam
b54ba4c952 FIX: mentions broken after adding an <abbr> tag
A previous shortcut used was not allowing for <abbr and other tags starting with a

If <abbr> appeared anywhere in the text all mentions would fail to link
2018-07-05 09:27:11 +10:00
Michael Brown
ae5d255f83 FIX: Reference example.com instead of somesite.com in examples
* somesite.com actually exists...
* example.com should be used in examples and is harmless to visit
2018-06-19 10:37:24 -04:00
Neil Lalonde
b8cf0788c6 FIX: broken mailto href's in emails 2018-06-08 13:11:58 -04:00
Andrew Schleifer
4be0e31459 fix s3_cdn_url when the s3 bucket contains a folder 2018-05-23 15:51:02 -05:00
Sam
a0cd54750c FIX: inline [code] not handled properly
The text

a
[code]test[/code]

Would eat up the `test` text cause translation from inline to block
for replace rule was not properly handled
2018-04-26 15:18:22 +10:00
Arpit Jalan
c61ce66411 fix the build 2018-04-18 13:38:45 +05:30
Robin Ward
d690ae0281 FIX: Broken specs too. Quotes are fragile! 2018-04-02 14:23:10 -04:00
Gerhard Schlager
d243b82fb3 FIX: Calculation of text length for <details> in excerpt was wrong 2018-03-12 16:55:23 +01:00
Sam
d39d2b9352 FEATURE: whitelist data for themes 2018-03-02 14:52:09 +11:00
Neil Lalonde
3313072957 Remove censored_pattern site setting, which is replaced by watched words 2018-02-26 16:29:27 -05:00
Régis Hanol
0559a4736a FIX: don't double request when downloading a file 2018-02-24 12:35:57 +01:00
Sam
94b2c70c0d PERF: remove oga gem
oga gem is automatically required by the aws gem
the oga gem retains about 1mb of memory, aws now uses nokogiri

This also removes the html normalize from the pretty text specs that was
a fair bit buggy as the polls test shows.
2018-02-15 14:36:40 +11:00
Sam
ee0d3f15c1 FEATURE: allow better fidelity for auto linkify, disable most tlds based linkify
New site settings:

enable_markdown_linkify: which is default on, auto links https:// and http:// and mail://

markdown_linkify_tlds: which allows control of what tlds get autolinked for cases such as www.site.com, default is com|net|gov
2018-02-01 13:22:38 +11:00
Maja Komel
330912e1e5 FIX: allowed href scheme link can start with a + (#5537)
* allowed href scheme link can start with a +

* allow tel:// links only to start with +

* add missing semicolon

* add test
2018-01-30 11:02:23 +11:00
Sam
3492a91056 FEATURE: allow site operators to disable emoji shortcuts 2018-01-24 12:21:44 +11:00
Sam
a9e2fc59c4 FIX: [constructor] bbcode would cause markdown crash 2017-12-27 16:11:30 +11:00
Robin Ward
21e1b05c7e FIX: Don't disable details when below truncate limit 2017-12-20 15:45:00 -05:00
Robin Ward
a0aca83c12 FIX: Broken spec 2017-12-19 17:55:41 -05:00
Robin Ward
b3fda0ea86 FIX: details tags broke excerpts 2017-12-19 17:28:55 -05:00
Gerhard Schlager
44ee388070 FEATURE: omit images from og and twitter description tags 2017-11-28 21:34:02 +01:00
Sam
8a66446849 FEATURE: add overflow-y auto to Markdown tables 2017-11-13 17:52:15 +11:00
ckeboss
93633865d9 Adds primary user group as a class to quote (#5285)
* Adds primary user group as a class to quote

This feature addition will add the class `group-PRIMARY_USER_GROUP` to
the quote `aside`. `PRIMARY_USER_GROUP` will be the primary user group
of the user being quoted. This is similar to the class that is added to
a `topic-post`.

* Remove trailing whitespace

* Fix avatar in test

* Address PR comments

* Fix trailing whitespace
2017-11-03 09:51:40 -04:00
Sam
70bb2aa426 FEATURE: allow specifying s3 config via globals
This refactors handling of s3 so it can be specified via GlobalSetting

This means that in a multisite environment you can configure s3 uploads
without actual sites knowing credentials in s3

It is a critical setting for situations where assets are mirrored to s3.
2017-10-06 16:20:01 +11:00
Guo Xiang Tan
3e53dbcade UX: Only include tag hashtag postfix when necessary.
https://meta.discourse.org/t/links-to-tags-not-working-in-final-post-unless-autocompleted/69884/6?u=tgxworld
2017-10-03 13:54:50 +08:00
Sam Saffron
e283e6aea0 FEATURE: allowed_iframes site setting for allowing iframes
This allows you to whitelist custom iframes if needed in posts
2017-09-01 10:15:44 -04:00
Sam
d4c28f6fcb remove stray puts from spec 2017-08-22 11:46:51 -04:00
Sam
bcf7dc38c2 FEATURE: server side support for upload:// markdown
This allows uploads to be specified using short sha1 hash instead of full
URL

Client side change is pending
2017-08-22 11:46:23 -04:00
Sam
5942ad83c0 Skip inline onebox for domain only 2017-08-02 17:27:58 -04:00
Sam
a4e1920604 FIX: inline oneboxer not applying to lists 2017-08-02 16:10:08 -04:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam
4f574e7c93 FEATURE: support inline bbcode blocks eg: [quote]test[/quote] 2017-07-27 17:55:17 -04:00
Neil Lalonde
68b3dd43ce fix intermittent failing tests, some watched word refactoring 2017-07-27 12:27:01 -04:00
Neil Lalonde
24cb950432 FEATURE: Watched Words: when posts contain words, do one of flag, require approval, censor, or block 2017-07-26 11:01:09 -04:00
David Taylor
27539e15e1 Allow whitespace around % scaler in markdown image syntax (#5000) 2017-07-26 10:54:20 -04:00
Sam
4da98cdcfe FIX: allow bbcode to be unquoted for default 2017-07-24 18:36:17 -04:00
Sam
c08a7aee8f clean up skipped tests
tighter connection handling in scheduler
2017-07-24 15:06:24 -04:00
Sam
40abcfc2f5 FIX: smart quote edge case with quotes 2017-07-24 12:21:49 -04:00
Sam
e1ce47a901 Pass the full CommonMark spec 2017-07-21 13:20:52 -04:00
Sam
fc90dd3592 Work in progress, CommonMark spec 2017-07-20 17:47:16 -04:00
Robin Ward
3882722195 FEATURE: Inline (Mini) Oneboxing
see:
https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400?source_topic_id=66066
2017-07-20 15:38:04 -04:00
Sam
44fb2a2833 DEV: support multiple capture groups for text post process 2017-07-20 15:33:44 -04:00
Sam
1096dcd602 correct bbcode parsing edge case 2017-07-20 13:02:40 -04:00
Sam
911ffbb98c Correct mention edge cases
@sam. was not correctly detected as a mention
2017-07-20 11:52:38 -04:00
Sam
c7b0764089 correct more edge cases with new engine 2017-07-17 17:44:48 -04:00
Sam
8921058c67 FIX: medium URL with @ incorrectly handled as mention
Also:

- remove unused code
- rename bbcode_ruler to bbcode.ruler
- add md.core.textPostProcess.ruler to apply at end of chain (excluding links)
2017-07-17 17:44:48 -04:00
Sam Saffron
d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Sam
79a084dd58 Revert "remove old markdown engine work-in-progress"
This reverts commit ee470b5317.
2017-07-12 18:10:51 -04:00
Sam
bcbb9f208d Revert "Integrate new engine, correct old specs"
This reverts commit f1b38ba4fb.
2017-07-12 18:10:07 -04:00
Sam
f1b38ba4fb Integrate new engine, correct old specs
corrects edge cases with

- full quotes
- [url] with nested tags
- engine overrides
- onebox applying to non http srcs
2017-07-12 17:44:40 -04:00
Sam Saffron
ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
Sam
5d139e461c FIX: multi option poll not working (new engine) 2017-07-11 14:43:34 -04:00
Sam
3f950a756a FEATURE: support image dimensions via Markdown image 2017-07-11 12:13:03 -04:00
Sam
ba9898c5a9 FIX: smarter newline handling for <img> tags on line alone
Run all of pretty text spec on new engine
2017-07-10 12:22:15 -04:00
Sam
6f09df0deb FIX: don't create tags autolinks inside links (new engine) 2017-07-07 13:04:25 -04:00
Sam
74f6894606 oops checked in a broken test 2017-07-07 11:08:04 -04:00
Sam
436b894f7a FIX: mention not working after a newline (new engine) 2017-07-07 11:06:50 -04:00
Guo Xiang Tan
13f3de4bf6 Nuke all SiteSetting.stubs from our codebase. 2017-07-07 15:09:14 +09:00
Sam
d2ba543ed3 correct case where %20 decoded as space in auto link
correct case where onebox is not applied when url is decoded
2017-07-03 16:32:53 -04:00