github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-12-11 23:33:56 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
eef93ac926
discourse/spec/lib/cooked_post_processor_spec.rb

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2190 lines
80 KiB
Ruby
Raw Normal View History

DEV: use #frozen_string_literal: true on all spec This change both speeds up specs (less strings to allocate) and helps catch cases where methods in Discourse are mutating inputs. Overall we will be migrating everything to use #frozen_string_literal: true it will take a while, but this is the first and safest move in this direction
2019-04-30 08:27:42 +08:00
# frozen_string_literal: true
pull hotlinked images
2013-11-06 02:04:47 +08:00
require "cooked_post_processor"
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
require "file_store/s3_store"
Initial release of Discourse
2013-02-06 03:16:51 +08:00
Add RSpec 4 compatibility (#17652) * Remove outdated option https://github.com/rspec/rspec-core/commit/04078317ba6577699d06cf4dccf014254dcde7a6 * Use the non-globally exposed RSpec syntax https://github.com/rspec/rspec-core/pull/2803 * Use the non-globally exposed RSpec syntax, cont https://github.com/rspec/rspec-core/pull/2803 * Comply to strict predicate matchers See: - https://github.com/rspec/rspec-expectations/pull/1195 - https://github.com/rspec/rspec-expectations/pull/1196 - https://github.com/rspec/rspec-expectations/pull/1277
2022-07-28 10:27:38 +08:00
RSpec.describe CookedPostProcessor do
DEV: Allow fab! without block (#24314) The most common thing that we do with fab! is: fab!(:thing) { Fabricate(:thing) } This commit adds a shorthand for this which is just simply: fab!(:thing) i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-10 06:47:59 +08:00
fab!(:upload)
fab!(:large_image_upload)
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
let(:upload_path) { Discourse.store.upload_path }
FIX: Link post to uploads in `PostCreator`. * This ensures that uploads are linked to their post on creation instead of a background job which may be delayed if Sidekiq is facing difficulties.
2018-09-06 09:58:01 +08:00
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#post_process" do
FIX: Link post to uploads in `PostCreator`. * This ensures that uploads are linked to their post on creation instead of a background job which may be delayed if Sidekiq is facing difficulties.
2018-09-06 09:58:01 +08:00
fab!(:post) { Fabricate(:post, raw: <<~RAW) }
<img src="#{upload.url}">
RAW
Initial release of Discourse
2013-02-06 03:16:51 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
let(:post_process) { sequence("post_process") }
Initial release of Discourse
2013-02-06 03:16:51 +08:00
update back-end specs
2013-07-11 04:59:07 +08:00
it "post process in sequence" do
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
cpp.expects(:post_process_oneboxes).in_sequence(post_process)
FIX: properly handle too large & broken images in posts
2017-11-16 22:45:07 +08:00
cpp.expects(:post_process_images).in_sequence(post_process)
pull hotlinked images
2013-11-06 02:04:47 +08:00
cpp.expects(:optimize_urls).in_sequence(post_process)
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
cpp.post_process
FIX: Link post to uploads in `PostCreator`. * This ensures that uploads are linked to their post on creation instead of a background job which may be delayed if Sidekiq is facing difficulties.
2018-09-06 09:58:01 +08:00
FEATURE: Create upload_references table (#16146) This table holds associations between uploads and other models. This can be used to prevent removing uploads that are still in use. * DEV: Create upload_references * DEV: Use UploadReference instead of PostUpload * DEV: Use UploadReference for SiteSetting * DEV: Use UploadReference for Badge * DEV: Use UploadReference for Category * DEV: Use UploadReference for CustomEmoji * DEV: Use UploadReference for Group * DEV: Use UploadReference for ThemeField * DEV: Use UploadReference for ThemeSetting * DEV: Use UploadReference for User * DEV: Use UploadReference for UserAvatar * DEV: Use UploadReference for UserExport * DEV: Use UploadReference for UserProfile * DEV: Add method to extract uploads from raw text * DEV: Use UploadReference for Draft * DEV: Use UploadReference for ReviewableQueuedPost * DEV: Use UploadReference for UserProfile's bio_raw * DEV: Do not copy user uploads to upload references * DEV: Copy post uploads again after deploy * DEV: Use created_at and updated_at from uploads table * FIX: Check if upload site setting is empty * DEV: Copy user uploads to upload references * DEV: Make upload extraction less strict
2022-06-09 07:24:30 +08:00
expect(UploadReference.exists?(target: post, upload: upload)).to eq(true)
Initial release of Discourse
2013-02-06 03:16:51 +08:00
end
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
describe "when post contains oneboxes and inline oneboxes" do
let(:url_hostname) { "meta.discourse.org" }
let(:url) { "https://#{url_hostname}/t/mini-inline-onebox-support-rfc/66400" }
UX: Strip class when link is not oneboxed due to site setting limits.
2018-11-29 14:33:01 +08:00
let(:not_oneboxed_url) { "https://#{url_hostname}/t/random-url" }
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
let(:title) { "some title" }
let(:post) { Fabricate(:post, raw: <<~RAW) }
#{url}
This is a #{url} with path
UX: Strip class when link is not oneboxed due to site setting limits.
2018-11-29 14:33:01 +08:00
#{not_oneboxed_url}
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
This is a https://#{url_hostname}/t/another-random-url test
This is a #{url} with path
#{url}
RAW
before do
SiteSetting.enable_inline_onebox_on_all_domains = true
FIX: Make sure max_oneboxes_per_post is enforced (#16215) PostAnalyzer and CookedPostProcessor both replace URLs with oneboxes. PostAnalyzer did not use the max_oneboxes_per_post site and setting and CookedPostProcessor replaced at most max_oneboxes_per_post URLs ignoring the oneboxes that were replaced already by PostAnalyzer.
2022-03-23 23:36:08 +08:00
Oneboxer.stubs(:cached_onebox).with(url).returns <<~HTML
<aside class="onebox allowlistedgeneric" data-onebox-src="https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400">
<header class="source">
<a href="https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400" target="_blank" rel="noopener">meta.discourse.org</a>
</header>
<article class="onebox-body">
<h3><a href="https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400" target="_blank" rel="noopener">some title</a></h3>
<p>some description</p>
</article>
<div class="onebox-metadata"></div>
<div style="clear: both"></div>
</aside>
HTML
Oneboxer.stubs(:cached_onebox).with(not_oneboxed_url).returns(nil)
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
%i[head get].each do |method|
stub_request(method, url).to_return(status: 200, body: <<~RAW)
<html>
<head>
<title>#{title}</title>
<meta property='og:title' content="#{title}">
<meta property='og:description' content="some description">
</head>
</html>
RAW
end
end
after do
FIX: properly invalidate inline oneboxes when rebaking When rebaking a post we were invalidating _regular_ oneboxes but not inline oneboxes. DEV: also renamed 'InlineOneboxer.purge' to 'InlineOneboxer.invalidate' to keep the API consistent with 'Oneboxer.invalidate'
2020-06-24 17:54:54 +08:00
InlineOneboxer.invalidate(url)
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
Oneboxer.invalidate(url)
end
it "should respect SiteSetting.max_oneboxes_per_post" do
SiteSetting.max_oneboxes_per_post = 2
SiteSetting.add_rel_nofollow_to_user_content = false
cpp.post_process
expect(cpp.html).to have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
with: {
href: url,
class: "inline-onebox",
},
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
text: title,
count: 2,
)
FIX: Make sure max_oneboxes_per_post is enforced (#16215) PostAnalyzer and CookedPostProcessor both replace URLs with oneboxes. PostAnalyzer did not use the max_oneboxes_per_post site and setting and CookedPostProcessor replaced at most max_oneboxes_per_post URLs ignoring the oneboxes that were replaced already by PostAnalyzer.
2022-03-23 23:36:08 +08:00
expect(cpp.html).to have_tag("aside.onebox a", text: title, count: 1)
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
expect(cpp.html).to have_tag("aside.onebox a", text: url_hostname, count: 1)
expect(cpp.html).to have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
without: {
class: "inline-onebox-loading",
},
UX: Strip class when link is not oneboxed due to site setting limits.
2018-11-29 14:33:01 +08:00
text: not_oneboxed_url,
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
count: 1,
)
UX: Strip class when link is not oneboxed due to site setting limits.
2018-11-29 14:33:01 +08:00
expect(cpp.html).to have_tag(
"a",
without: {
class: "onebox",
},
text: not_oneboxed_url,
count: 1,
)
FEATURE: Reimplement `SiteSetting.max_oneboxes_per_post`. (#6668) Previously, the site setting was only effective on the client side of things. Once the site setting was been reached, all oneboxes are not rendered. This commit changes it such that the site setting is respected both on the client and server side. The first N oneboxes are rendered and once the limit has been reached, subsequent oneboxes will not be rendered.
2018-11-27 16:00:31 +08:00
end
end
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
describe "when post contains inline oneboxes" do
before { SiteSetting.enable_inline_onebox_on_all_domains = true }
describe "internal links" do
DEV: Allow fab! without block (#24314) The most common thing that we do with fab! is: fab!(:thing) { Fabricate(:thing) } This commit adds a shorthand for this which is just simply: fab!(:thing) i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-10 06:47:59 +08:00
fab!(:topic)
DEV: Prefabrication (test optimization) (#7414) * Introduced fab!, a helper that creates database state for a group It's almost identical to let_it_be, except: 1. It creates a new object for each test by default, 2. You can disable it using PREFABRICATION=0
2019-05-07 11:12:20 +08:00
fab!(:post) { Fabricate(:post, raw: "Hello #{topic.url}") }
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
let(:url) { topic.url }
it "includes the topic title" do
cpp.post_process
expect(cpp.html).to have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
with: {
href: UrlHelper.cook_url(url),
},
without: {
class: "inline-onebox-loading",
},
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
text: topic.title,
count: 1,
)
topic.update!(title: "Updated to something else")
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
cpp.post_process
expect(cpp.html).to have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
with: {
href: UrlHelper.cook_url(url),
},
without: {
class: "inline-onebox-loading",
},
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
text: topic.title,
count: 1,
)
end
end
describe "external links" do
let(:url_with_path) { "https://meta.discourse.org/t/mini-inline-onebox-support-rfc/66400" }
let(:url_with_query_param) { "https://meta.discourse.org?a" }
let(:url_no_path) { "https://meta.discourse.org/" }
let(:urls) { [url_with_path, url_with_query_param, url_no_path] }
SECURITY: escape title HTML for inline onebox
2019-01-10 09:02:05 +08:00
let(:title) { "<b>some title</b>" }
let(:escaped_title) { CGI.escapeHTML(title) }
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
let(:post) { Fabricate(:post, raw: <<~RAW) }
This is a #{url_with_path} topic
This should not be inline #{url_no_path} oneboxed
- #{url_with_path}
- #{url_with_query_param}
RAW
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
let(:staff_post) { Fabricate(:post, user: Fabricate(:admin), raw: <<~RAW) }
This is a #{url_with_path} topic
RAW
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
before do
urls.each do |url|
stub_request(:get, url).to_return(
status: 200,
SECURITY: escape title HTML for inline onebox
2019-01-10 09:02:05 +08:00
body: "<html><head><title>#{escaped_title}</title></head></html>",
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
)
end
end
FIX: properly invalidate inline oneboxes when rebaking When rebaking a post we were invalidating _regular_ oneboxes but not inline oneboxes. DEV: also renamed 'InlineOneboxer.purge' to 'InlineOneboxer.invalidate' to keep the API consistent with 'Oneboxer.invalidate'
2020-06-24 17:54:54 +08:00
after { urls.each { |url| InlineOneboxer.invalidate(url) } }
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
it "should convert the right links to inline oneboxes" do
cpp.post_process
html = cpp.html
expect(html).to_not have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
with: {
href: url_no_path,
},
without: {
class: "inline-onebox-loading",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
},
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
text: title,
)
expect(html).to have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
with: {
href: url_with_path,
},
without: {
class: "inline-onebox-loading",
},
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
text: title,
count: 2,
)
expect(html).to have_tag(
"a",
DEV: Remove INLINE_ONEBOX_* constants There were two constants here, `INLINE_ONEBOX_LOADING_CSS_CLASS` and `INLINE_ONEBOX_CSS_CLASS` that were both longer than the strings they were DRYing up: `inline-onebox-loading` and `inline-onebox` I normally appreciate constants, but in this case it meant that we had a lot of JS imports resulting in many more lines of code (and CPU cycles spent figuring them out.) It also meant we had an `.erb` file and had to invoke Ruby to create the JS file, which meant the app was harder to port to Ember CLI. I removed the constants. It's less DRY but faster and simpler, and arguably the loss of DRYness is not significant as you can still search for the `inline-onebox-loading` and `inline-onebox` strings easily if you are refactoring.
2020-05-08 04:08:48 +08:00
with: {
href: url_with_query_param,
},
without: {
class: "inline-onebox-loading",
},
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
text: title,
count: 1,
)
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
expect(html).to have_tag("a[rel='noopener nofollow ugc']")
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
end
it "removes nofollow if user is staff/tl3" do
cpp = CookedPostProcessor.new(staff_post, invalidate_oneboxes: true)
cpp.post_process
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
expect(cpp.html).to_not have_tag("a[rel='noopener nofollow ugc']")
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
end
end
end
FEATURE: Show a blurry preview when lazy loading images This generates a 10x10 PNG thumbnail for each lightboxed image. If Image Lazy Loading is enabled (IntersectionObserver API) then we'll load the low res version when offscreen. As the image scrolls in we'll swap it for the high res version. We use a WeakMap to track the old image attributes. It's much less memory than storing them as `data-*` attributes and swapping them back and forth all the time.
2018-12-15 06:44:38 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "when processing images" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before { SiteSetting.responsive_post_image_sizes = "" }
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with responsive images" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before { SiteSetting.responsive_post_image_sizes = "1|1.5|3" }
it "includes responsive images on demand" do
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
upload.update!(width: 2000, height: 1500, filesize: 10_000, dominant_color: "FFFFFF")
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
post = Fabricate(:post, raw: "hello <img src='#{upload.url}'>")
# fake some optimized images
OptimizedImage.create!(
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
url: "/#{upload_path}/666x500.jpg",
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
width: 666,
height: 500,
upload_id: upload.id,
sha1: SecureRandom.hex,
extension: ".jpg",
filesize: 500,
version: OptimizedImage::VERSION,
)
FEATURE: add support for responsive images in posts When creating lightboxes we will attempt to create 1.5x and 2x thumbnails for retina screens, this can be controlled with a new hidden site setting called responsice_post_image_sizes, if you wish to create 3x images run SiteSetting.responsive_post_image_sizes = "1|1.5|2|3" The default should be good for most of the setups as it balances filesize with quality. 3x thumbs can get big.
2018-10-03 11:44:53 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
# fake 3x optimized image, we lose 2 pixels here over original due to rounding on downsize
OptimizedImage.create!(
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
url: "/#{upload_path}/1998x1500.jpg",
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
width: 1998,
height: 1500,
upload_id: upload.id,
sha1: SecureRandom.hex,
extension: ".jpg",
filesize: 800,
)
FEATURE: add support for responsive images in posts When creating lightboxes we will attempt to create 1.5x and 2x thumbnails for retina screens, this can be controlled with a new hidden site setting called responsice_post_image_sizes, if you wish to create 3x images run SiteSetting.responsive_post_image_sizes = "1|1.5|2|3" The default should be good for most of the setups as it balances filesize with quality. 3x thumbs can get big.
2018-10-03 11:44:53 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp = CookedPostProcessor.new(post)
FIX: Use CDN for optimized loading images. (#7006) We missed a few spots in the cooked post processor where images where not loaded using CDN, causing uneeded load and requests against the server
2019-02-20 10:55:08 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.add_to_size_cache(upload.url, 2000, 1500)
cpp.post_process
FIX: Use CDN for optimized loading images. (#7006) We missed a few spots in the cooked post processor where images where not loaded using CDN, causing uneeded load and requests against the server
2019-02-20 10:55:08 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
html = cpp.html
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
expect(html).to include(%Q|data-dominant-color="FFFFFF"|)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
# 1.5x is skipped cause we have a missing thumb
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
expect(html).to include(
"srcset=\"//test.localhost/#{upload_path}/666x500.jpg, //test.localhost/#{upload_path}/1998x1500.jpg 3x\"",
)
expect(html).to include("src=\"//test.localhost/#{upload_path}/666x500.jpg\"")
FIX: Use CDN for optimized loading images. (#7006) We missed a few spots in the cooked post processor where images where not loaded using CDN, causing uneeded load and requests against the server
2019-02-20 10:55:08 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
# works with CDN
set_cdn_url("http://cdn.localhost")
FIX: don't use srcset on cropped thumbnails
2018-10-25 22:08:10 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp = CookedPostProcessor.new(post)
cpp.add_to_size_cache(upload.url, 2000, 1500)
cpp.post_process
FIX: don't use srcset on cropped thumbnails
2018-10-25 22:08:10 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
html = cpp.html
FEATURE: add support for responsive images in posts When creating lightboxes we will attempt to create 1.5x and 2x thumbnails for retina screens, this can be controlled with a new hidden site setting called responsice_post_image_sizes, if you wish to create 3x images run SiteSetting.responsive_post_image_sizes = "1|1.5|2|3" The default should be good for most of the setups as it balances filesize with quality. 3x thumbs can get big.
2018-10-03 11:44:53 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
expect(html).to include(%Q|data-dominant-color="FFFFFF"|)
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
expect(html).to include(
"srcset=\"//cdn.localhost/#{upload_path}/666x500.jpg, //cdn.localhost/#{upload_path}/1998x1500.jpg 3x\"",
)
expect(html).to include("src=\"//cdn.localhost/#{upload_path}/666x500.jpg\"")
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
end
FEATURE: add support for responsive images in posts When creating lightboxes we will attempt to create 1.5x and 2x thumbnails for retina screens, this can be controlled with a new hidden site setting called responsice_post_image_sizes, if you wish to create 3x images run SiteSetting.responsive_post_image_sizes = "1|1.5|2|3" The default should be good for most of the setups as it balances filesize with quality. 3x thumbs can get big.
2018-10-03 11:44:53 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "doesn't include response images for cropped images" do
DEV: Reduce object fabrication in `CookedPostProcessor` spec.
2019-05-30 08:49:37 +08:00
upload.update!(width: 200, height: 4000, filesize: 12_345)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
post = Fabricate(:post, raw: "hello <img src='#{upload.url}'>")
# fake some optimized images
OptimizedImage.create!(
url: "http://a.b.c/200x500.jpg",
width: 200,
height: 500,
upload_id: upload.id,
sha1: SecureRandom.hex,
extension: ".jpg",
filesize: 500,
)
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp = CookedPostProcessor.new(post)
cpp.add_to_size_cache(upload.url, 200, 4000)
cpp.post_process
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
expect(cpp.html).to_not include('srcset="')
end
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
end
FIX stretched thumbnails
2013-11-26 01:36:13 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
shared_examples "leave dimensions alone" do
it "doesn't use them" do
expect(cpp.html).to match(%r{src="http://foo.bar/image.png" width="" height=""})
FIX: picture would be hidden when [details] block was closed in the composer
2015-03-17 01:57:15 +08:00
expect(cpp.html).to match(%r{src="http://domain.com/picture.jpg" width="50" height="42"})
expect(cpp).to be_dirty
end
end
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
context "with image_sizes" do
fab!(:post) { Fabricate(:post_with_image_urls) }
let(:cpp) { CookedPostProcessor.new(post, image_sizes: image_sizes) }
FIX: picture would be hidden when [details] block was closed in the composer
2015-03-17 01:57:15 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before do
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
end
FIX: picture would be hidden when [details] block was closed in the composer
2015-03-17 01:57:15 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "when valid" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:image_sizes) do
{ "http://foo.bar/image.png" => { "width" => 111, "height" => 222 } }
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "uses them" do
expect(cpp.html).to match(%r{src="http://foo.bar/image.png" width="111" height="222"})
expect(cpp.html).to match(
%r{src="http://domain.com/picture.jpg" width="50" height="42"},
)
expect(cpp).to be_dirty
end
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with invalid width" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:image_sizes) { { "http://foo.bar/image.png" => { "width" => 0, "height" => 222 } } }
include_examples "leave dimensions alone"
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with invalid height" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:image_sizes) { { "http://foo.bar/image.png" => { "width" => 111, "height" => 0 } } }
include_examples "leave dimensions alone"
end
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with invalid width & height" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:image_sizes) { { "http://foo.bar/image.png" => { "width" => 0, "height" => 0 } } }
include_examples "leave dimensions alone"
end
keep uploads reverse index up to date
2013-06-18 04:46:48 +08:00
end
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
context "with unsized images" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 123, height: 456) }
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}">
HTML
keep uploads reverse index up to date
2013-06-18 04:46:48 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:cpp) { CookedPostProcessor.new(post) }
Move a bunch of callbacks into PostCreator
2013-03-19 01:55:34 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "adds the width and height to images that don't have them" do
cpp.post_process
expect(cpp.html).to match(/width="123" height="456"/)
expect(cpp).to be_dirty
end
UX: Don't add light box for SVG images.
2018-06-18 17:10:23 +08:00
end
BUGFIX: broken spec
2014-05-26 19:17:20 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
context "with large images" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 1750, height: 2000) }
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}">
HTML
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FIX: Do not add lightbox to onebox images (#6479)
2018-10-11 05:57:21 +08:00
before do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
SiteSetting.max_image_height = 2000
SiteSetting.create_thumbnails = true
FIX: Do not add lightbox to onebox images (#6479)
2018-10-11 05:57:21 +08:00
end
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "generates overlay information" do
cpp.post_process
FIX: Do not add lightbox to onebox images (#6479)
2018-10-11 05:57:21 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost#{upload.url}" data-download-href="//test.localhost/#{upload_path}/#{upload.sha1}" title="logo.png"><img src="//test.localhost/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_690x788.png" width="690" height="788"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">logo.png</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
HTML
FIX: Do not add lightbox to onebox images (#6479)
2018-10-11 05:57:21 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
expect(cpp).to be_dirty
UX: Don't add light box for SVG images.
2018-06-18 17:10:23 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "when image is inside onebox" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:url) { "https://image.com/my-avatar" }
let(:post) { Fabricate(:post, raw: url) }
UX: Don't add light box for SVG images.
2018-06-18 17:10:23 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before do
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
Oneboxer
.stubs(:onebox)
.with(url, anything)
.returns(
"<img class='onebox' src='/#{upload_path}/original/1X/1234567890123456.jpg' />",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
end
it "should not add lightbox" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
FastImage.expects(:size).returns([1750, 2000])
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><img class="onebox" src="//test.localhost/#{upload_path}/original/1X/1234567890123456.jpg" width="690" height="788"></p>
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
HTML
end
UX: Don't add light box for SVG images.
2018-06-18 17:10:23 +08:00
end
FIX: URL with params for svg images should not be light boxed.
2018-06-20 10:47:14 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "when image is an svg" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
fab!(:post) do
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
Fabricate(
:post,
raw: "<img src=\"/#{Discourse.store.upload_path}/original/1X/1234567890123456.svg\">",
)
FIX: URL with params for svg images should not be light boxed.
2018-06-20 10:47:14 +08:00
end
it "should not add lightbox" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
FastImage.expects(:size).returns([1750, 2000])
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><img src="//test.localhost/#{upload_path}/original/1X/1234567890123456.svg" width="690" height="788"></p>
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
HTML
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "when image src is an URL" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:post) do
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
Fabricate(
:post,
raw:
"<img src=\"http://test.discourse/#{upload_path}/original/1X/1234567890123456.svg?somepamas\">",
)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
end
it "should not add lightbox" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
FastImage.expects(:size).returns([1750, 2000])
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
FIX: URL with params for svg images should not be light boxed.
2018-06-20 10:47:14 +08:00
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
expect(cpp.html).to match_html(
"<p><img src=\"http://test.discourse/#{upload_path}/original/1X/1234567890123456.svg?somepamas\" width=\"690\"\ height=\"788\"></p>",
)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
end
FIX: URL with params for svg images should not be light boxed.
2018-06-20 10:47:14 +08:00
end
end
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with s3_uploads" do
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
let(:upload) { Fabricate(:secure_upload_s3) }
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 19:32:25 +08:00
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
before do
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 19:32:25 +08:00
setup_s3
SiteSetting.s3_cdn_url = "https://s3.cdn.com"
SiteSetting.authorized_extensions = "png|jpg|gif|mov|ogg|"
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
stored_path = Discourse.store.get_path_for_upload(upload)
upload.update_column(:url, "#{SiteSetting.Upload.absolute_base_url}/#{stored_path}")
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 19:32:25 +08:00
stub_upload(upload)
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
SiteSetting.login_required = true
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
SiteSetting.secure_uploads = true
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
end
DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490) Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0. - [Release notes](https://github.com/aws/aws-sdk-ruby/releases) - [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 22:13:17 +08:00
let(:optimized_size) { "600x500" }
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
let(:post) do
DEV: Bump aws-sdk-sns from 1.13.0 to 1.21.0 (#8490) Bumps [aws-sdk-sns](https://github.com/aws/aws-sdk-ruby) from 1.13.0 to 1.21.0. - [Release notes](https://github.com/aws/aws-sdk-ruby/releases) - [Changelog](https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-sns/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-ruby/compare/1.13.0...1.21.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 22:13:17 +08:00
Fabricate(:post, raw: "![large.png|#{optimized_size}](#{upload.short_url})")
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
end
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
let(:cooked_html) { <<~HTML }
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost/secure-uploads/original/1X/#{upload.sha1}.png" data-download-href="//test.localhost/uploads/short-url/#{upload.base62_sha1}.unknown?dl=1" title="large.png"><img src="" alt="large.png" data-base62-sha1="#{upload.base62_sha1}" width="600" height="500"><div class="meta">
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">large.png</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg>
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
</div></a></div></p>
HTML
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
context "when the upload is attached to the correct post" do
before do
Discourse
.store
.class
.any_instance
.expects(:has_been_uploaded?)
.at_least_once
.returns(true)
FIX: Secure upload post processing race condition (#23968) * FIX: Secure upload post processing race condition This commit fixes a couple of issues. A little background -- when uploads are created in the composer for posts, regardless of whether the upload will eventually be marked secure or not, if secure_uploads is enabled we always mark the upload secure at first. This is so the upload is by default protected, regardless of post type (regular or PM) or category. This was causing issues in some rare occasions though because of the order of operations of our post creation and processing pipeline. When creating a post, we enqueue a sidekiq job to post-process the post which does various things including converting images to lightboxes. We were also enqueuing a job to update the secure status for all uploads in that post. Sometimes the secure status job would run before the post process job, marking uploads as _not secure_ in the background and changing their ACL before the post processor ran, which meant the users would see a broken image in their posts. This commit fixes that issue by always running the upload security changes inline _within_ the cooked_post_processor job. The other issue was that the lightbox wrapper link for images in the post would end up with a URL like this: ``` href="/secure-uploads/original/2X/4/4e1f00a40b6c952198bbdacae383ba77932fc542.jpeg" ``` Since we weren't actually using the `upload.url` to pass to `UrlHelper.cook_url` here, we weren't converting this href to the CDN URL if the post was not in a secure context (the UrlHelper does not know how to convert a secure-uploads URL to a CDN one). Now we always end up with the correct lightbox href. This was less of an issue than the other one, since the secure-uploads URL works even when the upload has become non-secure, but it was a good inconsistency to fix anyway.
2023-10-19 07:48:01 +08:00
upload.update!(secure: true, access_control_post: post)
post.link_post_uploads
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
end
DEV: Skip erratic spec for now Spec fails intermittently due to CDN state.
2020-04-25 11:20:04 +08:00
# TODO fix this spec, it is sometimes getting CDN links when it runs concurrently
xit "handles secure images with the correct lightbox link href" do
FIX: Secure upload post processing race condition (#23968) * FIX: Secure upload post processing race condition This commit fixes a couple of issues. A little background -- when uploads are created in the composer for posts, regardless of whether the upload will eventually be marked secure or not, if secure_uploads is enabled we always mark the upload secure at first. This is so the upload is by default protected, regardless of post type (regular or PM) or category. This was causing issues in some rare occasions though because of the order of operations of our post creation and processing pipeline. When creating a post, we enqueue a sidekiq job to post-process the post which does various things including converting images to lightboxes. We were also enqueuing a job to update the secure status for all uploads in that post. Sometimes the secure status job would run before the post process job, marking uploads as _not secure_ in the background and changing their ACL before the post processor ran, which meant the users would see a broken image in their posts. This commit fixes that issue by always running the upload security changes inline _within_ the cooked_post_processor job. The other issue was that the lightbox wrapper link for images in the post would end up with a URL like this: ``` href="/secure-uploads/original/2X/4/4e1f00a40b6c952198bbdacae383ba77932fc542.jpeg" ``` Since we weren't actually using the `upload.url` to pass to `UrlHelper.cook_url` here, we weren't converting this href to the CDN URL if the post was not in a secure context (the UrlHelper does not know how to convert a secure-uploads URL to a CDN one). Now we always end up with the correct lightbox href. This was less of an issue than the other one, since the secure-uploads URL works even when the upload has become non-secure, but it was a good inconsistency to fix anyway.
2023-10-19 07:48:01 +08:00
FastImage.expects(:size).returns([1750, 2000])
OptimizedImage.expects(:resize).returns(true)
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
cpp.post_process
expect(cpp.html).to match_html cooked_html
end
FIX: Secure upload post processing race condition (#23968) * FIX: Secure upload post processing race condition This commit fixes a couple of issues. A little background -- when uploads are created in the composer for posts, regardless of whether the upload will eventually be marked secure or not, if secure_uploads is enabled we always mark the upload secure at first. This is so the upload is by default protected, regardless of post type (regular or PM) or category. This was causing issues in some rare occasions though because of the order of operations of our post creation and processing pipeline. When creating a post, we enqueue a sidekiq job to post-process the post which does various things including converting images to lightboxes. We were also enqueuing a job to update the secure status for all uploads in that post. Sometimes the secure status job would run before the post process job, marking uploads as _not secure_ in the background and changing their ACL before the post processor ran, which meant the users would see a broken image in their posts. This commit fixes that issue by always running the upload security changes inline _within_ the cooked_post_processor job. The other issue was that the lightbox wrapper link for images in the post would end up with a URL like this: ``` href="/secure-uploads/original/2X/4/4e1f00a40b6c952198bbdacae383ba77932fc542.jpeg" ``` Since we weren't actually using the `upload.url` to pass to `UrlHelper.cook_url` here, we weren't converting this href to the CDN URL if the post was not in a secure context (the UrlHelper does not know how to convert a secure-uploads URL to a CDN one). Now we always end up with the correct lightbox href. This was less of an issue than the other one, since the secure-uploads URL works even when the upload has become non-secure, but it was a good inconsistency to fix anyway.
2023-10-19 07:48:01 +08:00
context "when the upload was not secure" do
before { upload.update!(secure: false) }
it "changes the secure status" do
cpp.post_process
expect(upload.reload.secure).to eq(true)
end
end
context "when the upload should no longer be considered secure" do
before { SiteSetting.login_required = false }
it "changes the secure status" do
cpp.post_process
expect(upload.reload.secure).to eq(false)
end
it "does not use a secure-uploads URL for the lightbox href" do
SiteSetting.create_thumbnails = false
SiteSetting.max_image_width = 10
SiteSetting.max_image_height = 10
cpp.post_process
expect(cpp.html).not_to have_tag(
"a",
with: {
class: "lightbox",
href: "//test.localhost/secure-uploads/original/1X/#{upload.sha1}.png",
},
)
end
end
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
end
context "when the upload is attached to a different post" do
before do
FastImage.size(upload.url)
upload.update(secure: true, access_control_post: Fabricate(:post))
end
it "does not create thumbnails or optimize images" do
CookedPostProcessor.any_instance.expects(:optimize_image!).never
Upload.any_instance.expects(:create_thumbnail!).never
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size
FIX: Mitigate issue where legacy pre-secure hotlinked media would not be redownloaded (#8802) Basically, say you had already downloaded a certain image from a certain URL using pull_hotlinked_images and the onebox. The upload would be stored by its sha as an upload record. Whenever you linked to the same URL again in a post (e.g. in our case an og:image on review.discourse) we would would reuse the original upload record because of the sha1. However when you turned on secure media this could cause problems as the first post that uses that upload after secure media is enabled will set the access control post for the upload to the new post. Then if the post is deleted every single onebox/link to that same image URL will fail forever with 403 as the secure-media-uploads URL fails if the access control post has been deleted. To fix this when cooking posts and pulling hotlinked images, we only allow using an original upload by URL if its access control post matches the current post, and if the original_sha1 is filled in, meaning it was uploaded AFTER secure media was enabled. otherwise we just redownload the media again to be safe, as the URL will always be new then.
2020-01-29 08:11:38 +08:00
cpp.post_process
expect(cpp.html).not_to match_html cooked_html
end
end
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
end
UX: Don't add light box for SVG images.
2018-06-18 17:10:23 +08:00
end
DEV: Allow for taller images in posts and oneboxes (#15395) The previous default aspect ratio for cropping tall images was a little too strict and was cutting off images. This new setting should allow for a larger range of image sizes before cropping them.
2021-12-23 07:01:34 +08:00
context "with tall images > default aspect ratio" do
fab!(:upload) { Fabricate(:image_upload, width: 500, height: 2200) }
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}">
HTML
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
DEV: Allow for taller images in posts and oneboxes (#15395) The previous default aspect ratio for cropping tall images was a little too strict and was cutting off images. This new setting should allow for a larger range of image sizes before cropping them.
2021-12-23 07:01:34 +08:00
before { SiteSetting.create_thumbnails = true }
it "resizes the image instead of crop" do
cpp.post_process
expect(cpp.html).to match(/width="113" height="500">/)
expect(cpp).to be_dirty
end
end
context "with taller images < default aspect ratio" do
fab!(:upload) { Fabricate(:image_upload, width: 500, height: 2300) }
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}">
HTML
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before { SiteSetting.create_thumbnails = true }
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "crops the image" do
cpp.post_process
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
DEV: Allow for taller images in posts and oneboxes (#15395) The previous default aspect ratio for cropping tall images was a little too strict and was cutting off images. This new setting should allow for a larger range of image sizes before cropping them.
2021-12-23 07:01:34 +08:00
expect(cpp.html).to match(/width="500" height="500">/)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
expect(cpp).to be_dirty
end
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
end
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
context "with iPhone X screenshots" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 1125, height: 2436) }
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}">
HTML
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before { SiteSetting.create_thumbnails = true }
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "crops the image" do
cpp.post_process
FIX: don't crop iPhone X screenshots
2017-11-15 03:37:27 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost#{upload.url}" data-download-href="//test.localhost/#{upload_path}/#{upload.sha1}" title="logo.png"><img src="//test.localhost/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_230x500.png" width="230" height="500"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">logo.png</span><span class="informations">1125×2436 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
HTML
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
expect(cpp).to be_dirty
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
end
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
context "with large images when using subfolders" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 1750, height: 2000) }
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="/subfolder#{upload.url}">
HTML
Adds large image upload on subfolder install spec
2016-03-10 11:48:50 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
Adds large image upload on subfolder install spec
2016-03-10 11:48:50 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before do
DEV: improve usability of subfolder specs Previously people were not consistent about mocking which left internals in a fragile state when running subfolder specs. This introduces a simple helper `set_subfolder` which you can use to set the subfolder for the spec. It takes care of proper configuration of subfolder and teardown. ``` # usage set_subfolder "/my_amazing_subfolder" ``` You should no longer stub base_uri or global_settings
2019-11-15 13:48:24 +08:00
set_subfolder "/subfolder"
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
stub_request(
:get,
"http://#{Discourse.current_hostname}/subfolder#{upload.url}",
).to_return(status: 200, body: File.new(Discourse.store.path_for(upload)))
DEV: improve usability of subfolder specs Previously people were not consistent about mocking which left internals in a fragile state when running subfolder specs. This introduces a simple helper `set_subfolder` which you can use to set the subfolder for the spec. It takes care of proper configuration of subfolder and teardown. ``` # usage set_subfolder "/my_amazing_subfolder" ``` You should no longer stub base_uri or global_settings
2019-11-15 13:48:24 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
SiteSetting.max_image_height = 2000
SiteSetting.create_thumbnails = true
end
Adds large image upload on subfolder install spec
2016-03-10 11:48:50 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "generates overlay information" do
cpp.post_process
Adds large image upload on subfolder install spec
2016-03-10 11:48:50 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost/subfolder#{upload.url}" data-download-href="//test.localhost/subfolder/#{upload_path}/#{upload.sha1}" title="logo.png"><img src="//test.localhost/subfolder/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_690x788.png" width="690" height="788"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">logo.png</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
HTML
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
expect(cpp).to be_dirty
end
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "should escape the filename" do
upload.update!(original_filename: "><img src=x onerror=alert('haha')>.png")
cpp.post_process
Initial release of Discourse
2013-02-06 03:16:51 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: prevent lightbox images from double escaping titles (#23458) This change fixes an issue where lightbox images are showing escaped text in the link title and lightbox image description area.
2023-09-13 14:33:08 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost/subfolder#{upload.url}" data-download-href="//test.localhost/subfolder/#{upload_path}/#{upload.sha1}" title="><img src=x onerror=alert('haha')>.png"><img src="//test.localhost/subfolder/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_690x788.png" width="690" height="788"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">&gt;&lt;img src=x onerror=alert('haha')&gt;.png</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
HTML
end
SECURITY: Escape HTML in filename.
2016-08-11 11:27:12 +08:00
end
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
context "with title and alt" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 1750, height: 2000) }
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}" title="WAT" alt="RED">
HTML
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
before do
SiteSetting.max_image_height = 2000
SiteSetting.create_thumbnails = true
end
it "generates overlay information using image title and ignores alt" do
cpp.post_process
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost#{upload.url}" data-download-href="//test.localhost/#{upload_path}/#{upload.sha1}" title="WAT"><img src="//test.localhost/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_690x788.png" title="WAT" alt="RED" width="690" height="788"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">WAT</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
HTML
expect(cpp).to be_dirty
end
end
context "with title only" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 1750, height: 2000) }
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}" title="WAT">
HTML
FEATURE: use img's title attribute in overlay information when provided
2014-11-04 05:03:06 +08:00
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FEATURE: use img's title attribute in overlay information when provided
2014-11-04 05:03:06 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
before do
SiteSetting.max_image_height = 2000
SiteSetting.create_thumbnails = true
end
FIX: images aren't lightboxed anymore (partially reverts 646c6eb7cd781b47663d104de4f37cdd99b465bf)
2017-10-19 05:54:36 +08:00
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
it "generates overlay information using image title" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
FEATURE: use img's title attribute in overlay information when provided
2014-11-04 05:03:06 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost#{upload.url}" data-download-href="//test.localhost/#{upload_path}/#{upload.sha1}" title="WAT"><img src="//test.localhost/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_690x788.png" title="WAT" width="690" height="788"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">WAT</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
HTML
expect(cpp).to be_dirty
end
end
context "with alt only" do
FIX: Perform crop using user-specified image sizes (#9224) * FIX: Perform crop using user-specified image sizes It used to resize the images to max width and height first and then perform the crop operation. This is wrong because it ignored the user specified image sizes from the Markdown. * DEV: Use real images in test
2020-03-26 22:40:00 +08:00
fab!(:upload) { Fabricate(:image_upload, width: 1750, height: 2000) }
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
fab!(:post) { Fabricate(:post, raw: <<~HTML) }
<img src="#{upload.url}" alt="RED">
HTML
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FEATURE: fallback to image alt before filename if there's no title in lightboxes (#8286) * use image alt as a fallback when there's no title * update spec we used to check that the overlay information is added when the image has a titie. This adds 2 more scenarios. One where an image has both a title and an alt, in which case the title should be used and alt ignored. The other is when there's only an alt, it should then be used to generate the overlay
2019-11-04 07:15:14 +08:00
before do
SiteSetting.max_image_height = 2000
SiteSetting.create_thumbnails = true
end
it "generates overlay information using image alt" do
cpp.post_process
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: Remove xlink hrefs (#15059)
2021-11-25 12:22:43 +08:00
<p><div class="lightbox-wrapper"><a class="lightbox" href="//test.localhost#{upload.url}" data-download-href="//test.localhost/#{upload_path}/#{upload.sha1}" title="RED"><img src="//test.localhost/#{upload_path}/optimized/1X/#{upload.sha1}_#{OptimizedImage::VERSION}_690x788.png" alt="RED" width="690" height="788"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">RED</span><span class="informations">1750×2000 1.21 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
HTML
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
expect(cpp).to be_dirty
end
FEATURE: use img's title attribute in overlay information when provided
2014-11-04 05:03:06 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with topic image" do
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
fab!(:post) { Fabricate(:post_with_uploaded_image) }
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:cpp) { CookedPostProcessor.new(post) }
it "adds a topic image if there's one in the first post" do
FastImage.stubs(:size)
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(post.topic.image_upload_id).to eq(nil)
FEATURE: use img's title attribute in overlay information when provided
2014-11-04 05:03:06 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
post.topic.reload
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(post.topic.image_upload_id).to be_present
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
end
FIX: Remove post/topic image_url on post edits - resets image_url when image is removed from first post on edit - excludes onebox icons from being featured as topic/post images
2020-02-07 00:19:40 +08:00
it "removes image if post is edited and no longer has an image" do
FastImage.stubs(:size)
cpp.post_process
post.topic.reload
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(post.topic.image_upload_id).to be_present
expect(post.image_upload_id).to be_present
FIX: Remove post/topic image_url on post edits - resets image_url when image is removed from first post on edit - excludes onebox icons from being featured as topic/post images
2020-02-07 00:19:40 +08:00
post.update!(raw: "This post no longer has an image.")
CookedPostProcessor.new(post).post_process
post.topic.reload
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(post.topic.image_upload_id).not_to be_present
expect(post.image_upload_id).not_to be_present
FIX: Remove post/topic image_url on post edits - resets image_url when image is removed from first post on edit - excludes onebox icons from being featured as topic/post images
2020-02-07 00:19:40 +08:00
end
FIX: Don't remove the topic image if posts don't have them
2020-02-14 03:00:30 +08:00
it "won't remove the original image if another post doesn't have an image" do
topic = post.topic
cpp.post_process
topic.reload
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(topic.image_upload_id).to be_present
expect(post.image_upload_id).to be_present
FIX: Don't remove the topic image if posts don't have them
2020-02-14 03:00:30 +08:00
post = Fabricate(:post, topic: topic, raw: "this post doesn't have an image")
CookedPostProcessor.new(post).post_process
topic.reload
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(post.topic.image_upload_id).to be_present
expect(post.image_upload_id).to be_blank
end
it "generates thumbnails correctly" do
PERF: stop downloading images from post processor and lean on uploads Previously we would unconditionally fetch all images via HTTP to grab original sizing from cooked post processor in 2 different spots. This was wasteful as we already calculate and cache this info in upload records. This also simplifies some specs and reduces use of mocks.
2022-11-25 09:40:31 +08:00
# image size in cooked is 1500*2000
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
topic = post.topic
cpp.post_process
topic.reload
expect(topic.image_upload_id).to be_present
expect(post.image_upload_id).to be_present
post = Fabricate(:post, topic: topic, raw: "this post doesn't have an image")
CookedPostProcessor.new(post).post_process
topic.reload
expect(post.topic.image_upload_id).to be_present
expect(post.image_upload_id).to be_blank
FIX: Don't remove the topic image if posts don't have them
2020-02-14 03:00:30 +08:00
end
Don't append the base url to absolute image urls without protocol.
2013-05-30 23:17:07 +08:00
end
FEATURE: include post image in OpenGraph image tag
2016-10-31 17:41:33 +08:00
FEATURE: Allow post/topic thumbnails to be prioritized via markdown (#12044) Previously we would always take the first image in a post to use as the thumbnail. On media-heavy sites, users may want to manually select a specific image as the topic thumbnail. This commit allows this to be done via a `|thumbnail` attribute in markdown. For example, in this case, bbb would be chosen as the thumbnail: ``` ![alttext|100x100](upload://aaa) ![alttext|100x100|thumbnail](upload://bbb) ```
2021-02-11 23:44:41 +08:00
it "prioritizes data-thumbnail images" do
upload1 = Fabricate(:image_upload, width: 1750, height: 2000)
upload2 = Fabricate(:image_upload, width: 1750, height: 2000)
post = Fabricate(:post, raw: <<~MD)
![alttext|1750x2000](#{upload1.url})
![alttext|1750x2000|thumbnail](#{upload2.url})
MD
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
CookedPostProcessor.new(post, disable_dominant_color: true).post_process
FEATURE: Allow post/topic thumbnails to be prioritized via markdown (#12044) Previously we would always take the first image in a post to use as the thumbnail. On media-heavy sites, users may want to manually select a specific image as the topic thumbnail. This commit allows this to be done via a `|thumbnail` attribute in markdown. For example, in this case, bbb would be chosen as the thumbnail: ``` ![alttext|100x100](upload://aaa) ![alttext|100x100|thumbnail](upload://bbb) ```
2021-02-11 23:44:41 +08:00
expect(post.reload.image_upload_id).to eq(upload2.id)
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with post image" do
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
let(:reply) { Fabricate(:post_with_uploaded_image, post_number: 2) }
let(:cpp) { CookedPostProcessor.new(reply) }
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
it "adds a post image if there's one in the post" do
FastImage.stubs(:size)
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(reply.image_upload_id).to eq(nil)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
reply.reload
FEATURE: Include optimized thumbnails for topics (#9215) This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this: - Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact - A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake - Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued - Topic lists and topics now include a `thumbnails` key, which includes all the available images: ``` "thumbnails": [ { "max_width": null, "max_height": null, "url": "//example.com/original-image.png", "width": 1380, "height": 1840 }, { "max_width": 1024, "max_height": 1024, "url": "//example.com/optimized-image.png", "width": 768, "height": 1024 } ] ``` - Themes can request additional thumbnail sizes by using a modifier in their `about.json` file: ``` "modifiers": { "topic_thumbnail_sizes": [ [200, 200], [800, 800] ], ... ``` Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated - Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 16:07:50 +08:00
expect(reply.image_upload_id).to be_present
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
end
FEATURE: include post image in OpenGraph image tag
2016-10-31 17:41:33 +08:00
end
Don't append the base url to absolute image urls without protocol.
2013-05-30 23:17:07 +08:00
end
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
end
do not lightbox oneboxed images
2013-06-24 08:10:21 +08:00
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#extract_images" do
FIX: don't use emojis as topic image
2015-08-05 18:57:31 +08:00
let(:post) { build(:post_with_plenty_of_images) }
pull hotlinked images
2013-11-06 02:04:47 +08:00
let(:cpp) { CookedPostProcessor.new(post) }
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
FIX: don't use emojis as topic image
2015-08-05 18:57:31 +08:00
it "does not extract emojis or images inside oneboxes or quotes" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.extract_images.length).to eq(0)
do not lightbox oneboxed images
2013-06-24 08:10:21 +08:00
end
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#get_size_from_attributes" do
FIX: rescale image during cooked_post_processor when only img height or width is specified
2015-08-30 05:56:25 +08:00
let(:post) { build(:post) }
let(:cpp) { CookedPostProcessor.new(post) }
it "returns the size when width and height are specified" do
img = { "src" => "http://foo.bar/image3.png", "width" => 50, "height" => 70 }
expect(cpp.get_size_from_attributes(img)).to eq([50, 70])
end
it "returns the size when width and height are floats" do
img = { "src" => "http://foo.bar/image3.png", "width" => 50.2, "height" => 70.1 }
expect(cpp.get_size_from_attributes(img)).to eq([50, 70])
end
it "resizes when only width is specified" do
img = { "src" => "http://foo.bar/image3.png", "width" => 100 }
FastImage.expects(:size).returns([200, 400])
expect(cpp.get_size_from_attributes(img)).to eq([100, 200])
end
it "resizes when only height is specified" do
img = { "src" => "http://foo.bar/image3.png", "height" => 100 }
FastImage.expects(:size).returns([100, 300])
expect(cpp.get_size_from_attributes(img)).to eq([33, 100])
end
FIX: Don't raise an error when the img attribute cannot be found
2017-05-09 03:35:31 +08:00
it "doesn't raise an error with a weird url" do
img = { "src" => nil, "height" => 100 }
expect(cpp.get_size_from_attributes(img)).to be_nil
end
FIX: rescale image during cooked_post_processor when only img height or width is specified
2015-08-30 05:56:25 +08:00
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#get_size_from_image_sizes" do
pull hotlinked images
2013-11-06 02:04:47 +08:00
let(:post) { build(:post) }
let(:cpp) { CookedPostProcessor.new(post) }
FIX: Check that the node has a src attr when getting size (#19696)
2023-01-04 02:27:05 +08:00
let(:image_sizes) do
{ "http://my.discourse.org/image.png" => { "width" => 111, "height" => 222 } }
end
pull hotlinked images
2013-11-06 02:04:47 +08:00
it "returns the size" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.get_size_from_image_sizes("/image.png", image_sizes)).to eq([111, 222])
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
end
FIX: Check that the node has a src attr when getting size (#19696)
2023-01-04 02:27:05 +08:00
it "returns nil whe img node has no src" do
expect(cpp.get_size_from_image_sizes(nil, image_sizes)).to eq(nil)
end
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#get_size" do
pull hotlinked images
2013-11-06 02:04:47 +08:00
let(:post) { build(:post) }
let(:cpp) { CookedPostProcessor.new(post) }
it "ensures urls are absolute" do
cpp.expects(:is_valid_image_url?).with("http://test.localhost/relative/url/image.png")
cpp.get_size("/relative/url/image.png")
end
it "ensures urls have a default scheme" do
cpp.expects(:is_valid_image_url?).with("http://schemaless.url/image.jpg")
cpp.get_size("//schemaless.url/image.jpg")
end
it "caches the results" do
FastImage.expects(:size).returns([200, 400])
cpp.get_size("http://foo.bar/image3.png")
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.get_size("http://foo.bar/image3.png")).to eq([200, 400])
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
end
soften the lightbox overlay
2013-06-26 08:44:20 +08:00
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#is_valid_image_url?" do
pull hotlinked images
2013-11-06 02:04:47 +08:00
let(:post) { build(:post) }
let(:cpp) { CookedPostProcessor.new(post) }
it "validates HTTP(s) urls" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.is_valid_image_url?("http://domain.com")).to eq(true)
expect(cpp.is_valid_image_url?("https://domain.com")).to eq(true)
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
it "doesn't validate other urls" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.is_valid_image_url?("ftp://domain.com")).to eq(false)
expect(cpp.is_valid_image_url?("ftps://domain.com")).to eq(false)
expect(cpp.is_valid_image_url?("/tmp/image.png")).to eq(false)
expect(cpp.is_valid_image_url?("//domain.com")).to eq(false)
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
it "doesn't throw an exception with a bad URI" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.is_valid_image_url?("http://do<main.com")).to eq(nil)
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#get_filename" do
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
let(:post) { build(:post) }
let(:cpp) { CookedPostProcessor.new(post) }
it "returns the filename of the src when there is no upload" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.get_filename(nil, "http://domain.com/image.png")).to eq("image.png")
soften the lightbox overlay
2013-06-26 08:44:20 +08:00
end
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
it "returns the original filename of the upload when there is an upload" do
upload = build(:upload, original_filename: "upload.jpg")
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.get_filename(upload, "http://domain.com/image.png")).to eq("upload.jpg")
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
it "returns a generic name for pasted images" do
upload = build(:upload, original_filename: "blob.png")
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.get_filename(upload, "http://domain.com/image.png")).to eq(
I18n.t("upload.pasted_image_filename"),
)
basic lightbox support
2013-02-19 14:57:14 +08:00
end
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#convert_to_link" do
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
fab!(:thumbnail) { Fabricate(:optimized_image, upload: upload, width: 512, height: 384) }
it "adds lightbox and optimizes images" do
PERF: stop downloading images from post processor and lean on uploads Previously we would unconditionally fetch all images via HTTP to grab original sizing from cooked post processor in 2 different spots. This was wasteful as we already calculate and cache this info in upload records. This also simplifies some specs and reduces use of mocks.
2022-11-25 09:40:31 +08:00
post = Fabricate(:post, raw: "![image|1024x768, 50%](#{large_image_upload.short_url})")
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
cpp = CookedPostProcessor.new(post, disable_dominant_color: true)
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
cpp.post_process
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
doc = Nokogiri::HTML5.fragment(cpp.html)
PERF: stop downloading images from post processor and lean on uploads Previously we would unconditionally fetch all images via HTTP to grab original sizing from cooked post processor in 2 different spots. This was wasteful as we already calculate and cache this info in upload records. This also simplifies some specs and reduces use of mocks.
2022-11-25 09:40:31 +08:00
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
expect(doc.css(".lightbox-wrapper").size).to eq(1)
expect(doc.css("img").first["srcset"]).to_not eq(nil)
end
FIX: Disable lightboxing of animated images (#13099)
2021-05-21 03:19:44 +08:00
it "processes animated images correctly" do
# skips optimization
# skips lightboxing
# adds "animated" class to element
FIX: Do not optimize animated images in cooked posts (#11214) CookedPostProcessor replaces all large images with their optimized versions, but for GIF images the optimized version is limited to first frame only. This caused animations it cooked posts to require a click to show up the lightbox and start playing.
2020-11-13 03:47:30 +08:00
upload.update!(animated: true)
post = Fabricate(:post, raw: "![image|1024x768, 50%](#{upload.short_url})")
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
cpp = CookedPostProcessor.new(post, disable_dominant_color: true)
FIX: Do not optimize animated images in cooked posts (#11214) CookedPostProcessor replaces all large images with their optimized versions, but for GIF images the optimized version is limited to first frame only. This caused animations it cooked posts to require a click to show up the lightbox and start playing.
2020-11-13 03:47:30 +08:00
cpp.post_process
doc = Nokogiri::HTML5.fragment(cpp.html)
FIX: Disable lightboxing of animated images (#13099)
2021-05-21 03:19:44 +08:00
expect(doc.css(".lightbox-wrapper").size).to eq(0)
FIX: Do not optimize animated images in cooked posts (#11214) CookedPostProcessor replaces all large images with their optimized versions, but for GIF images the optimized version is limited to first frame only. This caused animations it cooked posts to require a click to show up the lightbox and start playing.
2020-11-13 03:47:30 +08:00
expect(doc.css("img").first["src"]).to include(upload.url)
expect(doc.css("img").first["srcset"]).to eq(nil)
FEATURE: Allow pausing animated images in posts (#12795) Co-authored-by: Jarek Radosz <jradosz@gmail.com>
2021-04-22 23:28:35 +08:00
expect(doc.css("img.animated").size).to eq(1)
FIX: Do not optimize animated images in cooked posts (#11214) CookedPostProcessor replaces all large images with their optimized versions, but for GIF images the optimized version is limited to first frame only. This caused animations it cooked posts to require a click to show up the lightbox and start playing.
2020-11-13 03:47:30 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with giphy/tenor images" do
FEATURE: Enable pausing images from Giphy and Tenor (#13185)
2021-05-28 03:00:38 +08:00
before do
CookedPostProcessor
.any_instance
.stubs(:get_size)
.with("https://media2.giphy.com/media/7Oifk90VrCdNe/giphy.webp")
.returns([311, 280])
CookedPostProcessor
.any_instance
.stubs(:get_size)
.with("https://media1.tenor.com/images/20c7ddd5e84c7427954f430439c5209d/tenor.gif")
.returns([833, 104])
end
it "marks giphy images as animated" do
post =
Fabricate(
:post,
raw: "![tennis-gif|311x280](https://media2.giphy.com/media/7Oifk90VrCdNe/giphy.webp)",
)
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
cpp = CookedPostProcessor.new(post, disable_dominant_color: true)
FEATURE: Enable pausing images from Giphy and Tenor (#13185)
2021-05-28 03:00:38 +08:00
cpp.post_process
doc = Nokogiri::HTML5.fragment(cpp.html)
expect(doc.css("img.animated").size).to eq(1)
end
it "marks giphy images as animated" do
post =
Fabricate(
:post,
raw:
"![cat](https://media1.tenor.com/images/20c7ddd5e84c7427954f430439c5209d/tenor.gif)",
)
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
cpp = CookedPostProcessor.new(post, disable_dominant_color: true)
FEATURE: Enable pausing images from Giphy and Tenor (#13185)
2021-05-28 03:00:38 +08:00
cpp.post_process
doc = Nokogiri::HTML5.fragment(cpp.html)
expect(doc.css("img.animated").size).to eq(1)
end
end
FIX: Quoted images should be lightboxed (#21332) An older change about optimising images caused the selector that adds lightboxing not to apply on quoted images. This fixes that. The selector is now not applicable as optimisation occurs in a separate place. This change allows quoted images to be opened in a lightbox.
2023-05-02 17:02:19 +08:00
it "optimizes and wraps images in quotes with lightbox wrapper" do
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
post = Fabricate(:post, raw: <<~MD)
[quote]
PERF: stop downloading images from post processor and lean on uploads Previously we would unconditionally fetch all images via HTTP to grab original sizing from cooked post processor in 2 different spots. This was wasteful as we already calculate and cache this info in upload records. This also simplifies some specs and reduces use of mocks.
2022-11-25 09:40:31 +08:00
![image|1024x768, 50%](#{large_image_upload.short_url})
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
[/quote]
MD
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
cpp = CookedPostProcessor.new(post, disable_dominant_color: true)
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
cpp.post_process
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
doc = Nokogiri::HTML5.fragment(cpp.html)
FIX: Quoted images should be lightboxed (#21332) An older change about optimising images caused the selector that adds lightboxing not to apply on quoted images. This fixes that. The selector is now not applicable as optimisation occurs in a separate place. This change allows quoted images to be opened in a lightbox.
2023-05-02 17:02:19 +08:00
expect(doc.css(".lightbox-wrapper").size).to eq(1)
FIX: Optimize images in Onebox (#8471) This commit ensures that images in Onebox are being optimized, but not converted to lightbox too.
2019-12-09 21:39:25 +08:00
expect(doc.css("img").first["srcset"]).to_not eq(nil)
end
it "optimizes images in Onebox" do
Oneboxer
.expects(:onebox)
.with("https://discourse.org", anything)
PERF: stop downloading images from post processor and lean on uploads Previously we would unconditionally fetch all images via HTTP to grab original sizing from cooked post processor in 2 different spots. This was wasteful as we already calculate and cache this info in upload records. This also simplifies some specs and reduces use of mocks.
2022-11-25 09:40:31 +08:00
.returns(
"<aside class='onebox'><img src='#{large_image_upload.url}' width='512' height='384'></aside>",
)
FIX: Optimize images in Onebox (#8471) This commit ensures that images in Onebox are being optimized, but not converted to lightbox too.
2019-12-09 21:39:25 +08:00
post = Fabricate(:post, raw: "https://discourse.org")
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
cpp = CookedPostProcessor.new(post, disable_dominant_color: true)
FIX: Optimize images in Onebox (#8471) This commit ensures that images in Onebox are being optimized, but not converted to lightbox too.
2019-12-09 21:39:25 +08:00
cpp.post_process
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
doc = Nokogiri::HTML5.fragment(cpp.html)
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
expect(doc.css(".lightbox-wrapper").size).to eq(0)
DEV: Skip srcset for onebox thumbnails (#22621) * DEV: Skip srcset for onebox thumbnails In an effort to preserve bandwidth especially for mobile devices this change will prevent upscaled srcset attributes from being added to onebox thumbnail images. Besides checking the html for onebox classes, our database structure for uploads does not distinguish between regular images and onebox thumbnail images, but all upload images in discourse do have a thumbnail. By default this thumbnail is what is used for the non-upscaled image for onebox images, so we should only use that thumbnail. Because the rendered onebox image size is likely smaller than the upload thumbnail size there really shouldn't be a need to upscale.
2023-07-20 02:21:34 +08:00
expect(doc.css("img").first["srcset"]).to eq(nil)
expect(doc.css("img").first["src"]).to include("optimized")
expect(doc.css("img").first["src"]).to include("512x384")
FIX: Optimize quoted images (#8427) Only images that were part of a lightbox used to be optimized. This patch ensures that quoted images are also optimized.
2019-11-29 21:18:42 +08:00
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#post_process_oneboxes" do
pull hotlinked images
2013-11-06 02:04:47 +08:00
let(:post) { build(:post_with_youtube, id: 123) }
let(:cpp) { CookedPostProcessor.new(post, invalidate_oneboxes: true) }
removed auto_link_images_wider_than setting
2013-07-07 01:10:53 +08:00
pull hotlinked images
2013-11-06 02:04:47 +08:00
before do
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
Oneboxer
.expects(:onebox)
SECURITY: correct local onebox category checks Also removes ugly "source_topic_id" from cooked posts Patch was authored by @zogstrip Signed-off-by: Sam <sam.saffron@gmail.com>
2018-02-14 07:39:44 +08:00
.with(
"http://www.youtube.com/watch?v=9bZkp7q19f0",
invalidate_oneboxes: true,
user_id: nil,
category_id: post.topic.category_id,
)
pull hotlinked images
2013-11-06 02:04:47 +08:00
.returns("<div>GANGNAM STYLE</div>")
PERF: Move processing of inline onebox out of V8 context. (#6658)
2018-11-26 09:21:38 +08:00
pull hotlinked images
2013-11-06 02:04:47 +08:00
cpp.post_process_oneboxes
Initial release of Discourse
2013-02-06 03:16:51 +08:00
end
SECURITY: correct local onebox category checks Also removes ugly "source_topic_id" from cooked posts Patch was authored by @zogstrip Signed-off-by: Sam <sam.saffron@gmail.com>
2018-02-14 07:39:44 +08:00
UX: generic onebox treats all square images as avatars and renders them smaller
2017-11-13 08:19:06 +08:00
it "inserts the onebox without wrapping p" do
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp).to be_dirty
UX: generic onebox treats all square images as avatars and renders them smaller
2017-11-13 08:19:06 +08:00
expect(cpp.html).to match_html "<div>GANGNAM STYLE</div>"
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
describe "replacing downloaded onebox image" do
let(:url) { "https://image.com/my-avatar" }
let(:image_url) { "https://image.com/avatar.png" }
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
it "successfully replaces the image" do
Oneboxer
.stubs(:onebox)
.with(url, anything)
.returns("<img class='onebox' src='#{image_url}' />")
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
post = Fabricate(:post, raw: url)
UX: Use dominant color while loading onebox images (#21091) When we "pull hotlinked images" on onebox images, they are added to the uploads table and their dominant color is calculated. This commit adds the data to the HTML so that it can be used by the client in the same way as non-onebox images. It also adds specific handling to the new `discourse-lazy-videos` plugin.
2023-04-13 19:04:46 +08:00
upload.update!(url: "https://test.s3.amazonaws.com/something.png", dominant_color: "00ffff")
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
DEV: Move hotlinked image information into a dedicated table (#16585) This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change. For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 20:53:32 +08:00
PostHotlinkedMedia.create!(
url: "//image.com/avatar.png",
post: post,
status: "downloaded",
upload: upload,
)
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size(width: 100, height: 200)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
cpp.post_process_oneboxes
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
expect(cpp.doc.to_s).to eq(
UX: Use dominant color while loading onebox images (#21091) When we "pull hotlinked images" on onebox images, they are added to the uploads table and their dominant color is calculated. This commit adds the data to the HTML so that it can be used by the client in the same way as non-onebox images. It also adds specific handling to the new `discourse-lazy-videos` plugin.
2023-04-13 19:04:46 +08:00
"<p><img class=\"onebox\" src=\"#{upload.url}\" data-dominant-color=\"00ffff\" width=\"100\" height=\"200\"></p>",
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
)
FIX: Use find_by_id method to prevent record not found exception
2018-12-15 05:49:45 +08:00
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
upload.destroy!
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size(width: 100, height: 200)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
cpp.post_process_oneboxes
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
expect(cpp.doc.to_s).to eq(
"<p><img class=\"onebox\" src=\"#{image_url}\" width=\"100\" height=\"200\"></p>",
)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
Oneboxer.unstub(:onebox)
end
FIX: Use find_by_id method to prevent record not found exception
2018-12-15 05:49:45 +08:00
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
context "when the post is with_secure_uploads and the upload is secure and secure uploads is enabled" do
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
before do
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 19:32:25 +08:00
setup_s3
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
upload.update(secure: true)
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 19:32:25 +08:00
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
SiteSetting.login_required = true
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
SiteSetting.secure_uploads = true
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
end
it "does not use the direct URL, uses the cooked URL instead (because of the private ACL preventing w/h fetch)" do
Oneboxer
.stubs(:onebox)
.with(url, anything)
.returns("<img class='onebox' src='#{image_url}' />")
post = Fabricate(:post, raw: url)
UX: Use dominant color while loading onebox images (#21091) When we "pull hotlinked images" on onebox images, they are added to the uploads table and their dominant color is calculated. This commit adds the data to the HTML so that it can be used by the client in the same way as non-onebox images. It also adds specific handling to the new `discourse-lazy-videos` plugin.
2023-04-13 19:04:46 +08:00
upload.update!(
url: "https://test.s3.amazonaws.com/something.png",
dominant_color: "00ffff",
)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
DEV: Move hotlinked image information into a dedicated table (#16585) This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change. For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 20:53:32 +08:00
PostHotlinkedMedia.create!(
url: "//image.com/avatar.png",
post: post,
status: "downloaded",
upload: upload,
)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
cooked_url = "https://localhost/secure-uploads/test.png"
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
UrlHelper.expects(:cook_url).with(upload.url, secure: true).returns(cooked_url)
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size(width: 100, height: 200)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
cpp.post_process_oneboxes
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
expect(cpp.doc.to_s).to eq(
UX: Use dominant color while loading onebox images (#21091) When we "pull hotlinked images" on onebox images, they are added to the uploads table and their dominant color is calculated. This commit adds the data to the HTML so that it can be used by the client in the same way as non-onebox images. It also adds specific handling to the new `discourse-lazy-videos` plugin.
2023-04-13 19:04:46 +08:00
"<p><img class=\"onebox\" src=\"#{cooked_url}\" data-dominant-color=\"00ffff\" width=\"100\" height=\"200\"></p>",
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
)
FIX: Use presigned URL to avoid 403 when pulling hotlinked images for secure media (#8764) When we were pulling hotlinked images for oneboxes in the CookedPostProcessor, we were using the direct S3 URL, which returned a 403 error and thus did not set widths and heights of the images. We now cook the URL first based on whether the upload is secure before handing off to FastImage.
2020-01-23 07:31:46 +08:00
end
end
FIX: Onebox images are not downloaded locally without css class
2018-08-01 05:21:02 +08:00
end
FIX: Display large/broken image placeholders for image oneboxes
2018-10-07 20:10:15 +08:00
it "replaces large image placeholder" do
FEATURE: Humanize file size error messages (#14398) The file size error messages for max_image_size_kb and max_attachment_size_kb are shown to the user in the KB format, regardless of how large the limit is. Since we are going to support uploading much larger files soon, this KB-based limit soon becomes unfriendly to the end user. For example, if the max attachment size is set to 512000 KB, this is what the user sees: > Sorry, the file you are trying to upload is too big (maximum size is 512000KB) This makes the user do math. In almost all file explorers that a regular user would be familiar width, the file size is shown in a format based on the maximum increment (e.g. KB, MB, GB). This commit changes the behaviour to output a humanized file size instead of the raw KB. For the above example, it would now say: > Sorry, the file you are trying to upload is too big (maximum size is 512 MB) This humanization also handles decimals, e.g. 1536KB = 1.5 MB
2021-09-22 05:59:45 +08:00
SiteSetting.max_image_size_kb = 4096
FIX: Show large image placeholder for image onebox (#21237) Large or broken images are removed from oneboxes, but sometimes images were removed when they were oneboxed too. The reason is that images can be oneboxed by the AllowlistedGenericOnebox or ImageOnebox and only AllowlistedGenericOnebox was handled correctly.
2023-04-27 01:05:22 +08:00
url = "https://image.com/avatar.png"
FIX: Display large/broken image placeholders for image oneboxes
2018-10-07 20:10:15 +08:00
FIX: Show large image placeholder for image onebox (#21237) Large or broken images are removed from oneboxes, but sometimes images were removed when they were oneboxed too. The reason is that images can be oneboxed by the AllowlistedGenericOnebox or ImageOnebox and only AllowlistedGenericOnebox was handled correctly.
2023-04-27 01:05:22 +08:00
Oneboxer.stubs(:onebox).with(url, anything).returns <<~HTML
<a href="#{url}" target="_blank" rel="noopener" class="onebox">
<img class='onebox' src='#{url}' />
</a>
HTML
FIX: Display large/broken image placeholders for image oneboxes
2018-10-07 20:10:15 +08:00
post = Fabricate(:post, raw: url)
DEV: Move hotlinked image information into a dedicated table (#16585) This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change. For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 20:53:32 +08:00
PostHotlinkedMedia.create!(url: "//image.com/avatar.png", post: post, status: "too_large")
FIX: Display large/broken image placeholders for image oneboxes
2018-10-07 20:10:15 +08:00
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
DEV: Make `CookedPostProcessor#post_process_images` method private.
2019-05-27 11:28:37 +08:00
cpp.post_process
FIX: Display large/broken image placeholders for image oneboxes
2018-10-07 20:10:15 +08:00
expect(cpp.doc.to_s).to match(/<div class="large-image-placeholder">/)
FEATURE: Humanize file size error messages (#14398) The file size error messages for max_image_size_kb and max_attachment_size_kb are shown to the user in the KB format, regardless of how large the limit is. Since we are going to support uploading much larger files soon, this KB-based limit soon becomes unfriendly to the end user. For example, if the max attachment size is set to 512000 KB, this is what the user sees: > Sorry, the file you are trying to upload is too big (maximum size is 512000KB) This makes the user do math. In almost all file explorers that a regular user would be familiar width, the file size is shown in a format based on the maximum increment (e.g. KB, MB, GB). This commit changes the behaviour to output a humanized file size instead of the raw KB. For the above example, it would now say: > Sorry, the file you are trying to upload is too big (maximum size is 512 MB) This humanization also handles decimals, e.g. 1536KB = 1.5 MB
2021-09-22 05:59:45 +08:00
expect(cpp.doc.to_s).to include(
I18n.t("upload.placeholders.too_large_humanized", max_size: "4 MB"),
)
FIX: Display large/broken image placeholders for image oneboxes
2018-10-07 20:10:15 +08:00
end
DEV: Move hotlinked image information into a dedicated table (#16585) This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change. For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 20:53:32 +08:00
FIX: Remove dead and large images from oneboxes (#17868) Dead and large images are replaced with a placeholder, either a broken chain icon or a short text. This commit no longer applies this transformation for images inside Oneboxes, but removes them instead.
2022-08-12 00:09:48 +08:00
it "removes large images from onebox" do
url = "https://example.com/article"
Oneboxer.stubs(:onebox).with(url, anything).returns <<~HTML
<aside class="onebox allowlistedgeneric" data-onebox-src="https://example.com/article">
<header class="source">
<img src="https://example.com/favicon.ico" class="site-icon">
<a href="https://example.com/article" target="_blank" rel="nofollow ugc noopener">Example Site</a>
</header>
<article class="onebox-body">
<img src="https://example.com/article.jpeg" class="thumbnail">
<h3><a href="https://example.com/article" target="_blank" rel="nofollow ugc noopener">Lorem Ispum</a></h3>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer tellus neque, malesuada ac neque ac, tempus tincidunt lectus.</p>
</article>
</aside>
HTML
post = Fabricate(:post, raw: url)
PostHotlinkedMedia.create!(url: "//example.com/favicon.ico", post: post, status: "too_large")
PostHotlinkedMedia.create!(url: "//example.com/article.jpeg", post: post, status: "too_large")
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
cpp.post_process
expect(cpp.doc).to match_html <<~HTML
<aside class="onebox allowlistedgeneric" data-onebox-src="https://example.com/article">
<header class="source">
<a href="https://example.com/article" target="_blank" rel="noopener nofollow ugc">Example Site</a>
</header>
<article class="onebox-body">
<h3><a href="https://example.com/article" target="_blank" rel="noopener nofollow ugc">Lorem Ispum</a></h3>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer tellus neque, malesuada ac neque ac, tempus tincidunt lectus.</p>
</article>
</aside>
HTML
end
DEV: Move hotlinked image information into a dedicated table (#16585) This will make future changes to the 'pull hotlinked images' system easier. This commit should not introduce any functional change. For now, the old post_custom_field data is kept in the database. This will be dropped in a future commit.
2022-05-03 20:53:32 +08:00
it "replaces broken image placeholder" do
url = "https://image.com/my-avatar"
image_url = "https://image.com/avatar.png"
Oneboxer
.stubs(:onebox)
.with(url, anything)
.returns("<img class='onebox' src='#{image_url}' />")
post = Fabricate(:post, raw: url)
PostHotlinkedMedia.create!(
url: "//image.com/avatar.png",
post: post,
status: "download_failed",
)
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
cpp.post_process
expect(cpp.doc.to_s).to have_tag("span.broken-image")
expect(cpp.doc.to_s).to include(I18n.t("post.image_placeholder.broken"))
end
FIX: Remove dead and large images from oneboxes (#17868) Dead and large images are replaced with a placeholder, either a broken chain icon or a short text. This commit no longer applies this transformation for images inside Oneboxes, but removes them instead.
2022-08-12 00:09:48 +08:00
it "removes broken images from onebox" do
url = "https://example.com/article"
Oneboxer.stubs(:onebox).with(url, anything).returns <<~HTML
<aside class="onebox allowlistedgeneric" data-onebox-src="https://example.com/article">
<header class="source">
<img src="https://example.com/favicon.ico" class="site-icon">
<a href="https://example.com/article" target="_blank" rel="nofollow ugc noopener">Example Site</a>
</header>
<article class="onebox-body">
<img src="https://example.com/article.jpeg" class="thumbnail">
<h3><a href="https://example.com/article" target="_blank" rel="nofollow ugc noopener">Lorem Ispum</a></h3>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer tellus neque, malesuada ac neque ac, tempus tincidunt lectus.</p>
</article>
</aside>
HTML
post = Fabricate(:post, raw: url)
PostHotlinkedMedia.create!(
url: "//example.com/favicon.ico",
post: post,
status: "download_failed",
)
PostHotlinkedMedia.create!(
url: "//example.com/article.jpeg",
post: post,
status: "download_failed",
)
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
cpp.post_process
expect(cpp.doc).to match_html <<~HTML
<aside class="onebox allowlistedgeneric" data-onebox-src="https://example.com/article">
<header class="source">
<a href="https://example.com/article" target="_blank" rel="noopener nofollow ugc">Example Site</a>
</header>
<article class="onebox-body">
<h3><a href="https://example.com/article" target="_blank" rel="noopener nofollow ugc">Lorem Ispum</a></h3>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer tellus neque, malesuada ac neque ac, tempus tincidunt lectus.</p>
</article>
</aside>
HTML
end
UX: generic onebox treats all square images as avatars and renders them smaller
2017-11-13 08:19:06 +08:00
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#post_process_oneboxes removes nofollow if add_rel_nofollow_to_user_content is disabled" do
FIX: respect nofollow settings for onebox links
2018-03-26 18:24:39 +08:00
let(:post) { build(:post_with_youtube, id: 123) }
let(:cpp) { CookedPostProcessor.new(post, invalidate_oneboxes: true) }
before do
SiteSetting.add_rel_nofollow_to_user_content = false
Oneboxer
.expects(:onebox)
.with(
"http://www.youtube.com/watch?v=9bZkp7q19f0",
invalidate_oneboxes: true,
user_id: nil,
category_id: post.topic.category_id,
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
)
.returns(
'<aside class="onebox"><a href="https://www.youtube.com/watch?v=9bZkp7q19f0" rel="noopener nofollow ugc">GANGNAM STYLE</a></aside>',
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
)
FIX: respect nofollow settings for onebox links
2018-03-26 18:24:39 +08:00
cpp.post_process_oneboxes
end
it "removes nofollow noopener from links" do
expect(cpp).to be_dirty
FIX: Don't strip `noopener` from oneboxes
2020-07-14 04:54:15 +08:00
expect(
cpp.html,
).to match_html '<aside class="onebox"><a href="https://www.youtube.com/watch?v=9bZkp7q19f0" rel="noopener">GANGNAM STYLE</a></aside>'
FIX: respect nofollow settings for onebox links
2018-03-26 18:24:39 +08:00
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#post_process_oneboxes removes nofollow if user is tl3" do
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
let(:post) { build(:post_with_youtube, id: 123) }
let(:cpp) { CookedPostProcessor.new(post, invalidate_oneboxes: true) }
before do
post.user.trust_level = TrustLevel[3]
post.user.save!
SiteSetting.add_rel_nofollow_to_user_content = true
SiteSetting.tl3_links_no_follow = false
Oneboxer
.expects(:onebox)
.with(
"http://www.youtube.com/watch?v=9bZkp7q19f0",
invalidate_oneboxes: true,
user_id: nil,
category_id: post.topic.category_id,
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
)
.returns(
'<aside class="onebox"><a href="https://www.youtube.com/watch?v=9bZkp7q19f0" rel="noopener nofollow ugc">GANGNAM STYLE</a></aside>',
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
)
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
cpp.post_process_oneboxes
end
FIX: Don't strip `noopener` from oneboxes
2020-07-14 04:54:15 +08:00
it "removes nofollow ugc from links" do
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
expect(cpp).to be_dirty
FIX: Don't strip `noopener` from oneboxes
2020-07-14 04:54:15 +08:00
expect(
cpp.html,
).to match_html '<aside class="onebox"><a href="https://www.youtube.com/watch?v=9bZkp7q19f0" rel="noopener">GANGNAM STYLE</a></aside>'
FIX: respect `tl3 links no follow` setting (#8232)
2019-10-23 01:11:04 +08:00
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#post_process_oneboxes with oneboxed image" do
FIX: Process image onebox correctly when image is wrapped in a link The instagram onebox sometimes surrounds the image with an `<a>` tag, which was breaking the aspect ratio logic, and therefore causing posts to change height on load.
2019-05-10 17:02:15 +08:00
let(:post) { build(:post_with_youtube, id: 123) }
let(:cpp) { CookedPostProcessor.new(post, invalidate_oneboxes: true) }
it "applies aspect ratio to container" do
Oneboxer
.expects(:onebox)
.with(
"http://www.youtube.com/watch?v=9bZkp7q19f0",
invalidate_oneboxes: true,
user_id: nil,
category_id: post.topic.category_id,
)
.returns(
"<aside class='onebox'><div class='scale-images'><img src='/img.jpg' width='400' height='500'/></div></div>",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
)
FIX: Process image onebox correctly when image is wrapped in a link The instagram onebox sometimes surrounds the image with an `<a>` tag, which was breaking the aspect ratio logic, and therefore causing posts to change height on load.
2019-05-10 17:02:15 +08:00
cpp.post_process_oneboxes
expect(cpp.html).to match_html(
'<aside class="onebox"><div class="aspect-image-full-size" style="--aspect-ratio:400/500;"><img src="/img.jpg"></div></aside>',
)
end
it "applies aspect ratio when wrapped in link" do
Oneboxer
.expects(:onebox)
.with(
"http://www.youtube.com/watch?v=9bZkp7q19f0",
invalidate_oneboxes: true,
user_id: nil,
category_id: post.topic.category_id,
)
.returns(
"<aside class='onebox'><div class='scale-images'><a href='https://example.com'><img src='/img.jpg' width='400' height='500'/></a></div></div>",
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
)
FIX: Process image onebox correctly when image is wrapped in a link The instagram onebox sometimes surrounds the image with an `<a>` tag, which was breaking the aspect ratio logic, and therefore causing posts to change height on load.
2019-05-10 17:02:15 +08:00
cpp.post_process_oneboxes
expect(cpp.html).to match_html(
'<aside class="onebox"><div class="aspect-image-full-size" style="--aspect-ratio:400/500;"><a href="https://example.com"><img src="/img.jpg"></a></div></aside>',
)
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#post_process_oneboxes with square image" do
UX: generic onebox treats all square images as avatars and renders them smaller
2017-11-13 08:19:06 +08:00
it "generates a onebox-avatar class" do
url = "https://square-image.com/onebox"
body = <<~HTML
<html>
<head>
<meta property='og:title' content="Page awesome">
<meta property='og:image' content="https://image.com/avatar.png">
<meta property='og:description' content="Page awesome desc">
</head>
</html>
HTML
FIX: don't double request when downloading a file
2018-02-24 19:35:57 +08:00
stub_request(:head, url)
stub_request(:get, url).to_return(body: body)
UX: generic onebox treats all square images as avatars and renders them smaller
2017-11-13 08:19:06 +08:00
# not an ideal stub but shipping the whole image to fast image can add
# a lot of cost to this test
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size(width: 200, height: 200)
UX: generic onebox treats all square images as avatars and renders them smaller
2017-11-13 08:19:06 +08:00
post = Fabricate.build(:post, raw: url)
cpp = CookedPostProcessor.new(post, invalidate_oneboxes: true)
cpp.post_process_oneboxes
expect(cpp.doc.to_s).not_to include("aspect-image")
expect(cpp.doc.to_s).to include("onebox-avatar")
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#optimize_urls" do
pull hotlinked images
2013-11-06 02:04:47 +08:00
let(:post) { build(:post_with_uploads_and_links) }
let(:cpp) { CookedPostProcessor.new(post) }
it "uses schemaless url for uploads" do
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><a href="//test.localhost/#{upload_path}/original/2X/2345678901234567.jpg">Link</a><br>
<img src="//test.localhost/#{upload_path}/original/1X/1234567890123456.jpg"><br>
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
<a href="http://www.google.com" rel="noopener nofollow ugc">Google</a><br>
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
<img src="http://foo.bar/image.png"><br>
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<a class="attachment" href="//test.localhost/#{upload_path}/original/1X/af2c2618032c679333bebf745e75f9088748d737.txt">text.txt</a> (20 Bytes)<br>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<img src="//test.localhost/images/emoji/twitter/smile.png?v=#{Emoji::EMOJI_VERSION}" title=":smile:" class="emoji only-emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
FIX: store the topic links using the cooked upload url
2018-08-14 18:23:32 +08:00
HTML
refactor CookedPostProcessor & specs
2013-07-08 07:39:08 +08:00
end
pull hotlinked images
2013-11-06 02:04:47 +08:00
context "when CDN is enabled" do
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
it "uses schemaless CDN url for http uploads" do
pull hotlinked images
2013-11-06 02:04:47 +08:00
Rails.configuration.action_controller.stubs(:asset_host).returns("http://my.cdn.com")
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><a href="//my.cdn.com/#{upload_path}/original/2X/2345678901234567.jpg">Link</a><br>
<img src="//my.cdn.com/#{upload_path}/original/1X/1234567890123456.jpg"><br>
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
<a href="http://www.google.com" rel="noopener nofollow ugc">Google</a><br>
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
<img src="http://foo.bar/image.png"><br>
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<a class="attachment" href="//my.cdn.com/#{upload_path}/original/1X/af2c2618032c679333bebf745e75f9088748d737.txt">text.txt</a> (20 Bytes)<br>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<img src="//my.cdn.com/images/emoji/twitter/smile.png?v=#{Emoji::EMOJI_VERSION}" title=":smile:" class="emoji only-emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
FIX: store the topic links using the cooked upload url
2018-08-14 18:23:32 +08:00
HTML
pull hotlinked images
2013-11-06 02:04:47 +08:00
end
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
it "doesn't use schemaless CDN url for https uploads" do
FIX: properly support HTTPS CDN on HTTP site Previously we changed all CDN links to schemaless. This is desirable for non HTTPS sites, to ease migration to HTTPS. It is not desirable for secure sites. Once site is secure or CDN is secure a rebake should be required to move it back to non-secure.
2015-07-24 12:08:32 +08:00
Rails.configuration.action_controller.stubs(:asset_host).returns("https://my.cdn.com")
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><a href="https://my.cdn.com/#{upload_path}/original/2X/2345678901234567.jpg">Link</a><br>
<img src="https://my.cdn.com/#{upload_path}/original/1X/1234567890123456.jpg"><br>
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
<a href="http://www.google.com" rel="noopener nofollow ugc">Google</a><br>
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
<img src="http://foo.bar/image.png"><br>
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<a class="attachment" href="https://my.cdn.com/#{upload_path}/original/1X/af2c2618032c679333bebf745e75f9088748d737.txt">text.txt</a> (20 Bytes)<br>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<img src="https://my.cdn.com/images/emoji/twitter/smile.png?v=#{Emoji::EMOJI_VERSION}" title=":smile:" class="emoji only-emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
FIX: store the topic links using the cooked upload url
2018-08-14 18:23:32 +08:00
HTML
FIX: don't use the CDN for attachments when login is required
2016-01-07 04:54:01 +08:00
end
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
it "doesn't use CDN when login is required" do
FIX: don't use the CDN for attachments when login is required
2016-01-07 04:54:01 +08:00
SiteSetting.login_required = true
Rails.configuration.action_controller.stubs(:asset_host).returns("http://my.cdn.com")
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><a href="//my.cdn.com/#{upload_path}/original/2X/2345678901234567.jpg">Link</a><br>
<img src="//my.cdn.com/#{upload_path}/original/1X/1234567890123456.jpg"><br>
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
<a href="http://www.google.com" rel="noopener nofollow ugc">Google</a><br>
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
<img src="http://foo.bar/image.png"><br>
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<a class="attachment" href="//test.localhost/#{upload_path}/original/1X/af2c2618032c679333bebf745e75f9088748d737.txt">text.txt</a> (20 Bytes)<br>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<img src="//my.cdn.com/images/emoji/twitter/smile.png?v=#{Emoji::EMOJI_VERSION}" title=":smile:" class="emoji only-emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
FIX: store the topic links using the cooked upload url
2018-08-14 18:23:32 +08:00
HTML
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
end
it "doesn't use CDN when preventing anons from downloading files" do
SiteSetting.prevent_anons_from_downloading_files = true
Rails.configuration.action_controller.stubs(:asset_host).returns("http://my.cdn.com")
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<p><a href="//my.cdn.com/#{upload_path}/original/2X/2345678901234567.jpg">Link</a><br>
<img src="//my.cdn.com/#{upload_path}/original/1X/1234567890123456.jpg"><br>
FIX: Make sure rel attributes are correctly set. (#10645) We must guarantee that "rel=noopener" was set if "target=_blank" is present, which is not always the case for trusted users. Also, if the link contains the "nofollow" attribute, it has to have the "ugc" attribute as well.
2020-09-10 23:59:51 +08:00
<a href="http://www.google.com" rel="noopener nofollow ugc">Google</a><br>
FIX: don't put attachments on the CDN when 'prevent anons from downloading files' is enabled
2017-02-08 01:06:44 +08:00
<img src="http://foo.bar/image.png"><br>
FIX: parallel spec system needs a dedicated upload folder for each worker. (#8547)
2019-12-18 13:51:57 +08:00
<a class="attachment" href="//test.localhost/#{upload_path}/original/1X/af2c2618032c679333bebf745e75f9088748d737.txt">text.txt</a> (20 Bytes)<br>
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<img src="//my.cdn.com/images/emoji/twitter/smile.png?v=#{Emoji::EMOJI_VERSION}" title=":smile:" class="emoji only-emoji" alt=":smile:" loading="lazy" width="20" height="20"></p>
FIX: store the topic links using the cooked upload url
2018-08-14 18:23:32 +08:00
HTML
FIX: properly support HTTPS CDN on HTTP site Previously we changed all CDN links to schemaless. This is desirable for non HTTPS sites, to ease migration to HTTPS. It is not desirable for secure sites. Once site is secure or CDN is secure a rebake should be required to move it back to non-secure.
2015-07-24 12:08:32 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with s3_uploads" do
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
before do
DEV: Clean up S3 specs, stubs, and helpers Extracted commonly used spec helpers into spec/support/uploads_helpers.rb, removed unused stubs and let definitions. Makes it easier to write new S3-related specs without copy and pasting setup steps from other specs.
2020-09-14 19:32:25 +08:00
Rails.configuration.action_controller.stubs(:asset_host).returns("https://local.cdn.com")
setup_s3
SiteSetting.s3_cdn_url = "https://s3.cdn.com"
SiteSetting.authorized_extensions = "png|jpg|gif|mov|ogg|"
FIX: Secure Upload URLs in lightbox (#8451) This fixes the following issues: * The link element on the lightbox which pops open the lightbox was linking to the S3 URL with a private ACL instead of the secure media URL for the image * Change to use `@post.with_secure_media?` in `CookedPostProcessor` for URL cooking, as in some cases, like when a post is edited and an upload is added, `upload.secure?` can be false which resulted in `srcset` URLs not being cooked correctly to secure media upload urls.
2019-12-05 07:13:09 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
uploaded_file = file_from_fixtures("smallest.png")
upload_sha1 = Digest::SHA1.hexdigest(File.read(uploaded_file))
upload.update!(
original_filename: "smallest.png",
width: 10,
height: 20,
sha1: upload_sha1,
extension: "png",
)
end
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
it "uses the right CDN when uploads are on S3" do
stored_path = Discourse.store.get_path_for_upload(upload)
upload.update_column(:url, "#{SiteSetting.Upload.absolute_base_url}/#{stored_path}")
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
the_post =
Fabricate(
:post,
raw:
%Q{This post has a local emoji :+1: and an external upload\n\n![smallest.png|10x20](#{upload.short_url})},
)
DEV: Reduce object fabrication in `CookedPostProcessor` spec.
2019-05-30 08:49:37 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
cpp = CookedPostProcessor.new(the_post)
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<p>This post has a local emoji <img src="https://local.cdn.com/images/emoji/twitter/+1.png?v=#{Emoji::EMOJI_VERSION}" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"> and an external upload</p>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
<p><img src="https://s3.cdn.com/#{stored_path}" alt="smallest.png" data-base62-sha1="#{upload.base62_sha1}" width="10" height="20"></p>
HTML
end
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
it "doesn't use CDN for secure uploads" do
SiteSetting.secure_uploads = true
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
stored_path = Discourse.store.get_path_for_upload(upload)
upload.update_column(:url, "#{SiteSetting.Upload.absolute_base_url}/#{stored_path}")
upload.update_column(:secure, true)
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
the_post =
Fabricate(
:post,
raw:
%Q{This post has a local emoji :+1: and an external upload\n\n![smallest.png|10x20](#{upload.short_url})},
)
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
cpp = CookedPostProcessor.new(the_post)
cpp.optimize_urls
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: add native lazy loading for emojis (#15830)
2022-02-09 19:18:59 +08:00
<p>This post has a local emoji <img src="https://local.cdn.com/images/emoji/twitter/+1.png?v=#{Emoji::EMOJI_VERSION}" title=":+1:" class="emoji" alt=":+1:" loading="lazy" width="20" height="20"> and an external upload</p>
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
<p><img src="/secure-uploads/#{stored_path}" alt="smallest.png" data-base62-sha1="#{upload.base62_sha1}" width="10" height="20"></p>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
HTML
end
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
it "doesn't use the secure uploads URL for custom emoji" do
FIX: Cooking custom emojis should not use a secure URL (#15929) When a site has secure media enabled and a post is with secure media, we were incorrectly cooking custom emoji URLs and using the secure URL for those emojis, even though they should not be considered secure (their corresponding upload records in the database are _not_ secure). Now instead of the blanket post.with_secure_media? boolean for the secure: param, we also want to make sure the image whose URL is being cooked is also _not_ a custom emoji.
2022-02-14 11:02:42 +08:00
CustomEmoji.create!(name: "trout", upload: upload)
Emoji.clear_cache
Emoji.load_custom
stored_path = Discourse.store.get_path_for_upload(upload)
upload.update_column(:url, "#{SiteSetting.Upload.absolute_base_url}/#{stored_path}")
upload.update_column(:secure, true)
the_post = Fabricate(:post, raw: "This post has a custom emoji :trout:")
the_post.cook(the_post.raw)
cpp = CookedPostProcessor.new(the_post)
cpp.optimize_urls
upload_url = upload.url.gsub(SiteSetting.Upload.absolute_base_url, "https://s3.cdn.com")
expect(cpp.html).to match_html <<~HTML
<p>This post has a custom emoji <img src="#{upload_url}?v=#{Emoji::EMOJI_VERSION}" title=":trout:" class="emoji emoji-custom" alt=":trout:" loading="lazy" width="20" height="20"></p>
HTML
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with media uploads" do
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
fab!(:image_upload) { Fabricate(:upload) }
fab!(:audio_upload) { Fabricate(:upload, extension: "ogg") }
fab!(:video_upload) { Fabricate(:upload, extension: "mov") }
before do
video_upload.update!(
url: "#{SiteSetting.s3_cdn_url}/#{Discourse.store.get_path_for_upload(video_upload)}",
)
stub_request(:head, video_upload.url)
end
it "ignores prevent_anons_from_downloading_files and oneboxes video uploads" do
SiteSetting.prevent_anons_from_downloading_files = true
the_post =
Fabricate(:post, raw: "This post has an S3 video onebox:\n#{video_upload.url}")
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
cpp = CookedPostProcessor.new(the_post.reload)
cpp.post_process_oneboxes
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
expect(cpp.html).to match_html <<~HTML
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
<p>This post has an S3 video onebox:</p>
<div class="onebox video-onebox">
<video width="100%" height="100%" controls="">
<source src="#{video_upload.url}">
<a href="#{video_upload.url}" rel="nofollow ugc noopener">#{video_upload.url}</a>
</video>
</div>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
HTML
end
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
it "oneboxes video using secure url when secure_uploads is enabled" do
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
SiteSetting.login_required = true
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
SiteSetting.secure_uploads = true
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
video_upload.update_column(:secure, true)
FIX: ensure local images use local CDN when uploads are stored on S3 When the S3 store was enabled, we were only applying the S3 CDN. So all images stored locally, like the emojis, were never put on the local CDN. Fixed a bunch of CookedPostProcessor test by adding a call to 'optimize_urls' in order to get final URLs. I also removed the unnecessary PrettyText.add_s3_cdn method since this is already handled in the CookedPostProcessor.
2019-02-21 02:24:38 +08:00
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
the_post =
Fabricate(:post, raw: "This post has an S3 video onebox:\n#{video_upload.url}")
cpp = CookedPostProcessor.new(the_post)
cpp.post_process_oneboxes
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
secure_url =
video_upload.url.sub(SiteSetting.s3_cdn_url, "#{Discourse.base_url}/secure-uploads")
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
<p>This post has an S3 video onebox:</p><div class="onebox video-onebox">
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
<video width="100%" height="100%" controls="">
<source src="#{secure_url}">
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
<a href="#{secure_url}">#{secure_url}</a>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
</video>
</div>
HTML
end
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
it "oneboxes only audio/video and not images when secure_uploads is enabled" do
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
SiteSetting.login_required = true
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
SiteSetting.secure_uploads = true
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
video_upload.update_column(:secure, true)
audio_upload.update!(
url: "#{SiteSetting.s3_cdn_url}/#{Discourse.store.get_path_for_upload(audio_upload)}",
secure: true,
)
image_upload.update!(
url: "#{SiteSetting.s3_cdn_url}/#{Discourse.store.get_path_for_upload(image_upload)}",
secure: true,
)
stub_request(:head, audio_upload.url)
stub_request(:get, image_upload.url)
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
raw = <<~RAW.rstrip
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
This post has a video upload.
#{video_upload.url}
This post has an audio upload.
#{audio_upload.url}
And an image upload.
![logo.png](upload://#{image_upload.base62_sha1}.#{image_upload.extension})
RAW
the_post = Fabricate(:post, raw: raw)
cpp = CookedPostProcessor.new(the_post)
cpp.post_process_oneboxes
DEV: Rename secure_media to secure_uploads (#18376) This commit renames all secure_media related settings to secure_uploads_* along with the associated functionality. This is being done because "media" does not really cover it, we aren't just doing this for images and videos etc. but for all uploads in the site. Additionally, in future we want to secure more types of uploads, and enable a kind of "mixed mode" where some uploads are secure and some are not, so keeping media in the name is just confusing. This also keeps compatibility with the `secure-media-uploads` path, and changes new secure URLs to be `secure-uploads`. Deprecated settings: * secure_media -> secure_uploads * secure_media_allow_embed_images_in_emails -> secure_uploads_allow_embed_images_in_emails * secure_media_max_email_embed_image_size_kb -> secure_uploads_max_email_embed_image_size_kb
2022-09-29 07:24:33 +08:00
secure_video_url =
video_upload.url.sub(SiteSetting.s3_cdn_url, "#{Discourse.base_url}/secure-uploads")
secure_audio_url =
audio_upload.url.sub(SiteSetting.s3_cdn_url, "#{Discourse.base_url}/secure-uploads")
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
FIX: Make Oneboxer#apply insert block Oneboxes correctly (#11449) It used to insert block Oneboxes inside paragraphs which resulted in invalid HTML. This needed an additional parsing for removal of empty paragraphs and the resulting HTML could still be invalid. This commit ensure that block Oneboxes are inserted correctly, by splitting the paragraph containing the link and putting the block between the two. Paragraphs left with nothing but whitespaces will be removed. Follow up to 7f3a30d79fa63aca063e4186a21f983e23d759e8.
2020-12-14 23:49:37 +08:00
expect(cpp.html).to match_html <<~HTML
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-20 09:49:22 +08:00
<p>This post has a video upload.</p><div class="onebox video-onebox">
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
<video width="100%" height="100%" controls="">
<source src="#{secure_video_url}">
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-20 09:49:22 +08:00
<a href="#{secure_video_url}">
#{secure_video_url}
</a>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
</video>
</div>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-20 09:49:22 +08:00
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
<p>This post has an audio upload.<br>
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-20 09:49:22 +08:00
<audio controls="">
<source src="#{secure_audio_url}">
<a href="#{secure_audio_url}">
#{secure_audio_url}
</a>
</audio>
</p>
FEATURE: Add support for secure media (#7888) This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. A few notes: - the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads - the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured - upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status - when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error - when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 09:25:42 +08:00
<p>And an image upload.<br>
<img src="#{image_upload.url}" alt="#{image_upload.original_filename}" data-base62-sha1="#{image_upload.base62_sha1}"></p>
HTML
end
end
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#remove_user_ids" do
FEATURE: Remove user IDs from internal URLs. (#7406)
2019-04-23 10:45:41 +08:00
let(:topic) { Fabricate(:topic) }
let(:post) { Fabricate(:post, raw: <<~RAW) }
link to a topic: #{topic.url}?u=foo
a tricky link to a topic: #{topic.url}?bob=bob;u=sam&jane=jane
link to an external topic: https://google.com/?u=bar
FIX: rescue InvalidURIError when removing user ids from links
2019-04-25 15:06:31 +08:00
a malformed url: https://www.example.com/#123#4
FEATURE: Remove user IDs from internal URLs. (#7406)
2019-04-23 10:45:41 +08:00
RAW
UX: Use dominant color as image loading placeholder (#18248) We previously had a system which would generate a 10x10px preview of images and add their URLs in a data-small-upload attribute. The client would then use that as the background-image of the `<img>` element. This works reasonably well on fast connections, but on slower connections it can take a few seconds for the placeholders to appear. The act of loading the placeholders can also break or delay the loading of the 'real' images. This commit replaces the placeholder logic with a new approach. Instead of a 10x10px preview, we use imagemagick to calculate the average color of an image and store it in the database. The hex color value then added as a `data-dominant-color` attribute on the `<img>` element, and the client can use this as a `background-color` on the element while the real image is loading. That means no extra HTTP request is required, and so the placeholder color can appear instantly. Dominant color will be calculated: 1. When a new upload is created 2. During a post rebake, if the dominant color is missing from an upload, it will be calculated and stored 3. Every 15 minutes, 25 old upload records are fetched and their dominant color calculated and stored. (part of the existing PeriodicalUpdates job) Existing posts will continue to use the old 10x10px placeholder system until they are next rebaked
2022-09-20 17:28:17 +08:00
let(:cpp) { CookedPostProcessor.new(post, disable_dominant_color: true) }
FEATURE: Remove user IDs from internal URLs. (#7406)
2019-04-23 10:45:41 +08:00
it "does remove user ids" do
cpp.remove_user_ids
expect(cpp.html).to have_tag("a", with: { href: topic.url })
expect(cpp.html).to have_tag("a", with: { href: "#{topic.url}?bob=bob&jane=jane" })
expect(cpp.html).to have_tag("a", with: { href: "https://google.com/?u=bar" })
FIX: rescue InvalidURIError when removing user ids from links
2019-04-25 15:06:31 +08:00
expect(cpp.html).to have_tag("a", with: { href: "https://www.example.com/#123#4" })
FEATURE: Remove user IDs from internal URLs. (#7406)
2019-04-23 10:45:41 +08:00
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#is_a_hyperlink?" do
FIX markdown hotlinked images were not properly pulled
2013-11-20 20:10:08 +08:00
let(:post) { build(:post) }
let(:cpp) { CookedPostProcessor.new(post) }
FEATURE: Nokogumbo (#9577) * FEATURE: Nokogumbo Use Nokogumbo HTML parser.
2020-05-05 11:46:57 +08:00
let(:doc) do
Nokogiri::HTML5.fragment(
'<body><div><a><img id="linked_image"></a><p><img id="standard_image"></p></div></body>',
)
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
end
FIX markdown hotlinked images were not properly pulled
2013-11-20 20:10:08 +08:00
it "is true when the image is inside a link" do
img = doc.css("img#linked_image").first
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.is_a_hyperlink?(img)).to eq(true)
FIX markdown hotlinked images were not properly pulled
2013-11-20 20:10:08 +08:00
end
it "is false when the image is not inside a link" do
img = doc.css("img#standard_image").first
few components with rspec3 syntax
2015-01-10 00:34:37 +08:00
expect(cpp.is_a_hyperlink?(img)).to eq(false)
FIX markdown hotlinked images were not properly pulled
2013-11-20 20:10:08 +08:00
end
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
describe "grant badges" do
FIX: FirstReplyByEmail badge wasn't granted DEPRECATED: PostProcess badge trigger
2016-08-11 01:24:01 +08:00
let(:cpp) { CookedPostProcessor.new(post) }
FEATURE: New "First Onebox" badge
2016-04-13 02:09:59 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with emoji inside a quote" do
Feature: Change markdown engine to markdown it This commit removes the old evilstreak markdownjs engine. - Adds specs to WhiteLister and changes it to stop using globals (Fixes large memory leak) - Fixes edge cases around bbcode handling - Removes mdtest which is no longer valid (to be replaced with CommonMark) - Updates MiniRacer to correct minor unmanaged memory leak - Fixes plugin specs
2017-07-14 20:27:28 +08:00
let(:post) do
Fabricate(:post, raw: "time to eat some sweet \n[quote]\n:candy:\n[/quote]\n mmmm")
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
end
Remove `UserFirst` table and grant Emoji badge directly
2016-04-08 00:27:26 +08:00
it "doesn't award a badge when the emoji is in a quote" do
cpp.grant_badges
expect(post.user.user_badges.where(badge_id: Badge::FirstEmoji).exists?).to eq(false)
end
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with emoji in the text" do
Remove `UserFirst` table and grant Emoji badge directly
2016-04-08 00:27:26 +08:00
let(:post) { Fabricate(:post, raw: "time to eat some sweet :candy: mmmm") }
it "awards a badge for using an emoji" do
cpp.grant_badges
expect(post.user.user_badges.where(badge_id: Badge::FirstEmoji).exists?).to eq(true)
end
end
FEATURE: New "First Onebox" badge
2016-04-13 02:09:59 +08:00
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with onebox" do
FIX: Award 'First Onebox' badge just for Oneboxed URLs. (#7974)
2019-08-09 00:45:18 +08:00
before do
Oneboxer.stubs(:onebox).with(anything, anything).returns(nil)
FIX: use allowlist and blocklist terminology (#10209) This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 08:23:54 +08:00
Oneboxer
.stubs(:onebox)
.with("https://discourse.org", anything)
.returns("<aside class=\"onebox allowlistedgeneric\">the rest of the onebox</aside>")
FIX: Award 'First Onebox' badge just for Oneboxed URLs. (#7974)
2019-08-09 00:45:18 +08:00
end
FEATURE: New "First Onebox" badge
2016-04-13 02:09:59 +08:00
FIX: Award 'First Onebox' badge just for Oneboxed URLs. (#7974)
2019-08-09 00:45:18 +08:00
it "awards the badge for using an onebox" do
post = Fabricate(:post, raw: "onebox me:\n\nhttps://discourse.org\n")
cpp = CookedPostProcessor.new(post)
FEATURE: New "First Onebox" badge
2016-04-13 02:09:59 +08:00
cpp.post_process_oneboxes
cpp.grant_badges
expect(post.user.user_badges.where(badge_id: Badge::FirstOnebox).exists?).to eq(true)
end
FIX: Don't ever grant badges when they're disabled
2016-08-20 03:16:37 +08:00
FIX: Award 'First Onebox' badge just for Oneboxed URLs. (#7974)
2019-08-09 00:45:18 +08:00
it "does not award the badge when link is not oneboxed" do
post = Fabricate(:post, raw: "onebox me:\n\nhttp://example.com\n")
cpp = CookedPostProcessor.new(post)
cpp.post_process_oneboxes
cpp.grant_badges
expect(post.user.user_badges.where(badge_id: Badge::FirstOnebox).exists?).to eq(false)
end
it "does not award the badge when the badge is disabled" do
FIX: Don't ever grant badges when they're disabled
2016-08-20 03:16:37 +08:00
Badge.where(id: Badge::FirstOnebox).update_all(enabled: false)
FIX: Award 'First Onebox' badge just for Oneboxed URLs. (#7974)
2019-08-09 00:45:18 +08:00
post = Fabricate(:post, raw: "onebox me:\n\nhttps://discourse.org\n")
cpp = CookedPostProcessor.new(post)
FIX: Don't ever grant badges when they're disabled
2016-08-20 03:16:37 +08:00
cpp.post_process_oneboxes
cpp.grant_badges
expect(post.user.user_badges.where(badge_id: Badge::FirstOnebox).exists?).to eq(false)
end
FEATURE: New "First Onebox" badge
2016-04-13 02:09:59 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with reply_by_email" do
FIX: FirstReplyByEmail badge wasn't granted DEPRECATED: PostProcess badge trigger
2016-08-11 01:24:01 +08:00
let(:post) do
Fabricate(:post, raw: "This is a **reply** via email ;)", via_email: true, post_number: 2)
DEV: Apply syntax_tree formatting to `spec/*`
2023-01-09 19:18:21 +08:00
end
FIX: FirstReplyByEmail badge wasn't granted DEPRECATED: PostProcess badge trigger
2016-08-11 01:24:01 +08:00
it "awards a badge for replying via email" do
cpp.grant_badges
expect(post.user.user_badges.where(badge_id: Badge::FirstReplyByEmail).exists?).to eq(true)
end
end
Remove `UserFirst` table and grant Emoji badge directly
2016-04-08 00:27:26 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
describe "quote processing" do
FEATURE: Add `quote-modified` class if a quote has been modified
2018-03-14 01:07:51 +08:00
let(:cpp) { CookedPostProcessor.new(cp) }
let(:pp) { Fabricate(:post, raw: "This post is ripe for quoting!") }
context "with an unmodified quote" do
let(:cp) do
Fabricate(
:post,
raw:
"[quote=\"#{pp.user.username}, post: #{pp.post_number}, topic:#{pp.topic_id}]\nripe for quoting\n[/quote]\ntest",
)
end
it "should not be marked as modified" do
cpp.post_process_quotes
expect(cpp.doc.css("aside.quote.quote-modified")).to be_blank
end
end
context "with a modified quote" do
let(:cp) do
Fabricate(
:post,
raw:
"[quote=\"#{pp.user.username}, post: #{pp.post_number}, topic:#{pp.topic_id}]\nmodified\n[/quote]\ntest",
)
end
it "should be marked as modified" do
cpp.post_process_quotes
expect(cpp.doc.css("aside.quote.quote-modified")).to be_present
end
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
context "with external discourse instance quote" do
FIX: handle quote rendering for external Discourse instance (#16722) Gracefully handle quotes from an external discourse instance by stripping quote-controls and including username in the title
2022-05-12 23:07:43 +08:00
let(:external_raw) { <<~RAW.strip }
[quote="random_guy_not_from_our_discourse, post:2004, topic:401"]
this quote is not from our discourse
[/quote]
and this is a reply
RAW
let(:cp) { Fabricate(:post, raw: external_raw) }
it "it should be marked as missing" do
cpp.post_process_quotes
expect(cpp.doc.css("aside.quote.quote-post-not-found")).to be_present
end
end
FEATURE: Add `quote-modified` class if a quote has been modified
2018-03-14 01:07:51 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
describe "full quote on direct reply" do
DEV: Allow fab! without block (#24314) The most common thing that we do with fab! is: fab!(:thing) { Fabricate(:thing) } This commit adds a shorthand for this which is just simply: fab!(:thing) i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-10 06:47:59 +08:00
fab!(:topic)
FIX: remove full quote on direct replies when "typographed" Use the cooked version of the post and the quote to compare their content in order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 23:49:29 +08:00
let!(:post) { Fabricate(:post, topic: topic, raw: 'this is the "first" post') }
FEATURE: Remove full quote only if first paragraph. (#6793)
2018-12-18 22:46:20 +08:00
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
let(:raw) { <<~RAW.strip }
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
[quote="#{post.user.username}, post:#{post.post_number}, topic:#{topic.id}"]
FIX: remove full quote on direct replies when "typographed" Use the cooked version of the post and the quote to compare their content in order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 23:49:29 +08:00
this is the “first” post
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
[/quote]
and this is the third reply
RAW
FEATURE: Remove full quotes of direct replies. (#6729)
2018-12-07 20:07:11 +08:00
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
let(:raw2) { <<~RAW.strip }
FEATURE: Remove full quote only if first paragraph. (#6793)
2018-12-18 22:46:20 +08:00
and this is the third reply
[quote="#{post.user.username}, post:#{post.post_number}, topic:#{topic.id}"]
FIX: remove full quote on direct replies when "typographed" Use the cooked version of the post and the quote to compare their content in order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 23:49:29 +08:00
this is the ”first” post
FEATURE: Remove full quote only if first paragraph. (#6793)
2018-12-18 22:46:20 +08:00
[/quote]
RAW
FIX: Remove full nested quotes on direct reply (#8581) It used to check how many quotes were inside a post, without taking considering that some quotes can contain other quotes. This commit selects only top level quotes. I had to use XPath because I could not find an equivalent CSS selector.
2019-12-20 16:24:34 +08:00
let(:raw3) { <<~RAW.strip }
[quote="#{post.user.username}, post:#{post.post_number}, topic:#{topic.id}"]
this is the “first” post
[/quote]
[quote="#{post.user.username}, post:#{post.post_number}, topic:#{topic.id}"]
this is the “first” post
[/quote]
and this is the third reply
RAW
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
before { SiteSetting.remove_full_quote = true }
FEATURE: Remove full quotes of direct replies. (#6729)
2018-12-07 20:07:11 +08:00
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
it "works" do
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
hidden = Fabricate(:post, topic: topic, hidden: true, raw: "this is the second post after")
small_action = Fabricate(:post, topic: topic, post_type: Post.types[:small_action])
FEATURE: Remove full quotes of direct replies. (#6729)
2018-12-07 20:07:11 +08:00
reply = Fabricate(:post, topic: topic, raw: raw)
DEV: Improve flaky time-sensitive specs (#9141)
2020-03-11 05:13:17 +08:00
freeze_time do
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
topic.bumped_at = 1.day.ago
FIX: remove full quote on direct replies when "typographed" Use the cooked version of the post and the quote to compare their content in order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 23:49:29 +08:00
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
Fix the build.
2019-04-08 21:30:04 +08:00
expect(topic.ordered_posts.pluck(:id)).to eq(
[post.id, hidden.id, small_action.id, reply.id],
)
DEV: Improve flaky time-sensitive specs (#9141)
2020-03-11 05:13:17 +08:00
expect(topic.bumped_at).to eq_time(1.day.ago)
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
expect(reply.raw).to eq("and this is the third reply")
expect(reply.revisions.count).to eq(1)
expect(reply.revisions.first.modifications["raw"]).to eq([raw, reply.raw])
expect(reply.revisions.first.modifications["edit_reason"][1]).to eq(
I18n.t(:removed_direct_reply_full_quotes),
)
end
end
FIX: Remove full nested quotes on direct reply (#8581) It used to check how many quotes were inside a post, without taking considering that some quotes can contain other quotes. This commit selects only top level quotes. I had to use XPath because I could not find an equivalent CSS selector.
2019-12-20 16:24:34 +08:00
it "does nothing if there are multiple quotes" do
reply = Fabricate(:post, topic: topic, raw: raw3)
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
expect(topic.ordered_posts.pluck(:id)).to eq([post.id, reply.id])
expect(reply.raw).to eq(raw3)
end
FEATURE: Remove full quote only if first paragraph. (#6793)
2018-12-18 22:46:20 +08:00
it "does not delete quote if not first paragraph" do
reply = Fabricate(:post, topic: topic, raw: raw2)
FIX: remove full quote on direct replies when "typographed" Use the cooked version of the post and the quote to compare their content in order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 23:49:29 +08:00
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
Fix the build.
2019-04-08 21:30:04 +08:00
expect(topic.ordered_posts.pluck(:id)).to eq([post.id, reply.id])
FEATURE: Remove full quote only if first paragraph. (#6793)
2018-12-18 22:46:20 +08:00
expect(reply.raw).to eq(raw2)
end
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
it "does nothing when 'remove_full_quote' is disabled" do
SiteSetting.remove_full_quote = false
reply = Fabricate(:post, topic: topic, raw: raw)
FEATURE: Remove full quotes of direct replies. (#6729)
2018-12-07 20:07:11 +08:00
FIX: remove full quote on direct replies when "typographed" Use the cooked version of the post and the quote to compare their content in order to take into account the "typographer" option of the markdown pipeline.
2019-05-15 23:49:29 +08:00
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
* FEATURE: Adds site setting to let quotes on direct replies. * DEV: Added test. * FIX: Do not bump topic when removing full quotes.
2018-12-12 22:42:53 +08:00
expect(reply.raw).to eq(raw)
FEATURE: Remove full quotes of direct replies. (#6729)
2018-12-07 20:07:11 +08:00
end
FIX: blank cooked markdown could raise an exception in logs Previously if somehow a user created a blank markdown document using tag tricks (eg `<p></p><p></p><p></p><p></p><p></p><p></p>`) and so on, we would completely strip the document down to blank on post process due to onebox hack. Needs a followup cause I am still unclear about the reason for empty p stripping and it can cause some unclear cases when we re-cook posts.
2020-01-29 08:37:04 +08:00
it "does not generate a blank HTML document" do
post = Fabricate(:post, topic: topic, raw: "<sunday><monday>")
cp = CookedPostProcessor.new(post)
cp.post_process
expect(cp.html).to eq("<p></p>")
end
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
it "works only on new posts" do
FIX: Slow test (deadlock) It was getting caught in a `DistributedMutex` deadlock (twice!), which meant this test was taking 120s to run. I'm not sure why queue jobs was turned off here, because when I turn it on the test passes and takes <2s instead.
2019-02-20 01:29:50 +08:00
Fabricate(:post, topic: topic, hidden: true, raw: "this is the second post after")
Fabricate(:post, topic: topic, post_type: Post.types[:small_action])
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
reply = PostCreator.create!(topic.user, topic_id: topic.id, raw: raw)
FIX: Slow test (deadlock) It was getting caught in a `DistributedMutex` deadlock (twice!), which meant this test was taking 120s to run. I'm not sure why queue jobs was turned off here, because when I turn it on the test passes and takes <2s instead.
2019-02-20 01:29:50 +08:00
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
CookedPostProcessor.new(reply).post_process
expect(reply.raw).to eq(raw)
PostRevisor.new(reply).revise!(
Discourse.system_user,
raw: raw,
edit_reason: "put back full quote",
)
FIX: remove 'crawl_images' site setting (#14646)
2021-10-19 19:42:29 +08:00
stub_image_size
FEATURE: Remove full quotes only from new posts. (#6862)
2019-01-17 10:24:32 +08:00
CookedPostProcessor.new(reply).post_process(new_post: true)
expect(reply.raw).to eq("and this is the third reply")
end
FIX: Remove full nested quotes on direct reply (#8581) It used to check how many quotes were inside a post, without taking considering that some quotes can contain other quotes. This commit selects only top level quotes. I had to use XPath because I could not find an equivalent CSS selector.
2019-12-20 16:24:34 +08:00
it "works with nested quotes" do
reply1 = Fabricate(:post, topic: topic, raw: raw)
reply2 = Fabricate(:post, topic: topic, raw: <<~RAW.strip)
[quote="#{reply1.user.username}, post:#{reply1.post_number}, topic:#{topic.id}"]
#{raw}
[/quote]
quoting a post with a quote
RAW
CookedPostProcessor.new(reply2).remove_full_quote_on_direct_reply
expect(reply2.raw).to eq("quoting a post with a quote")
end
FEATURE: Remove full quotes of direct replies. (#6729)
2018-12-07 20:07:11 +08:00
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
describe "full quote on direct reply with full name prioritization" do
DEV: Prioritize full name when display_name_on_posts active (#16078) See: https://meta.discourse.org/t/display-full-name-not-username-when-attributing-quote-or-reply/203533?u=isaacjanzen for context The initial release [broke quoting](https://meta.discourse.org/t/quoting-broken-when-name-matches-username/217633?u=isaacjanzen) but we now pass the username when ``` siteSettings.display_name_on_posts && !siteSettings.prioritize_username_in_ux && post.name ``` as well as the full name to guarantee that we are not getting any mismatches when querying for user / avatar. eg. ``` [quote="Isaac Janzen, post:3, topic:7, full:true, username:isaac.janzen"] bing bong [/quote] ```
2022-04-20 23:07:51 +08:00
fab!(:user) { Fabricate(:user, name: "james, john, the third") }
DEV: Allow fab! without block (#24314) The most common thing that we do with fab! is: fab!(:thing) { Fabricate(:thing) } This commit adds a shorthand for this which is just simply: fab!(:thing) i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-10 06:47:59 +08:00
fab!(:topic)
DEV: Prioritize full name when display_name_on_posts active (#16078) See: https://meta.discourse.org/t/display-full-name-not-username-when-attributing-quote-or-reply/203533?u=isaacjanzen for context The initial release [broke quoting](https://meta.discourse.org/t/quoting-broken-when-name-matches-username/217633?u=isaacjanzen) but we now pass the username when ``` siteSettings.display_name_on_posts && !siteSettings.prioritize_username_in_ux && post.name ``` as well as the full name to guarantee that we are not getting any mismatches when querying for user / avatar. eg. ``` [quote="Isaac Janzen, post:3, topic:7, full:true, username:isaac.janzen"] bing bong [/quote] ```
2022-04-20 23:07:51 +08:00
let!(:post) { Fabricate(:post, user: user, topic: topic, raw: 'this is the "first" post') }
let(:raw) { <<~RAW.strip }
[quote="#{post.user.name}, post:#{post.post_number}, topic:#{topic.id}, username:#{post.user.username}"]
this is the “first” post
[/quote]
and this is the third reply
RAW
let(:raw2) { <<~RAW.strip }
and this is the third reply
[quote="#{post.user.name}, post:#{post.post_number}, topic:#{topic.id}, username:#{post.user.username}"]
this is the ”first” post
[/quote]
RAW
let(:raw3) { <<~RAW.strip }
[quote="#{post.user.name}, post:#{post.post_number}, topic:#{topic.id}, username:#{post.user.username}"]
this is the “first” post
[/quote]
[quote="#{post.user.name}, post:#{post.post_number}, topic:#{topic.id}, username:#{post.user.username}"]
this is the “first” post
[/quote]
and this is the third reply
RAW
before do
SiteSetting.remove_full_quote = true
SiteSetting.display_name_on_posts = true
SiteSetting.prioritize_username_in_ux = false
end
it "removes direct reply with full quotes" do
hidden = Fabricate(:post, topic: topic, hidden: true, raw: "this is the second post after")
small_action = Fabricate(:post, topic: topic, post_type: Post.types[:small_action])
reply = Fabricate(:post, topic: topic, raw: raw)
freeze_time do
topic.bumped_at = 1.day.ago
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
expect(topic.ordered_posts.pluck(:id)).to eq(
[post.id, hidden.id, small_action.id, reply.id],
)
expect(topic.bumped_at).to eq_time(1.day.ago)
expect(reply.raw).to eq("and this is the third reply")
expect(reply.revisions.count).to eq(1)
expect(reply.revisions.first.modifications["raw"]).to eq([raw, reply.raw])
expect(reply.revisions.first.modifications["edit_reason"][1]).to eq(
I18n.t(:removed_direct_reply_full_quotes),
)
end
end
it "does nothing if there are multiple quotes" do
reply = Fabricate(:post, topic: topic, raw: raw3)
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
expect(topic.ordered_posts.pluck(:id)).to eq([post.id, reply.id])
expect(reply.raw).to eq(raw3)
end
it "does not delete quote if not first paragraph" do
reply = Fabricate(:post, topic: topic, raw: raw2)
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
expect(topic.ordered_posts.pluck(:id)).to eq([post.id, reply.id])
expect(reply.raw).to eq(raw2)
end
it "does nothing when 'remove_full_quote' is disabled" do
SiteSetting.remove_full_quote = false
reply = Fabricate(:post, topic: topic, raw: raw)
CookedPostProcessor.new(reply).remove_full_quote_on_direct_reply
expect(reply.raw).to eq(raw)
end
it "does not generate a blank HTML document" do
post = Fabricate(:post, topic: topic, raw: "<sunday><monday>")
cp = CookedPostProcessor.new(post)
cp.post_process
expect(cp.html).to eq("<p></p>")
end
it "works only on new posts" do
Fabricate(:post, topic: topic, hidden: true, raw: "this is the second post after")
Fabricate(:post, topic: topic, post_type: Post.types[:small_action])
reply = PostCreator.create!(topic.user, topic_id: topic.id, raw: raw)
stub_image_size
CookedPostProcessor.new(reply).post_process
expect(reply.raw).to eq(raw)
PostRevisor.new(reply).revise!(
Discourse.system_user,
raw: raw,
edit_reason: "put back full quote",
)
stub_image_size
CookedPostProcessor.new(reply).post_process(new_post: true)
expect(reply.raw).to eq("and this is the third reply")
end
it "works with nested quotes" do
reply1 = Fabricate(:post, user: user, topic: topic, raw: raw)
reply2 = Fabricate(:post, topic: topic, raw: <<~RAW.strip)
[quote="#{reply1.user.name}, post:#{reply1.post_number}, topic:#{topic.id}, username:#{reply1.user.username}"]
#{raw}
[/quote]
quoting a post with a quote
RAW
CookedPostProcessor.new(reply2).remove_full_quote_on_direct_reply
expect(reply2.raw).to eq("quoting a post with a quote")
end
end
DEV: Use proper wording for contexts in specs
2022-07-28 00:14:14 +08:00
describe "prioritizes full name in quotes" do
DEV: Prioritize full name when display_name_on_posts active (#16078) See: https://meta.discourse.org/t/display-full-name-not-username-when-attributing-quote-or-reply/203533?u=isaacjanzen for context The initial release [broke quoting](https://meta.discourse.org/t/quoting-broken-when-name-matches-username/217633?u=isaacjanzen) but we now pass the username when ``` siteSettings.display_name_on_posts && !siteSettings.prioritize_username_in_ux && post.name ``` as well as the full name to guarantee that we are not getting any mismatches when querying for user / avatar. eg. ``` [quote="Isaac Janzen, post:3, topic:7, full:true, username:isaac.janzen"] bing bong [/quote] ```
2022-04-20 23:07:51 +08:00
fab!(:user) { Fabricate(:user, name: "james, john, the third") }
DEV: Allow fab! without block (#24314) The most common thing that we do with fab! is: fab!(:thing) { Fabricate(:thing) } This commit adds a shorthand for this which is just simply: fab!(:thing) i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-10 06:47:59 +08:00
fab!(:topic)
DEV: Prioritize full name when display_name_on_posts active (#16078) See: https://meta.discourse.org/t/display-full-name-not-username-when-attributing-quote-or-reply/203533?u=isaacjanzen for context The initial release [broke quoting](https://meta.discourse.org/t/quoting-broken-when-name-matches-username/217633?u=isaacjanzen) but we now pass the username when ``` siteSettings.display_name_on_posts && !siteSettings.prioritize_username_in_ux && post.name ``` as well as the full name to guarantee that we are not getting any mismatches when querying for user / avatar. eg. ``` [quote="Isaac Janzen, post:3, topic:7, full:true, username:isaac.janzen"] bing bong [/quote] ```
2022-04-20 23:07:51 +08:00
let!(:post) { Fabricate(:post, user: user, topic: topic, raw: 'this is the "first" post') }
before do
SiteSetting.display_name_on_posts = true
SiteSetting.prioritize_username_in_ux = false
end
it "maintains full name post processing" do
reply = Fabricate(:post, user: user, topic: topic, raw: <<~RAW.strip)
[quote="#{user.name}, post:#{post.id}, topic:#{topic.id}, username:#{user.username}"]
quoting a post with a quote
[/quote]
quoting a post with a quote
RAW
doc = Nokogiri::HTML5.fragment(CookedPostProcessor.new(reply).html)
expect(doc.css(".title").text).to eq("\n\n #{user.name}:")
end
end
DEV: Use `describe` for methods in specs
2022-07-27 18:21:10 +08:00
describe "#html" do
DEV: use HTML5 version of loofah (#21522) https://meta.discourse.org/t/markdown-preview-and-result-differ/263878 The result of this markdown had different results in the composer preview and the post. This is solved by updating Loofah to the latest version and using html5 fragments like our user had reported. While the change was only needed in cooked_post_processor.rb for this fix, other areas also had to be updated due to various side effects.
2023-06-20 09:49:22 +08:00
it "escapes html entities in attributes per html5" do
post = Fabricate(:post, raw: '<img alt="&<something>">')
expect(post.cook(post.raw)).to eq('<p><img alt="&amp;<something>"></p>')
expect(CookedPostProcessor.new(post).html).to eq('<p><img alt="&amp;<something>"></p>')
SECURITY: Prefer Loofah for processing cooked HTML
2021-02-24 23:14:43 +08:00
end
end
Initial release of Discourse
2013-02-06 03:16:51 +08:00
end
Reference in New Issue Copy Permalink