github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-12-14 15:13:44 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
104ca5c325
discourse/spec/lib/onebox
History
Alan Guo Xiang Tan c8da2a33e8
FIX: Attempt to onebox even if response body exceeds max_download_kb (#26929) 
In 95a82d608d, we lowered the default for
`Onebox.options.max_download_kb` from 10mb to 2mb for security hardening
purposes. However, this resulted in multiple bug reports where seemingly
nomral URLs stopped being oneboxed. It turns out that lowering
`Onebox.options.max_download_kb` resulted in `Onebox::Helpers::DownloadTooLarge` being raised
more often for more URLs  in `Onebox::Helpers.fetch_response` which
`Onebox::Helpers.fetch_html_doc` relies on. When
`Onebox::Helpers::DownloadTooLarge` is raised in
`Onebox::Helpers.fetch_response`, we throw away whatever response body
which we have already downloaded at that point. This is not ideal
because Nokogiri can parse incomplete HTML documents and there is a
really high chance that the incomplete HTML document still contains the
information which we need for oneboxing.

Therefore, this commit updates `Onebox::Helpers.fetch_html_doc` to not
throw away the response body when the size of the response body exceeds
`Onebox.options.max_download_size`. Instead, we just take whatever
response which we have and get Nokogiri to parse it.
2024-05-09 07:00:34 +08:00
..
engine FEATURE: Add onebox for loom (#26016) 2024-03-04 15:12:08 -07:00
domain_checker_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
engine_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
helpers_spec.rb FIX: Attempt to onebox even if response body exceeds max_download_kb (#26929) 2024-05-09 07:00:34 +08:00
json_ld_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
layout_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
matcher_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
movie_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
oembed_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
open_graph_spec.rb FEATURE: Show more context in Discourse topic oneboxes 2023-01-11 14:22:53 +01:00
preview_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
status_check_spec.rb DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00