This reverts commit ab74a50d85.
We really want to upgrade redis, but discovered some edge cases
around failover we need to test.
Holding off on the upgrade till a bit more testing happens
Previously some local micro-benchmarks revealed it was not giving any perf
benefits.
Now that we upgraded to 2.6.5 we are seeing some segfaults.
No need to carry this dependency around anymore.
We can re-evaluate in future if it improves perf and fix the segfaults.
Bump onebox version, and add new styling
Commit, PR and Issue oneboxes are updated with a new design. Timestamps are now localized using local-dates (if installed).
Bump onebox version to include new github rendering, and add relevant CSS
Avatars are reduced in size significantly, and icons are added to easily differentiate PRs and commits. The 'Issue:' prefix is removed from issue oneboxes, to make them consistent with commits and PRs.
We preload to ensure as much memory as possible is reused from unicorn master
to various workers using copy-on-write (sidekiq, unicorn)
This migrates the preloading code into the Discourse module for easier
reuse and adds 3 notable preloading changes
1. We attempt to localize a string on each site, ensuring we warmup
the i18n
2. We preload all our templates (compiling .erb to class)
3. We warm-up our search tokenizer which uses cppjieba which is a large
memory consumer, this will only cause a warmup on CJK sites or sites with
the special site setting enabled.
Adds 2 factor authentication method via second factor security keys over [web authn](https://developer.mozilla.org/en-US/docs/Web/API/Web_Authentication_API).
Allows a user to authenticate a second factor on login, login-via-email, admin-login, and change password routes. Adds registration area within existing user second factor preferences to register multiple security keys. Supports both external (yubikey) and built-in (macOS/android fingerprint readers).
This updates rubyzip library so that callers can trust entries when
extracting files avoiding situations where a rogues zip imported by a rogue
admin could cause a disk space issue.
This corrects an XSS in ?pp=help.
Also removes the jQuery dependency from rack-mini-profiler and restricts
memory sensitive profiling methods development only.
* Adjustments to pass specs on Rails 6.0.0
* Use classic autoloader instead of Zeitwerk
* Update Rails 6.0.0 deprecated methods
* Rails 6.0.0 not allowing column with integer name
* Drop freedom_patches/rails6.rb
* Default value for trigger_transactional_callbacks? is true
* Bump rspec-rails version to 4.0.0.beta2
New version of mini scheduler allows you to select the name of a schedule
in the history page in `/sidekiq/scheduler/history`.
This is handy for quickly looking up timing trends.
This commit introduces 2 features:
1. DISCOURSE_COMPRESS_ANON_CACHE (true|false, default false): this allows
you to optionally compress the anon cache body entries in Redis, can be
useful for high load sites with Redis that lives on a separate server to
to webs
2. DISCOURSE_ANON_CACHE_STORE_THRESHOLD (default 2), only pop entries into
redis if we observe them more than N times. This avoids situations where
a crawler can walk a big pile of topics and store them all in Redis never
to be used. Our default anon cache time for topics is only 60 seconds. Anon
cache is in place to avoid the "slashdot" effect where a single topic is
hit by 100s of people in one minute.
Prior to this commit if we ever checked in a per_host directive and scheduled
job was previously global, total chaos could ensue as a feedback loop would
start
We very rarely used per_host to date, but just started making use of it
for heartbeats
Under extreme load on large databases certain regular jobs can take quite
a while to run. We need to ensure we never starve a sidekiq from running
mini scheduler, cause without it we are unable to queue stuff such as
heartbeat jobs.
This adds a 1 minute rate limit to all JS error reporting per IP. Previously
we would only use the global rate limit.
This also introduces DISCOURSE_ENABLE_JS_ERROR_REPORTING, if it is set to
false then no JS error reporting will be allowed on the site.
The message_bus performs a fair amount of work prior to hijacking requests
this change ensures that if there is a situation where the server is flooded
message_bus will inform client to back off for 30 seconds + random(120 secs)
This back-off is ultra cheap and happens very early in the middleware.
It corrects a situation where a flood to message bus could cause the app
to become unresponsive
MessageBus update is here to ensure message_bus gem properly respects
Retry-After header and status 429.
Under normal state this code should never trigger, to disable raise the
value of DISCOURSE_REJECT_MESSAGE_BUS_QUEUE_SECONDS, default is to tell
message bus to go away if we are queueing for 100ms or longer
This feature adds the ability to customize the HTML part of all emails using a custom HTML template and optionally some CSS to style it. The CSS will be parsed and converted into inline styles because CSS is poorly supported by email clients. When writing the custom HTML and CSS, be aware of what email clients support. Keep customizations very simple.
Customizations can be added and edited in Admin > Customize > Email Style.
Since the summary email is already heavily styled, there is a setting to disable custom styles for summary emails called "apply custom styles to digest" found in Admin > Settings > Email.
As part of this work, RTL locales are now rendered correctly for all emails.
* Revert "Revert "FEATURE: admin/user exports are compressed using the zip format (#7784)""
This reverts commit f89bd55576.
* Replace .tar.zip with .zip
* FEATURE: admin/user exports are compressed using the zip format
* Update translations. Theme exporter now exports .zip file. Theme importer supports .zip and .gz files
* Fix controller test, updated locale and skip saving the csv export to disk
Overcommit uses prebuilt hooks and require global installation.
To avoid this issues replace it with Lefthook.
Lefthook will be installed with npm packages. New contributors
will have fully consistent git hooks.
* Updated test-prof
* Made rails_helper.rb use new test-prof APIs
Instead of the previous temporary hacks.
* Added environment option to disable prefabrication
It was removed mistakenly
We recently noticed a leak, this introduces a new method on MiniRacer::Context
```
context.write_heap_snapshot(path)
```
To dump current memory in v8 context to a file, this can then be analyzed
in chrome and other similar tools
Can be triggered in production using rbtrace
AWS is a big moving target, this fills gaps in the API. Technically we
use such a tiny surface area that it probably does not matter, but it is
good to be up to date here.
This updates another batch of gems that are lowish risk
Most of the gem changes are here for Rails 6 / Ruby 2.6.3 support
Excon did some stuff around better cipher ordering
- Update github_blob engine to support displaying stl files
- FEATURE: add `data-original-href` attribute to Vimeo iframes
- Add poster image for video oneboxes
Adds the parallel_tests gem, and redis/postgres configuration for running rspec tests in parallel. To use:
```
rake parallel:rake[db:create]
rake parallel:rake[db:migrate]
rake parallel:spec
```
This brings the test suite from 12m20s to 3m11s on my macOS machine
- Fixes deprecation regarding usage of BigDecimal in dev
- Handle edge case where query_hash would clear a non existent result
- Minor perf improvement to query_single
Most important thing though is that we are now on the latest gem
The only reason this was done was to remove this deprecation when running
tests.
```
/home/sam/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/mini_sql-0.1.10/lib/mini_sql/coders.rb:5: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
```
Ruby 2.5.3 has an upatched issue that crashes unicorn after fork:
https://bugs.ruby-lang.org/issues/14634
This will be patched in 2.5.4 however for now just warn people dev is slower
and disable async logging on the older rubies
Another group of gems updated, better_errors is fairly safe as its for dev
chunky_png fairly safe, used for getting image info from pngs, erubi update
is very safe. Sanitize is used by onebox and should always be on latest.
Long term we should think of making sanitize an optional dependency on onebox
cause we are happy to just provide methods from core to do this and it would
remove nokogumbo and sanitize deps.
This follows up on 03376224 which reverted the redis update which is not
supported, rest of the gems should be fine.(unicorn / nokogiri / sidekiq /
pg)
This updates a few more lower risk gems, the main goal here is to have
nothing outdated. Avoiding a giant commit is going to make it slightly
easier to partially roll back if something goes wrong
Two very low risk updates, message_bus has been released no need to depend
on pre-release.
mini_racer update is for a very minor change (shared isolates are not used
in discourse so it is not a fix we technically need)
Launching Discourse no longer should require foreman in dev. We can simply
use `bin/unicorn` which automatically launches a sidekiq worker.
The foreman gem depends on thor ~> 0.19.1 which is no longer supported in
rails 6. So this pre-emptively prepares us for it.
This temporary revert is here cause a certain immune system in logster
when a large number of events are groupes is no longer working in
production.
What this means is that if you have 10 thousand errors grouped it can
impose an very large amount of load on redis.
This reverts commit abee39ecd0.
Fixes linear-gradient CSS issue on iOS
Increases z-index for the actions menu to prevent the screen divider/splitter from obstructing the menu
e43e98dc61
This provides us with instrumentation missing after rails upgrade
Latest version of rails uses exec_params internally which is no longer
routed to intercepted methods in mini profiler 1.0.0
Message bus uses a key to keep track of the last id for each channel
this key was never expired even if channel data expired
This change ensures we also expire the tracking key, it means a lot to us
cause each topic has a channel, so if you have 1 million topics that is
1 million keys that may persist forever
This release contains security fixes to the underlying rack library
used by Discourse.
Impact is not too high as we do not use request.scheme in our templates
* FEATURE: Added MaxMindDb to resolve IP information.
* FEATURE: Added browser detection based on user agent.
* FEATURE: Added recently used devices in user preferences.
* DEV: Added acceptance test for recently used devices.
* UX: Do not show 'Show more' button if there aren't more tokens.
* DEV: Fix unit tests.
* DEV: Make changes after code review.
* Add more detailed unit tests.
* Improve logging messages.
* Minor coding style fixes.
* DEV: Use DropdownSelectBoxComponent and run Prettier.
* DEV: Fix unit tests.
