Commit Graph

843 Commits

Author SHA1 Message Date
Sam
e456ae1b6c Revert "Upgrade mail gem to drop dependency on mime-types."
This reverts commit 498cc15aa9.

Noticed a certain amount of extra allocations due to this upgrade,
reverting to confirm if this is the case
2018-02-16 07:04:22 +11:00
Guo Xiang Tan
498cc15aa9 Upgrade mail gem to drop dependency on mime-types.
Boot Memory Usage

```
Total allocated: 225695379 bytes (1990663 objects)
Total retained:  32795116 bytes (291607 objects)
```

```
Total allocated: 218342892 bytes (1897350 objects)
Total retained:  30123426 bytes (261198 objects)
```
2018-02-15 16:24:26 +08:00
Sam
94b2c70c0d PERF: remove oga gem
oga gem is automatically required by the aws gem
the oga gem retains about 1mb of memory, aws now uses nokogiri

This also removes the html normalize from the pretty text specs that was
a fair bit buggy as the polls test shows.
2018-02-15 14:36:40 +11:00
Sam
c89b42c488 PERF: only require the rss library if used
Before:

Total allocated: 257909321 bytes (2514134 objects)
Total retained:  39681579 bytes (343387 objects)

allocated memory by gem
-----------------------------------
  42875979  rss

retained memory by gem
-----------------------------------
   2080188  rss

retained objects by gem
-----------------------------------
     13052  rss

After:

Total allocated: 210562047 bytes (2252030 objects)
Total retained:  37433816 bytes (328635 objects)

----

So, 2 less megabytes on boot and 13000 objects stuck in ruby heaps forever.
2018-02-15 13:11:33 +11:00
Rafael dos Santos Silva
218c8e8623
Merge pull request #5595 from discourse/new-gfycat-onebox
New gfycat onebox
2018-02-14 19:03:28 -02:00
Rafael dos Santos Silva
c3606fdef6 Update onebox 2018-02-14 17:43:13 -02:00
Robin Ward
0b13b82e94 Update nokogiri to latest version 2018-02-13 19:36:19 -05:00
Guo Xiang Tan
19b4116416 Bump rails_multisite to 2.0.4. 2018-02-12 15:39:21 +08:00
Guo Xiang Tan
5dabcc9081 Fix Gemfile.lock. 2018-02-12 14:08:02 +08:00
Guo Xiang Tan
87b0cc382b Bump rails_multisite to 2.0.3. 2018-02-12 14:03:04 +08:00
Régis Hanol
508a8e29ab update email_reply_trimmer to latest version 2018-02-06 01:04:45 +01:00
Sam
073d072aa6 correct lockfile 2018-01-29 18:28:59 +11:00
Sam
d2025e33cf FEATURE: bump rbtrace to latest 2018-01-23 15:36:47 +11:00
Sam
36ed8f5176 Correct regression where /sidekiq/scheduler was not working 2018-01-22 10:48:35 +11:00
Guo Xiang Tan
54dc191a91 Update rails_multisite to 2.0.1. 2018-01-19 10:19:16 +08:00
Sam
12872d03be PERF: run post timings in background
This means that if a very large amount of registered users hit
a single topic we will handle it gracefully, even if db gets slow.
2018-01-19 08:27:29 +11:00
Gerhard Schlager
5d7a33cd6d Update email_reply_trimmer to improve performance 2018-01-17 12:04:56 +01:00
Sam
8c47eb2951 update coderay 2018-01-17 15:50:41 +11:00
Arpit Jalan
a0f0ed576a bump onebox version 2018-01-11 23:15:25 +05:30
Arpit Jalan
afb18a5c41 bump onebox version 2018-01-11 14:30:35 +05:30
Sam
d5fd6d2639 PERF: defer message bus on hidden tabs so only one in 1.5 seconds happens 2018-01-09 12:39:17 +11:00
Guo Xiang Tan
74ec100aff Upgrade pg gem. 2018-01-08 10:27:19 +08:00
Régis Hanol
e3f8182125 FIX: Google Calendar oneboxes weren't working 2018-01-07 19:15:11 +01:00
Sam
8d4cd57662 Update libv8 from 5.9 to 6.3 2018-01-05 17:12:04 +11:00
Jano Suchal
05ec1be101 Drop the fork usage of fastimage (#5461)
Fork is not needed anymore since fastimage 2.1.1 has been released with the needed PR merged in.
2018-01-04 09:53:53 +01:00
Sam
030a0cd698 bump onebox for vimeo private link support 2017-12-29 09:38:25 +11:00
Arpit Jalan
b74e933cfb bump onebox version 2017-12-23 01:17:30 +05:30
Arpit Jalan
16076f9ab8 bump onebox version 2017-12-21 14:49:59 +05:30
Sam
fd67508497 FIX: don't count users as replying long after they are done typing
Also, don't leak a large amount of redis usage for presence stuff
2017-12-18 15:41:32 +11:00
Arpit Jalan
f9990eeb13 bump onebox version 2017-12-07 12:05:19 +05:30
Kyle Zhao
5f318a5241 FEATURE: Replace SimpleRSS with Ruby RSS module (#5311)
* SPEC: PollFeedJob parsing atom feed

* add FeedItemAccessor

It is to provide a consistent interface to access a feed item's tag
content.

* add FeedElementInstaller

to install non-standard and non-namespaced feed elements

* FEATURE: replace SimpleRSS with Ruby RSS module

* get FinalDestination and download with Excon

* support namespaced element with FeedElementInstaller
2017-12-06 10:45:09 +11:00
Arpit Jalan
7bbf137238 bump onebox version 2017-12-05 07:28:45 +05:30
Guo Xiang Tan
f5d45f797a Fix broken Gemfile.lock. 2017-12-04 11:28:45 +08:00
Guo Xiang Tan
78f09f0489 Logster 1.2.9 is the version we want to be at. 2017-12-04 11:05:14 +08:00
Guo Xiang Tan
846b047847 Revert "Revert "Upgrade logster.""
This reverts commit 57ca4b20e2.
2017-12-04 09:30:21 +08:00
Guo Xiang Tan
57ca4b20e2 Revert "Upgrade logster."
This reverts commit 1376a86bcc.
2017-12-01 17:32:49 +08:00
Guo Xiang Tan
1376a86bcc Upgrade logster. 2017-12-01 16:03:00 +08:00
Guo Xiang Tan
3053499375 Update mini_suffix. 2017-11-30 11:14:06 +08:00
Guo Xiang Tan
fffb7561c4 Update mini_suffix. 2017-11-29 17:07:41 +08:00
Guo Xiang Tan
8491c5fba5 UX: Replace heuristic solution for extracting root domain. 2017-11-29 16:26:46 +08:00
Arpit Jalan
64cdb4f7e8 bump onebox version 2017-11-27 12:21:47 +05:30
Guo Xiang Tan
c8157f6b07 Update seed-fu with PG10 support.
https://meta.discourse.org/t/discourse-appears-to-be-broken-with-postgres-10/71723
2017-11-27 10:26:59 +08:00
Arpit Jalan
abbb2b28eb bump onebox version 2017-11-25 15:53:18 +05:30
Sam
407d80f1dd FEATURE: update rack-mini-profiler to latest 2017-11-24 10:28:35 +11:00
Sam
8fd09fe616 FEATURE: upgrade message bus
new version recovers very cleanly from redis flush
2017-11-24 09:23:12 +11:00
Arpit Jalan
961a09ff14 bump onebox version 2017-11-23 17:45:08 +05:30
Arpit Jalan
211dac6f71 bump onebox version 2017-11-22 14:55:47 +05:30
Arpit Jalan
b8c9975429 bump onebox version 2017-11-21 11:10:44 +05:30
Guo Xiang Tan
89d9ffa884 EXPERIMENTAL: Allow logs to be shipped via different methods.
See https://github.com/dwbutler/logstash-logger#uri-configuration for
a list of available methods.
2017-11-15 09:11:33 +08:00
Sam
5427ca13b7 FEATURE: update rails multisite so we error out if RAILS_DB is invalid 2017-11-13 16:37:25 +11:00
Sam
86e6732f78 FEATURE: update rails multisite
always allow /srv/status through even if host does not match
2017-11-09 15:40:56 +11:00
Sam
fc7dca58fe UX: oneboxes with avatars now display consistently
Onebox avatar size is reduced to 60px

Also fixes regression with some oneboxes not cooking after post
2017-11-08 11:50:01 +11:00
Guo Xiang Tan
a97273e1a5 Update Unicorn to 5.3.1. 2017-11-06 11:22:58 +08:00
Arpit Jalan
253de41166 bump onebox gem version 2017-11-03 22:38:36 +05:30
Arpit Jalan
91438849d2 bump onebox version 2017-11-02 00:51:21 +05:30
Arpit Jalan
6b5bb9d664 bump onebox gem version 2017-11-01 15:50:23 +05:30
Guo Xiang Tan
d888d3c54c EXPERIMENTAL: Allow Logstash formatter to be enable for lograge logs. 2017-11-01 08:38:06 +08:00
Guo Xiang Tan
58bb3c14f6 Remove gem that is no longer used. 2017-10-31 14:13:54 +08:00
Guo Xiang Tan
d487be759c EXPERIMENTAL: Allow app to Logstash directly. 2017-10-27 22:56:32 +08:00
Guo Xiang Tan
90d6677d97 EXPERIMENTAL: Allow lograge to be selected as the logging library. 2017-10-27 17:54:45 +08:00
Guo Xiang Tan
db579f7917 Upgrade logster. 2017-10-27 12:13:52 +08:00
Guo Xiang Tan
09e7c1ce8a Update Rubocop. 2017-10-23 12:03:31 +08:00
Guo Xiang Tan
a88f910469 Bump message_bus to 2.0.8. 2017-10-06 08:33:46 +08:00
Guo Xiang Tan
a25851032a Update Sidekiq. 2017-10-05 13:59:21 +08:00
Sam
ac01885b60 FEATURE: rake tasks for uploading assets to S3
This opens the door to serving application.js and so on from s3.

Also updates s3 gem for some tagging support
2017-10-03 18:00:53 +11:00
Guo Xiang Tan
f1d8ed6aaf Update lock file. 2017-10-03 14:59:25 +08:00
Guo Xiang Tan
7e059a5a6e Upgrade Rails to 5.1.4. 2017-10-03 14:56:44 +08:00
Sam
9fa575dca1 Update message bus
This corrects a rare race condition.
2017-10-02 15:21:45 +11:00
Sam
41261b32a5 FIX: update message bus
- Corrects broken short polling
- Corrects after fork
2017-09-29 16:58:12 +10:00
Guo Xiang Tan
d442da9fdd SECURITY: Update Nokogiri. 2017-09-25 20:06:58 +08:00
Guo Xiang Tan
77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan
898ee93547 Upgrade to Rails 5. 2017-09-25 10:01:22 +08:00
Guo Xiang Tan
d58de2b484 Install Ruby-Prof. 2017-09-25 10:01:22 +08:00
Gerhard Schlager
0960e2dd67 bump onebox version 2017-09-18 14:25:56 +02:00
Gerhard Schlager
619864bc70 update email_reply_trimmer to latest 2017-09-14 17:20:55 +02:00
Arpit Jalan
eafab41a2c bump onebox version 2017-08-17 13:10:11 +05:30
Sam
91f0f76fb1 update message bus to ruby 2.0 packaged version 2017-08-16 15:05:39 -04:00
Sam Saffron
d27e81a296 Perf: update message_bus to latest 2017-08-16 07:50:12 -04:00
Bianca Nenciu
1c079bec33 Remove Spork dependency. 2017-08-10 22:54:52 +01:00
Régis Hanol
9aa49e922c let's use the latest omniauth-github gem 2017-08-07 10:43:54 +02:00
Régis Hanol
ad20f59897 FIX: couldn't upload SVG with a DOCTYPE 2017-08-07 10:32:35 +02:00
Arpit Jalan
447e4386ee bump onebox version 2017-08-03 13:37:22 +05:30
Guo Xiang Tan
54f09ca59d Install rubocop in development mode. 2017-08-01 10:51:44 +09:00
Erick Guan
f1eccd113c Replace rmmseg gem for cppjieba_rb since better dictionary (#5006)
* Rename locale to ts config in search module to make it clear

* Replace rmmese-cpp for cppjieba_rb
2017-07-31 15:28:48 -04:00
Régis Hanol
cae437850e update email_reply_trimmer to latest 2017-07-31 11:21:41 +02:00
Guo Xiang Tan
358d245dd8 Upgrade Rails to 4.2.9. 2017-07-31 15:37:46 +09:00
Sam
9d774a951a Update libv8 and benchmark 2017-07-26 10:59:41 -04:00
Sam Saffron
045a2abcec FEATURE: remove the timecop gem
We should only have one way of mocking time, misuse of timecop
was causing build stability issues
2017-07-24 12:11:10 -04:00
Arpit Jalan
eb6c5a54ea bump onebox version 2017-07-20 22:31:01 +05:30
Sam
38daa61fc7 FEATURE: libv8 upgraded to version 5.7
corresponds more or less with Chrome 57
2017-07-18 10:55:51 -04:00
Sam Saffron
d0c5205a52 Feature: Change markdown engine to markdown it
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
2017-07-17 11:41:34 -04:00
Robin Ward
c786700817 Upgrade onebox for a simple video fix 2017-07-10 14:20:07 -04:00
Robin Ward
7906c9ce29 Update onebox gem for Trello support 2017-07-07 10:45:22 -04:00
Robin Ward
2f480e21a0 Stop using Babel source gem, which is no longer updated 2017-07-05 14:19:16 -04:00
Sam
234694b50f Feature: CommonMark support
This adds the markdown.it engine to Discourse.
https://github.com/markdown-it/markdown-it

As the migration is going to take a while the new engine is default
disabled. To enable it you must change the hidden site setting:
enable_experimental_markdown_it.

This commit is a squash of many other commits, it also includes some
improvements to autospec (ability to run plugins), and a dev dependency
on the og gem for html normalization.
2017-06-23 12:01:33 -04:00
Robin Ward
94ab194a7f Adds support for Github oneboxes to commits in pull requests 2017-06-22 15:46:07 -04:00
Sam
2181e995f1 remove sinatra 2017-06-19 13:49:11 -04:00
Guo Xiang Tan
be44e4b730 Update Sidekiq. 2017-06-19 14:20:54 +09:00
Guo Xiang Tan
d82dbd565b FIX: Remove dependency on rest-client. 2017-06-16 09:42:41 +09:00
Robin Ward
8ae445766f Upgrade Ember to 2.13 2017-06-15 10:12:07 -04:00
Régis Hanol
5d63a7f4a6 FIX: pull hotlinked images even when they have no extension 2017-06-13 13:27:05 +02:00
Guo Xiang Tan
3acc9941dd Merge pull request #4920 from greysteil/dependabot/bundler/mail-2.6.6
Bump mail from 2.6.6.rc1 to 2.6.6
2017-06-12 18:00:08 +09:00
dependabot[bot]
d43b46d09b Bump mail from 2.6.6.rc1 to 2.6.6
Bumps [mail](https://github.com/mikel/mail) from 2.6.6.rc1 to 2.6.6.
- [Changelog](https://github.com/mikel/mail/blob/master/CHANGELOG.rdoc)
- [Commits](https://github.com/mikel/mail/commits/v2.6.6)
2017-06-10 07:03:22 +00:00
Robin Ward
b5bead1fe3 FIX: Don't use target=_blank for local oneboxes 2017-06-09 12:15:36 -04:00
dependabot[bot]
63eb298089 Bump puma from 3.6.0 to 3.9.1
Bumps [puma](https://github.com/puma/puma) from 3.6.0 to 3.9.1.
- [Release notes](https://github.com/puma/puma/releases/tag/v3.9.1)
- [Changelog](https://github.com/puma/puma/blob/master/History.md)
- [Commits](https://github.com/puma/puma/compare/v3.6.0...v3.9.1)
2017-06-08 14:37:02 +00:00
Guo Xiang Tan
ff1154b523 Update Sidekiq to 5.0.x. 2017-06-08 07:15:13 +09:00
Guo Xiang Tan
22ecbf1249 More gem updates. 2017-06-08 07:14:22 +09:00
Guo Xiang Tan
61a4711e83 Update i18n. 2017-06-07 11:13:21 +09:00
Guo Xiang Tan
24cf302753 Update nokogiri. 2017-06-07 11:13:07 +09:00
Guo Xiang Tan
5332f25aa9 Update gems in development group. 2017-06-07 11:12:44 +09:00
Robin Ward
369bb78f8e FIX: Support for cookies in onebox redirects 2017-06-06 15:02:11 -04:00
Robin Ward
4b82d4fb15 Upgrade Ember to latest patch 2017-06-05 14:39:04 -04:00
Robin Ward
3acabf3e2f FIX: Improved Amazon oneboxing for some URLs 2017-06-02 16:53:40 -04:00
Robin Ward
c4b6420455 Update Onebox for better Twitch support 2017-06-01 16:23:47 -04:00
Robin Ward
19d5eb903c SECURITY: Vunerability in mail gem
(see https://github.com/mikel/mail/pull/1097)
2017-06-01 14:51:30 -04:00
Sam
3384a7dd2a proper bootsnap release 2017-05-26 13:26:45 -04:00
Sam
076f079d17 bootsnap upgrade
see: https://github.com/Shopify/bootsnap/pull/43#issuecomment-304281474
2017-05-26 09:37:18 -04:00
Guo Xiang Tan
56f98de7b2 Use webmock to stub external web requests. 2017-05-26 15:19:09 +08:00
Robin Ward
8bc93c0b01 FEATURE: Add custom User-Agent to Discourse Oneboxes 2017-05-24 12:19:36 -04:00
Robin Ward
29755be139 Remove bundle-audit, it is not updated frequently enough 2017-05-18 12:31:19 -04:00
Sam
851876acfa update miniprofiler to clean up errors 2017-05-17 15:35:18 -04:00
Robin Ward
1870089cf1 Include latest onebox fix 2017-05-17 15:21:40 -04:00
Sam
17f75bc95e FIX: bump mini profiler to support sub folders 2017-05-17 13:51:31 -04:00
Sam
bdd4a9e69d update nokogiri 2017-05-17 11:33:20 -04:00
Robin Ward
e865a44b93 FIX: Forgot to lock gem 2017-05-15 13:35:28 -04:00
Guo Xiang Tan
d5f823dae5 Fix the build on travis. 2017-05-15 19:48:14 +08:00
Robin Ward
49a30a7532 FIX: Include new onebox fixes 2017-05-12 14:25:15 -04:00
Sam
961dc0480d update bootsnap 2017-05-12 09:45:28 -04:00
Sam
704101253d add bootsnap to speed up test and dev 2017-05-11 13:51:38 -04:00
Régis Hanol
9641d2413d REFACTOR: upload workflow creation into UploadCreator
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
2017-05-11 00:16:57 +02:00
Guo Xiang Tan
a470bce8fb Update mail gem to 2.6.5 for a slightly smaller memory footprint. 2017-05-03 17:02:47 +08:00
Guo Xiang Tan
8d04afc01b Update OJ. 2017-05-03 16:52:02 +08:00
Guo Xiang Tan
862052fca8 Update rack-mini-profiler. 2017-05-03 16:52:02 +08:00
Guo Xiang Tan
f81d39114d Update RSpec's dependency to fix deprecation warning. 2017-04-26 16:21:51 +08:00
Guo Xiang Tan
b00886f5c1 Remove sidekiq-statistic gem.
* We don't really use it and there is an on-going issue
  with the gem not expiring keys in a Redis list which
  hasn't been fully resolved.

  https://github.com/davydovanton/sidekiq-statistic/issues/73
2017-04-26 14:54:47 +08:00
Sam Saffron
587653ad73 remove bootsnap till we sort out linux 2017-04-25 14:55:18 -07:00
Sam Saffron
1ae67a21f5 FEATURE: add support for bootsnap in dev
To use bootsnap which speeds up rails boot add

BOOTSNAP=1 to your env
2017-04-25 10:15:12 -07:00
Sam
0b3aec9c94 FEATURE: set UNICORN_STATS_SOCKET_DIR for status socket
eg:

sam@ubuntu stats_sockets % socat - UNIX-CONNECT:9622.sock
gc_stat
{"count":46,"heap_allocated_pages":2459,"heap_sorted_length":2460,"heap_allocatable_pages":0,"heap_available_slots":1002267,"heap_live_slots":647293,"heap_free_slots":354974,"heap_final_slots":0,"heap_marked_slots":503494,"heap_swept_slots":498773,"heap_eden_pages":2459,"heap_tomb_pages":0,"total_allocated_pages":2459,"total_freed_pages":0,"total_allocated_objects":4337014,"total_freed_objects":3689721,"malloc_increase_bytes":6448248,"malloc_increase_bytes_limit":29188387,"minor_gc_count":36,"major_gc_count":10,"remembered_wb_unprotected_objects":19958,"remembered_wb_unprotected_objects_limit":39842,"old_objects":462019,"old_objects_limit":895782,"oldmalloc_increase_bytes":6448696,"oldmalloc_increase_bytes_limit":19350882}
2017-04-21 11:37:03 -04:00
Guo Xiang Tan
d7b446c174 Install discourse_image_optim gem for now. 2017-04-20 15:14:39 +08:00
Sam
6e5296a510 FEATURE: upgrade sprockets to latest stable
This unlocks Rails upgrades, so we can now upgrade to latest Rails
2017-04-17 10:12:05 -04:00
Guo Xiang Tan
04016f0dec Support Ruby 2.4. 2017-04-15 12:29:00 +08:00
Guo Xiang Tan
86efc57390 FIX: Onebox's dependencies were not updated. 2017-04-15 11:39:51 +08:00
Sam
22214f5e5c Upgrade the Listen gem 2017-04-13 10:39:36 -04:00
Guo Xiang Tan
f48022b674 Upgrade Rails to 4.2.8. 2017-04-13 10:17:09 +08:00
Régis Hanol
a5f1c57a70 bump onebox 2017-04-13 00:49:22 +02:00
Guo Xiang Tan
0165b22b76 Bump onebox. 2017-04-13 06:31:25 +08:00
Sam
a3e8c3cd7b FEATURE: Native theme support
This feature introduces the concept of themes. Themes are an evolution
of site customizations.

Themes introduce two very big conceptual changes:

- A theme may include other "child themes", children can include grand
children and so on.

- A theme may specify a color scheme

The change does away with the idea of "enabled" color schemes.

It also adds a bunch of big niceties like

- You can source a theme from a git repo

- History for themes is much improved

- You can only have a single enabled theme. Themes can be selected by
    users, if you opt for it.

On a technical level this change comes with a whole bunch of goodies

- All CSS is now compiled using a custom pipeline that uses libsass
    see /lib/stylesheet

- There is a single pipeline for css compilation (in the past we used
    one for customizations and another one for the rest of the app

- The stylesheet pipeline is now divorced of sprockets, there is no
   reliance on sprockets for CSS bundling

- CSS is generated with source maps everywhere (including themes) this
    makes debugging much easier

- Our "live reloader" is smarter and avoid a flash of unstyled content
   we run a file watcher in "puma" in dev so you no longer need to run
   rake autospec to watch for CSS changes
2017-04-12 10:53:49 -04:00
Guo Xiang Tan
4cb6880766 Upgrade image_optim. 2017-04-12 10:38:17 -04:00
Robin Ward
28da0549b6 Update facebook login gem 2017-03-27 16:52:09 -04:00
Guo Xiang Tan
9f299b6842 Use after_commit since after_save runs within the transaction as well. 2017-03-16 16:02:34 +08:00