Commit Graph

15 Commits

Author SHA1 Message Date
Bianca Nenciu
a835fd99bd FIX: Truncate bookmarks.name when remapping
The new name may be too long for the bookmarks.name column and raise an
exception. This changes allows the remapper to truncate the new value to
fit (truncates to 100 characters).
2024-12-11 18:53:17 -05:00
Selase Krakani
a20b7fa83f
DEV: Gracefully handle regex_replace max column length violations ()
* DEV: Gracefully handle `regex_replace` violations of column length constraints

This is a follow-up to the `remap` [refactor](9b0cfa99c5).
Similar to `remap`, the entire `regex_replace` operation fails if the new content exceeds the column’s max length.

This change introduces an optional mode, controlled by the new `skip_max_length_violations` param
to skip records eligible for `regex_replace`  where the new content violates the max column length constraint.

It also includes updates to the exception message raised when `regex_replace` fails to include more details

* DEV: Remove string escapes in heredoc text
2024-11-25 11:39:53 +00:00
Selase Krakani
9b0cfa99c5
DEV: Gracefully handle remaps which violate DB column constraints ()
* DEV: Gracefully handle remaps which violate DB column constraints

This change implements length constraint enforcement to skip remaps
which exceed column max lengths

* DEV: Only perform skipped column stats lookup when verbose is true

* DEV: Tidy up specs

* DEV: Make skipping violating remap behaviour opt-in

This change introduces a new `skip_max_length_violations` param for
`remap`, set to `false` by default to ensure we still continue to fail
hard when max lenth constraints are violated.

To aid in quick resolution when remaps fail, this change also
adds more context to the exception message to include the offending table
and column information

* Apply suggestions from code review

Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>

* FIX: Various fixes

- Linter errors
- Remap status "logger" early return condition

---------

Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2024-11-15 10:42:25 +00:00
David Taylor
cb932d6ee1
DEV: Apply syntax_tree formatting to spec/* 2023-01-09 11:49:28 +00:00
David Taylor
c9dab6fd08
DEV: Automatically require 'rails_helper' in all specs ()
It's very easy to forget to add `require 'rails_helper'` at the top of every core/plugin spec file, and omissions can cause some very confusing/sporadic errors.

By setting this flag in `.rspec`, we can remove the need for `require 'rails_helper'` entirely.
2022-03-01 17:50:50 +00:00
David Taylor
03818e642a
FEATURE: Include optimized thumbnails for topics ()
This introduces new APIs for obtaining optimized thumbnails for topics. There are a few building blocks required for this:

- Introduces new `image_upload_id` columns on the `posts` and `topics` table. This replaces the old `image_url` column, which means that thumbnails are now restricted to uploads. Hotlinked thumbnails are no longer possible. In normal use (with pull_hotlinked_images enabled), this has no noticeable impact

- A migration attempts to match existing urls to upload records. If a match cannot be found then the posts will be queued for rebake

- Optimized thumbnails are generated during post_process_cooked. If thumbnails are missing when serializing a topic list, then a sidekiq job is queued

- Topic lists and topics now include a `thumbnails` key, which includes all the available images:
   ```
   "thumbnails": [
   {
     "max_width": null,
     "max_height": null,
     "url": "//example.com/original-image.png",
     "width": 1380,
     "height": 1840
   },
   {
     "max_width": 1024,
     "max_height": 1024,
     "url": "//example.com/optimized-image.png",
     "width": 768,
     "height": 1024
   }
   ]
  ```

- Themes can request additional thumbnail sizes by using a modifier in their `about.json` file:
   ```
    "modifiers": {
      "topic_thumbnail_sizes": [
        [200, 200],
        [800, 800]
      ],
      ...
  ```
  Remember that these are generated asynchronously, so your theme should include logic to fallback to other available thumbnails if your requested size has not yet been generated

- Two new raw plugin outlets are introduced, to improve the customisability of the topic list. `topic-list-before-columns` and `topic-list-before-link`
2020-05-05 09:07:50 +01:00
Krzysztof Kotlarek
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk ()
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
Régis Hanol
b5c8f5f900 SPEC: ensure never remap readonly columns
Follow-up to bfcbfd78
2019-05-09 18:01:35 +02:00
Sam Saffron
4ea21fa2d0 DEV: use #frozen_string_literal: true on all spec
This change both speeds up specs (less strings to allocate) and helps catch
cases where methods in Discourse are mutating inputs.

Overall we will be migrating everything to use #frozen_string_literal: true
it will take a while, but this is the first and safest move in this direction
2019-04-30 10:27:42 +10:00
Guo Xiang Tan
19c3c25db1 FIX: Handle BBCode in migrate_to_s3 task as well. 2019-03-22 16:47:06 +08:00
Régis Hanol
5381096bfd PERF: new 'migrate_to_s3' rake task 2018-12-26 17:34:49 +01:00
Guo Xiang Tan
57f92ac808 Revert "Swtich to regexp for DbHelper.remap."
Regexp is so much slower.

This reverts commit c3f89e3cd7.
2018-11-08 14:20:09 +08:00
Guo Xiang Tan
c3f89e3cd7 Swtich to regexp for DbHelper.remap. 2018-11-08 14:08:38 +08:00
Guo Xiang Tan
9737938a4a Add option to skip tabels when using DbHelper.remap. 2018-11-08 12:29:37 +08:00
Guo Xiang Tan
3365753bd0 PERF: Reduce number of database queries for DbHelper.remap
* Cuts number of queries from 273 to 89
* Add some specs
* For a table with 500 posts, benchmarks locally shows a runtime
  reduction from 0.046929135 to 0.032694705.
2018-11-08 10:54:39 +08:00