github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-30 13:45:59 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
a8eb607162
discourse/spec/lib
History
Martin Brennan d3f02a1270
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) 
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.

**Serverside**

We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.

Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.

The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.

When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.

**Markdown**

The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.

This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.

**Chat Channels**

This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.

------

**Known Rough Edges**

- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 08:37:06 +10:00
..
auth DEV: Update rubocop (#18754) 2022-10-26 09:05:15 +08:00
backup_restore FIX: Backup/Restore didn't use correct Redis namespace in multisite (#18060) 2022-08-24 01:43:42 +02:00
common_passwords DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
compression SECURITY: Prevent arbitrary file write when decompressing files (#18421) 2022-09-29 20:00:38 +02:00
concern DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
content_security_policy Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
email FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) 2022-11-21 08:37:06 +10:00
file_store DEV: Fix mocha deprecations (#18828) 2022-11-02 10:47:59 +01:00
final_destination SECURITY: Expand and improve SSRF Protections (#18815) 2022-11-01 16:33:17 +00:00
freedom_patches DEV: update to patched discourse-seed-fu (#18493) 2022-10-07 09:16:04 +11:00
guardian FEATURE: hidden site setting to suppress unsecured categories from admins (#19098) 2022-11-18 14:37:36 +11:00
highlight_js Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
i18n Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
imap DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
import Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
middleware FEATURE: new site setting to set locale from cookie for anonymous users. (#18377) 2022-09-27 14:26:06 +05:30
migration Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
onebox DEV: Fix mocha deprecations (#18828) 2022-11-02 10:47:59 +01:00
plugin DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
pretty_text FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) 2022-11-21 08:37:06 +10:00
rate_limiter DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
scheduler Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
second_factor DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
seed_data FEATURE: Default Composer Category Site Setting (#18967) 2022-11-14 11:09:57 -07:00
site_settings DEV: Fix mocha deprecations (#18828) 2022-11-02 10:47:59 +01:00
stylesheet PERF: Make stylesheet hashes consistent between deploys (#18909) 2022-11-07 16:13:35 +00:00
svg_sprite DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
theme_store DEV: Reinstate --single-branch when cloning themes (#19026) 2022-11-15 13:23:57 -06:00
topic_query DEV: Remove usages of enable_personal_messages (#18437) 2022-10-05 10:50:20 +10:00
validators DEV: Remove remaining hardcoded ids (#18735) 2022-10-25 15:29:09 +08:00
webauthn Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
wizard FIX: do not prefill default site title value on wizard introduction step (#18496) 2022-10-06 12:02:48 +05:30
admin_confirmation_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
admin_user_index_query_spec.rb FEATURE: Introduce personal_message_enabled_groups setting (#18042) 2022-09-26 13:58:40 +10:00
archetype_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
bookmark_manager_spec.rb FIX: Do not save default auto_delete_preference for bookmark (#18119) 2022-08-30 09:21:41 +10:00
bookmark_query_spec.rb FEATURE: whispers available for groups (#17170) 2022-06-30 10:18:12 +10:00
bookmark_reminder_notification_handler_spec.rb FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
browser_detection_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
cache_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
category_badge_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
composer_messages_finder_spec.rb FEATURE: add composer warning when user haven't been seen in a long time (#18340) 2022-09-27 22:06:40 +05:30
content_buffer_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
content_security_policy_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
cooked_post_processor_spec.rb SECURITY: Expand and improve SSRF Protections (#18815) 2022-11-01 16:33:17 +00:00
crawler_detection_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
current_user_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
db_helper_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
directory_helper_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
discourse_diff_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
discourse_event_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
discourse_hub_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
discourse_js_processor_spec.rb Build(deps): Bump @babel/standalone from 7.19.3 to 7.19.6 in /app/assets/javascripts (#18689) 2022-10-21 14:48:27 +01:00
discourse_plugin_registry_spec.rb DEV: Fix mocha deprecations (#18828) 2022-11-02 10:47:59 +01:00
discourse_redis_spec.rb Revert "DEV: Upgrade Redis to 4.8" (#18099) 2022-08-26 15:16:50 +10:00
discourse_sourcemapping_url_processor_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
discourse_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
discourse_tagging_spec.rb FIX: Revert recursively tag lookup with missing ancestor tags (#18439) 2022-09-30 08:28:09 +00:00
discourse_updates_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
distributed_cache_spec.rb DEV: Make the first argument to the top-level describe a constant in specs 2022-08-08 18:07:49 +02:00
distributed_memoizer_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
distributed_mutex_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
email_cook_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
email_updater_spec.rb FEATURE: Add setting to always confirm old email (#18417) 2022-09-30 00:49:17 +03:00
ember_cli_spec.rb DEV: Add test for EmberCli#ember_version (#18140) 2022-08-30 20:33:08 +01:00
encodings_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
enum_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
excerpt_parser_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
feed_element_installer_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
feed_item_accessor_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
file_helper_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
filter_best_posts_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
final_destination_spec.rb SECURITY: Expand and improve SSRF Protections (#18815) 2022-11-01 16:33:17 +00:00
flag_settings_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
gaps_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
git_url_spec.rb SECURITY: Expand and improve SSRF Protections (#18815) 2022-11-01 16:33:17 +00:00
global_path_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
group_email_credentials_check_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
guardian_spec.rb FIX: Ensure moderators_manage_categories_and_groups is respected (#18884) 2022-11-11 11:06:05 +00:00
has_errors_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
hijack_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
html_prettify_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
html_to_markdown_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
image_sizer_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
inline_oneboxer_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
js_locale_helper_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
json_error_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
letter_avatar_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
message_id_service_spec.rb FEATURE: Overhaul email threading (#17996) 2022-09-26 09:14:24 +10:00
method_profiler_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
mini_sql_multisite_connection_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
mobile_detection_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
new_post_manager_spec.rb DEV: Remove usages of enable_personal_messages (#18437) 2022-10-05 10:50:20 +10:00
new_post_result_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
onebox_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
oneboxer_spec.rb DEV: Fix busted twitter onebox spec (#18009) 2022-08-20 11:19:19 +02:00
onpdiff_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
pbkdf2_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
pinned_check_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
plain_text_to_markdown_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
post_action_creator_spec.rb FEATURE: Introduce personal_message_enabled_groups setting (#18042) 2022-09-26 13:58:40 +10:00
post_action_destroyer_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
post_creator_spec.rb DEV: Remove usages of enable_personal_messages (#18437) 2022-10-05 10:50:20 +10:00
post_destroyer_spec.rb DEV: Use AR enums in reviewables related code 2022-09-22 14:44:27 +02:00
post_jobs_enqueuer_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
post_locker_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
post_merger_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
post_revisor_spec.rb FIX: Skip all post validations if necessary (#18625) 2022-10-19 18:54:32 +03:00
presence_channel_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
pretty_text_spec.rb FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) 2022-11-21 08:37:06 +10:00
promotion_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
quote_comparer_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
rate_limiter_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
retrieve_title_spec.rb FIX: ignore malformed HTML for title extraction (#18040) 2022-08-23 15:03:57 +10:00
rtl_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
s3_cors_rulesets_spec.rb DEV: Automatically require 'rails_helper' in all specs (#16077) 2022-03-01 17:50:50 +00:00
s3_helper_spec.rb PERF: Update s3:expire_missing_assets to delete in batches (#18908) 2022-11-07 12:53:14 +00:00
s3_inventory_multisite_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
s3_inventory_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
score_calculator_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
scss_checker_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
search_spec.rb FIX: Support unicode in search filter @username (#18804) 2022-11-16 10:42:37 +01:00
secure_session_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
shrink_uploaded_image_spec.rb FIX: Shrinking images where smaller image upload exists (#18965) 2022-11-10 12:43:56 +01:00
site_icon_manager_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
site_setting_extension_multisite_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
site_setting_extension_spec.rb DEV: Remove remaining hardcoded ids (#18735) 2022-10-25 15:29:09 +08:00
slug_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
spam_handler_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
suggested_topics_builder_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
system_message_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
text_cleaner_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
text_sentinel_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
theme_flag_modifier_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
theme_javascript_compiler_spec.rb DEV: Apply theme template transforms to colocated components (#18704) 2022-10-21 19:05:34 +01:00
theme_settings_manager_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
theme_settings_parser_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
timeline_lookup_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
tiny_japanese_segmenter_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
topic_creator_spec.rb DEV: Remove usages of enable_personal_messages (#18437) 2022-10-05 10:50:20 +10:00
topic_publisher_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
topic_query_spec.rb DEV: Remove usages of enable_personal_messages (#18437) 2022-10-05 10:50:20 +10:00
topic_retriever_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
topic_upload_security_manager_spec.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
topic_view_spec.rb DEV: Fix a flaky spec (#18995) 2022-11-16 10:46:30 +01:00
topics_bulk_action_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
trashable_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
trust_level_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00
twitter_api_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
unread_spec.rb DEV: Remove remaining hardcoded ids (#18735) 2022-10-25 15:29:09 +08:00
upload_creator_spec.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
upload_markdown_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
upload_recovery_spec.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
upload_security_spec.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
url_helper_spec.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
user_comm_screener_spec.rb FIX: Handle actor not having preferences in UserCommScreener (#17790) 2022-08-04 10:16:54 +10:00
user_lookup_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
user_name_suggester_spec.rb Add RSpec 4 compatibility (#17652) 2022-07-28 10:27:38 +08:00
version_spec.rb DEV: Use proper wording for contexts in specs 2022-08-04 11:05:02 +02:00