discourse/app/serializers
Martin Brennan d3f02a1270
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.

**Serverside**

We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.

Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.

The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.

When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.

**Markdown**

The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.

This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.

**Chat Channels**

This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.

------

**Known Rough Edges**

- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 08:37:06 +10:00
..
concerns SECURITY: Restrict display of topic titles associated with user badges (#18768) 2022-10-27 11:26:14 +08:00
about_serializer.rb FIX: do not include contact url & email in client site settings payload (#13004) 2021-05-19 16:15:24 +10:00
admin_badge_serializer.rb FEATURE: Link to text customization when editing system badges (#11345) 2020-12-08 11:55:49 -08:00
admin_badges_serializer.rb
admin_detailed_user_serializer.rb FIX: do not lock account if backup codes are available (#18982) 2022-11-11 13:00:06 +11:00
admin_email_template_serializer.rb
admin_plugin_serializer.rb UX: show plugin descriptions on admin plugins page 2021-02-12 11:38:50 -05:00
admin_user_action_serializer.rb FIX: Posts can belong to hard-deleted topics (#17329) 2022-07-05 10:51:21 +03:00
admin_user_list_serializer.rb FEATURE: Rename 'Discourse SSO' to DiscourseConnect (#11978) 2021-02-08 10:04:33 +00:00
admin_user_serializer.rb
admin_web_hook_event_serializer.rb
admin_web_hook_serializer.rb
api_key_scope_serializer.rb UX: Help users understand the meaning of each scope. (#10468) 2020-08-18 15:12:04 -03:00
api_key_serializer.rb FEATURE: Add scopes to API keys (#9844) 2020-07-16 15:51:24 -03:00
application_serializer.rb FIX: Clear post action types application serializer fragment cache. 2021-06-04 09:14:49 +08:00
archetype_serializer.rb
associated_group_serializer.rb FEATURE: Experimental support for group membership via google auth (#14835) 2021-12-09 12:30:27 +00:00
auth_provider_serializer.rb DEV: Deprecate message parameter in auth provider plugin API (#12523) 2021-03-25 13:23:48 +00:00
backup_file_serializer.rb
badge_grouping_serializer.rb
badge_index_serializer.rb
badge_serializer.rb UX: Add image uploader widget for uploading badge images (#12377) 2021-03-17 08:55:23 +03:00
badge_type_serializer.rb
basic_category_serializer.rb FEATURE: Add dark mode option for category logos (#18460) 2022-10-07 11:00:44 -04:00
basic_group_history_serializer.rb
basic_group_serializer.rb FIX: N+1 for admins viewing groups page 2020-09-16 14:58:52 -04:00
basic_group_user_serializer.rb
basic_post_serializer.rb
basic_reviewable_flagged_post_serializer.rb DEV: Add reviewables tab to the new user menu (#17630) 2022-07-28 11:16:33 +03:00
basic_reviewable_queued_post_serializer.rb DEV: Add reviewables tab to the new user menu (#17630) 2022-07-28 11:16:33 +03:00
basic_reviewable_serializer.rb DEV: Include pending reviewables in the main tab in the user menu (#18471) 2022-10-05 12:30:02 +03:00
basic_reviewable_user_serializer.rb DEV: Add reviewables tab to the new user menu (#17630) 2022-07-28 11:16:33 +03:00
basic_topic_serializer.rb
basic_user_badge_serializer.rb
basic_user_serializer.rb FIX: Remove tags from experimental sidebar on notification level changed (#17083) 2022-06-14 15:39:56 +08:00
basic_user_with_status_serializer.rb DEV: move BasicUserWithStatusSerializer from Discourse Chat (#18745) 2022-10-26 16:41:31 +04:00
category_and_topic_lists_serializer.rb FEATURE: Allow setting avatar flair for automatic groups (#12586) 2021-04-06 11:13:06 -04:00
category_detailed_serializer.rb FEATURE: Return subcategories on categories endpoint (#14492) 2021-10-05 12:12:31 -06:00
category_group_serializer.rb FEATURE: Group category permissions tab (#10388) 2020-08-10 09:49:05 -05:00
category_list_serializer.rb DEV: Remove draft attributes from topic lists (#12525) 2021-03-30 11:42:26 -04:00
category_required_tag_group_serializer.rb DEV: Ensure a broken tag_group relation doesn't raise an error (#16529) 2022-04-21 18:18:35 +01:00
category_serializer.rb SECURITY: Category group permissions leaked to normal users. 2022-04-08 13:46:20 +08:00
category_upload_serializer.rb
color_scheme_color_serializer.rb
color_scheme_selectable_serializer.rb FEATURE: User selectable color schemes (#10544) 2020-08-28 10:36:52 -04:00
color_scheme_serializer.rb FEATURE: User selectable color schemes (#10544) 2020-08-28 10:36:52 -04:00
current_user_serializer.rb FIX: do not lock account if backup codes are available (#18982) 2022-11-11 13:00:06 +11:00
detailed_tag_serializer.rb
detailed_user_badge_serializer.rb SECURITY: Restrict display of topic titles associated with user badges (#18768) 2022-10-27 11:26:14 +08:00
directory_column_serializer.rb DEV: Plugin API to add directory columns (#13440) 2021-06-22 13:00:04 -05:00
directory_item_serializer.rb FIX: Always serialize the correct attributes for DirectoryItems (#13510) 2021-06-23 14:55:17 -05:00
discourse_version_check_serializer.rb
draft_serializer.rb
edit_directory_column_serializer.rb DEV: Plugin API to add directory columns (#13440) 2021-06-22 13:00:04 -05:00
email_log_serializer.rb FEATURE: Show SMTP response on admin email sent list and rearrange columns (#17143) 2022-08-03 08:11:54 +10:00
email_style_serializer.rb
embeddable_host_serializer.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
embedding_serializer.rb
emoji_serializer.rb FIX: Don't raise an error if a custom emoji image was deleted 2020-12-09 15:40:40 -05:00
flagged_topic_serializer.rb
flagged_topic_summary_serializer.rb
flagged_user_serializer.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
flair_group_serializer.rb FEATURE: Let users select flair (#13587) 2021-07-08 10:46:21 +03:00
found_user_serializer.rb DEV: return user status on the user search route (#17716) 2022-08-09 14:54:33 +04:00
found_user_with_status_serializer.rb DEV: return user status on the user search route (#17716) 2022-08-09 14:54:33 +04:00
gap_serializer.rb
group_post_serializer.rb FIX: Posts with Staff Colors were excluded from the group activity 2020-12-11 11:52:03 -05:00
group_post_user_serializer.rb Add primary group classes (#12807) 2021-04-22 15:00:23 -07:00
group_requester_serializer.rb
group_show_serializer.rb FEATURE: Allow sending group SMTP emails with from alias (#15687) 2022-02-07 13:52:01 +10:00
group_user_serializer.rb FIX: include_ serializer methods must end with ? (#14407) 2021-09-22 16:01:25 +03:00
grouped_screened_url_serializer.rb
grouped_search_result_serializer.rb
hidden_profile_serializer.rb
incoming_email_details_serializer.rb
incoming_email_serializer.rb
invite_link_serializer.rb FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
invite_serializer.rb FEATURE: Show error if invite to topic is invalid (#15959) 2022-02-16 18:35:02 +02:00
invited_serializer.rb FEATURE: Various improvements to invite system (#12298) 2021-03-06 13:29:35 +02:00
invited_user_record_serializer.rb FIX: include_ serializer methods must end with ? (#14407) 2021-09-22 16:01:25 +03:00
invited_user_serializer.rb FEATURE: multiple use invite links (#9813) 2020-06-09 20:49:32 +05:30
listable_topic_serializer.rb FEATURE: whispers available for groups (#17170) 2022-06-30 10:18:12 +10:00
new_post_result_serializer.rb FIX: Update post's raw from server response (#13438) 2021-06-18 16:26:57 +03:00
notification_serializer.rb FIX: Maintain notification order by priority (#13186) 2021-05-31 09:27:13 +03:00
penalty_counts_serializer.rb
pending_post_serializer.rb FEATURE: Display pending posts on user’s page 2021-11-29 10:26:33 +01:00
permalink_serializer.rb
post_action_type_serializer.rb Fix i18n issues reported on Crowdin (#10925) 2020-10-16 15:24:58 +02:00
post_action_user_serializer.rb FEATURE: Don't display muted/ignored users under "who liked" (#10084) 2020-06-19 10:44:21 -04:00
post_item_excerpt.rb
post_revision_serializer.rb FIX: Do not attempt to serialize Tag objects when tagging disabled (#18264) 2022-09-15 16:17:48 +01:00
post_serializer.rb FIX: Move show like logic to client side (#18025) 2022-08-22 18:40:09 +03:00
post_stream_serializer_mixin.rb DEV: Remove TopicView#first_post_id. (#14631) 2021-10-18 14:47:47 +08:00
post_wordpress_serializer.rb
poster_serializer.rb FEATURE: Allow setting avatar flair for automatic groups (#12586) 2021-04-06 11:13:06 -04:00
presence_channel_state_serializer.rb DEV: Introduce PresenceChannel API for core and plugin use 2021-08-27 16:26:06 +01:00
primary_group_serializer.rb FEATURE: Let users select flair (#13587) 2021-07-08 10:46:21 +03:00
private_message_topic_tracking_state_serializer.rb FEATURE: Display unread and new counts for messages. (#14059) 2021-08-25 11:17:56 +08:00
published_page_serializer.rb FEATURE: allows published pages to be public (#10053) 2020-06-17 12:42:20 +02:00
queued_post_serializer.rb DEV: Use AR enums in reviewables related code 2022-09-22 14:44:27 +02:00
reviewable_action_serializer.rb DEV: APIs for plugin to add custom reviewable confirm modal (#12246) 2021-03-02 10:28:27 -06:00
reviewable_bundled_action_serializer.rb
reviewable_conversation_post_serializer.rb
reviewable_conversation_serializer.rb
reviewable_editable_field_serializer.rb
reviewable_explanation_serializer.rb
reviewable_flagged_post_serializer.rb
reviewable_history_serializer.rb DEV: Use AR enums in reviewables related code 2022-09-22 14:44:27 +02:00
reviewable_perform_result_serializer.rb DEV: Combine all header notification bubbles into one in the new user menu (#17718) 2022-08-03 08:57:59 +03:00
reviewable_post_serializer.rb FEATURE: Review every post using the review queue. (#12734) 2021-04-21 08:41:36 -03:00
reviewable_queued_post_serializer.rb
reviewable_score_explanation_serializer.rb
reviewable_score_serializer.rb DEV: Use AR enums in reviewables related code 2022-09-22 14:44:27 +02:00
reviewable_score_type_serializer.rb Revert "FEATURE: Let reviewables override the score type title. (#16234)" (#16238) 2022-03-21 16:32:47 -03:00
reviewable_serializer.rb DEV: Use AR enums in reviewables related code 2022-09-22 14:44:27 +02:00
reviewable_settings_serializer.rb
reviewable_topic_serializer.rb
reviewable_user_serializer.rb FEATURE: reason to reject user signup (#11700) 2021-01-15 09:43:26 +11:00
screened_email_serializer.rb
screened_ip_address_serializer.rb
screened_url_serializer.rb
search_logs_serializer.rb
search_post_serializer.rb DEV: Add plugin API to extend search results (#12966) 2021-06-15 15:32:41 +10:00
search_result_user_serializer.rb FEATURE: the ability to search users by custom fields (#12762) 2021-04-27 15:52:45 +10:00
search_topic_list_item_serializer.rb Revert "PERF: Reduce size of search payload by removing unused topic attributes." 2020-07-23 09:25:31 +08:00
similar_topic_serializer.rb
single_sign_on_record_serializer.rb FIX: hide sso payload behind a button click and log views (#12110) 2021-02-17 21:27:51 +05:30
site_category_serializer.rb FEATURE: Allow multiple required tag groups for a category (#16381) 2022-04-06 14:08:06 +01:00
site_serializer.rb FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937) 2022-11-21 08:37:06 +10:00
site_text_serializer.rb FIX: I18n couldn't find translations. (#11774) 2021-01-20 17:43:00 -03:00
skipped_email_log_serializer.rb
suggested_topic_serializer.rb
suggested_topics_mixin.rb FEATURE: Display new/unread count in browse more messages for PMs. (#14188) 2021-09-07 12:30:40 +08:00
tag_group_serializer.rb
tag_serializer.rb FEATURE: ability to add description to tags (#15125) 2021-12-01 09:18:56 +11:00
theme_serializer.rb FIX: add theme field errors (#12880) 2021-04-28 15:00:37 -07:00
theme_settings_serializer.rb DEV: support json_schema in theme settings (#12294) 2021-03-10 20:15:04 -05:00
theme_translation_serializer.rb
topic_embed_serializer.rb
topic_flag_type_serializer.rb
topic_link_serializer.rb
topic_list_item_serializer.rb DEV: Remove PostAction/UserAction bookmark refs (#16681) 2022-05-10 10:42:18 +10:00
topic_list_serializer.rb DEV: Remove draft attributes from topic lists (#12525) 2021-03-30 11:42:26 -04:00
topic_pending_post_serializer.rb
topic_post_count_serializer.rb FEATURE: Let users select flair (#13587) 2021-07-08 10:46:21 +03:00
topic_poster_serializer.rb FEATURE: Let users select flair (#13587) 2021-07-08 10:46:21 +03:00
topic_timer_serializer.rb FEATURE: Allow durations < 1 hour and < 1 day for topic timers where duration is specified (auto delete replies, close based on last post) (#11961) 2021-02-05 10:12:56 +10:00
topic_tracking_state_serializer.rb FIX: Improve reliability of topic tracking state (#17387) 2022-07-14 13:44:58 +08:00
topic_view_details_serializer.rb FIX: can_permanently_delete should check for admin (#16348) 2022-04-01 12:03:39 +11:00
topic_view_posts_serializer.rb
topic_view_serializer.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
topic_view_wordpress_serializer.rb FEATURE: Add category_id to TopicViewWordpressSerializer (#10372) 2020-08-05 11:50:15 +10:00
trust_level3_requirements_serializer.rb
upload_serializer.rb DEV: Rename secure_media to secure_uploads (#18376) 2022-09-29 09:24:33 +10:00
user_action_serializer.rb FIX: rename action_code_href to action_code_path (#14834) 2021-11-08 14:32:17 +11:00
user_auth_token_serializer.rb
user_badge_serializer.rb SECURITY: Restrict display of topic titles associated with user badges (#18768) 2022-10-27 11:26:14 +08:00
user_badges_serializer.rb
user_bookmark_base_serializer.rb FIX: Allow .ics for polymorphic bookmarks (#16694) 2022-05-11 09:29:24 +10:00
user_bookmark_list_serializer.rb FIX: Return next bookmarks page only if it exists (#18139) 2022-09-01 13:04:00 +03:00
user_card_serializer.rb FIX: Make can_send_private_messages not reliant on system user (#18812) 2022-11-07 09:11:18 +10:00
user_field_serializer.rb FEATURE: the ability to search users by custom fields (#12762) 2021-04-27 15:52:45 +10:00
user_history_serializer.rb
user_name_serializer.rb
user_option_serializer.rb FEATURE: Add button to reset seen popups (#18586) 2022-10-20 09:06:39 +08:00
user_post_bookmark_serializer.rb FIX: Allow .ics for polymorphic bookmarks (#16694) 2022-05-11 09:29:24 +10:00
user_post_topic_bookmark_base_serializer.rb PERF: Only load the current user's topic_user for bookmarks list (#17873) 2022-08-17 09:40:24 +08:00
user_serializer.rb FIX: do not lock account if backup codes are available (#18982) 2022-11-11 13:00:06 +11:00
user_status_serializer.rb FEATURE: auto remove user status after predefined period (#17236) 2022-07-05 19:12:22 +04:00
user_summary_serializer.rb FEATURE: Let users select flair (#13587) 2021-07-08 10:46:21 +03:00
user_tag_notifications_serializer.rb FIX: Remove tags from experimental sidebar on notification level changed (#17083) 2022-06-14 15:39:56 +08:00
user_topic_bookmark_serializer.rb PERF: Rely on preload for first_post for TopicBookmarkable (#18066) 2022-08-24 16:01:29 +10:00
user_with_custom_fields_serializer.rb FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
user_wordpress_serializer.rb
watched_word_list_serializer.rb FEATURE: Add support for case-sensitive Watched Words (#17445) 2022-08-02 10:06:03 +02:00
watched_word_serializer.rb FEATURE: Add support for case-sensitive Watched Words (#17445) 2022-08-02 10:06:03 +02:00
web_hook_category_serializer.rb
web_hook_flag_serializer.rb
web_hook_group_serializer.rb
web_hook_group_user_serializer.rb FEATURE: trigger webhook when a user added/removed in a group. (#12653) 2021-04-08 21:16:34 +05:30
web_hook_like_serializer.rb FEATURE: add support for like webhooks (#12917) 2021-04-30 17:08:38 -07:00
web_hook_post_serializer.rb FEATURE: Let users select flair (#13587) 2021-07-08 10:46:21 +03:00
web_hook_topic_view_serializer.rb FEATURE: Topic-level bookmarks (#14353) 2021-09-21 08:45:47 +10:00
web_hook_user_serializer.rb FEATURE: user status (#16875) 2022-05-27 13:15:14 +04:00
wizard_field_choice_serializer.rb
wizard_field_serializer.rb FEATURE: allow wizard checkbox field to be disabled (#17916) 2022-08-15 05:52:07 +05:30
wizard_serializer.rb UX: Add Styling step to wizard (#14132) 2021-08-25 17:10:12 -04:00
wizard_step_serializer.rb