2022-11-02 21:41:30 +08:00
|
|
|
# frozen_string_literal: true
|
|
|
|
|
|
|
|
class ChatMessage < ActiveRecord::Base
|
|
|
|
include Trashable
|
|
|
|
attribute :has_oneboxes, default: false
|
|
|
|
|
|
|
|
BAKED_VERSION = 2
|
|
|
|
|
|
|
|
belongs_to :chat_channel
|
|
|
|
belongs_to :user
|
|
|
|
belongs_to :in_reply_to, class_name: "ChatMessage"
|
2022-11-07 07:04:47 +08:00
|
|
|
belongs_to :last_editor, class_name: "User"
|
2022-11-02 21:41:30 +08:00
|
|
|
has_many :replies, class_name: "ChatMessage", foreign_key: "in_reply_to_id", dependent: :nullify
|
|
|
|
has_many :revisions, class_name: "ChatMessageRevision", dependent: :destroy
|
|
|
|
has_many :reactions, class_name: "ChatMessageReaction", dependent: :destroy
|
|
|
|
has_many :bookmarks, as: :bookmarkable, dependent: :destroy
|
|
|
|
has_many :chat_uploads, dependent: :destroy
|
|
|
|
has_many :uploads, through: :chat_uploads
|
|
|
|
has_one :chat_webhook_event, dependent: :destroy
|
|
|
|
has_one :chat_mention, dependent: :destroy
|
|
|
|
|
|
|
|
scope :in_public_channel,
|
|
|
|
-> {
|
|
|
|
joins(:chat_channel).where(
|
|
|
|
chat_channel: {
|
|
|
|
chatable_type: ChatChannel.public_channel_chatable_types,
|
|
|
|
},
|
|
|
|
)
|
|
|
|
}
|
|
|
|
|
|
|
|
scope :in_dm_channel,
|
2022-11-02 22:53:36 +08:00
|
|
|
-> { joins(:chat_channel).where(chat_channel: { chatable_type: "DirectMessage" }) }
|
2022-11-02 21:41:30 +08:00
|
|
|
|
|
|
|
scope :created_before, ->(date) { where("chat_messages.created_at < ?", date) }
|
|
|
|
|
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 06:37:06 +08:00
|
|
|
before_save { ensure_last_editor_id }
|
2022-11-07 07:04:47 +08:00
|
|
|
|
2022-11-02 21:41:30 +08:00
|
|
|
def validate_message(has_uploads:)
|
|
|
|
WatchedWordsValidator.new(attributes: [:message]).validate(self)
|
2022-12-14 08:48:23 +08:00
|
|
|
|
|
|
|
if self.new_record? || self.changed.include?("message")
|
|
|
|
Chat::DuplicateMessageValidator.new(self).validate
|
|
|
|
end
|
2022-11-02 21:41:30 +08:00
|
|
|
|
|
|
|
if !has_uploads && message_too_short?
|
|
|
|
self.errors.add(
|
|
|
|
:base,
|
|
|
|
I18n.t(
|
|
|
|
"chat.errors.minimum_length_not_met",
|
|
|
|
minimum: SiteSetting.chat_minimum_message_length,
|
|
|
|
),
|
|
|
|
)
|
|
|
|
end
|
2022-11-28 08:48:30 +08:00
|
|
|
|
|
|
|
if message_too_long?
|
|
|
|
self.errors.add(
|
|
|
|
:base,
|
2022-12-14 08:48:23 +08:00
|
|
|
I18n.t("chat.errors.message_too_long", maximum: SiteSetting.chat_maximum_message_length),
|
2022-11-28 08:48:30 +08:00
|
|
|
)
|
|
|
|
end
|
2022-11-02 21:41:30 +08:00
|
|
|
end
|
|
|
|
|
|
|
|
def attach_uploads(uploads)
|
|
|
|
return if uploads.blank?
|
|
|
|
|
|
|
|
now = Time.now
|
|
|
|
record_attrs =
|
|
|
|
uploads.map do |upload|
|
|
|
|
{ upload_id: upload.id, chat_message_id: self.id, created_at: now, updated_at: now }
|
|
|
|
end
|
|
|
|
ChatUpload.insert_all!(record_attrs)
|
|
|
|
end
|
|
|
|
|
|
|
|
def excerpt
|
|
|
|
# just show the URL if the whole message is a URL, because we cannot excerpt oneboxes
|
|
|
|
return message if UrlHelper.relaxed_parse(message).is_a?(URI)
|
|
|
|
|
|
|
|
# upload-only messages are better represented as the filename
|
|
|
|
return uploads.first.original_filename if cooked.blank? && uploads.present?
|
|
|
|
|
|
|
|
# this may return blank for some complex things like quotes, that is acceptable
|
|
|
|
PrettyText.excerpt(cooked, 50, {})
|
|
|
|
end
|
|
|
|
|
|
|
|
def cooked_for_excerpt
|
|
|
|
(cooked.blank? && uploads.present?) ? "<p>#{uploads.first.original_filename}</p>" : cooked
|
|
|
|
end
|
|
|
|
|
|
|
|
def push_notification_excerpt
|
|
|
|
Emoji.gsub_emoji_to_unicode(message).truncate(400)
|
|
|
|
end
|
|
|
|
|
|
|
|
def to_markdown
|
|
|
|
markdown = []
|
|
|
|
|
|
|
|
if self.message.present?
|
|
|
|
msg = self.message
|
|
|
|
|
|
|
|
self.chat_uploads.any? ? markdown << msg + "\n" : markdown << msg
|
|
|
|
end
|
|
|
|
|
|
|
|
self
|
|
|
|
.chat_uploads
|
|
|
|
.order(:created_at)
|
|
|
|
.each { |chat_upload| markdown << UploadMarkdown.new(chat_upload.upload).to_markdown }
|
|
|
|
|
|
|
|
markdown.reject(&:empty?).join("\n")
|
|
|
|
end
|
|
|
|
|
|
|
|
def cook
|
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 06:37:06 +08:00
|
|
|
ensure_last_editor_id
|
|
|
|
|
|
|
|
self.cooked = self.class.cook(self.message, user_id: self.last_editor_id)
|
2022-11-02 21:41:30 +08:00
|
|
|
self.cooked_version = BAKED_VERSION
|
|
|
|
end
|
|
|
|
|
|
|
|
def rebake!(invalidate_oneboxes: false, priority: nil)
|
2022-12-19 09:05:37 +08:00
|
|
|
ensure_last_editor_id
|
|
|
|
|
2022-11-02 21:41:30 +08:00
|
|
|
previous_cooked = self.cooked
|
2022-12-19 09:05:37 +08:00
|
|
|
new_cooked =
|
|
|
|
self.class.cook(
|
|
|
|
message,
|
|
|
|
invalidate_oneboxes: invalidate_oneboxes,
|
|
|
|
user_id: self.last_editor_id,
|
|
|
|
)
|
2022-11-02 21:41:30 +08:00
|
|
|
update_columns(cooked: new_cooked, cooked_version: BAKED_VERSION)
|
|
|
|
args = { chat_message_id: self.id }
|
|
|
|
args[:queue] = priority.to_s if priority && priority != :normal
|
|
|
|
args[:is_dirty] = true if previous_cooked != new_cooked
|
|
|
|
|
|
|
|
Jobs.enqueue(:process_chat_message, args)
|
|
|
|
end
|
|
|
|
|
|
|
|
def self.uncooked
|
|
|
|
where("cooked_version <> ? or cooked_version IS NULL", BAKED_VERSION)
|
|
|
|
end
|
|
|
|
|
|
|
|
MARKDOWN_FEATURES = %w[
|
|
|
|
anchor
|
|
|
|
bbcode-block
|
|
|
|
bbcode-inline
|
|
|
|
code
|
|
|
|
category-hashtag
|
|
|
|
censored
|
|
|
|
chat-transcript
|
|
|
|
discourse-local-dates
|
|
|
|
emoji
|
|
|
|
emojiShortcuts
|
|
|
|
inlineEmoji
|
|
|
|
html-img
|
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 06:37:06 +08:00
|
|
|
hashtag-autocomplete
|
2022-11-02 21:41:30 +08:00
|
|
|
mentions
|
|
|
|
unicodeUsernames
|
|
|
|
onebox
|
|
|
|
quotes
|
|
|
|
spoiler-alert
|
|
|
|
table
|
|
|
|
text-post-process
|
|
|
|
upload-protocol
|
|
|
|
watched-words
|
|
|
|
]
|
|
|
|
|
|
|
|
MARKDOWN_IT_RULES = %w[
|
|
|
|
autolink
|
|
|
|
list
|
|
|
|
backticks
|
|
|
|
newline
|
|
|
|
code
|
|
|
|
fence
|
|
|
|
image
|
|
|
|
table
|
|
|
|
linkify
|
|
|
|
link
|
|
|
|
strikethrough
|
|
|
|
blockquote
|
|
|
|
emphasis
|
|
|
|
]
|
|
|
|
|
|
|
|
def self.cook(message, opts = {})
|
2022-12-19 09:05:37 +08:00
|
|
|
# A rule in our Markdown pipeline may have Guardian checks that require a
|
|
|
|
# user to be present. The last editing user of the message will be more
|
|
|
|
# generally up to date than the creating user. For example, we use
|
|
|
|
# this when cooking #hashtags to determine whether we should render
|
|
|
|
# the found hashtag based on whether the user can access the channel it
|
|
|
|
# is referencing.
|
2022-11-02 21:41:30 +08:00
|
|
|
cooked =
|
|
|
|
PrettyText.cook(
|
|
|
|
message,
|
|
|
|
features_override: MARKDOWN_FEATURES + DiscoursePluginRegistry.chat_markdown_features.to_a,
|
|
|
|
markdown_it_rules: MARKDOWN_IT_RULES,
|
|
|
|
force_quote_link: true,
|
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 06:37:06 +08:00
|
|
|
user_id: opts[:user_id],
|
2022-12-14 08:48:23 +08:00
|
|
|
hashtag_context: "chat-composer",
|
2022-11-02 21:41:30 +08:00
|
|
|
)
|
|
|
|
|
|
|
|
result =
|
|
|
|
Oneboxer.apply(cooked) do |url|
|
|
|
|
if opts[:invalidate_oneboxes]
|
|
|
|
Oneboxer.invalidate(url)
|
|
|
|
InlineOneboxer.invalidate(url)
|
|
|
|
end
|
|
|
|
onebox = Oneboxer.cached_onebox(url)
|
|
|
|
onebox
|
|
|
|
end
|
|
|
|
|
|
|
|
cooked = result.to_html if result.changed?
|
|
|
|
cooked
|
|
|
|
end
|
|
|
|
|
|
|
|
def full_url
|
|
|
|
"#{Discourse.base_url}#{url}"
|
|
|
|
end
|
|
|
|
|
|
|
|
def url
|
|
|
|
"/chat/message/#{self.id}"
|
|
|
|
end
|
|
|
|
|
|
|
|
private
|
|
|
|
|
|
|
|
def message_too_short?
|
|
|
|
message.length < SiteSetting.chat_minimum_message_length
|
|
|
|
end
|
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 06:37:06 +08:00
|
|
|
|
2022-11-28 08:48:30 +08:00
|
|
|
def message_too_long?
|
|
|
|
message.length > SiteSetting.chat_maximum_message_length
|
|
|
|
end
|
|
|
|
|
FEATURE: Generic hashtag autocomplete lookup and markdown cooking (#18937)
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
2022-11-21 06:37:06 +08:00
|
|
|
def ensure_last_editor_id
|
|
|
|
self.last_editor_id ||= self.user_id
|
|
|
|
end
|
2022-11-02 21:41:30 +08:00
|
|
|
end
|
|
|
|
|
|
|
|
# == Schema Information
|
|
|
|
#
|
|
|
|
# Table name: chat_messages
|
|
|
|
#
|
|
|
|
# id :bigint not null, primary key
|
|
|
|
# chat_channel_id :integer not null
|
|
|
|
# user_id :integer
|
|
|
|
# created_at :datetime not null
|
|
|
|
# updated_at :datetime not null
|
|
|
|
# deleted_at :datetime
|
|
|
|
# deleted_by_id :integer
|
|
|
|
# in_reply_to_id :integer
|
|
|
|
# message :text
|
|
|
|
# cooked :text
|
|
|
|
# cooked_version :integer
|
2022-11-08 07:06:13 +08:00
|
|
|
# last_editor_id :integer not null
|
2022-11-02 21:41:30 +08:00
|
|
|
#
|
|
|
|
# Indexes
|
|
|
|
#
|
|
|
|
# idx_chat_messages_by_created_at_not_deleted (created_at) WHERE (deleted_at IS NULL)
|
|
|
|
# index_chat_messages_on_chat_channel_id_and_created_at (chat_channel_id,created_at)
|
PERF: Add index for chat unread counts query (#19516)
This commit adds an index for the query which the chat plugin executes
multiple times when preloading user data in `Chat::ChatChannelFetcher.unread_counts`.
Sample query plan from a query I grabbed from one of our production
instance.
Before:
```
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GroupAggregate (cost=10.77..696.67 rows=7 width=16) (actual time=7.735..7.736 rows=0 loops=1)
Group Key: cc.id
-> Nested Loop (cost=10.77..696.54 rows=12 width=8) (actual time=7.734..7.735 rows=0 loops=1)
Join Filter: (cc.id = cm.chat_channel_id)
-> Nested Loop (cost=0.56..76.44 rows=1 width=16) (actual time=0.011..0.037 rows=7 loops=1)
-> Index Only Scan using chat_channels_pkey on chat_channels cc (cost=0.28..22.08 rows=7 width=8) (actual time=0.004..0.014 rows=7 loops=1)
Index Cond: (id = ANY ('{192,300,228,727,8,612,1633}'::bigint[]))
Heap Fetches: 0
-> Index Scan using user_chat_channel_unique_memberships on user_chat_channel_memberships uccm (cost=0.28..7.73 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=7)
Index Cond: ((user_id = 1338) AND (chat_channel_id = cc.id))
-> Bitmap Heap Scan on chat_messages cm (cost=10.21..618.98 rows=89 width=12) (actual time=1.096..1.097 rows=0 loops=7)
Recheck Cond: (chat_channel_id = uccm.chat_channel_id)
Filter: ((deleted_at IS NULL) AND (user_id <> 1338) AND (id > COALESCE(uccm.last_read_message_id, 0)))
Rows Removed by Filter: 2085
Heap Blocks: exact=7106
-> Bitmap Index Scan on index_chat_messages_on_chat_channel_id_and_created_at (cost=0.00..10.19 rows=270 width=0) (actual time=0.114..0.114 rows=2085 loops=7)
Index Cond: (chat_channel_id = uccm.chat_channel_id)
Planning Time: 0.408 ms
Execution Time: 7.762 ms
(19 rows)
```
After:
```
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GroupAggregate (cost=5.84..367.39 rows=7 width=16) (actual time=0.130..0.131 rows=0 loops=1)
Group Key: cc.id
-> Nested Loop (cost=5.84..367.26 rows=12 width=8) (actual time=0.129..0.130 rows=0 loops=1)
Join Filter: (cc.id = cm.chat_channel_id)
-> Nested Loop (cost=0.56..76.44 rows=1 width=16) (actual time=0.038..0.069 rows=7 loops=1)
-> Index Only Scan using chat_channels_pkey on chat_channels cc (cost=0.28..22.08 rows=7 width=8) (actual time=0.011..0.022 rows=7 loops=1)
Index Cond: (id = ANY ('{192,300,228,727,8,612,1633}'::bigint[]))
Heap Fetches: 0
-> Index Scan using user_chat_channel_unique_memberships on user_chat_channel_memberships uccm (cost=0.28..7.73 rows=1 width=8) (actual time=0.006..0.006 rows=1 loops=7)
Index Cond: ((user_id = 1338) AND (chat_channel_id = cc.id))
-> Bitmap Heap Scan on chat_messages cm (cost=5.28..289.71 rows=89 width=12) (actual time=0.008..0.008 rows=0 loops=7)
Recheck Cond: ((chat_channel_id = uccm.chat_channel_id) AND (id > COALESCE(uccm.last_read_message_id, 0)) AND (deleted_at IS NULL))
Filter: (user_id <> 1338)
-> Bitmap Index Scan on index_chat_messages_on_chat_channel_id_and_id (cost=0.00..5.26 rows=90 width=0) (actual time=0.008..0.008 rows=0 loops=7)
Index Cond: ((chat_channel_id = uccm.chat_channel_id) AND (id > COALESCE(uccm.last_read_message_id, 0)))
Planning Time: 1.217 ms
Execution Time: 0.188 ms
(17 rows)
```
2022-12-20 05:10:53 +08:00
|
|
|
# index_chat_messages_on_chat_channel_id_and_id (chat_channel_id,id) WHERE (deleted_at IS NULL)
|
2022-11-07 07:04:47 +08:00
|
|
|
# index_chat_messages_on_last_editor_id (last_editor_id)
|
2022-11-02 21:41:30 +08:00
|
|
|
#
|