This commit introduces the possibility to stream messages. To allow plugins to use streaming this commit also ships a `ChatSDK` library to allow to interact with few parts of discourse chat.
```ruby
ChatSDK::Message.create_with_stream(raw: "test") do |helper|
5.times do |i|
is_streaming = helper.stream(raw: "more #{i}")
next if !is_streaming
sleep 2
end
end
```
This commit also introduces all the frontend parts:
- messages can now be marked as streaming
- when streaming their content will be updated when a new content is appended
- a special UI will be showing (a blinking indicator)
- a cancel button allows the user to stop the streaming, when cancelled `helper.stream(...)` will return `false`, and the plugin can decide exit early
The service `Chat::CreateMessage` will now accept `context_post_ids` and `context_topic_id` as params. These values represent the topic which might be visible when sending a message (for now, this is only possible when using the drawer).
The `DiscourseEvent` `chat_message_created` will now have the following signature:
```ruby
on(:chat_message_created) do | message, channel, user, meta|
p meta[:context][:post_ids]
end
```
This commit includes several changes to make hashtags work when "lazy
load categories" is enabled. The previous hashtag implementation use the
category colors CSS variables, but these are not defined when the site
setting is enabled because categories are no longer preloaded.
This commit implements two fundamental changes:
1. load colors together with the other hashtag information
2. load cooked hashtag data asynchronously
The first change is implemented by adding "colors" to the HashtagItem
model. It is a list because two colors are returned for subcategories:
the color of the parent category and subcategory.
The second change is implemented on the server-side in a new route
/hashtags/by-ids and on the client side by loading previously unseen
hashtags, generating the CSS on the fly and injecting it into the page.
There have been minimal changes outside of these two fundamental ones,
but a refactoring will be coming soon to reuse as much of the code
and maybe favor use of `style` rather than injecting CSS into the page,
which can lead to page rerenders and indefinite grow of the styles.
We've changed access settings to be group membership based rather than based on the TL value directly. We kept both conditions here while we updated any plugins and themes. It should now be safe to remove.
We usually don't enforce foreign key relationships on the database level.
Because of that, occasionally it's possible to see a chat message that
references to a non-existent chat_channel or user. MessagesExporter
failed in such case before, this PR fixes that.
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_tag_topics site setting to tag_topic_allowed_groups.
We have all these calls to Group.refresh_automatic_groups! littered throughout the tests. Including tests that are seemingly unrelated to groups. This is because automatic group memberships aren't fabricated when making a vanilla user. There are two places where you'd want to use this:
You have fabricated a user that needs a certain trust level (which is now based on group membership.)
You need the system user to have a certain trust level.
In the first case, we can pass refresh_auto_groups: true to the fabricator instead. This is a more lightweight operation that only considers a single user, instead of all users in all groups.
The second case is no longer a thing after #25400.
At the moment, when someone is mentioning a group, or using here or
all mention, we create a chat_mention record per user. What we want
instead is to have special kinds of mentions, so we can create only one
chat_mention record in such cases. This PR implements that.
Note, that such mentions will still have N related notifications, one
notification per a user. We don't expect we'll have performance
problems on the notifications side, but if at some point we do, we
should be able to solve them on the side of notifications
(notifications are handled in jobs, also some little delays with
the notifications are acceptable, so we can make sure notifications
are properly queued, and that processing of every notification is
fast enough to make delays small enough).
The preparation work for this PR was done in fbd24fa, where we make
it possible for one mention to have several related notifications.
A pretty tricky part of this PR is schema and data migration, I've explained
related details inline on the migration files.
We're changing the implementation of trust levels to use groups. Part of this is to have site settings that reference trust levels use groups instead. It converts the min_trust_level_to_tag_topics site setting to tag_topic_allowed_groups.
This adds the following chat metrics:
- _chat_open_channels_with_threads_enabled_ — a count of open channels
where threading is enabled.
- _chat_channel_messages_ — a count of messages sent in a chat channel
(i.e. not a personal chat / direct message), within a thread or outside of a thread.
- _chat_threaded_messages_ — a count of messages sent within a thread
in a chat channel (i.e. not a personal chat / direct messages).
- _chat_direct_messages_ — a count of messages sent in a personal chat / direct messages.
The metrics added using the plugin API introduced in 098ab29d,
and extended in d91456fd.
Note that these stats won't be exposed at the `about.json`
and the `site/statistics.json` routes.
This PR introduces thread support for channel archives. Now, threaded messages are rendered inside a `details` HTML tag in posts.
The transcript markdown rules now support two new attributes: `threadId` and `threadTitle`.
- If `threadId` is present, all nested `chat` tags are rendered inside the first one.
- `threadTitle` (optional) defines the summary content.
```
[chat threadId=19 ... ]
thread OM
[chat ... ]
thread reply
[/chat]
[/chat]
```
If threads are split across multiple posts when archiving, the range of messages in each part will be displayed alongside the thread title. For example: `(message 1 to 16 of 20)` and `(message 17 to 20 of 20)`.
Group channels will allow users to create channels with a name and invite people. It's possible to add people even after creation of the channel. Removing users is not yet possible but will be added in the near future.
Technically a group channel is `direct_message_channel` with a group attribute set to true on its direct message (chatable). This model might evolve in the future but offers much flexibility for now without having to rely on a complex migration.
The commit essentially consists of:
- a migration to set existing direct message channels with more than 2 users to a group
- a new message creator which allows to search, add members, and create groups
- a new `AddUsersToChannel` service
- a modified `SearchChatable` service
The most common thing that we do with fab! is:
fab!(:thing) { Fabricate(:thing) }
This commit adds a shorthand for this which is just simply:
fab!(:thing)
i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
This adds the ability to collect stats without exposing them
among other stats via API.
The most important thing I wanted to achieve is to provide
an API where stats are not exposed by default, and a developer
has to explicitly specify that they should be
exposed (`expose_via_api: true`). Implementing an opposite
solution would be simpler, but that's less safe in terms of
potential security issues.
When working on this, I had to refactor the current solution.
I would go even further with the refactoring, but the next steps
seem to be going too far in changing the solution we have,
and that would also take more time. Two things that can be
improved in the future:
1. Data structures for holding stats can be further improved
2. Core stats are hard-coded in the About template (it's hard
to fix it without correcting data structures first, see point 1):
63a0700d45/app/views/about/index.html.erb (L61-L101)
The most significant refactorings are:
1. Introducing the `Stat` model
2. Aligning the way the core and the plugin stats' are registered
There is an edge case where the following occurs:
1. The user sets a bookmark reminder on a post/topic
2. The post/topic is changed to a PM before or after the reminder
fires, and the notification remains unread by the user
3. The user opens their bookmark reminder notification list
and they can still see the notification even though they cannot
access the topic anymore
There is a very low chance for information leaking here, since
the only thing that could be exposed is the topic title if it
changes to something sensitive.
This commit filters the bookmark unread notifications by using
the bookmarkable can_see? methods and also prevents sending
reminder notifications for bookmarks the user can no longer see.
This change allows users to edit their chat messages based on the criteria added to Site Settings.
If the grace period conditions are met then there will be no (edited) text applied to the message.
The following site settings are added to chat:
chat editing grace period (seconds since message created)
chat editing grace period max diff for low trust levels (number of characters changed)
chat editing grace period max diff for high trust levels (number of characters changed)
This feature adds notifications for chat messages that are sent within personal chats (1:1 and personal group chats).
To prevent notification spam we make use of consolidated notifications to combine updated message information in a meaningful way that allows the receiver to quickly jump into the chat to see what they missed.
This update respects muted channels, muted and blocked users. It will only create a new notification when the user has not muted the channel and the notified user is not muting or ignoring the message sender.
- moves the onebox logic away from `plugin.rb` to a new `onebox_handler` lib
- splits the `discourse_chat_message` template into two: one for channels, and one for messages
- refactors the logic code slightly to send only the necessary arguments to each template
This commit shouldn't change end-user behavior.
This PR swaps out the custom pathway to publishing and rendering mention warnings after a message is sent.
ChatPublisher#publish_notice is used, and expanded. Now, instead of only accepting text_content as an argument, component and component_args are accepted and there is a renderer for these components.
Translations moved to server, as notices expect text to be passed in unless a component is rendered
The warnings are rendered at the top now, outside of the scope of the single message that sent it.
I entirely removed the jit_messages_spec b/c it's duplicate testing of other parts of the app. IMO we don't need a backend test for a feature, a component test for the feature AND a system test (that is slow and potentially even flakey due to timing issues with wait) to test the same thing. So jit_messages_spec is gone.
This is extracted from #22390.
This patch aims to ease the transition to the new message creation
service. (in progress in #22390) Indeed, the new service patch is
breaking some specs from `discourse-ai` and `discourse-templates`
because these plugins are using either `Chat::MessageCreator` or the
`chat_message` fabricator.
This patch addresses theses issues by normalizing how we create a chat
message in specs. To do so, the preferred way is to use
`Fabricate(:chat_message)` with a new `:use_service` option allowing to
call the service under the hood. While this patch will obviously call
`Chat::MessageCreator`, the new service patch will now be able to simply
change the call to `Chat::CreateMessage` without breaking any specs from
other plugins.
Another thing this patch does is to not create chat messages using the
service for specs that aren’t system ones, thus speeding the execution
time a bit in the process.
In #22914 we added a fix to stop creating reviewables in the review queue when flagging a chat message and choosing the "notify user" option. By mistake we also stopped creating it when selecting the "something else" option.
This change makes it so a "something else" flag once again creates a reviewable. (Same behaviour as posts.)
This is also fixes the issue of chat composer warnings persisting across channels. Currently if you try to mention more groups than is allowed for example, a mention warning pops up. When you change channels the mention warning will not disappear even if there is no text in the composer.
This adds a reset function to the chat-composer-warnings-tracker.js, which is called when the channel is changed and the message is empty. In the event that the message is not empty we call captureMentions to check the loaded drafts' mentions.
This PR would be nicer if the post-send notice used the new chat notices API to publish the mention warnings but we would have to change the existing ones and I thought that would be too much change for this PR. It'd be a good followup though.
This commit removes any logic in the app and in specs around
enable_experimental_hashtag_autocomplete and deletes some
old category hashtag code that is no longer necessary.
It also adds a `slug_ref` category instance method, which
will generate a reference like `parent:child` for a category,
with an optional depth, which hashtags use. Also refactors
PostRevisor which was using CategoryHashtagDataSource directly
which is a no-no.
Deletes the old hashtag markdown rule as well.
Prior to this commit we were loading a large number of thread messages without any pagination. This commit attempts to fix this and also improves the following points:
- code sharing between channels and threads:
Attempts to reuse/share the code use in channels for threads. To make it possible part of this code has been extracted in dedicated helpers or has been improved to reduce the duplication needed.
Examples of extracted helpers:
- `stackingContextFix`: the ios hack for rendering bug when momentum scrolling is interrupted
- `scrollListToMessage`, `scrollListToTop`, `scrollListToBottom`: a series of helper to correctly scroll to a specific position in the list of messages
- better general performance of listing messages:
One of the main changes which has been made is to remove the computation of visible message during scroll, it will only happen when needed (update last read for example). This constant recomputation of `message.visible` on intersection observer event while scrolling was consuming a lot of CPU time.
This is extracted from #22390.
This patch simplifies a little how we handle uploads in chat, relying on
ActiveRecord mechanisms instead of calling custom methods.
This also makes `Chat::Message#validate_message` a “real” AR validation,
meaning it will run automatically when `#valid?` is called.
This is extracted from #22390.
This patch adds a new `optional` option to the `model` step. This
means if an optional model returns something blank (`nil` or an empty
collection) then the service won’t fail and will execute the next step.
However if a model is properly returned, the step will try to check if
it is valid or not (if it responds to `#invalid?`). If the model isn’t
valid, then the step will fail (so no change here).
Someone who cannot chat is also not able to join chat channels,
so we may not check all the time user.can_chat? && user.can_join_chat_channel?
and just call user.can_join_chat_channel? instead.
We need a nice way to only return some hashtag data
sources based on various site settings. This commit
adds an enabled? method that every hashtag data source
must implement. If this returns false the data source
will not be used at all for hashtag lookups or search.
It could only occur on message created by the user itself and deleted while the user was looking at the channel.
It more generally fix the trash service which was not correctly setting the author of the delete.
`SiteSetting.enable_public_channels` allows site admin to decide if public channels are available at all. There's no distinction between admins or not as we expect admins to create private category channels if they want to limit usage.
Initial migration and changes to models as well as
changing the following services to update last_message_id:
* Chat::MessageCreator
* Chat::RestoreMessage
* Chat::TrashMessage
The data migration will set the `last_message_id` for all existing
threads and channels in the database.
When we query the thread list as well as the channel,
we look at the last message ID for the following:
* Channel - Sorting DM channels, and channel metadata for the list of channels
* Thread - Last reply details for thread indicators and thread list
This commit makes sure we don't load all data into memory when doing CSV exports.
The most important change here made to the recently introduced export of chat
messages (3ea31f4). We were loading all data into memory in the first version, with
this commit it's not the case anymore.
Speaking of old exports. Some of them already use find_each, and it worked as
expected, without loading all data into memory. And it will proceed working as
expected after this commit.
In general, I made sure this change didn't break other CSV exports, first manually, and
then by writing system specs for them. Sadly, I haven't managed yet to make those
specs stable, they work fine locally, but flaky in GitHub actions, so I've disabled them
for now.
I'll be making more changes to the CSV exports code soon, those system specs will be
very helpful. I'll be running them locally, and I hope I'll manage to make them stable
while doing that work.
This commit replaces two existing screens:
- draft
- channel selection modal
Main features compared to existing solutions
- features are now combined, meaning you can for example create multi users DM
- it will show users with chat disabled
- it shows unread state
- hopefully a better look/feel
- lots of small details and fixes...
Other noticeable fixes
- starting a DM with a user, even from the user card and clicking <kbd>Chat</kbd> will not show a green dot for the target user (or even the channel) until a message is actually sent
- it should almost never do a full page reload anymore
---------
Co-authored-by: Martin Brennan <mjrbrennan@gmail.com>
Co-authored-by: Jordan Vidrine <30537603+jordanvidrine@users.noreply.github.com>
Co-authored-by: chapoi <101828855+chapoi@users.noreply.github.com>
Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
This will be used when we move the channel creation for DMs
to happen when we first send a message in a DM channel to avoid
a double-request. For now we can just have a new API endpoint
for creating this that the existing frontend code can use,
that uses the new service pattern.
This also uses the new policy pattern for services where the policy
can be defined in a class so a more dynamic reason for the policy
failing can be sent to the controller.
Co-authored-by: Loïc Guitaut <loic@discourse.org>
