When chat is enabled, there's a scheduled job that runs every 5 minutes to check whether we need to send a "chat summary" email to users with unread chat messages or mentions.
On Discourse with a large number of users, the query used wasn't optimal and sometimes taking minutes. Which isn't good when the query is called every 5 minutes 😬
This PR reworks the query in `Chat::Mailer.send_unread_mentions_summary`.
Instead of starting from the `users` table, it starts from the `user_chat_channel_memberships` table which is the main piece tying everything together.
The new query is mostly similar to the previous one, with some bug fixes (like ensuring the user has `allow_private_messages` enabled for direct messages) and is also slightly simpler since it doesn't keep track of the `memberships_with_unread_messages` anymore. That part has been moved to the `user_notifications.chat_summary` email method.
The `UserEmailExtension` has been deleted since that was using to N+1 update the `user_chat_channel_memberships.last_unread_mention_when_emailed_it`(quite a mouthful 😛) but that's now done directly in the `user_notifications.chat_summary` email method.
The "plat de résistance" of that PR - the `user_notifications.chat_summary` method has been re-worked for improved performances 🚀
Instead of doing everything in one query, it does 4 tiny ones.
- One to retrieve the list of unread mentions (@something) in "category" channels
- One to retrieve the list of unread messages in "direct message" channels (aka. 1-1 and group discussions)
- One to load all the chat messages for each "category" channels from the last unread mention
- One to load all the chat messages for each "direct message" channels from the last unread message
All the specs for both `Chat::Mailer` and `UserNotification.chat_summary` have been rewriten for easier comprehension and faster execution (mostly by not using chat services which makes the specs go 10x slower...)
Internal ref - t/129848
For both `chat_allowed_groups` and `chat_message_flag_allowed_groups`,
this commit removes the `is_staff?` guardian check, and instead
adds both `moderators` and `admins` auto groups as `mandatory_values`
to those settings, as part of an ongoing effort to do this for
group-based setting values.
`chat_preferred_mobile_index` allows to set the preferred default tab when loading chat on mobile.
Current choices are:
- channels
- direct_messages
- my_threads
This PR introduces thread support for channel archives. Now, threaded messages are rendered inside a `details` HTML tag in posts.
The transcript markdown rules now support two new attributes: `threadId` and `threadTitle`.
- If `threadId` is present, all nested `chat` tags are rendered inside the first one.
- `threadTitle` (optional) defines the summary content.
```
[chat threadId=19 ... ]
thread OM
[chat ... ]
thread reply
[/chat]
[/chat]
```
If threads are split across multiple posts when archiving, the range of messages in each part will be displayed alongside the thread title. For example: `(message 1 to 16 of 20)` and `(message 17 to 20 of 20)`.
Chat will now check for the state of `SiteSetting.private_email` when sending the summary, when enabled, the mail will not display user information, channel information other than the ID and no message information, only the count of messages.
Group channels will allow users to create channels with a name and invite people. It's possible to add people even after creation of the channel. Removing users is not yet possible but will be added in the near future.
Technically a group channel is `direct_message_channel` with a group attribute set to true on its direct message (chatable). This model might evolve in the future but offers much flexibility for now without having to rely on a complex migration.
The commit essentially consists of:
- a migration to set existing direct message channels with more than 2 users to a group
- a new message creator which allows to search, add members, and create groups
- a new `AddUsersToChannel` service
- a modified `SearchChatable` service
This change allows users to edit their chat messages based on the criteria added to Site Settings.
If the grace period conditions are met then there will be no (edited) text applied to the message.
The following site settings are added to chat:
chat editing grace period (seconds since message created)
chat editing grace period max diff for low trust levels (number of characters changed)
chat editing grace period max diff for high trust levels (number of characters changed)
This PR swaps out the custom pathway to publishing and rendering mention warnings after a message is sent.
ChatPublisher#publish_notice is used, and expanded. Now, instead of only accepting text_content as an argument, component and component_args are accepted and there is a renderer for these components.
Translations moved to server, as notices expect text to be passed in unless a component is rendered
The warnings are rendered at the top now, outside of the scope of the single message that sent it.
I entirely removed the jit_messages_spec b/c it's duplicate testing of other parts of the app. IMO we don't need a backend test for a feature, a component test for the feature AND a system test (that is slow and potentially even flakey due to timing issues with wait) to test the same thing. So jit_messages_spec is gone.
This moves the "delete message" action (if it is available) of a flagged chat message under the "ignore" menu. This puts it on par with the menu for flagged posts.
`SiteSetting.enable_public_channels` allows site admin to decide if public channels are available at all. There's no distinction between admins or not as we expect admins to create private category channels if they want to limit usage.
* FEATURE: Inline topic summary. Cached version accessible to everyone.
Anons and non-members of the `custom_summarization_allowed_groups_map` groups can see cached summaries for any accessible topic. After the first 12 hours and if the posts to summarize have changed, allowed users clicking on the button will automatically re-generate it.
* Ensure chat summaries work and prevent model hallucinations when there are no messages.
* FIX: Use pluralized string
* REFACTOR: Fix misuse of pluralized string
* REFACTOR: Fix misuse of pluralized string
* DEV: Remove linting of `one` key in MessageFormat string, it doesn't work
* REFACTOR: Fix misuse of pluralized string
This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff. The string is quite complicated, so the best option was to switch to MessageFormat.
* REFACTOR: Fix misuse of pluralized string
* FIX: Use pluralized string
This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff.
* REFACTOR: Correctly pluralize reaction tooltips in chat
This also ensures that maximum 5 usernames are shown and fixes the number of "others" which was off by 1 if the current user reacted on a message.
* REFACTOR: Use translatable string as comma separator
* DEV: Add comment to translation to clarify the meaning of `%{identifier}`
* REFACTOR: Use translatable comma separator and use explicit interpolation keys
* REFACTOR: Don't interpolate lowercase channel status
* REFACTOR: Fix misuse of pluralized string
* REFACTOR: Don't interpolate channel status
* REFACTOR: Use %{count} interpolation key
* REFACTOR: Fix misuse of pluralized string
* REFACTOR: Correctly pluralize DM chat channel titles
Whenever we create a chat message that is `in_reply_to` another
message, we want to lazily populate the thread record for the
message chain.
If there is no thread yet for the root message in the reply chain,
we create a new thread with the appropriate details, and use that
thread ID for every message in the chain that does not yet have
a thread ID.
* Root message (ID 1) - no thread ID
* Message (ID 2, in_reply_to 1) - no thread ID
* When I as a user create a message in reply to ID 2, we create a thread and apply it to ID 1, ID 2, and the new message
If there is a thread for the root message in the reply chain, we
do not create one, and use the thread ID for the newly created chat
message.
* Root message (ID 1) - thread ID 700
* Message (ID 2, in_reply_to 1) - thread ID 700
* When I as a user create a message in reply to ID 2, we use the existing thread ID 700 for the new message
We also support passing in the `thread_id` to `ChatMessageCreator`,
which will be used when replying to a message that is already part of
a thread, and we validate whether that `thread_id` is okay in the context
of the channel and also the reply chain.
This work is always done, regardless of channel `thread_enabled` settings
or the `enable_experimental_chat_threaded_discussions` site setting.
This commit does not include a large data migration to backfill threads for
all existing reply chains, its unnecessary to do this so early in the project,
we can do this later if necessary.
This commit also includes thread considerations in the `MessageMover` class:
* If the original message and N other messages of a thread is moved,
the remaining messages in the thread have a new thread created in
the old channel and are moved to it.
* The reply chain is not preserved for moved messages, so new threads are
not created in the destination channel.
In addition to this, I added a fix to also clear the `in_reply_to_id` of messages
in the old channel which are moved out of that channel for data cleanliness.
Whenever we create a chat message that is `in_reply_to` another
message, we want to lazily populate the thread record for the
message chain.
If there is no thread yet for the root message in the reply chain,
we create a new thread with the appropriate details, and use that
thread ID for every message in the chain that does not yet have
a thread ID.
* Root message (ID 1) - no thread ID
* Message (ID 2, in_reply_to 1) - no thread ID
* When I as a user create a message in reply to ID 2, we create a thread and apply it to ID 1, ID 2, and the new message
If there is a thread for the root message in the reply chain, we
do not create one, and use the thread ID for the newly created chat
message.
* Root message (ID 1) - thread ID 700
* Message (ID 2, in_reply_to 1) - thread ID 700
* When I as a user create a message in reply to ID 2, we use the existing thread ID 700 for the new message
We also support passing in the `thread_id` to `ChatMessageCreator`,
which will be used when replying to a message that is already part of
a thread, and we validate whether that `thread_id` is okay in the context
of the channel and also the reply chain.
This work is always done, regardless of channel `thread_enabled` settings
or the `enable_experimental_chat_threaded_discussions` site setting.
This commit does not include a large data migration to backfill threads for
all existing reply chains, its unnecessary to do this so early in the project,
we can do this later if necessary.
This commit also includes thread considerations in the `MessageMover` class:
* If the original message and N other messages of a thread is moved,
the remaining messages in the thread have a new thread created in
the old channel and are moved to it.
* The reply chain is not preserved for moved messages, so new threads are
not created in the destination channel.
In addition to this, I added a fix to also clear the `in_reply_to_id` of messages
in the old channel which are moved out of that channel for data cleanliness.
Only allow maximum of `50_000` characters for chat drafts. A hidden `max_chat_draft_length` setting can control this limit. A migration is also provided to delete any abusive draft in the database.
The number of drafts loaded on current user has also been limited and ordered by most recent update.
Note that spec files moved are not directly related to the fix.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Co-authored-by: Régis Hanol <regis@hanol.fr>
If the enable_experimental_hashtag_autocomplete setting is
enabled, then we should autolink hashtag references to the
archived channels (e.g. #blah::channel) for a nicer UX, and
just show the channel name if not (since doing #channelName
can lead to weird inconsistent results).
There was an issue with channel archiving, where at times the topic
creation could fail which left the archive in a bad state, as read-only
instead of archived. This commit does several things:
* Changes the ChatChannelArchiveService to validate the topic being
created first and if it is not valid report the topic creation errors
in the PM we send to the user
* Changes the UI message in the channel with the archive status to reflect
that topic creation failed
* Validate the new topic when starting the archive process from the UI,
and show the validation errors to the user straight away instead of
creating the archive record and starting the process
This also fixes another issue in the discourse_dev config which was
failing because YAML parsing does not enable all classes by default now,
which was making the seeding rake task for chat fail.
* FEATURE: Enforce mention limits for chat messages
The first part of these changes adds a new setting called `max_mentions_per_chat_message`, which skips notifications when the message contains too many mentions. It also respects the `max_users_notified_per_group_mention` setting
and skips notifications if expanding a group mention would exceed it.
We also include a new component to display JIT warning for these limits to the user while composing a message.
* Simplify ignoring/muting filter in chat_notifier
* Post-send warnings for unsent warnings
* Improve pluralization
* Address review feedback
* Fix test
* Address second feedback round
* Third round of feedback
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Only allow maximum of 6000 characters for chat messages when they
are created or edited. A hidden setting can control this limit,
6000 is the default.
There is also a migration here to truncate any existing messages to
6000 characters if the message is already over that and if the
chat_messages table exists. We also set cooked_version to NULL
for those messages so we can identify them for rebake.
This setting limits the number of users in a direct message. 0 means you can only create a direct message with yourself.
Co-authored-by: David McClure <dave@xerotrope.org>
This commit automatically ensures that category channels
have slugs when they are created or updated based on the
channel name, category name, or existing slug. The behaviour
has been copied from the Category model.
We also include a backfill here with a simplified version
of Slug.for with deduplication to fill the slugs for already
created Category chat channels.
The channel slug is also now used for chat notifications,
and for the UI and navigation for chat. `slugifyChannel`
is still used, but now does the following fallback:
* Uses channel.slug if it is present
* Uses channel.escapedTitle if it is present
* Uses channel.title if it is present
In future we may want to remove this altogether
and always rely on the slug being present, but this
is currently not possible because we are not generating
slugs for DM channels at this point.
