Turns out making a html4 fragment and then operating on parts of it using html5 fragments is a bad idea. ;)
This seems to fix the issue with occasionally missing GH icons in oneboxes.
Someone who cannot chat is also not able to join chat channels,
so we may not check all the time user.can_chat? && user.can_join_chat_channel?
and just call user.can_join_chat_channel? instead.
We need a nice way to only return some hashtag data
sources based on various site settings. This commit
adds an enabled? method that every hashtag data source
must implement. If this returns false the data source
will not be used at all for hashtag lookups or search.
It could only occur on message created by the user itself and deleted while the user was looking at the channel.
It more generally fix the trash service which was not correctly setting the author of the delete.
`SiteSetting.enable_public_channels` allows site admin to decide if public channels are available at all. There's no distinction between admins or not as we expect admins to create private category channels if they want to limit usage.
Initial migration and changes to models as well as
changing the following services to update last_message_id:
* Chat::MessageCreator
* Chat::RestoreMessage
* Chat::TrashMessage
The data migration will set the `last_message_id` for all existing
threads and channels in the database.
When we query the thread list as well as the channel,
we look at the last message ID for the following:
* Channel - Sorting DM channels, and channel metadata for the list of channels
* Thread - Last reply details for thread indicators and thread list
This commit makes sure we don't load all data into memory when doing CSV exports.
The most important change here made to the recently introduced export of chat
messages (3ea31f4). We were loading all data into memory in the first version, with
this commit it's not the case anymore.
Speaking of old exports. Some of them already use find_each, and it worked as
expected, without loading all data into memory. And it will proceed working as
expected after this commit.
In general, I made sure this change didn't break other CSV exports, first manually, and
then by writing system specs for them. Sadly, I haven't managed yet to make those
specs stable, they work fine locally, but flaky in GitHub actions, so I've disabled them
for now.
I'll be making more changes to the CSV exports code soon, those system specs will be
very helpful. I'll be running them locally, and I hope I'll manage to make them stable
while doing that work.
Trying to fix two issues:
1. Sometimes the publish_new! event for update_thread_original_message
finishes running on the UI before the one for thread_created, in this
case we just want to do nothing because thread_created will fetch the
new thread along with its preview from the server if needed
2. Sometimes the thread GET and /read events were erroring because
last_reply on the thread was nil, this was potentially occuring because
the thread_created event was coming through to the UI before the rest
of MessageCreator was done, so we just move that after the big update
to set thread_id for the new and existing messages in the reply
chain
This commit replaces two existing screens:
- draft
- channel selection modal
Main features compared to existing solutions
- features are now combined, meaning you can for example create multi users DM
- it will show users with chat disabled
- it shows unread state
- hopefully a better look/feel
- lots of small details and fixes...
Other noticeable fixes
- starting a DM with a user, even from the user card and clicking <kbd>Chat</kbd> will not show a green dot for the target user (or even the channel) until a message is actually sent
- it should almost never do a full page reload anymore
---------
Co-authored-by: Martin Brennan <mjrbrennan@gmail.com>
Co-authored-by: Jordan Vidrine <30537603+jordanvidrine@users.noreply.github.com>
Co-authored-by: chapoi <101828855+chapoi@users.noreply.github.com>
Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
Followup to 3f1024de76
The ActiveModel::Types.register(:array) call for chat was
called too late in the Zeitwerk load order in production,
causing this error:
> `lookup': Unknown type :array (ArgumentError)
> raise ArgumentError, "Unknown type #{symbol.inspect}"
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We need to load the type and register it manually before the rest
of the chat files are loaded via the engine and Zeitwerk.
This will be used when we move the channel creation for DMs
to happen when we first send a message in a DM channel to avoid
a double-request. For now we can just have a new API endpoint
for creating this that the existing frontend code can use,
that uses the new service pattern.
This also uses the new policy pattern for services where the policy
can be defined in a class so a more dynamic reason for the policy
failing can be sent to the controller.
Co-authored-by: Loïc Guitaut <loic@discourse.org>
* FEATURE: Sort thread list by unread threads first
This commit changes the thread list to show the threads that
have unread messages at the top of the list sorted by the
last reply date + time, then all other threads sorted by
last reply date + time.
This also fixes some issues by removing the last_reply
relationship on the thread, which did not work for complex
querying scenarios because its order would be discarded.
* FIX: Various fixes for thread list loading
* Use the channel.threadsManager and find the channel first rather
than use activeChannel in the threads manager, otherwise we may
be looking at differenct channels.
* Look at threadsManager directly instead of storing result for threads
list otherwise it can get out of sync because of replace: true in
other places we are loading threads into the store.
* Fix sorting for thread.last_reply, needed a resort.
This small patch registers a new `ActiveModel` type: `array`.
It will split a string on `,` to create a new array. If the value is
already an array, nothing will happen and for all other types, it will
wrap the value in an array.
Here’s an example on an existing contract:
```ruby
attribute :target_usernames
before_validation do
self.target_usernames =
(
if target_usernames.is_a?(String)
target_usernames.split(",")
else
target_usernames
end
)
end
# can be rewritten as:
attribute :target_usernames, :array
```
To export chat messages, go to `/admin/plugins/chat` and click the Create export
button in the _Export chat messages_ section. You'll receive a direct message
when the export is finished.
Currently, this exports all messages from the last 6 months, but not more than
10000 messages.
This exports all chat messages, including messages from private channels and
users' direct conversations. This also exports messages that were deleted.
We had a bug in this code recently, sometimes users saw weird notifications
like:
User mentioned all_mentioned_user_ids in the help chat channel
We fixed that bug in b85d057.
This refactoring is a follow-up to that fix. As that bug showed, it’s quite easy
to introduce a key that may end up being sent to the `NotifyMentioned` job,
which can lead to such weird notifications. This refactoring makes sure that
the `to_notify` hash contains only IDs of users that should be notified about
mentions.
Previously this was defining methods like `UserOption#never?`, `UserOption#all_new?`, `UserOption#dm_and_mentions?`. Now they will be prefixed like `UserOption#chat_header_indicator_never?`
Unfortunately, Discourse's `UserOption` model is not currently autoloaded. That means that modifying it via a `reloadable_patch` will try to apply the changes repeatedly. Normally this doesn't matter since the changes are idempotent. However, introducing ActiveRecord enums is not idempotent - they raise an error if the same enum already exists on the model. This commit adds a check to avoid hitting this 'duplicate definition' error.
Reproduced
```
rails runner 'Rails.application.reloader.reload!'
```
This commit adds a tracking dropdown to each individual thread, similar to topics,
that allows the user to change the notification level for a thread manually. Previously
the user had to reply to a thread to track it and see unread indicators.
Since the user can now manually track threads, the thread index has also been changed
to only show threads that the user is a member of, rather than threads that they had sent
messages in.
Unread indicators also respect the notification level -- Normal level thread tracking
will not show unread indicators in the UI when new messages are sent in the thread.
This commit adds the initial part of thread indicator improvements:
* Show the reply count, last reply date and excerpt,
and the participants of the thread's avatars and
count of additional participants
* Add a participants component for the thread that
can be reused for the list
* Add a query class to get the thread participants
* Live update the thread indicator more consistently
with the last reply and participant details
image image
In subsequent PRs we will cache the participants since
they do not change often, and improve the thread list
further with participants.
This commit also adds a showPresence boolean (default
true) to ChatUserAvatar, since we don't want to show the
online indicator for thread participants.
---------
Co-authored-by: chapoi <charlie@discourse.org>
When a thread is created / a new message is created in the
thread, we want to make sure that the original message user
has a membership for that thread, otherwise they will not
receive unread indicators for messages in the thread.
This patch introduces policy objects to chat services. It allows putting
more complex logic in a dedicated class, which will make services
thinner. It also allows providing a reason why the policy failed.
Some change has been made to the service runner too to use more easily
these new policy objects: when matching a failing policy (or any failing
step actually), the result object is now provided to the block. This
way, instead of having to access the reason why the policy failed by
doing `result["result.policy.policy_name"].reason` inside the block,
this one can be simply written like this:
```ruby
on_failed_policy(:policy_name) { |policy| policy.reason }
```
This commit makes some fundamental changes to how hashtag cooking and
icon generation works in the new experimental hashtag autocomplete mode.
Previously we cooked the appropriate SVG icon with the cooked hashtag,
though this has proved inflexible especially for theming purposes.
Instead, we now cook a data-ID attribute with the hashtag and add a new
span as an icon placeholder. This is replaced on the client side with an
icon (or a square span in the case of categories) on the client side via
the decorateCooked API for posts and chat messages.
This client side logic uses the generated hashtag, category, and channel
CSS classes added in a previous commit.
This is missing changes to the sidebar to use the new generated CSS
classes and also colors and the split square for categories in the
hashtag autocomplete menu -- I will tackle this in a separate PR so it
is clearer.
This moves chat tracking state calculation for channels
and threads into a central Chat::TrackingStateManager service, that
serves a similar purpose to the TopicTrackingState model
in core.
This service calls down to these query classes:
* ThreadUnreadsQuery
* ChannelUnreadsQuery
To get the unread_count and mention_count for the appropriate
channels and threads.
As well as this, this commit refactors the client-side chat
tracking state.
Now, there is a central ChatTrackingStateManager Ember Service
so all tracking is accessible and can be counted from one place,
which can also initialize tracking from an initial payload.
The actual tracking counts are now maintained in a ChatTrackingState
class that is initialized on the `.tracking` property of both channel and
thread objects.
This removes the attributes on UserChatChannelMembership and decoration
of said membership from ChannelFetcher, preferring instead to have an additional
object for tracking in the JSON.
A follow-up to 54b2a85b. That commit didn't fix the issue because the to_notify hash that we return from the notify_edit method isn't used anywhere apart from tests (that's confusing, we're going to fix that soon).
In the past, we create a `chat_mention` records only when we wanted to notify a user about a mention. Since we don't send notifications when a user mentioning himself, we didn't create a `chat_mention` records in those cases.
Now we use `chat_mentions` records in other scenarios too, so when a user is mentioning himself we want to:
1. Create a `chat_mention` record for that mention
2. Do not create a notification for that mention
When the user sends a message in a thread, we want to
create a membership for them in the background (default
to notification level of Watching) so we can track whether
they have read the thread.
Then, for now since we don't have granular message reading/
scrolling in the thread panel, we just update the thread
last_read_message_id for the user to the latest reply in the
thread when they open the thread panel. This at least will
mark the thread as read.
In future PRs we want to show the blue dot indicator in various
places in the UI for unread threads which will also require
some MessageBus functionality.
This takes into account the same issue fixed for channels
in ae3231e140
When setting DISCOURSE_ZEITWERK_EAGER_LOAD=1 to enable
eager loading the previous solution to adding chat_levels
to the core NotificationLevels would break with a module
loading error (c.f. cc2570fce3)
We don't actually _need_ to extend the core class, we can just
make our own for chat, let's do this instead.
This will enable us to begin work on user tracking
state for a thread so we can show thread-specific
unreads and mentions indicators. In this case are following
the core notification_level paradigm rather than the solution
UserChatChannelMembership went with, and eventually we
will want to refactor the other table to match this as well.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
After this change, in order to join a chat channel, a user needs to be in a group with at least “Reply” permission for the category. If the user only has “See” permission, they are able to preview the channel, but not join it or send messages. The auto-join function also follows this new restriction.
---------
Co-authored-by: Martin Brennan <martin@discourse.org>
This commit adds an initial thread list UI. There are several limitations
with this that will be addressed in future PRs:
* There is no MessageBus reactivity, so e.g. if someone edits the original
message of the thread it will not be reflected in the list. However if
the thread title is updated the original message indicator will be updated.
* There is no unread functionality for threads in the list, if new messages
come into the thread there is no indicator in the UI.
* There is no unread indicator on the actual button to open the thread list.
* No pagination.
In saying that, this is the functionality so far:
* We show a list of the 50 threads that the user has most recently participated
in (i.e. sent a message) for the channel in descending order.
* Each thread we show a rich excerpt, the title, and the user who is the OM creator.
* The title is editable by staff and by the OM creator.
* Thread indicators show a title. We also replace emojis in the titles.
* Thread list works in the drawer/mobile.
When making the list of users to notify we set `all_mentioned_user_ids` key on the `to_notify` Hash.
This hash will be passed around until the actual moment where we send the notifications:
```ruby
identifier_text =
case identifier_type
when :here_mentions
"@here"
when :global_mentions
"@all"
when :direct_mentions
""
else
"@#{identifier_type}"
end
```
As not found `all_mentioned_user_ids` would end up being sent as `@all_mentioned_user_ids` which is obviously incorrect.
This commit is a direct fix to the issue and will remove the key as soon as we have used it sooner up in the chain.
This bug was reproducible when doing this sequence of events:
- create a message with a direct mention: `@bob hi`
- edit this message into a global mention `@all hi`
Every replies creates a thread, even when threading is disabled. This is how we ensure we can go back and forth. However, a message bus event should only be published when threading is enabled, otherwise frontend will attempt to display a thread which is not possible when disabled.
This fixes a silent background 404 when doing a reply in a direct message channel or a non threading enabled category channel.
What is the problem?
Previously, this was the query used to move change messages into another
channel.
```
INSERT INTO chat_messages(
chat_channel_id, user_id, last_editor_id, message, cooked, cooked_version, created_at, updated_at
)
SELECT :destination_channel_id,
user_id,
last_editor_id,
message,
cooked,
cooked_version,
CLOCK_TIMESTAMP(),
CLOCK_TIMESTAMP()
FROM chat_messages
WHERE id IN (:message_ids)
RETURNING id
```
The problem is that this incorrectly assumes that the insertion will be based on the order of `message_ids`. However, that
is not the case as PostgreSQL provides no such guarantee. Instead we need to explicitly order the messages to ensure
the right order of insertion.
This problem was discovered by a flaky test which exposed the non-guarantee order of insertion.
We need to create and update `chat_mentions` records for messages earlier. They should be created or updated before we call `Chat::Publisher.publish_new!` `Chat::Publisher.publish_edit!` to send the message to message bus subscribers).
This logic is covered with tests in `message_creator_spec.rb`, `message_updater_spec.rb`, `notifier_spec.rb` and `notify_mentioned_spec.rb`.
See the commits history for steps of refactoring.
This commit implements all the necessary logic to create thread seamlessly. For this it relies on the same logic used for messages and generates a `staged-id`(using the format: `staged-thread-CHANNEL_ID-MESSAGE_ID` which is used to re-conciliate state client sides once the thread has been persisted on the backend.
Part of this change the client side is now always using real thread and channel objects instead of sometimes relying on a flat `threadId` or `channelId`.
This PR also brings three UX changes:
- thread starts from top
- number of buttons on message actions is dependent of the width of the enclosing container
- <kbd>shift + ArrowUp</kbd> will reply to the last message
This was reverted in 38cebd3ed5.
The issue was that I was using Discourse.redis.delete_prefixed
which does a slow redis KEYS lookup, which is not advised in
production. This commit removes that, and also ensures the periodical
thread count update only happens if threading is enabled.
I changed to use a redis INCR/DECR for reply count
cache. This avoids a round trip to redis to GET the current
count, and also avoids multi-process issues, where
if there's two processes trying to increment at the
same time, they may both receive the same value, add one
to it, then both write the same value back.
Then, it's only n+1 instead of n+2.
This also prevents almost all chat scheduled jobs from
running if chat is disabled, the only one remaining is
the message retention job.
This commit moves the category channel creation out
of the Chat::Api::Channel controller and into a
dedicated CreateCategoryChannel service. A follow up
commit will move the DM channel creation out of
the old DirectMessageChannelCreator service.
Also includes a new on_model_errors helper
for chat service class usage, that collects model
validation errors to present in a nice way.
---------
Co-authored-by: Loïc Guitaut <loic@discourse.org>
Steps to reproduce the bug:
1. Send a chat message
2. Edit the message and add a mention to it
3. The mentioned user won't receive a notification
This PR fixes the problem.
Also:
1. There's no need anymore to have a code for removing notifications in the `notify_edit` method, because a call to `@chat_message.update_mentions` in the first line of the `notify_edit` method does that job:
ff56f403a2/plugins/chat/lib/chat/notifier.rb (L90)
2. There's no need to load mention records from database, it's enough to pluck user ids
This is to help generate random channels and chat
messages for local dev. This was removed in 12a18d4d55
presumably because it was not worth refactoring at the
time.
I've only added these tasks:
- `rake chat:message:populate\[113,20\]` (channel_id, count)
- Generates the count of messages for a channel ID provided,
otherwise uses a random channel and 200 count.
- `rake chat:category_channel:populate`
- Creates a chat channel for a random category.
- `rake chat🧵populate\[132,5\]` (channel_id, message_count)
- Creates a thread with N messages in the specified channel,
and enables threading in that channel if necessary
We currently don't have a nice UI to show unread messages for the thread,
and it will take some time to create one. For now, this commit makes it so
new messages inside a thread do not count towards a chat channel's unread
counts, and new messages sent in a thread do not update a user's `last_read_message_id`
for a channel.
In addition, this PR refactors the `Chat::ChannelFetcher` to use the `Chat::ChannelUnreadsQuery`
query class for consistency, and made said class able to return zeroed-out records
for channels the user is not a member of.
Finally, a small bug is fixed here where if a user's `last_read_message_id` for
a channel was a thread's OM ID, then the thread OM would not show in the
main channel stream for them until another reply to the channel was posted.
We've found these exceptions in logs:
Job exception: undefined method `destroy!' for nil:NilClass
/var/www/discourse/plugins/chat/lib/chat/notifier.rb:102:in `block in notify_edit'
/var/www/discourse/plugins/chat/lib/chat/notifier.rb💯in `each'
/var/www/discourse/plugins/chat/lib/chat/notifier.rb💯in `notify_edit'
/var/www/discourse/plugins/chat/app/jobs/regular/chat/send_message_notifications.rb:18:in `execute'
In the past, we were creating `chat_mention` records only for sending notifications, so every mention record had a related notification. It isn't the case anymore (since fa543cda). This PR fixes the problem by making sure the notification exists before trying to remove it. Also, we shouldn't be deleting a `chat_mention` record itself, only a notification, this PR fixes that too.
It's quite hard to reproduce this bug locally, I wasn't able to do so, the logic in this class is quite complicated, that's why I'm not adding a test. Also, when looking at this I realized that this method isn't in a fully correct state now, I suspect sometimes some notifications may not be delivered after someone edits a chat message and adds new mentions to it. I'm going to refactor and simplify the method in a subsequent PR.
This commit introduces a redis cache over the top of the thread
replies_count DB cache, so that we can quickly and accurately
increment/decrement the reply count for all users and not have
to constantly update the database-level count. This is done so
the UI can have a count that is displayed to the users on each
thread indicator, that appears to live update on each chat
message create/trash/recover inside the thread.
This commit also introduces the `Chat::RestoreMessage` service
and moves the restore endpoint into the `Api::ChannelMessages`
controller as part of incremental migrations to move things out
of ChatController.
Finally, this commit refactors `Chat::Publisher` to be less repetitive
with its `MessageBus` sending code.
This commit introduces a new thread indicator for channels with `threading_enabled`
set to true and the `enable_exp` site setting set to true. In addition, in the main channel
stream we now hide all messages that are linked to threads except for the original message,
disabling the concept of an "echo mode" for now, we may revisit this in future. We also
remove the jigsaw puzzle "Open Thread" button for message actions, since the thread
indicator can just be used instead.
This also stops the `Chat::Publisher` from sending any messages related to chat
messages that are linked to a thread, unless that chat message is the OM of the
thread. A subsequent PR will link up all MessageBus events within the thread panel,
and for the message indicators.
Another subsequent PR will add the excerpt of the latest message in each thread,
as well as the avatars of the users messaging in the thread.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
Followup to 0924f874bd,
we migrated Chat::Upload records to UploadReference records
there and have not been making new Chat::Upload records
for some time, we can now delete the model and table.
This commit adds a keyboard shortcut (Shift+ESC) for chat which marks all
of the chat channels that the user is currently a following member of as read,
updating their `last_read_message_id`. This is done via a new service.
It also includes some refactors and controller changes:
* The old mark message read route from `ChatController` is now supplanted
by the `Chat::Api::ReadsController#update` route.
* The new controller can handle either marking a single or all messages read,
and uses the correct service based on the route and params.
* The `UpdateUserLastRead` service is now used (it wasn't before), and has been slightly
updated to just use the guardian user ID.
There are many situations that may cause users to lose permission to
send messages in a chat channel. Until now we have relied on security
checks in `Chat::ChatChannelFetcher` to remove channels which the
user may have a `UserChatChannelMembership` record for but which
they do not have access to.
This commit takes a more proactive approach. Now any of these following
`DiscourseEvent` triggers may cause `UserChatChannelMembership`
records to be deleted:
* `category_updated` - Permissions of the category changed
(i.e. CategoryGroup records changed)
* `user_removed_from_group` - Means the user may not be able to access the
channel based on `GroupUser` or also `chat_allowed_groups`
* `site_setting_changed` - The `chat_allowed_groups` was updated, some
users may no longer be in groups that can access chat.
* `group_destroyed` - Means the user may not be able to access the
channel based on `GroupUser` or also `chat_allowed_groups`
All of these are handled in a distinct service run in a background
job. Users removed are logged via `StaffActionLog` and then we
publish messages on a per-channel basis to users who had their
memberships deleted.
When the user has a channel they are kicked from open, we show
a dialog saying "You no longer have access to this channel".
When they click OK we redirect them either:
* To their first other public channel, if they have any followed
* The chat browse page if they don't
This is to save on tons of requests from kicked out users getting messages
from other channels.
When the user does not have the kicked channel open, we can just
silently yoink it out of their sidebar and turn off subscriptions.
This refactoring simplifies ChatNotifier a bit. I wanted to drop
that argument for expand_direct_mentions too, but that needs
a bit deeper refactoring, so it's better to do it separately.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
This commit main goal was to comply with Zeitwerk and properly rely on autoloading. To achieve this, most resources have been namespaced under the `Chat` module.
- Given all models are now namespaced with `Chat::` and would change the stored types in DB when using polymorphism or STI (single table inheritance), this commit uses various Rails methods to ensure proper class is loaded and the stored name in DB is unchanged, eg: `Chat::Message` model will be stored as `"ChatMessage"`, and `"ChatMessage"` will correctly load `Chat::Message` model.
- Jobs are now using constants only, eg: `Jobs::Chat::Foo` and should only be enqueued this way
Notes:
- This commit also used this opportunity to limit the number of registered css files in plugin.rb
- `discourse_dev` support has been removed within this commit and will be reintroduced later
<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
This regressed with the commit fa543cd. Starting from that commit, we create mention records even if a user shouldn't be notified. So when sending emails, we should be making sure if a notification was actually created for a mention. This is essentially the whole fix that we need here. Tests will be provided in a following PR.
Before this commit, we created a chat mention record only in case we wanted to send a notification about that mention to the user. Notifications were the only use case for the chat_mention db table. Now we want to use that table for other features, so we have to always create a chat_mention record.
This commit allows the user to set their preference vis-a-vis
the chat icon in the header of the page. There are three options:
- All New (default) - This maintains the existing behaviour where
all new messages in the channel show a blue dot on the icon
- Direct Messages and Mentions - Only show the green dot on the
icon when you are directly messaged or mentioned, the blue dot
is never shown
- Never - Never show any dot on the chat icon, for those who
want tractor-beam-laser-focus
This patch introduces a new `ServiceJob` class allowing the use of
`with_service` in jobs.
This way, it’s easier to use the chat service objects in jobs and
provides the same level of functionality than the one we have in
controllers.
* FIX: Use pluralized string
* REFACTOR: Fix misuse of pluralized string
* REFACTOR: Fix misuse of pluralized string
* DEV: Remove linting of `one` key in MessageFormat string, it doesn't work
* REFACTOR: Fix misuse of pluralized string
This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff. The string is quite complicated, so the best option was to switch to MessageFormat.
* REFACTOR: Fix misuse of pluralized string
* FIX: Use pluralized string
This also ensures that the URL works on subfolder and shows the site setting link only for admins instead of staff.
* REFACTOR: Correctly pluralize reaction tooltips in chat
This also ensures that maximum 5 usernames are shown and fixes the number of "others" which was off by 1 if the current user reacted on a message.
* REFACTOR: Use translatable string as comma separator
* DEV: Add comment to translation to clarify the meaning of `%{identifier}`
* REFACTOR: Use translatable comma separator and use explicit interpolation keys
* REFACTOR: Don't interpolate lowercase channel status
* REFACTOR: Fix misuse of pluralized string
* REFACTOR: Don't interpolate channel status
* REFACTOR: Use %{count} interpolation key
* REFACTOR: Fix misuse of pluralized string
* REFACTOR: Correctly pluralize DM chat channel titles
Initially, the ChatMention model / db table was introduced to better support notifications (see discourse/discourse-chat@0801d10). That means that currently, we create a new chat_mention record only if a user will be notified about the mention.
Now we plan to start using the ChatMention model in other scenarios (for example for implementing user status on mentions) so we need to always create a new record in the chat_mention table. This PR does the first step into that direction by decoupling the logic for extracting and expanding mentions from the code related to notifications.
This doesn't change any behavior, only extracts code from ChatNotifier.
We were calling the job with a symbol as one of the values:
```ruby
Jobs.enqueue(
:send_message_notifications,
chat_message_id: 1,
timestamp: Time.now.iso8601(6),
reason: :new,
)
```
Which is a bad pattern as when the job serialisation will happen, `:new` will become `"new"` and you have to deal with a string in your job and not a symbol, which can be confusing and lead to bugs.
This commit fixes the UpdateUserLastRead spec which was checking
for a message ID that did not exist -- this could fail at times
since message ID 2 could exist. Better to create + destroy a message
since then it's guaranteed we have a unique ID.
This also attempts to clarify a step that we expect to fail which
succeeds instead by adding another emoji next to the success tick and
an explanation text.
Also removes some uses of unless in Services::Base, we generally prefer
to use alternatives, since unless can be hard to parse in a lot of
cases.
Co-authored-by: Loïc Guitaut <loic@discourse.org>
This commit introduces the skeleton of the chat thread UI. The
structure of the components looks like this. Its done this way
so the side panel can be used for other things as well if we wish,
not just for threads:
```
.main-chat-outlet
<ChatLivePane />
<ChatSidePanel>
<-- rendered with {{outlet}} -->
<ChatThread />
</ChatSidePanel>
```
Later on the `ChatThreadList` will be rendered here as well.
Now, when you go to a channel you can open a thread by clicking
on either the Open Thread message action button or by clicking on
the reply indicator. This will take you to a route like `chat/c/:slug/:channelId/t/:threadId`.
This works on mobile as well.
This commit includes basic serializers and routes for threads,
as well as a new `ChatThreadsManager` service in JS that caches
threads for a channel the same way the channel threads manager does.
The chat messages inside the thread are intentionally left out
until a later PR.
**NOTE: These changes are gated behind the site setting enable_experimental_chat_threaded_discussions
and the threading_enabled boolean on a ChatChannel**
This is a combined work of Martin Brennan, Loïc Guitaut, and Joffrey Jaffeux.
---
This commit implements a base service object when working in chat. The documentation is available at https://discourse.github.io/discourse/chat/backend/Chat/Service.html
Generating documentation has been made as part of this commit with a bigger goal in mind of generally making it easier to dive into the chat project.
Working with services generally involves 3 parts:
- The service object itself, which is a series of steps where few of them are specialized (model, transaction, policy)
```ruby
class UpdateAge
include Chat::Service::Base
model :user, :fetch_user
policy :can_see_user
contract
step :update_age
class Contract
attribute :age, :integer
end
def fetch_user(user_id:, **)
User.find_by(id: user_id)
end
def can_see_user(guardian:, **)
guardian.can_see_user(user)
end
def update_age(age:, **)
user.update!(age: age)
end
end
```
- The `with_service` controller helper, handling success and failure of the service within a service and making easy to return proper response to it from the controller
```ruby
def update
with_service(UpdateAge) do
on_success { render_serialized(result.user, BasicUserSerializer, root: "user") }
end
end
```
- Rspec matchers and steps inspector, improving the dev experience while creating specs for a service
```ruby
RSpec.describe(UpdateAge) do
subject(:result) do
described_class.call(guardian: guardian, user_id: user.id, age: age)
end
fab!(:user) { Fabricate(:user) }
fab!(:current_user) { Fabricate(:admin) }
let(:guardian) { Guardian.new(current_user) }
let(:age) { 1 }
it { expect(user.reload.age).to eq(age) }
end
```
Note in case of unexpected failure in your spec, the output will give all the relevant information:
```
1) UpdateAge when no channel_id is given is expected to fail to find a model named 'user'
Failure/Error: it { is_expected.to fail_to_find_a_model(:user) }
Expected model 'foo' (key: 'result.model.user') was not found in the result object.
[1/4] [model] 'user' ❌
[2/4] [policy] 'can_see_user'
[3/4] [contract] 'default'
[4/4] [step] 'update_age'
/Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/update_age.rb:32:in `fetch_user': missing keyword: :user_id (ArgumentError)
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `instance_exec'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:202:in `call'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:219:in `call'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `block in run!'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `each'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:417:in `run!'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:411:in `run'
from <internal:kernel>:90:in `tap'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/app/services/base.rb:302:in `call'
from /Users/joffreyjaffeux/Code/pr-discourse/plugins/chat/spec/services/update_age_spec.rb:15:in `block (3 levels) in <main>'
```
Whenever we create a chat message that is `in_reply_to` another
message, we want to lazily populate the thread record for the
message chain.
If there is no thread yet for the root message in the reply chain,
we create a new thread with the appropriate details, and use that
thread ID for every message in the chain that does not yet have
a thread ID.
* Root message (ID 1) - no thread ID
* Message (ID 2, in_reply_to 1) - no thread ID
* When I as a user create a message in reply to ID 2, we create a thread and apply it to ID 1, ID 2, and the new message
If there is a thread for the root message in the reply chain, we
do not create one, and use the thread ID for the newly created chat
message.
* Root message (ID 1) - thread ID 700
* Message (ID 2, in_reply_to 1) - thread ID 700
* When I as a user create a message in reply to ID 2, we use the existing thread ID 700 for the new message
We also support passing in the `thread_id` to `ChatMessageCreator`,
which will be used when replying to a message that is already part of
a thread, and we validate whether that `thread_id` is okay in the context
of the channel and also the reply chain.
This work is always done, regardless of channel `thread_enabled` settings
or the `enable_experimental_chat_threaded_discussions` site setting.
This commit does not include a large data migration to backfill threads for
all existing reply chains, its unnecessary to do this so early in the project,
we can do this later if necessary.
This commit also includes thread considerations in the `MessageMover` class:
* If the original message and N other messages of a thread is moved,
the remaining messages in the thread have a new thread created in
the old channel and are moved to it.
* The reply chain is not preserved for moved messages, so new threads are
not created in the destination channel.
In addition to this, I added a fix to also clear the `in_reply_to_id` of messages
in the old channel which are moved out of that channel for data cleanliness.
Whenever we create a chat message that is `in_reply_to` another
message, we want to lazily populate the thread record for the
message chain.
If there is no thread yet for the root message in the reply chain,
we create a new thread with the appropriate details, and use that
thread ID for every message in the chain that does not yet have
a thread ID.
* Root message (ID 1) - no thread ID
* Message (ID 2, in_reply_to 1) - no thread ID
* When I as a user create a message in reply to ID 2, we create a thread and apply it to ID 1, ID 2, and the new message
If there is a thread for the root message in the reply chain, we
do not create one, and use the thread ID for the newly created chat
message.
* Root message (ID 1) - thread ID 700
* Message (ID 2, in_reply_to 1) - thread ID 700
* When I as a user create a message in reply to ID 2, we use the existing thread ID 700 for the new message
We also support passing in the `thread_id` to `ChatMessageCreator`,
which will be used when replying to a message that is already part of
a thread, and we validate whether that `thread_id` is okay in the context
of the channel and also the reply chain.
This work is always done, regardless of channel `thread_enabled` settings
or the `enable_experimental_chat_threaded_discussions` site setting.
This commit does not include a large data migration to backfill threads for
all existing reply chains, its unnecessary to do this so early in the project,
we can do this later if necessary.
This commit also includes thread considerations in the `MessageMover` class:
* If the original message and N other messages of a thread is moved,
the remaining messages in the thread have a new thread created in
the old channel and are moved to it.
* The reply chain is not preserved for moved messages, so new threads are
not created in the destination channel.
In addition to this, I added a fix to also clear the `in_reply_to_id` of messages
in the old channel which are moved out of that channel for data cleanliness.
`last_message_sent_at` could be equal and as a result the order would be random causing random spec failures in plugins/chat/spec/mailers/user_notifications_spec.rb:182
* DEV: Rnemae channel path to just c
Also swap the channel id and channel slug params to be consistent with core.
* linting
* channel_path
* params in wrong order
* Drop slugify helper and channel route without slug
* Request slug and route models through the channel model if possible
* Add client side redirection for backwards-compatibility
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
We've had the UploadReference table for some time now in core,
but it was added after ChatUpload was and chat was just never
moved over to this new system.
This commit changes all chat code dealing with uploads to create/
update/delete/query UploadReference records instead of ChatUpload
records for consistency. At a later date we will drop the ChatUpload
table, but for now keeping it for data backup.
The migration + post migration are the same, we need both in case
any chat uploads are added/removed during deploy.
Note this commit also slightly changes internal API: channel instead of getChannel and updateCurrentUserChannelNotificationsSettings instead of updateCurrentUserChatChannelNotificationsSettings.
Also destroyChannel takes a second param which is the name confirmation instead of an optional object containing this confirmation. This is to enforce the fact that it's required.
In the future a top level jsdoc config file could be used instead of the hack tempfile, but while it's only an experiment for chat, it's probably good enough.
If the enable_experimental_hashtag_autocomplete setting is
enabled, then we should autolink hashtag references to the
archived channels (e.g. #blah::channel) for a nicer UX, and
just show the channel name if not (since doing #channelName
can lead to weird inconsistent results).
There was an issue with channel archiving, where at times the topic
creation could fail which left the archive in a bad state, as read-only
instead of archived. This commit does several things:
* Changes the ChatChannelArchiveService to validate the topic being
created first and if it is not valid report the topic creation errors
in the PM we send to the user
* Changes the UI message in the channel with the archive status to reflect
that topic creation failed
* Validate the new topic when starting the archive process from the UI,
and show the validation errors to the user straight away instead of
creating the archive record and starting the process
This also fixes another issue in the discourse_dev config which was
failing because YAML parsing does not enable all classes by default now,
which was making the seeding rake task for chat fail.
`last_message_sent_at` has a `NOT_NULL` constraint in the DB so it should be safe to use for sorting.
This was causing two flakeys:
```
1) UserNotifications.chat_summary with public channel email subject with regular mentions includes both channel titles when there are exactly two with unread mentions
Failure/Error: example.run
expected: "[Discourse] New message in Random 62 and Test channel"
got: "[Discourse] New message in Test channel and Random 62"
(compared using ==)
# ./plugins/chat/spec/mailers/user_notifications_spec.rb:203:in `block (6 levels) in <main>'
# ./spec/rails_helper.rb:356:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
2) UserNotifications.chat_summary with public channel email subject with regular mentions displays a count when there are more than two channels with unread mentions
Failure/Error: example.run
expected: "[Discourse] New message in Random 62 and 2 others"
got: "[Discourse] New message in Test channel 0 and 2 others"
(compared using ==)
# ./plugins/chat/spec/mailers/user_notifications_spec.rb:236:in `block (6 levels) in <main>'
# ./spec/rails_helper.rb:356:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```
This PR removes the limit added to max_users_notified_per_group_mention during #19034 and improve the performance when expanding mentions for large channel or groups by removing some N+1 queries and making the whole process async.
* Fully async chat message notifications
* Remove mention setting limit and get rid of N+1 queries
Note this is a very large PR, and some of it could have been splited, but keeping it one chunk made it to merge conflicts and to revert if necessary. Actual new code logic is also not that much, as most of the changes are removing js tests, adding system specs or moving things around.
To make it possible this commit is doing the following changes:
- converting (and adding new) existing js acceptances tests into system tests. This change was necessary to ensure as little regressions as possible while changing paradigm
- moving away from store. Using glimmer and tracked properties requires to have class objects everywhere and as a result works well with models. However store/adapters are suffering from many bugs and limitations. As a workaround the `chat-api` and `chat-channels-manager` are an answer to this problem by encapsulating backend calls and frontend storage logic; while still using js models.
- dropping `appEvents` as much as possible. Using tracked properties and a better local storage of channel models, allows to be much more reactive and doesn’t require arbitrary manual updates everywhere in the app.
- while working on replacing store, the existing work of a chat api (backend) has been continued to support more cases.
- removing code from the `chat` service to separate concerns, `chat-subscriptions-manager` and `chat-channels-manager`, being the largest examples of where the code has been rewritten/moved.
Future wok:
- improve behavior when closing/deleting a channel, it's already slightly buggy on live, it's rare enough that it's not a big issue, but should be improved
- improve page objects used in chat
- move more endpoints to the API
- finish temporarily skipped tests
- extract more code from the `chat` service
- use glimmer for `chat-messages`
- separate concerns in `chat-live-pane`
- eventually add js tests for `chat-api`, `chat-channels-manager` and `chat-subscriptions-manager`, they are indirectly heavy tested through system tests but it would be nice to at least test the public API
<!-- NOTE: All pull requests should have tests (rspec in Ruby, qunit in JavaScript). If your code does not include test coverage, please include an explanation of why it was omitted. -->
Follow up to a review in #18937, this commit changes the HashtagAutocompleteService to no longer use class variables to register hashtag data sources or types in context priority order. This is to address multisite concerns, where one site could e.g. have chat disabled and another might not. The filtered plugin registers I added will not be included if the plugin is disabled.
In both ChatMessage#rebake! and in ChatMessageProcessor
when we were calling ChatMessage.cook we were missing the
user_id to cook with, which causes missed hashtag cooks
because of missing permissions.
Previously, restricted category chat channel was available for all groups - even `readonly`. From now on, only user who belong to group with `create_post` or `full` permissions can access that chat channel.
This introduces another "section" of queries to the
hashtag autocomplete search, which returns results for
each type that start with the search term. So now results
will be in this order, and within these sections ordered
by the types in priority order:
1. Exact matches sorted by type
2. "starts with" sorted by type
3. Everything else sorted by type then name within type
Instead of passing `user` to `guardian.can_chat?`, we
can just use the inner `@user` that is part of the guardian
instance already to determine whether that user can chat,
since this is how it works for all other usages of guardian
even within chat.
Previously with this experimental feature a user would be
able to search for public channels for public categories
using the new #hashtag system even if they couldn't chat.
This commit fixes the hole.
This commit allows us to type # in the UI and present autocomplete
results immediately with the following logic for the topic composer,
and reversed for the chat composer:
* Categories the user can access and has not muted sorted by `topic_count`
* Tags the user can access and has not muted sorted by `topic_count`
* Chat channels the user is a member of sorted by `messages_count`
So in effect, we allow searching for hashtags without a search term.
To do this we add a new `search_without_term` to each data source so
each one can define how it wants to handle this logic.
* FEATURE: Enforce mention limits for chat messages
The first part of these changes adds a new setting called `max_mentions_per_chat_message`, which skips notifications when the message contains too many mentions. It also respects the `max_users_notified_per_group_mention` setting
and skips notifications if expanding a group mention would exceed it.
We also include a new component to display JIT warning for these limits to the user while composing a message.
* Simplify ignoring/muting filter in chat_notifier
* Post-send warnings for unsent warnings
* Improve pluralization
* Address review feedback
* Fix test
* Address second feedback round
* Third round of feedback
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
The settings tab of each category channel should now present the option to allow or disallow channel wide mentions: @here and @all.
When disallowed, using these mentions in the channel should have no effect.
When searching for categories it is possible for
a child category to have a slug that matches the term
exactly, but will not be found by .lookup since we
don't return these categories unless the ref matches
parent:child.
Introduces a search_sort method to each hashtag data
source so they can provide their custom sort logic of
results, in category's case putting all matching slugs
to the top regardless of parent/child relationship
then sorting by text.
* Do not search category name when searching channels to avoid
confusing results
* Overflow text in autocomplete menu with ... if it is too long
* Make autocomplete menu less height
This changes the hashtag search to first do a lookup to find
results where the slug exactly matches the
search term. Now when we search for hashtags, the
exact matches will be found first and put at the top of
the results.
`ChatChannelFetcher` has also been modified here to allow
for more options for performance -- we do not need to
query DM channels for secured IDs when looking up or searching
channels for hashtags, since they should never show in
results there (they have no slugs). Nor do we need to include
the channel archive records.
Also changes the limit of hashtag results to 20 by default
with a hidden site setting, and makes it so the scroll for the
results is overflowed.
Adds the description as a title="" attribute on the hashtag
autocomplete search items for tags, categories, and channels.
These descriptions can be seen by the user since they are
able to see the results that are returned by the search via
Guardian checks.
Currently when a category is deleted, if it has an associated chat
channel, the latter won’t be deleted automatically.
The fix is quite simple as we were simply missing a `dependent:
:destroy` option on the existing relation.
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
This setting limits the number of users in a direct message. 0 means you can only create a direct message with yourself.
Co-authored-by: David McClure <dave@xerotrope.org>
The mailer in charge of sending chat summary emails applies a filter to ensure only members of groups listed in the `chat allowed groups` setting receive them. However, when you set it to `everyone`, nobody will be notified because
we treat this group differently and don't create `GroupUser` records for every user on the site.
This commit changes the mailer to skip the filter when the `everyone` ID is in the list.
Currently it’s not possible to delete a category if an associated chat
channel is present even if there are no messages in this channel.
This can lead to annoying situations for our users.
This patch addresses the issue by checking if the channel is empty
instead of just checking if there is a channel.
Follow up to 766bcbc684
Makes ChatMessage.last_editor_id and ChatMessageRevision.user_id
NOT NULL since they are always filled in now and the last commit
had a migration to backfill this data.
Follow up to 766bcbc684
This fixes a gaffe from that commit where I passed in the
guardian to ChatMessageUpdater but then forgot to remove
the old way of setting the guardian and user instance variables
from the chat_message that was passed in.
Also, it moves the ensure_can_edit_message! check from the
controller into ChatMessageUpdater so all the access
checks are in the same place.
This commit adds last_editor_id to ChatMessage for parity with Post in
core, as well as adding user_id to the ChatMessageRevision record since
we need to know who is making edits and revisions to messages, in case
in future we want to allow more than just the current user to edit chat
messages. The backfill for data here simply uses the record's creating
user ID, but in future if we allow other people to edit the messages it
will use their ID.
This is a followup of the previous refactor where we created two new
models to handle all the dedicated logic that was present in the
`ChatChannel` model.
For the sake of consistency, `DMChannel` has been renamed to
`DirectMessageChannel` and the previous `DirectMessageChannel` model is
now named `DirectMessage`. This should help reasoning about direct
messages.
