## What is the problem?
MessageBus by default uses long polling which keeps a connection
open for 25 seconds by default. The problem here is that Capybara does not know about these
connections being kept opened by MessageBus and hence does not know how
to stop these connections at the end of each test. As a result, the long polling MessageBus connections are kept opened by the browser and we hit chrome's limit of 6 concurrent requests per host, new request made in the browser is marked as "pending" until a request is freed up. Since we keep a MessageBus long polling connection opened for 25 seconds, our finders in Capybara end up hitting Capybara's wait time out causing the tests to fail.
## What is the fix?
Since we can't rely on Capybara to close all the existing Capybara
connections, we manually execute a script to stop all MessageBus
connections after each system test.
```
for i in {1..10}; do
echo "Running iteration $i"
PARALLEL_TEST_PROCESSORS=8 CAPYBARA_DEFAULT_MAX_WAIT_TIME=10 bin/turbo_rspec --seed=34908 --profile --verbose --format documentation spec/system
if [ $? -ne 0 ]; then
echo "Error encountered on iteration $i"
exit 1
fi
done
echo "All 10 iterations completed successfully"
```
Without the fix, the script fails consistently in the first few iterations. Running in non-headless mode with the "network" tab opened will reveal the requests that are marked as pending.
Why is this change required?
The flaky system test was due to the fact that we had to poll for the
user preferences interface page to reload after saving. However, this
turns out to be a bug on the user perferences interface page because the
page should only reload if the user has selected a new theme that is
different from the site's default but we were reloading the page for
users that did not have any user theme selected. Therefore there was an
unnecessary reload happening when saving other fields on the user
preferences interface page.
The fix use the SelectKit component in the spec and improves reliability of SelectKit component to ensure expanded/collapsed state effectively set/present.
Multiple lines have also been removed as they are not necessary, eg:
- check button is present
- click button
The check is un-necesssary as we won't find the button on click if not present. This kind of checks are only necessary when another element has to be present before the button is show, eg:
- check modal is displayed
- click button
FInally this commit changes the way the SelectKit component initializes component and now uses a css selector instead of a finder, it ensures we are always getting fresh object and allows to build complete selectors: ".context[data-id=1].foo:enabled"
Watching screenshots of failures it appears that sometimes the reply was stuck at: `this is a #` due to the autocomplete showing, given we only need to send a reply here? I think fill_in + click send should be more reliable here.
Note I also tweaked `send_reply` to accept a content param and use it to fill composer when given.
Page settings can be very long and the setting can be way out of viewport. This should ensure the setting can be found and clicked.
This was for example causing this flakey in discourse-topic-voting:
```
Failures:
1) Voting enables voting in category topics and votes
Failure/Error: find(".edit-category-tab .#{setting} label.checkbox-label", text: text).click
Capybara::ElementNotFound:
Unable to find css ".edit-category-tab .enable-topic-voting label.checkbox-label"
[Screenshot Image]: /__w/discourse/discourse/tmp/capybara/failures_r_spec_example_groups_voting_enables_voting_in_category_topics_and_votes_873.png
~~~~~~~ JS LOGS ~~~~~~~
http://localhost:31341/favicon.ico - Failed to load resource: the server responded with a status of 404 (Not Found)
~~~~~ END JS LOGS ~~~~~
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:312:in `block in synced_resolve'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/base.rb:84:in `synchronize'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:301:in `synced_resolve'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:60:in `find'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/session.rb:773:in `find'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/dsl.rb:52:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/dsl.rb:52:in `find'
# ./spec/system/page_objects/pages/category.rb:34:in `toggle_setting'
# ./plugins/discourse-topic-voting/spec/system/voting_spec.rb:39:in `block (2 levels) in <main>'
# ./spec/rails_helper.rb:368:in `block (3 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:189:in `block in timeout'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `block in catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:198:in `timeout'
# ./spec/rails_helper.rb:364:in `block (2 levels) in <top (required)>'
# ./spec/rails_helper.rb:356:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.2.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
Finished in 7 minutes 18 seconds (files took 0 seconds to load)
445 examples, 1 failure, 17 pending, 1 error occurred outside of examples
Failed examples:
rspec ./plugins/discourse-topic-voting/spec/system/voting_spec.rb:27 # Voting enables voting in category topics and votes
```
In tests we could attempt to click the preview button when it was not enabled yet, causing a noop and a future failure. This fix ensures we try to find (and wait) for an enabled button.
This should help with this specific flakey:
```
Failures:
1) Admin Customize Form Templates when visiting the page to create a new form template should render all the input field types in the preview
Failure/Error: find(".form-template-field__#{type}").present?
Capybara::ElementNotFound:
Unable to find css ".form-template-field__input"
[Screenshot Image]: /__w/discourse/discourse/tmp/capybara/failures_r_spec_example_groups_admin_customize_form_templates_when_visiting_the_page_to_create_a_new_form_template_should_render_all_the_input_field_types_in_the_preview_277.png
~~~~~~~ JS LOGS ~~~~~~~
http://localhost:31339/favicon.ico - Failed to load resource: the server responded with a status of 404 (Not Found)
~~~~~ END JS LOGS ~~~~~
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:312:in `block in synced_resolve'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/base.rb:84:in `synchronize'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:301:in `synced_resolve'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:60:in `find'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/session.rb:773:in `find'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/dsl.rb:52:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/dsl.rb:52:in `find'
# ./spec/system/page_objects/pages/form_template.rb:55:in `has_input_field?'
# ./spec/system/admin_customize_form_templates_spec.rb:114:in `block (3 levels) in <main>'
# ./spec/rails_helper.rb:368:in `block (3 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:189:in `block in timeout'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `block in catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:198:in `timeout'
# ./spec/rails_helper.rb:364:in `block (2 levels) in <top (required)>'
# ./spec/rails_helper.rb:356:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.2.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```
In tests we could attempt to click the button before it was enabled and it could also not be visible on screen. This commits attempts to solve both problems.
It should solve this failure:
```
Failures:
1) Admin Customize Form Templates when visiting the page to create a new form template should allow admin to create a new form template
Failure/Error: find(".form-templates__table tbody tr td", text: name).present?
Capybara::ElementNotFound:
Unable to find css ".form-templates__table tbody tr td"
[Screenshot Image]: /__w/discourse/discourse/tmp/capybara/failures_r_spec_example_groups_admin_customize_form_templates_when_visiting_the_page_to_create_a_new_form_template_should_allow_admin_to_create_a_new_form_template_911.png
~~~~~~~ JS LOGS ~~~~~~~
http://localhost:31339/favicon.ico - Failed to load resource: the server responded with a status of 404 (Not Found)
~~~~~ END JS LOGS ~~~~~
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:312:in `block in synced_resolve'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/base.rb:84:in `synchronize'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:301:in `synced_resolve'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/node/finders.rb:60:in `find'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/session.rb:773:in `find'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/dsl.rb:52:in `call'
# ./vendor/bundle/ruby/3.2.0/gems/capybara-3.39.1/lib/capybara/dsl.rb:52:in `find'
# ./spec/system/page_objects/pages/form_template.rb:21:in `has_form_template?'
# ./spec/system/admin_customize_form_templates_spec.rb:71:in `block (3 levels) in <main>'
# ./spec/rails_helper.rb:381:in `block (3 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:189:in `block in timeout'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `block in catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:36:in `catch'
# ./vendor/bundle/ruby/3.2.0/gems/timeout-0.3.2/lib/timeout.rb:198:in `timeout'
# ./spec/rails_helper.rb:377:in `block (2 levels) in <top (required)>'
# ./spec/rails_helper.rb:369:in `block (2 levels) in <top (required)>'
# ./vendor/bundle/ruby/3.2.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
```
added site toggle functionality through site settings
added tests to implemented feature
Introduced suggested correction
renamed find_new_topic method and deleted click_new_topic_button method
What is the problem?
We are relying on RSpec custom matchers in system tests by defining
predicates in page objects. The problem is that this can result in a
system test unnecessarily waiting up till the full duration of
Capybara's default wait time when the RSpec custom matcher is used with
`not_to`. Considering this topic page object where we have a `has_post?`
predicate defined.
```
class Topic < PageObject
def has_post?
has_css?('something')
end
end
```
The assertion `expect(Topic.new).not_to have_post` will end up waiting
the full Capybara's default wait time since the RSpec custom matcher is
calling Capybara's `has_css?` method which will wait until the selector
appear. If the selector has already disappeared by the time the
assertion is called, we end up waiting for something that will never
exists.
This commit fixes such cases by introducing new predicates that uses
the `has_no_*` versions of Capybara's node matchers.
For future reference, `to have_css` and `not_to have_css` is safe to sue
because the RSpec matcher defined by Capbyara is smart enough to call
`has_css?` or `has_no_css?` based on the expectation of the assertion.
Responding to negative behaviour tends to solicit more of the same. Common wisdom states: "don't feed the trolls".
This change codifies that advice by introducing a new nudge when hitting the reply button on a flagged post. It will be shown if either the current user, or two other users (configurable via a site setting) have flagged the post.
This feature will allow sites to define which emoji are not allowed. Emoji in this list should be excluded from the set we show in the core emoji picker used in the composer for posts when emoji are enabled. And they should not be allowed to be chosen to be added to messages or as reactions in chat.
This feature prevents denied emoji from appearing in the following scenarios:
- topic title and page title
- private messages (topic title and body)
- inserting emojis into a chat
- reacting to chat messages
- using the emoji picker (composer, user status etc)
- using search within emoji picker
It also takes into account the various ways that emojis can be accessed, such as:
- emoji autocomplete suggestions
- emoji favourites (auto populates when adding to emoji deny list for example)
- emoji inline translations
- emoji skintones (ie. for certain hand gestures)
When a category has default_list_filter=none, there were a number of issues which this commit resolves:
1. When using the breadcrumbs to navigate a `default_list_filter=none` category, adding a tag filter would not apply the no-subcategories filter, but the subcategories dropdown would still say 'none'. This commit adjusts `getCategoryAndTagUrl` so that `/none` is added to the URL
2. When landing on `/tags/c/{slug}/{id}/{tag}`, for a default_list_filter=none category, it would include subcategories. This commit introduces a client-side redirect to match the behavior of `/c/{slug}/{id}`
3. When directly navigating to `/c/{slug}/{id}`, it was correctly redirecting to `/c/{slug}/{id}/none`, BUT it was still using the preloaded data for the old route. This has been happening since e7a84948. Prior to that, the preloaded data was discarded and a new JSON request was made to the server. This commit restores that discarding behavior. In future we may want to look into making this more efficient.
System specs are introduced to provide end-end testing of this functionality
* DEV: Change sidebar header dropdown to use wait_for_animation
Introduced in 54351e1b8a, this
helper should remove the need to have to add the .animated
CSS class in JS for the sidebar.
* DEV: Revert spec change
Feature to allow adding new tags from the edit tag synonyms tag search field.
Previously new tags had to be created from the topic composer, and then added via the edit tag synonyms page.
/t/92741
What is the problem?
We have a hidden site setting `show_category_definitions_in_topic_lists`
which is set to false by default. What this means is that category
definition topics are not shown in the topic list by default. Only the
category definition topic for the category being viewed will be shown.
However, we have a bug where we would show that a category has new
topics when a new child category along with its category definition
topic is created even though the topic list does not list the child
category's category definition topic.
What is the fix here?
This commit fixes the problem by shipping down an additional
`is_category_topic` attribute in `TopicTrackingStateItemSerializer` when
the `show_category_definitions_in_topic_lists` site setting has been set
to false. With the new attribute, we can then exclude counting child
categories' category definition topics when counting new and unread
counts for a category.
Small js fix for fast edit to allow posts to save changes when the post contains apostrophes and quotation marks. Replaces unicode characters in text prior to saving the edit.
Includes system tests for fast edit and introduces a new system spec component for fast edit usage.
With the introduction of the sidebar navigation menu, the design team at
Discourse redesigned the user profile navigation to better coexist with
the sidebar.
The latter can be called directly from the Topic page object,
so we can remove some duplication between the two. There are
levels of page objects (e.g. entire page, component, complete flow)
and its perfectly valid to call one from another.
We are all in on system specs, so this commit moves all the chat quoting acceptance tests (some of which have been skipped for a while) into system specs.
Honestly seems like it's being in some weird loop for
discourse/hashtag_autocomplete_spec.rb for this:
```ruby
within topic_page.post_by_number(2) do
cooked_hashtags = page.all(".hashtag-cooked", count: 2)
expect(cooked_hashtags[0]["outerHTML"]).to eq(<<~HTML.chomp)
<a class=\"hashtag-cooked\" href=\"#{category.url}\" data-type=\"category\" data-slug=\"cool-cat\"><svg class=\"fa d-icon d-icon-folder svg-icon svg-node\"><use href=\"#folder\"></use></svg><span>Cool Category</span></a>
HTML
expect(cooked_hashtags[1]["outerHTML"]).to eq(<<~HTML.chomp)
<a class=\"hashtag-cooked\" href=\"#{tag.url}\" data-type=\"tag\" data-slug=\"cooltag\"><svg class=\"fa d-icon d-icon-tag svg-icon svg-node\"><use href=\"#tag\"></use></svg><span>cooltag</span></a>
HTML
end
```
I see this many times in the full logs with `SELENIUM_VERBOSE_DRIVER_LOGS=1`:
```
COMMAND FindElements {
"using": "css selector",
"value": "#post_2"
}
Followed by:
COMMAND FindChildElements {
"id": "26dfe542-659b-46cc-ac8c-a6c2d9cbdf0a",
"using": "css selector",
"value": ".hashtag-cooked"
}
```
Over and over and over, there are 58 such occurrences. I am beginning to
think `within` is just poison that should be avoided.
Fixes an issue on mobile where navigating away from search and returning
results in confusing UI where there are no results but headings says "N
results found".
Fixes broken behaviour of arrow buttons for certain users as the interval to scroll menu can be cancelled before the scrolling actually happens.
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
This commit fleshes out and adds functionality for the new `#hashtag` search and
lookup system, still hidden behind the `enable_experimental_hashtag_autocomplete`
feature flag.
**Serverside**
We have two plugin API registration methods that are used to define data sources
(`register_hashtag_data_source`) and hashtag result type priorities depending on
the context (`register_hashtag_type_in_context`). Reading the comments in plugin.rb
should make it clear what these are doing. Reading the `HashtagAutocompleteService`
in full will likely help a lot as well.
Each data source is responsible for providing its own **lookup** and **search**
method that returns hashtag results based on the arguments provided. For example,
the category hashtag data source has to take into account parent categories and
how they relate, and each data source has to define their own icon to use for the
hashtag, and so on.
The `Site` serializer has two new attributes that source data from `HashtagAutocompleteService`.
There is `hashtag_icons` that is just a simple array of all the different icons that
can be used for allowlisting in our markdown pipeline, and there is `hashtag_context_configurations`
that is used to store the type priority orders for each registered context.
When sending emails, we cannot render the SVG icons for hashtags, so
we need to change the HTML hashtags to the normal `#hashtag` text.
**Markdown**
The `hashtag-autocomplete.js` file is where I have added the new `hashtag-autocomplete`
markdown rule, and like all of our rules this is used to cook the raw text on both the clientside
and on the serverside using MiniRacer. Only on the server side do we actually reach out to
the database with the `hashtagLookup` function, on the clientside we just render a plainer
version of the hashtag HTML. Only in the composer preview do we do further lookups based
on this.
This rule is the first one (that I can find) that uses the `currentUser` based on a passed
in `user_id` for guardian checks in markdown rendering code. This is the `last_editor_id`
for both the post and chat message. In some cases we need to cook without a user present,
so the `Discourse.system_user` is used in this case.
**Chat Channels**
This also contains the changes required for chat so that chat channels can be used
as a data source for hashtag searches and lookups. This data source will only be
used when `enable_experimental_hashtag_autocomplete` is `true`, so we don't have
to worry about channel results suddenly turning up.
------
**Known Rough Edges**
- Onebox excerpts will not render the icon svg/use tags, I plan to address that in a follow up PR
- Selecting a hashtag + pressing the Quote button will result in weird behaviour, I plan to address that in a follow up PR
- Mixed hashtag contexts for hashtags without a type suffix will not work correctly, e.g. #ux which is both a category and a channel slug will resolve to a category when used inside a post or within a [chat] transcript in that post. Users can get around this manually by adding the correct suffix, for example ::channel. We may get to this at some point in future
- Icons will not show for the hashtags in emails since SVG support is so terrible in email (this is not likely to be resolved, but still noting for posterity)
- Additional refinements and review fixes wil
This commit introduces rails system tests run with chromedriver, selenium,
and headless chrome to our testing toolbox.
We use the `webdrivers` gem and `selenium-webdriver` which is what
the latest Rails uses so the tests run locally and in CI out of the box.
You can use `SELENIUM_VERBOSE_DRIVER_LOGS=1` to show extra
verbose logs of what selenium is doing to communicate with the system
tests.
By default JS logs are verbose so errors from JS are shown when
running system tests, you can disable this with
`SELENIUM_DISABLE_VERBOSE_JS_LOGS=1`
You can use `SELENIUM_HEADLESS=0` to run the system
tests inside a chrome browser instead of headless, which can be useful to debug things
and see what the spec sees. See note above about `bin/ember-cli` to avoid
surprises.
I have modified `bin/turbo_rspec` to exclude `spec/system` by default,
support for parallel system specs is a little shaky right now and we don't
want them slowing down the turbo by default either.
### PageObjects and System Tests
To make querying and inspecting parts of the page easier
and more reusable inbetween system tests, we are using the
concept of [PageObjects](https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/) in
our system tests. A "Page" here is generally corresponds to
an overarching ember route, e.g. "Topic" for `/t/324345/some-topic`,
and this contains logic for querying components within the topic
such as "Posts".
I have also split "Modals" into their own entity. Further down the
line we may want to explore creating independent "Component"
contexts.
Capybara DSL should be included in each PageObject class,
reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl
For system tests, since they are so slow, we want to focus on
the "happy path" and not do every different possible context
and branch check using them. They are meant to be overarching
tests that check a number of things are correct using the full stack
from JS and ember to rails to ruby and then the database.
### CI Setup
Whenever a system spec fails, a screenshot
is taken and a build artifact is produced _after the entire CI run is complete_,
which can be downloaded from the Actions UI in the repo.
Most importantly, a step to build the Ember app using Ember CLI
is needed, otherwise the JS assets cannot be found by capybara:
```
- name: Build Ember CLI
run: bin/ember-cli --build
```
A new `--build` argument has been added to `bin/ember-cli` for this
case, which is not needed locally if you already have the discourse
rails server running via `bin/ember-cli -u` since the whole server is built and
set up by default.
Co-authored-by: David Taylor <david@taylorhq.com>
