mirror of
https://github.com/discourse/discourse.git
synced 2024-11-27 09:33:37 +08:00
cfec408bc1
Why this change? Currently we only rerun failing tests to check if they are flaky tests when there are 10 or less failing tests. When there are more than 10 failing tests in the first run, we assume that the odds of those tests being flaky are low and do not rerun the tests. However, there was a bug where we do not clean up the potential flaky tests being logged when there are too many test failures. This resulted in those test failures being treated as flaky tests. What does this change do? Clean up the flaky tests report when we do not rerun the tests.
55 lines
1.8 KiB
Ruby
55 lines
1.8 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
module TurboTests
|
|
module Flaky
|
|
class Manager
|
|
PATH = Rails.root.join("tmp/turbo_rspec_flaky_tests.json")
|
|
|
|
def self.potential_flaky_tests
|
|
JSON
|
|
.parse(File.read(PATH))
|
|
.map { |failed_example| failed_example["location_rerun_argument"] }
|
|
end
|
|
|
|
def self.remove_flaky_tests
|
|
File.delete(PATH) if File.exist?(PATH)
|
|
end
|
|
|
|
# This method should only be called by a formatter registered with `TurboTests::Runner` and logs the failed examples
|
|
# to `PATH`. See `FailedExample#to_h` for the details of each example that is logged.
|
|
#
|
|
# @param [Array<TurboTests::FakeExample>] failed_examples
|
|
def self.log_potential_flaky_tests(failed_examples)
|
|
return if failed_examples.empty?
|
|
|
|
File.open(PATH, "w") do |file|
|
|
file.puts(
|
|
failed_examples.map { |failed_example| FailedExample.new(failed_example).to_h }.to_json,
|
|
)
|
|
end
|
|
end
|
|
|
|
# This method should only be called by a formatter registered with `RSpec::Core::Formatters.register` and removes
|
|
# the given examples from the log file at `PATH` by matching the `location_rerun_argument` of each example.
|
|
#
|
|
# @param [Array<RSpec::Core::Example>] failed_examples
|
|
def self.remove_example(failed_examples)
|
|
flaky_tests =
|
|
JSON
|
|
.parse(File.read(PATH))
|
|
.reject do |failed_example|
|
|
failed_examples.any? do |example|
|
|
failed_example["location_rerun_argument"] == example.location_rerun_argument
|
|
end
|
|
end
|
|
|
|
if flaky_tests.present?
|
|
File.write(PATH, flaky_tests.to_json)
|
|
else
|
|
File.delete(PATH)
|
|
end
|
|
end
|
|
end
|
|
end
|
|
end
|