discourse/app/controllers/admin/reports_controller.rb

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

161 lines
4.3 KiB
Ruby
Raw Normal View History

# frozen_string_literal: true
class Admin::ReportsController < Admin::StaffController
REPORTS_LIMIT = 50
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
HIDDEN_PAGEVIEW_REPORTS = ["site_traffic"]
HIDDEN_LEGACY_PAGEVIEW_REPORTS = %w[
consolidated_page_views
consolidated_page_views_browser_detection
page_view_anon_reqs
page_view_logged_in_reqs
]
def index
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
page_view_req_report_methods =
["page_view_total_reqs"] +
ApplicationRequest
.req_types
.keys
.select { |r| r =~ /\Apage_view_/ && r !~ /mobile/ }
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
.map { |r| r + "_reqs" }
reports_methods =
page_view_req_report_methods +
Report.singleton_methods.grep(/\Areport_(?!about|storage_stats)/)
reports =
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
reports_methods
.reduce([]) do |reports_acc, name|
type = name.to_s.gsub("report_", "")
description = I18n.t("reports.#{type}.description", default: "")
description_link = I18n.t("reports.#{type}.description_link", default: "")
if SiteSetting.use_legacy_pageviews
next reports_acc if HIDDEN_PAGEVIEW_REPORTS.include?(type)
else
next reports_acc if HIDDEN_LEGACY_PAGEVIEW_REPORTS.include?(type)
end
reports_acc << {
type: type,
title: I18n.t("reports.#{type}.title"),
description: description.presence ? description : nil,
description_link: description_link.presence ? description_link : nil,
}
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
reports_acc
end
.sort_by { |report| report[:title] }
render_json_dump(reports: reports)
end
def bulk
reports = []
hijack do
params[:reports].each do |report_type, report_params|
args = parse_params(report_params)
report = nil
report = Report.find_cached(report_type, args) if (report_params[:cache])
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
if SiteSetting.use_legacy_pageviews
if HIDDEN_PAGEVIEW_REPORTS.include?(report_type)
report = Report._get(report_type, args)
report.error = :not_found
end
else
if HIDDEN_LEGACY_PAGEVIEW_REPORTS.include?(report_type)
report = Report._get(report_type, args)
report.error = :not_found
end
end
if report
reports << report
else
report = Report.find(report_type, args)
Report.cache(report) if (report_params[:cache]) && report
if report.blank?
report = Report._get(report_type, args)
report.error = :not_found
end
reports << report
end
end
render_json_dump(reports: reports)
end
end
def show
report_type = params[:type]
raise Discourse::NotFound unless report_type =~ /\A[a-z0-9\_]+\z/
FEATURE: Switch to new methods of pageview measurement and reporting (#28729) ### UI changes All of the UI changes described are gated behind the `use_legacy_pageviews` site setting. This commit changes the admin dashboard pageviews report to use the "Consolidated Pageviews with Browser Detection" report introduced in 2f2da7274732cba30d03b6c5c3a4194652cb6783 with the following changes: * The report name is changed to "Site traffic" * The pageview count on the dashboard is counting only using the new method * The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews" * By default "known crawlers" and "other" sources of pageviews are hidden on the report When `use_legacy_pageviews` is `true`, we do not show or allow running the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`, we do not show or allow running the following legacy reports: * consolidated_page_views * consolidated_page_views_browser_detection * page_view_anon_reqs * page_view_logged_in_reqs ### Historical data changes Also part of this change is that, since we introduced our new "Consolidated Pageviews with Browser Detection" report, some admins are confused at either: * The lack of data before a certain date , which didn’t exist before we started collecting it * Comparing this and the current "Consolidated Pageviews" report data, which rolls up "Other Pageviews" into "Anonymous Browser" and so it appears inaccurate All pageview data in the new report before the date where the _first_ anon or logged in browser pageview was recorded is now hidden.
2024-09-10 07:51:49 +08:00
if SiteSetting.use_legacy_pageviews
raise Discourse::NotFound if HIDDEN_PAGEVIEW_REPORTS.include?(report_type)
else
raise Discourse::NotFound if HIDDEN_LEGACY_PAGEVIEW_REPORTS.include?(report_type)
end
args = parse_params(params)
report = nil
report = Report.find_cached(report_type, args) if (params[:cache])
return render_json_dump(report: report) if report
hijack do
report = Report.find(report_type, args)
raise Discourse::NotFound if report.blank?
Report.cache(report) if (params[:cache])
render_json_dump(report: report)
end
end
private
def parse_params(report_params)
begin
start_date =
(
if report_params[:start_date].present?
Time.parse(report_params[:start_date]).to_date
else
1.days.ago
end
).beginning_of_day
end_date =
(
if report_params[:end_date].present?
Time.parse(report_params[:end_date]).to_date
else
start_date + 30.days
end
).end_of_day
rescue ArgumentError => e
raise Discourse::InvalidParameters.new(e.message)
end
facets = nil
facets = report_params[:facets].map { |s| s.to_s.to_sym } if Array === report_params[:facets]
limit = fetch_limit_from_params(params: report_params, default: nil, max: REPORTS_LIMIT)
filters = nil
filters = report_params[:filters] if report_params.has_key?(:filters)
FEATURE: Exposing a way to add a generic report filter (#6816) * FEATURE: Exposing a way to add a generic report filter ## Why do we need this change? Part of the work discussed [here](https://meta.discourse.org/t/gain-understanding-of-file-uploads-usage/104994), and implemented a first spike [here](https://github.com/discourse/discourse/pull/6809), I am trying to expose a single generic filter selector per report. ## How does this work? We basically expose a simple, single generic filter that is computed and displayed based on backend values passed into the report. This would be a simple contract between the frontend and the backend. **Backend changes:** we simply need to return a list of dropdown / select options, and enable the report's newly introduced `custom_filtering` property. For example, for our [Top Uploads](https://github.com/discourse/discourse/pull/6809/files#diff-3f97cbb8726f3310e0b0c386dbe89e22R1423) report, it can look like this on the backend: ```ruby report.custom_filtering = true report.custom_filter_options = [{ id: "any", name: "Any" }, { id: "jpg", name: "JPEG" } ] ``` In our javascript report HTTP call, it will look like: ```js { "custom_filtering": true, "custom_filter_options": [ { "id": "any", "name": "Any" }, { "id": "jpg", "name": "JPG" } ] } ``` **Frontend changes:** We introduced a generic `filter` param and a `combo-box` which hooks up into the existing framework for fetching a report. This works alright, with the limitation of being a single custom filter per report. If we wanted to add, for an instance a `filesize filter`, this will not work for us. _I went through with this approach because it is hard to predict and build abstractions for requirements or problems we don't have yet, or might not have._ ## How does it look like? ![a1ktg1odde](https://user-images.githubusercontent.com/45508821/50485875-f17edb80-09ee-11e9-92dd-1454ab041fbb.gif) ## More on the bigger picture The major concern here I have is the solution I introduced might serve the `think small` version of the reporting work, but I don't think it serves the `think big`, I will try to shed some light into why. Within the current design, It is hard to maintain QueryParams for dynamically generated params (based on the idea of introducing more than one custom filter per report). To allow ourselves to have more than one generic filter, we will need to: a. Use the Route's model to retrieve the report's payload (we are now dependent on changes of the QueryParams via computed properties) b. After retrieving the payload, we can use the `setupController` to define our dynamic QueryParams based on the custom filters definitions we received from the backend c. Load a custom filter specific Ember component based on the definitions we received from the backend
2019-03-15 20:15:38 +08:00
{ start_date: start_date, end_date: end_date, filters: filters, facets: facets, limit: limit }
end
end