discourse/script
Michael Fitz-Payne 971409741f DEV: refactor cache_critical_dns for SRV RR awareness
Modify the cache_critical_dns script for SRV RR awareness. The new
behaviour is only enabled when one or more of the following environment
variables are present (and only for a host where the `DISCOURSE_*_HOST_SRV`
variable is present):
- `DISCOURSE_DB_HOST_SRV`
- `DISCOURSE_DB_REPLICA_HOST_SRV`
- `DISCOURSE_REDIS_HOST_SRV`
- `DISCOURSE_REDIS_REPLICA_HOST_SRV`

Some minor changes in refactor to original script behaviour:
- add Name and SRVName classes for storing resolved addresses for a hostname
- pass DNS client into main run loop instead of creating inside the loop
- ensure all times are UTC
- add environment override for system hosts file path and time between DNS
  checks mainly for testing purposes

The environment variable for `BUNDLE_GEMFILE` is set to enables Ruby to
load gems that are installed and vendored via the project's Gemfile.
This script is usually not run from the project directory as it is
configured as a system service (see
71ba9fb7b5/templates/cache-dns.template.yml (L19))
and therefore cannot load gems like `pg` or `redis` from the default
load paths. Setting this environment variable configures bundler to look
in the correct project directory during it's setup phase.

When a `DISCOURSE_*_HOST_SRV` environment variable is present, the
decision for which target to cache is as follows:
- resolve the SRV targets for the provided hostname
- lookup the addresses for all of the resolved SRV targets via the
  A and AAAA RRs for the target's hostname
- perform a protocol-aware healthcheck (PostgreSQL or Redis pings)
- pick the newest target that passes the healthcheck

From there, the resolved address for the SRV target is cached against
the hostname as specified by the original form of the environment
variable.

For example: The hostname specified by the `DISCOURSE_DB_HOST` record
is `database.example.com`, and the `DISCOURSE_DB_HOST_SRV` record is
`database._postgresql._tcp.sd.example.com`. An SRV RR lookup will return
zero or more targets. Each of the targets will be queried for A and AAAA
RRs. For each of the addresses returned, the newest address that passes
a protocol-aware healthcheck will be cached. This address is cached so
that if any newer address for the SRV target appears we can perform a
health check and prefer the newer address if the check passes.

All resolved SRV targets are cached for a minimum of 30 minutes in memory
so that we can prefer newer hosts over older hosts when more than one target
is returned. Any host in the cache that hasn't been seen for more than 30
minutes is purged.

See /t/61485.
2022-10-26 09:22:20 +10:00
..
benchmarks DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
bulk_import DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
demon_test FIX: Revert Demon::DemonBase back to Demon::Base (#8132) 2019-10-02 14:54:08 +10:00
import_scripts DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
.gitignore improvements to the mbox import script 2017-11-18 17:16:44 +01:00
alice.txt
analyze_sidekiq_queues.rb correct type 2019-05-22 12:28:17 +10:00
bench.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
biggest_objects.rb DEV: add debugging scripts for memory leaks 2019-10-03 16:36:31 +10:00
boot_mem.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
cache_critical_dns DEV: refactor cache_critical_dns for SRV RR awareness 2022-10-26 09:22:20 +10:00
check_forking.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
compile_hbs.rb Rename the server side widget hbs compiler 2020-03-27 12:06:14 -04:00
copyright-deposit Add script for compiling copyright deposits (#9646) 2020-05-06 12:51:45 -04:00
db_timestamps_mover.rb DEV: Make db_timestamp_mover work with tables with unique constraints (#14027) 2021-08-12 19:24:21 +04:00
diff_heaps.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
discourse DEV: Fix thor deprecation warning (#14680) 2021-10-21 21:01:05 +02:00
docker_test.rb DEV: Remove the remaining Travis code (#13255) 2021-06-02 20:29:47 +02:00
downsize_uploads.rb FIX: Ambiguous column in downsize_uploads (#14972) 2021-11-16 16:23:32 +01:00
i18n_lint.rb DEV: Clean up and refactor CI workflow(s) (#12144) 2021-02-22 10:28:32 +01:00
measure.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
memory-analysis DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
memstats.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
micro_bench.rb DEV: undo pluck_first changes to micro benchmark 2019-10-21 12:21:24 +01:00
mwrap_sidekiq PERF: avoid shelling to get hostname aggressively 2020-02-18 15:13:19 +11:00
profile_db_generator.rb FIX: ensure we consistently pick the same topic for bench 2020-01-08 16:33:45 +11:00
promote_migrations DEV: Update script/promote_migrations (#13513) 2021-06-24 13:57:23 +01:00
rails DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
redis_memory.rb DEV: s/\$redis/Discourse\.redis (#8431) 2019-12-03 10:05:53 +01:00
require_profiler.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
spawn_backup_restore.rb FIX: Killing a Unicorn worker shouldn't kill a running backup or restore process 2020-10-13 19:48:53 +02:00
start_test_db.rb DEV: Wait for initdb to complete in docker.rake (#15614) 2022-01-17 17:45:39 +00:00
switch_container_ruby DEV: add script to switch ruby version from inside container 2019-03-25 17:41:24 +11:00
test_email_settings.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
test_hbs_compiler.rb REFACTOR: Remove Discourse.__widget_helpers 2020-08-06 14:35:46 -04:00
test_mem.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
test_memory_leak.rb DEV: improve diagnostics on mem leak checker 2019-10-04 09:47:33 +10:00
test_pretty_text.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
thread_detective.rb PERF: avoid spinning a thread each time we close a connection 2019-09-12 17:34:04 +10:00
user_simulator.rb DEV: User simulator tried to modify frozen string. 2019-08-16 17:32:17 +03:00