Commit Graph

48 Commits

Author SHA1 Message Date
Guo Xiang Tan
bf21ebaecc DEV: Allow custom value when pausing sidekiq to aid in debugging.
Sometimes, it is useful to know what caused Sidekiq to be paused.
2019-02-19 10:55:53 +08:00
Guo Xiang Tan
fe92b496a7 FIX: Don't clear connections on failover.
Clearing connections can be really slow (up to 10 seconds).
2019-01-22 14:48:01 +08:00
Guo Xiang Tan
de197000e7 FIX: Clear connections before disabling readonly mode. 2019-01-22 14:14:13 +08:00
Guo Xiang Tan
426907cb88 FIX: Incorrect subscription in PostgreSQLFallbackHandler. 2019-01-21 17:39:31 +08:00
Guo Xiang Tan
cc83bc8fcf FIX: Avoid throwing an error when processing PG down message. 2019-01-21 11:48:54 +08:00
Guo Xiang Tan
ec58c33e9e DEV: Improve postgresql fallover and multisite tests. 2019-01-15 12:52:45 +08:00
Guo Xiang Tan
08b268c5bc Be more forceful in disconnecting connections during failover. 2018-09-04 10:32:43 +08:00
Guo Xiang Tan
7c173265d5 FIX: Don't clear connections on the same process. 2018-06-12 13:06:25 +08:00
Guo Xiang Tan
646ed87aba Clear all connections once master recovers. 2018-06-12 12:13:59 +08:00
Guo Xiang Tan
fd75e54793 Disconnect the pool during failover and fallback. 2018-06-12 11:09:19 +08:00
Guo Xiang Tan
a7be68500d Better error reporting in PostgreSQL fallover thread. 2018-06-12 09:39:11 +08:00
Guo Xiang Tan
2adae2a0b6 FIX: Be more aggressive in checking master when trying to fallback. 2018-06-12 00:09:15 +08:00
Guo Xiang Tan
c0b0c66a9e FIX: PostgreSQL fallback thread doesn't run server starts in readonly. 2018-06-11 15:23:58 +08:00
Guo Xiang Tan
d06af14c26 FIX: Notify all processes to fallback when PostgreSQL recovers. 2018-06-11 15:10:28 +08:00
Guo Xiang Tan
d192924876 FIX: Can't boot Discourse app with read-only PG connection take 2. 2018-06-11 13:25:51 +08:00
Guo Xiang Tan
5656e8f366 FIX: Can't boot Discourse with a read-only PG connection. 2018-06-11 12:29:23 +08:00
Guo Xiang Tan
a509f466a0 Expose replica_postgresql_connection to ActiveRecord::Base. 2017-11-24 09:35:45 +08:00
Guo Xiang Tan
c390169b71 Move monkey patch into freedom_patchers. 2017-11-21 14:15:11 +08:00
Guo Xiang Tan
337ccfa3a6 MONKEYPATCH: Allow us to disable the use of advisory locks during migration. 2017-11-21 14:06:22 +08:00
Guo Xiang Tan
54455fa40b Use MessageBus to get other processes to failover faster. 2017-10-23 17:58:24 +08:00
Guo Xiang Tan
7673684d91 Allow DistributedCache#delete to not publish changes. 2017-10-20 23:03:42 +08:00
Guo Xiang Tan
fe1e78ddf4 Make PostgreSQL failover work with distributed cache. 2017-10-20 17:15:29 +08:00
Guo Xiang Tan
d3003592cd Connection may be nil. 2017-10-20 12:10:03 +08:00
Guo Xiang Tan
057571d173 Raise errors in PostgreSQLFallbackHandler threads. 2017-10-20 12:06:03 +08:00
Guo Xiang Tan
24e3e321e1 Only initialize distributed cache once. 2017-10-20 07:17:41 +08:00
Guo Xiang Tan
c7402ef794 FIX: Use a distributed cache to store failover state. 2017-10-20 05:46:19 +08:00
Guo Xiang Tan
1b5ee0ae72 FIX: Couldn't boot Discourse app with a readonly postgres. 2017-10-17 13:44:30 +08:00
Guo Xiang Tan
5f76e5062d Pause Sidekiq when postgres failovers. 2017-10-16 19:48:31 +08:00
Guo Xiang Tan
d4388f54a2 FIX: Use exact patht to ensure we always redirect with the right format. 2017-09-28 10:29:41 +08:00
Guo Xiang Tan
9540b952fd Fix broken condition in PostgresqlFallbackAdapter. 2017-09-25 13:48:59 +08:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Guo Xiang Tan
cdd550e947 Use a different Redis key when PG failover sets site to readonly mode. 2017-01-11 16:38:49 +08:00
Guo Xiang Tan
1e7de826dc FIX: Remove unused code. 2016-11-30 16:39:38 +08:00
Guo Xiang Tan
f794c25f60 FIX: Ensure a Thread is always running. 2016-11-30 16:38:21 +08:00
Guo Xiang Tan
e8a3043129 Spawn a single thread that checks for PostgreSQL fallback. 2016-11-17 13:52:08 +08:00
Guo Xiang Tan
880b7e1053
FIX: Connections were incorrectly going to master when failing over. 2016-05-30 11:50:02 +08:00
Guo Xiang Tan
86ed6c7d5e
Revert "Clear read only timestamp in Redis when switching back to master."
This reverts commit f891430f32.
2016-05-27 18:02:26 +08:00
Guo Xiang Tan
f891430f32
Clear read only timestamp in Redis when switching back to master. 2016-05-18 10:17:03 +08:00
Guo Xiang Tan
d783a393ec Fix randomly failing test. 2016-03-08 12:51:51 +08:00
Guo Xiang Tan
fcc86d3a9d FIX: PostgreSQLFallbackHandler was bouncing in and out of readonly. 2016-03-08 10:02:38 +08:00
Guo Xiang Tan
b41aa27a84 FEATURE: Support multisite in PostgreSQL fallback adapter. 2016-03-02 21:37:37 +08:00
Guo Xiang Tan
c344bd3fd1 FIX: Make sure we write inside a lock. 2016-02-23 23:25:59 +08:00
Guo Xiang Tan
b0d2e69cc3 FIX: Update log level to warn. 2016-02-12 15:18:34 +08:00
Guo Xiang Tan
74dc838f5f FIX: Add a lock to ensure only a single thread is running each time. 2016-02-05 10:47:47 +08:00
Guo Xiang Tan
c532d7d1ae Internally AR::Base.establish_connection removes the current connection. 2016-02-05 08:51:50 +08:00
Guo Xiang Tan
a08496bb1a Remove Concurrent::TimerTask which spawns a long lasting Thread. 2016-02-05 08:51:50 +08:00
Guo Xiang Tan
0058d09e35 Second attempt which removes any kind proxying. 2016-02-05 08:51:10 +08:00
Guo Xiang Tan
46589a1a0c FEATURE: AR adapter to failover to a replica DB server. 2016-02-05 08:51:10 +08:00