discourse/script
Leonardo Mosquera bfecbde837
Fixes for vBulletin bulk importer (#17618)
* Allow taking table prefix from env var

* FIX: remove unused column references

The columns `filedata` and `extension` are not present in a v4.2.4
database, and they aren't used in the method anyways.

* FIX: report progress for tables without imported_id

* FIX: effectively check for AR validation errors

NOTE: other migration scripts also have this problem; see /t/58202

* FIX: properly count Posts when importing attachments

* FIX: improve logging

* Remove leftover comment

* FIX: show progress when exporting Permalink file

* PERF: stream Permalink file

The current way results in tons of memory usage; write once per line instead

* Document fixes needed

* WIP - deduplicate category names

* Ignore non alphanumeric chars for grouping

* FIX: properly deduplicate user emails by merging accounts

* FIX: don't merge empty UserEmails

* Improve logging

* Merge users AFTER fixing primary key sequences

* Parallelize user merging

* Save duplicated users structure for debugging purposes

* Add progress logging for the (multiple hour) user merging step
2022-11-28 16:30:19 -03:00
..
benchmarks DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
bulk_import Fixes for vBulletin bulk importer (#17618) 2022-11-28 16:30:19 -03:00
demon_test FIX: Revert Demon::DemonBase back to Demon::Base (#8132) 2019-10-02 14:54:08 +10:00
import_scripts Fixes for vBulletin bulk importer (#17618) 2022-11-28 16:30:19 -03:00
.gitignore
alice.txt
analyse_message_bus.rb DEV: Add message-bus analysis script (#15979) 2022-02-18 20:21:17 +00:00
analyze_sidekiq_queues.rb correct type 2019-05-22 12:28:17 +10:00
bench.rb DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
biggest_objects.rb DEV: add debugging scripts for memory leaks 2019-10-03 16:36:31 +10:00
boot_mem.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
cache_critical_dns DEV(cache_critical_dns): add caching for MessageBus Redis hostname 2022-10-12 10:11:22 +10:00
check_forking.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
copyright-deposit Add script for compiling copyright deposits (#9646) 2020-05-06 12:51:45 -04:00
db_timestamps_mover.rb DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
diff_heaps.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
discourse DEV: Be more lenient in CLI confirmation (#16290) 2022-03-25 20:14:41 +00:00
docker_test.rb DEV: Remove the remaining Travis code (#13255) 2021-06-02 20:29:47 +02:00
i18n_lint.rb DEV: Clean up and refactor CI workflow(s) (#12144) 2021-02-22 10:28:32 +01:00
measure.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
memory-analysis DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
memstats.rb DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
micro_bench.rb DEV: undo pluck_first changes to micro benchmark 2019-10-21 12:21:24 +01:00
mwrap_sidekiq PERF: avoid shelling to get hostname aggressively 2020-02-18 15:13:19 +11:00
profile_db_generator.rb FIX: ensure we consistently pick the same topic for bench 2020-01-08 16:33:45 +11:00
promote_migrations DEV: Promote historic post_deploy migrations (#16288) 2022-03-25 15:48:20 +00:00
rails DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
redis_memory.rb DEV: s/\$redis/Discourse\.redis (#8431) 2019-12-03 10:05:53 +01:00
require_profiler.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
spawn_backup_restore.rb FIX: Killing a Unicorn worker shouldn't kill a running backup or restore process 2020-10-13 19:48:53 +02:00
start_test_db.rb DEV: Wait for initdb to complete in docker.rake (#15614) 2022-01-17 17:45:39 +00:00
switch_container_ruby DEV: add script to switch ruby version from inside container 2019-03-25 17:41:24 +11:00
test_email_settings.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
test_mem.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
test_memory_leak.rb DEV: improve diagnostics on mem leak checker 2019-10-04 09:47:33 +10:00
test_pretty_text.rb DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
thread_detective.rb PERF: avoid spinning a thread each time we close a connection 2019-09-12 17:34:04 +10:00
user_simulator.rb DEV: User simulator tried to modify frozen string. 2019-08-16 17:32:17 +03:00