Commit Graph

498 Commits

Author SHA1 Message Date
Gerhard Schlager
941e096df4 Fix error in base import script
Follow-up to 655a08dbbd
2019-03-06 21:58:25 +01:00
maulkin
655a08dbbd FIX: Return actual errors if PostCreator fails (#7096) 2019-03-06 21:29:37 +01:00
Penar Musaraj
b1035cc691 FIX: NodeBB import details
- mark imported users as active

- do not strip @ from usernames in post content

- improve uploads path matching
2019-03-06 12:30:36 -05:00
Gerhard Schlager
c36c9c2ee5 FEATURE: Import script for AnswerBase
Improves the generic database used by some import scripts:
* Adds additional columns for users
* Adds support for attachments
* Allows setting the data type for keys (numeric or string) to ensure correct sorting
2019-02-28 22:08:12 +01:00
Gerhard Schlager
24369a8166 Improve phpBB3 importer
* Log errors when mapping of posts, messages, etc. fails
* Allow permalink normalizations for old subfolder installation
* Disable importing of polls for now. It's broken.
2019-02-17 23:20:20 +01:00
Gerhard Schlager
8d5dfe1e01 FIX: Don't import parts of the email address as name 2019-02-17 22:59:18 +01:00
Régis Hanol
1e67bcb456
PERF: bulk feature topic users & reset topic counters after an import 2019-01-17 21:48:23 +01:00
Régis Hanol
788719d271 DEV: speed up posts base imports 2019-01-04 15:30:17 +01:00
Arpit Jalan
71a5369fef FIX: do not convert quote tags to markdown 2018-12-11 20:09:46 +05:30
Régis Hanol
3c9c95ac83 Update Rubocop to 0.60 2018-12-04 10:48:16 +01:00
David Taylor
9248ad1905 DEV: Enable Style/SingleLineMethods and Style/Semicolon in Rubocop (#6717) 2018-12-04 11:48:13 +08:00
Guo Xiang Tan
5076487eaf Update discuz_x import script to not use Category#logo_url. 2018-11-09 14:15:31 +08:00
Gerhard Schlager
d6f89a85ef Make Rubocop happy 2018-10-31 01:30:14 +01:00
Gerhard Schlager
65db9326b4 FEATURE: Add download script for Google Groups 2018-10-31 01:12:05 +01:00
Gerhard Schlager
efa265cbc8 Rename mbox import script 2018-10-31 01:12:05 +01:00
Gerhard Schlager
edbc004a9a Remove old mbox import script 2018-10-31 01:12:05 +01:00
Régis Hanol
c39a1022cc PERF: user imports would slow down the more users were imported 2018-10-22 11:14:13 +02:00
Régis Hanol
afa22a0c6f REFACTOR: more 'fake_email' to base importer 2018-10-22 11:12:40 +02:00
Régis Hanol
8b20e2500a
Remove unnecessary line 2018-10-19 15:48:48 +02:00
Régis Hanol
637123ff6f Merge users based on their email in vBulletin importer 2018-10-19 15:16:45 +02:00
Régis Hanol
53aa0344bf FIX: properly import vBulletin's hashed password 2018-10-18 10:22:55 +02:00
Régis Hanol
5f2fb0fe33 Show original options when an error happens while importing an user 2018-10-18 10:21:12 +02:00
Gerhard Schlager
341836eb42 Fix the rake task and importer instead 2018-10-17 16:48:09 +02:00
Gerhard Schlager
ee18d9ace0 FIX: mbox importer and rake task were broken 2018-10-17 16:34:18 +02:00
Neil Lalonde
a68032835a FEATURE: XenForo importer can import categories from the xf_node table and convert sub-categories beyond second level to tags 2018-10-11 12:04:15 -04:00
Guo Xiang Tan
71185c13b5
Merge pull request #6377 from tgxworld/remove_tif_tiff
Drop `tif`, `tiff`, `webp` and `bmp` from supported images.
2018-09-12 09:32:32 +08:00
Guo Xiang Tan
e1b16e445e Rename FileHelper.is_image? -> FileHelper.is_supported_image?. 2018-09-12 09:22:28 +08:00
Carsten Brandt
921e2213b8 FEATURE: Updated IPB import script
* IPB import script replace PHP code tags with proper markdown

remove excess newlines in code blocks
decode HTML entities in code blocks
add replacement for list items
proper handling of attachments that are not images
fix typo
improved quote handling
fix code style complaint from travis-ci build
2018-09-12 11:12:28 +10:00
Guo Xiang Tan
434035f167 FIX: Link post to uploads in PostCreator.
* This ensures that uploads are linked to their post on creation
  instead of a background job which may be delayed if Sidekiq
  is facing difficulties.
2018-09-06 11:18:11 +08:00
Guo Xiang Tan
8dc1463ab3 Enable Lint/ShadowingOuterLocalVariable for Rubocop. 2018-09-04 10:16:42 +08:00
Neil Lalonde
15f657309a FEATURE: Zendesk importer that uses its API to get data 2018-08-28 10:21:39 -04:00
Gerhard Schlager
ac743dab10 Improve mbox import script
* emails weren't sorted in correct order
* better default regex for splitting mbox files
* output Message-ID if email is skipped because it doesn't have a Date
2018-08-23 09:46:28 +02:00
Gerhard Schlager
7f4ef3db9e Improve Telligent importer
* Try multiple filenames and do lots of guessing when searching for attachments
* Unescape HTML in filenames and replace invalid characters in filenames
* Existing permalinks prevented resuming of import
* Prevent duplicate attachments in same post
2018-08-13 15:28:11 +02:00
Gerhard Schlager
8513605421 Fix the import of avatars and attachments
This time for real ;-)
2018-08-12 22:26:07 +02:00
Gerhard Schlager
6d813c2b52 FIX: Importers failed to import avatars 2018-08-12 22:02:17 +02:00
Gerhard Schlager
1794aea939 FEATURE: Add import script for Telligent 2018-08-12 22:01:23 +02:00
Vinoth Kannan
1390eb2957 Disable bootstrap mode before start importing 2018-07-25 12:12:26 +05:30
Régis Hanol
e8e9b5cea4 FIX: clean URLs in SMF1 importer 2018-07-19 13:17:43 +02:00
Régis Hanol
63e5349209 FIX: [img] BBCode tags might have parameters 2018-07-19 13:11:01 +02:00
Régis Hanol
5434cf02a3 FIX: smf1 importer was swallowing some data 2018-07-19 10:29:54 +02:00
Régis Hanol
c818550172 Support custom avatar in SMF1 importer 2018-07-12 17:38:07 +02:00
Régis Hanol
5c4534d895 Update SMF1 import
- Properly import avatar when they use an external image
- Don't import the same attachment twice
2018-07-12 16:55:30 +02:00
Sam
f4f95ce956 correct linting 2018-06-29 16:04:38 +10:00
David Lee
8f43872bff Add Question2Answer import script 2018-06-29 15:48:01 +10:00
Arpit Jalan
c73f98c289 FIX: invert from and to user id in smf1 import script 2018-06-28 12:30:28 +05:30
Gerhard Schlager
fb022098f6 Base importer: Calculate category colors depending on parent category 2018-06-27 20:27:11 +02:00
Vinoth Kannan
652b32484f Assign default value for message template matches 2018-06-26 05:16:03 +05:30
Vinoth Kannan
f3011c709b Extract html content from lithium message template 2018-06-26 05:07:32 +05:30
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
2018-06-19 16:13:36 +10:00
Vinoth Kannan
4ffa4a28b0 FIX: duplicate_emails get overridden in new batch of import 2018-06-19 00:21:48 +05:30