github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-30 11:04:35 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
31,483 Commits 214 Branches 500 Tags 960 MiB
c50db76f5d
Commit Graph

111 Commits

Author SHA1 Message Date
Régis Hanol
1e67bcb456
PERF: bulk feature topic users & reset topic counters after an import 2019-01-17 21:48:23 +01:00
Régis Hanol
788719d271 DEV: speed up posts base imports 2019-01-04 15:30:17 +01:00
Arpit Jalan
71a5369fef FIX: do not convert quote tags to markdown 2018-12-11 20:09:46 +05:30
Régis Hanol
c39a1022cc PERF: user imports would slow down the more users were imported 2018-10-22 11:14:13 +02:00
Régis Hanol
afa22a0c6f REFACTOR: more 'fake_email' to base importer 2018-10-22 11:12:40 +02:00
Régis Hanol
5f2fb0fe33 Show original options when an error happens while importing an user 2018-10-18 10:21:12 +02:00
Neil Lalonde
15f657309a FEATURE: Zendesk importer that uses its API to get data 2018-08-28 10:21:39 -04:00
Vinoth Kannan
1390eb2957 Disable bootstrap mode before start importing 2018-07-25 12:12:26 +05:30
Gerhard Schlager
fb022098f6 Base importer: Calculate category colors depending on parent category 2018-06-27 20:27:11 +02:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql 
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
 2018-06-19 16:13:36 +10:00
Arpit Jalan
b4e0cddcc9 disable all outgoing emails in base importer 2018-06-07 22:49:38 +05:30
Arpit Jalan
f9ab3848ed FEATURE: support disabling emails for non-staff users 2018-06-07 18:31:08 +05:30
Régis Hanol
aeb511e8ff FEATURE: SMF1 importer 2018-05-28 11:02:19 +02:00
Gerhard Schlager
eceeef8413 Imported categories use colors from settings instead of brown 2018-05-22 21:55:54 +02:00
Neil Lalonde
6e8bf1271f increase purge site settings for imports in case jobs like clean_up_uploads cause a mess 2018-04-09 12:44:59 -04:00
Gerhard Schlager
2834705bd1 Improve base import script 
* Make sure the category description is imported correctly
  (the about topic usually had the wrong excerpt).
* Allow import scripts to mark topics as closed or archived.
* Allow import scripts to store the topic's original id.
  It will be stored in topic_custom_fields as import_topic_id.
 2018-03-16 22:39:06 +01:00
Arpit Jalan
25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
Gerhard Schlager
bb54eb1192 Improvements to mbox importer 
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
 2018-01-17 12:04:57 +01:00
Gerhard Schlager
a93bf186c5 FIX: validation of website failed in base importer 2017-12-28 14:51:43 +01:00
Gerhard Schlager
94ceb06b02 fix user lookup by email in disqus import script 2017-11-16 16:26:18 +01:00
Neil Lalonde
aec5de451b FIX: compare imported ids as string instead of converting to integer, same as LookupContainer 2017-10-23 14:01:57 -04:00
Régis Hanol
0d257058c9 make rubocop 🤖 happy 2017-10-23 10:21:50 +02:00
Jay Pfaffman
20e7e285d6 add users with invalid email addresses (#5224) 
* add users with invalid email addresses

* start to add suspend user stff

* don't create a suspend_user function

* include reason for suspension
 2017-10-23 10:18:44 +02:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Arpit Jalan
ed5d322196 FIX: handle existing users in imports 2017-07-24 16:22:56 +05:30
Guo Xiang Tan
d38b55e0c8 Merge pull request #4959 from farmdawgnation/minor-mbox-import-fix 
Add a nil check on the connection before attempting to exec it
 2017-07-21 09:45:15 +09:00
Leo McArdle
d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Matt Farmer
5b11391588 Add a nil check on the connection before attempting to exec it 2017-07-07 11:11:43 -04:00
Régis Hanol
e155cb6db1 FIX: JIVE API Importer 
- Added the 'hierarchical=false' query parameter to retrieve comments (replies) chronologically
- Calling 'create_post' (singular) wasn't updating the LookupContainer and thus we couldn't populate the 'reply_to_post_number' column
 2017-04-26 15:36:35 +02:00
Régis Hanol
0d250c3935 50% faster vBulletin 4 importer 2017-02-01 14:33:09 +01:00
Dmitry Demenchuk
f687bfb7e2 Allow to process imported content. (add additional callbacks) 2016-10-26 16:51:34 +01:00
Jay Pfaffman
30f2081c1b importer/base.rb: improve progress reporting 2016-09-20 11:27:45 -07:00
Robin Ward
72d7c055f4 Lots of ugprades to the Disqus importer script 2016-08-17 15:10:46 -04:00
Gerhard Schlager
fccc0c93c5 Store the original username during import (#4349) 2016-07-27 22:30:15 +02:00
Régis Hanol
56d2626187 support 'read_restricted' categories in importers 2016-07-27 18:38:23 +02:00
Neil Lalonde
3b119dac6d FIX: importer performance of update_tl0, and give feedback during vanilla_mysql's create_permalinks 2016-07-06 10:58:43 -04:00
Robin Ward
b42f28d4c3 Improved mailing list import. 
Now uses a SQLite database to store messages rather than JSON files
for performance and memory considerations.
 2016-06-14 11:45:17 -04:00
Neil Lalonde
f07b3e7df9 PERF: base importer updates user_stats with sql (mostly) 2016-05-17 18:39:04 -04:00
Régis Hanol
b716886240 update bbPress importer 2016-03-07 18:22:15 +01:00
Gerhard Schlager
8a486d8cea Allow importers to set empty names 2016-02-21 23:35:47 +01:00
Gerhard Schlager
eb166e78b6 Don't try to import invalid websites 2016-02-21 17:38:04 +01:00
Neil Lalonde
08e10c2a9d FEATURE: Zoho importer 2015-12-04 15:06:46 -05:00
Régis Hanol
26006f12a2 Increased 'error resistance' in base importer class 2015-12-01 15:38:21 +01:00
Robin Ward
6c6d406e63 Small fixes to import base. 2015-10-29 16:47:27 -04:00
Gerhard Schlager
6238a43f93 Fix base and vBulletin importer 2015-10-21 19:07:31 +02:00
Sam
606aeb9d55 improvements to importer 
- improve perf of test for existing posts
- always use a system guardian when importing posts
- for lithuim importer requery raw (transform is not repeatable)
 2015-10-16 10:41:21 +11:00
Gerhard Schlager
29d27ec2ef FIX: Prevent cast error when import_id contains strings and numbers 
Some importers insert numbers and strings as import_id into the same
custom_fields table. This change prevents cast errors during imports.
 2015-09-30 16:38:06 +02:00
Kane York
821124f3fd FEATURE: Skip batches if all records exist 
Update all import scripts to take advantage of all_records_exist?
 2015-09-21 17:13:02 -07:00
Kane York
200ee15991 FIX: Importer not handling usernames correctly 2015-09-21 16:27:47 -07:00
Gerhard Schlager
8c03dd16af Adds support for importing password hashes used by "migratepassword" plugin 
Adds setting to phpBB3 importer for importing passwords (default: off)
Plugin: https://github.com/discoursehosting/discourse-migratepassword
 2015-08-20 22:22:47 +02:00