github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-12-15 10:13:40 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
34,185 Commits 214 Branches 500 Tags 960 MiB
477bacb3ae
Commit Graph

520 Commits

Author SHA1 Message Date
Vinoth Kannan
0639b902dc Import lithium nodes into categories 2018-05-18 01:34:31 +05:30
Vinoth Kannan
ade7c94b43 Import lithium roles into groups 2018-05-16 17:26:15 +05:30
Régis Hanol
c11e8c9a64 add support for importing attachments in bbPress importer 2018-04-24 11:08:12 -04:00
Neil Lalonde
6e8bf1271f increase purge site settings for imports in case jobs like clean_up_uploads cause a mess 2018-04-09 12:44:59 -04:00
Gerhard Schlager
f2d00e5eff FEATURE: Use Message-ID for detecting email replies to group 
Ignores the site setting "find_related_post_with_key" and always tries to honor the `In-Reply-To` and `References` header for emails sent to a group.

The senders email address must be included in the `To` or `CC` header of a previous email sent to the group and the `Message-ID` of that email must be included in the current email's `In-Reply-To` or `References` header.
 2018-04-05 11:00:38 +02:00
Miro Michalicka
8d283e78e7 Fix broken queries in Drupal 6 migration. 2018-03-31 17:53:40 +02:00
Gerhard Schlager
cf9965fd54 Add rake task for reordering posts by creation_date 2018-03-29 14:42:41 +02:00
Gerhard Schlager
a0d5e243fd Improve the import script for "Get Satisfaction" 
* Works with the current column layout exported as Excel file
* Tries to fix invalid CSV when it wasn't exported from Excel
* Imports categories
* Imports topics into the correct category
* Allows skipping archived topics
* Allows skipping private topics
* Makes use of the latest features from the base importer
* Some minor fixes and documentation updates
 2018-03-19 14:58:17 +01:00
Régis Hanol
dad1e4b17d don't import data only available in a plugin in phpBB 3 importer 2018-03-17 00:36:12 +01:00
Gerhard Schlager
2834705bd1 Improve base import script 
* Make sure the category description is imported correctly
  (the about topic usually had the wrong excerpt).
* Allow import scripts to mark topics as closed or archived.
* Allow import scripts to store the topic's original id.
  It will be stored in topic_custom_fields as import_topic_id.
 2018-03-16 22:39:06 +01:00
Régis Hanol
bb6ac51607 Update PHPBB3 importer 
- Extract name from profile_fields_data table
- Use 'posts.poster_ip' to track topic views
 2018-03-16 18:32:27 +01:00
Gerhard Schlager
9b651adadb FIX: mbox importer should ignore emails without date 2018-03-13 13:42:57 +01:00
Sam
ddcca1d5cc lint 2018-03-08 15:16:12 +11:00
neerajmalve
4b8ff7a057 Update xenforo.rb (#5622) 
#Fix for the error: xenforo.rb: 160: in `gsub!': invalid byte sequence in UTF-8 (ArgumentError)
 2018-03-07 22:57:38 -05:00
Régis Hanol
07bc52dc69 import topic views in phpbb3 2018-03-07 19:27:03 +01:00
Gerhard Schlager
dc32ee5cbf Improvements to mbox import script 
* Ignore errors during indexing and show information about the message causing the problem
* Always activate imported users if they aren't staged
 2018-03-06 11:32:12 +01:00
Régis Hanol
49400337b6 Improve IPBoard3 importer 
- fix encoding issues
- close topics
- don't import queued & not approved topics
- don't import deleted posts
 2018-03-05 10:24:42 +01:00
Régis Hanol
ae9b1e8554 FEATURE: IP.Board 3 importer 2018-03-01 09:43:32 +01:00
Régis Hanol
82e68670bf FIX: SimplePress importer wasn't handling increment imports properly 2018-02-24 00:42:17 +01:00
Sam
720e1965e3 FEATURE: add category suppress from latest 
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
 2018-02-22 09:56:35 +11:00
Gerhard Schlager
479f7ed18f Ignore case when removing mailing list name from subject 2018-02-12 21:41:58 +01:00
Régis Hanol
31f5ca6180 improve simple press importer 2018-02-07 17:37:51 +01:00
Arpit Jalan
25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
Neil Lalonde
cc830f7119 sorry rubocop 2018-01-30 14:37:15 -05:00
Jay Pfaffman
db17c531d6 Feature: Invision Power Board Importer 2018-01-30 08:44:02 -08:00
Gerhard Schlager
192a0886e2 FIX: BBCode to Markdown conversion in phpBB3 importer was broken 
This fixes the conversion for quotes, code blocks and lists (except for nested lists). It also discourages the usage of the ruby-bbcode-to-md gem.
 2018-01-30 12:50:42 +01:00
Gerhard Schlager
c26db2116c FIX: phpBB3 importer imported some users as anonymous users 2018-01-30 12:50:42 +01:00
Neil Lalonde
db287ad58e FIX: invalid SQL in vanilla_mysql import script 2018-01-19 11:53:07 -05:00
Gerhard Schlager
6500343431 FIX: mbox importer didn't detected already indexed files 2018-01-17 17:03:53 +01:00
Gerhard Schlager
bb54eb1192 Improvements to mbox importer 
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
 2018-01-17 12:04:57 +01:00
Yaw Anokwa
77a92e8878 Allow user staging via setting (#5468) 2018-01-04 09:17:35 +01:00
Gerhard Schlager
a93bf186c5 FIX: validation of website failed in base importer 2017-12-28 14:51:43 +01:00
Jay Pfaffman
cc46ab5bb1 smf2: deal with duplicate category names (#5435) 2017-12-27 16:56:36 +11:00
Gerhard Schlager
cafe69caac Refactor mbox import script 2017-12-13 22:03:31 +01:00
Arpit Jalan
3190c13c22 import staged users as inactive in mbox import 2017-12-13 08:45:43 +05:30
Gerhard Schlager
16738cfb1b FEATURE: convert plain text emails to markdown 2017-12-06 01:47:51 +01:00
Gerhard Schlager
32dd1e66be improvements to the mbox import script 
* ignores dot-files and empty emails
* new setting to prefer HTML over plaintext emails during import
* restore original site settings at the end of import
* elided content of HTML mails was not put inside details block
 2017-11-18 17:16:44 +01:00
Gerhard Schlager
94ceb06b02 fix user lookup by email in disqus import script 2017-11-16 16:26:18 +01:00
Gerhard Schlager
06a6ddc3ba handle plaintext and HTML emails in mbox importer 2017-11-15 20:22:11 +01:00
Jon Bartlett
5d8508c523 New yahoo groups importer 2017-11-09 11:07:57 +01:00
Neil Lalonde
667b025d12 make rubocop happy 2017-11-07 14:20:46 -05:00
discoursehosting
4f0bdec370 some improvements for importers (#5295) 
* decode html entities within code blocks

* Only import users that actually participated in the bbpress part of Wordpress; import password hashes

* create permalinks for topics

* Better handling of [code] blocks
 2017-11-07 17:50:43 +01:00
Jay Pfaffman
9cbb90c5ed add importer for modx forum (#5239) 2017-10-31 10:50:34 +11:00
Neil Lalonde
aec5de451b FIX: compare imported ids as string instead of converting to integer, same as LookupContainer 2017-10-23 14:01:57 -04:00
Régis Hanol
0d257058c9 make rubocop 🤖 happy 2017-10-23 10:21:50 +02:00
Jay Pfaffman
20e7e285d6 add users with invalid email addresses (#5224) 
* add users with invalid email addresses

* start to add suspend user stff

* don't create a suspend_user function

* include reason for suspension
 2017-10-23 10:18:44 +02:00
Gerhard Schlager
6c829c24d7 escaping the subject isn't needed in the mbox importer 2017-10-19 15:25:20 +02:00
Gerhard Schlager
c41880ab19 Improvements to the experimental mbox importer 
* Disable journaling to improve performance in Docker
* Use the email cooking method
* Store IncomingEmail in order find related posts by Message-ID
* Escape HTML in imported messages
 2017-10-19 14:27:40 +02:00
Gerhard Schlager
e9fb2c5756
phpBB3 importer: Fix validation of site settings 2017-08-29 01:48:30 +02:00
Neil Lalonde
3f8e535692 FEATURE: bbpress importer can import private messages 2017-08-21 16:39:57 -04:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Gerhard Schlager
32ef30cff9
phpBB3 importer: Permalink for first post of each topic was missing 2017-07-25 23:29:56 +02:00
Gerhard Schlager
5be52fa821
phpBB3 importer: Use permalink normalization for category links 2017-07-25 21:14:46 +02:00
Arpit Jalan
ed5d322196 FIX: handle existing users in imports 2017-07-24 16:22:56 +05:30
Guo Xiang Tan
d38b55e0c8 Merge pull request #4959 from farmdawgnation/minor-mbox-import-fix 
Add a nil check on the connection before attempting to exec it
 2017-07-21 09:45:15 +09:00
Leo McArdle
d0b027d88d FEATURE: phase 1 of supporting multiple email addresses 2017-07-20 11:22:27 +09:00
Sam Saffron
d0c5205a52 Feature: Change markdown engine to markdown it 
This commit removes the old evilstreak markdownjs engine.

- Adds specs to WhiteLister and changes it to stop using globals
    (Fixes large memory leak)
- Fixes edge cases around bbcode handling
- Removes mdtest which is no longer valid (to be replaced with
    CommonMark)
- Updates MiniRacer to correct minor unmanaged memory leak
- Fixes plugin specs
 2017-07-17 11:41:34 -04:00
Sam
79a084dd58 Revert "remove old markdown engine work-in-progress" 
This reverts commit ee470b5317.
 2017-07-12 18:10:51 -04:00
Sam Saffron
ee470b5317 remove old markdown engine work-in-progress 2017-07-12 17:44:40 -04:00
Matt Farmer
5b11391588 Add a nil check on the connection before attempting to exec it 2017-07-07 11:11:43 -04:00
Jay Pfaffman
1c9cab0908 mybb: disable outgoing mail 2017-06-30 16:28:14 -07:00
Jay Pfaffman
a7d89d2a8a mybb: add permalink support 2017-06-30 14:11:49 -07:00
Melroy Neil Dsouza
420abce549 Optimize import script - vanilla_mysql.rb 
Use WHERE id > last_id instead of OFFSET because OFFSET causes performance issues
 2017-06-19 10:30:55 +05:30
Guo Xiang Tan
d82dbd565b FIX: Remove dependency on rest-client. 2017-06-16 09:42:41 +09:00
Régis Hanol
4e8e9c2d2f FIX: ensure we never collide ids in the JIVE API importer 2017-06-06 16:13:07 +02:00
Régis Hanol
b93a4e369d add support for importing content by id in JIVE API importer 2017-06-03 21:49:30 +02:00
Gerhard Schlager
8299e7e8c3
Add new, experimental version of mbox importer 2017-05-29 20:59:18 +02:00
lead4good
0d36fb0c2b bbpress import script: Adding support for anonymous users 2017-05-25 13:15:50 +02:00
Robin Ward
cdbe027c1c Refactor FileHelper to use keyword arguments. 2017-05-24 13:54:26 -04:00
Régis Hanol
2826f98bb9 update JIVE API importer 2017-05-16 10:22:44 +02:00
Régis Hanol
9641d2413d REFACTOR: upload workflow creation into UploadCreator 
- Automatically convert large-ish PNG/BMP to JPEG
- Updated fast_image to latest version
 2017-05-11 00:16:57 +02:00
Robin Ward
71414f7ba2 We don't need to log downloading avatars that are too large 2017-05-09 14:46:42 -04:00
Régis Hanol
4b9cc1ad17 import likes & bookmarks in JIVE API importer 2017-05-03 16:00:23 +02:00
Régis Hanol
e155cb6db1 FIX: JIVE API Importer 
- Added the 'hierarchical=false' query parameter to retrieve comments (replies) chronologically
- Calling 'create_post' (singular) wasn't updating the LookupContainer and thus we couldn't populate the 'reply_to_post_number' column
 2017-04-26 15:36:35 +02:00
Gerhard Schlager
8d9b5ed70a
phpBB3 importer: Disable file size validators during import 2017-04-26 01:05:36 +02:00
Régis Hanol
29ddb3a611 update JIVE API importer 2017-04-24 22:03:12 +02:00
Régis Hanol
ff6a98183f WIP: JIVE importer using the API 2017-04-12 22:30:36 +02:00
michael@discoursehosting.com
17b5303449 Convert nested quotes 2017-04-02 20:38:23 +00:00
michael@discoursehosting.com
6168bcb4c4 Someone broke/disabled? the php check 2017-04-02 20:35:43 +00:00
michael@discoursehosting.com
7e27559f3e We need to get rid of the ignore_quotes thing as well 2017-04-02 20:33:48 +00:00
michael@discoursehosting.com
60be3e0d38 Get rid of the weird and broken message graph 2017-04-02 20:31:06 +00:00
michael@discoursehosting.com
7d09af141e Apparently need to save 2017-04-02 20:28:16 +00:00
michael@discoursehosting.com
a72f56c3a7 Add migratepassword plugin hash 2017-04-02 20:27:57 +00:00
michael@discoursehosting.com
332d655a1d Use html_for_upload instead of broken v8 stuff 2017-04-02 20:25:08 +00:00
michael@discoursehosting.com
e788ac3351 Add permalinks creator for PrettyURL plugin 2017-04-02 20:23:03 +00:00
Robin Ward
14410b71fb Convert server side paths to use /u/ 2017-03-30 10:23:24 -04:00
David McClure
ed627c8233 FEATURE: Socialcast Importer: Set category and tags based on group 2017-02-26 04:54:07 -08:00
Gerhard Schlager
1fb5dac1bc
phpBB3 importer: Fix the link regex 
Thanks to Leo Davidson for the improved regex.
https://meta.discourse.org/t/importing-from-phpbb3/30810/153
 2017-02-05 17:13:31 +01:00
Gerhard Schlager
14a0dff469
phpBB3 importer: Fix import of anonymous users 2017-02-05 15:26:50 +01:00
Gerhard Schlager
b79eef32e7
phpBB3 importer: Make it harder to set wrong table prefix 2017-02-05 14:13:25 +01:00
Gerhard Schlager
5569afc48b
phpBB3 importer: Don't ignore users that belong to non-existent group 2017-02-05 14:13:25 +01:00
Régis Hanol
0d250c3935 50% faster vBulletin 4 importer 2017-02-01 14:33:09 +01:00
Jay Pfaffman
3a5c0c5605 add env variables for vBulletin import script 2017-01-31 10:31:18 +05:30
Robin Ward
4ed46a45ed FIX: In the case of a blank name, use a SHA1 2017-01-19 14:31:50 -05:00
Robin Ward
8d4cff642e FIX: Use the username scrubber when generating the emails 2017-01-18 14:23:23 -05:00
Robin Ward
775b70227d FIX: Better standardizing on username 2017-01-18 13:22:45 -05:00
Robin Ward
f1c242f61d A generic JSON importer that can be used for simple imports 2017-01-18 13:22:45 -05:00
Jay Pfaffman
ffbaf374c8 use .presence rather than DIY checking 2017-01-11 12:55:25 -08:00
Jay Pfaffman
e307bbccf9 Merge branch 'master' of github.com:discourse/discourse into bbpress-missing-display-name 2017-01-11 11:28:38 -08:00
Jay Pfaffman
c5d6bfe7e2 bbpress: Use nicename if display_name is missing 2017-01-11 11:26:55 -08:00
Rafael dos Santos Silva
3a3a464a32 Merge pull request #4642 from miromichalicka/master 
Add support for import from Drupal 6
 2017-01-11 12:56:52 -02:00
Arpit Jalan
6506c0d900 use table prefix in bbpress import script 2017-01-10 16:07:13 +05:30
Miroslav Michalicka
a313ddf8e3 Fix typos 2017-01-08 10:16:32 +01:00
Miroslav Michalicka
bde42e397c Migration script from Drupal 6 2017-01-08 10:07:46 +01:00
Mohamad Abras
bc621a704e fix close topics vb4 importer 2016-12-15 14:20:05 +02:00
Arpit Jalan
59523aef9d more improvements to vBulletin import script 2016-12-05 17:44:18 +05:30
Mohamad Abras
f68194cf8e fix nil for vb4 importer 2016-12-05 04:16:59 +02:00
Mohamad Abras
52749c0121 imporoving vb4 importer 2016-12-04 00:02:47 +02:00
Jay Pfaffman
10f82a9a8e new importer: my little forum 2016-11-01 12:17:36 -07:00
Dmitry Demenchuk
f687bfb7e2 Allow to process imported content. (add additional callbacks) 2016-10-26 16:51:34 +01:00
Régis Hanol
095170d6df Merge pull request #4493 from pfaffman/mbox-email-to-username 
MBOX: replace email address with @username
 2016-10-12 22:45:31 +02:00
Jay Pfaffman
4ccd224c26 Add link to HOWTO 2016-10-12 13:29:19 -07:00
Jay Pfaffman
df3ac5e2e9 MBOX: replace email address with @username 2016-10-12 13:18:34 -07:00
Sam
47733d4f94 Merge pull request #4480 from vinkas0/import/fluxbb 
FluxBB import script
 2016-10-10 16:38:53 +11:00
Jay Pfaffman
40814aa66d no replace_email_addresses in this branch 2016-10-07 09:41:30 -07:00
Jay Pfaffman
1886f021e2 mbox: Improve error checking and reporting 2016-10-07 09:38:54 -07:00
Vinoth Kannan
743ddf50a9 fluxbb.rb: skip admin & mod groups 2016-10-07 13:56:10 +05:30
Vinoth Kannan
ae8088ab99 fluxbb.rb: optional table prefix env variable added 2016-10-07 10:29:56 +05:30
Vinoth Kannan
470529d4c8 fluxbb.rb: importing groups & members 2016-10-06 21:08:40 +05:30
Vinoth Kannan
7bf75f8828 fluxbb.rb: move configuration to ENV 2016-10-06 20:09:40 +05:30
Vinoth Kannan
119746aaa0 new: FluxBB import script file added 2016-10-06 20:00:39 +05:30
Régis Hanol
3d7558c3b7 Merge pull request #4453 from pfaffman/mbox-config-from-env 
mbox: move configuration to ENV, add default user level
 2016-10-03 22:02:45 +02:00
Jay Pfaffman
bae5400042 All config can be done via environment variables 2016-09-30 14:24:58 -07:00
Jay Pfaffman
84de8e92e1 Rexexp.escape LISTNAME 2016-09-28 08:40:31 -07:00
Régis Hanol
11fdb8f038 Merge pull request #4465 from pfaffman/bbpress-env-config 
bbpress.rb: move configuration to ENV
 2016-09-28 11:20:41 +02:00
Jay Pfaffman
1c0e5ce523 bbpress.rb: move configuration to ENV 2016-09-26 15:03:42 -07:00
Jay Pfaffman
6e603c85a8 mbox: sort input files by title, sort messages by date 2016-09-26 12:18:11 -07:00
Sam
262297965f Merge pull request #4442 from pfaffman/kunena3-improvements 
KUNENA3 importer: parameterize database and fields, link to attachments
 2016-09-23 10:36:45 +10:00
Jay Pfaffman
99860910fc mbox: move configuration to ENV, add default user level 2016-09-21 12:01:57 -07:00
Jay Pfaffman
30f2081c1b importer/base.rb: improve progress reporting 2016-09-20 11:27:45 -07:00
Jay Pfaffman
152e5b9b7e parameterize database and fields, link to attachments 2016-09-15 15:24:06 -07:00
Arpit Jalan
92e716a1fd fix vbulletin import script 2016-09-14 08:15:48 +05:30
Arpit Jalan
ed0b355e15 generalize permalink URL in GetSatisfaction import script 2016-09-08 15:01:40 +05:30
Arpit Jalan
bade41db42 improve GetSatisfaction import script 2016-09-08 14:57:20 +05:30
timur
aeae63a56a Change reverse.each 
Change on reverse_each because reverse_each loops in reverse order (no
intermediate array created).
 2016-09-07 14:57:31 +08:00
Arpit Jalan
87e84756f0 improve Ning import script 2016-09-05 15:27:03 +05:30
Arpit Jalan
cdc1d7e46c Merge pull request #4413 from Mogztter/fix-optional-attachments 
FIX: Nabble importer, attachments can be not found
 2016-09-05 00:05:40 +05:30
Hanwen (Steinway) Wu
371f4f4be7 Fix SQLite datetime issues 
The raw datetime string from raw emails doesn't conform to SQLite standard. Therefore it can't be sorted. This fix will make it conforming and sort-able by SQLite.
 2016-08-26 12:47:03 -04:00
Guillaume Grossetie
4ba99bcc04 FIX: Attachments can be not found 2016-08-23 19:58:46 +02:00
Guillaume Grossetie
566ecae0db FIX: Update Nabble importer to use Email.Receiver new API 
Email.Receiver API changed in 3083657358
 2016-08-22 14:04:01 +02:00
David McClure
f7f8226b4c Add Socialcast importer 2016-08-20 17:25:19 -07:00
m.b
2fde6944e3 Add missing fat arrow in bbcode_to_md method rescue block 
Related to https://meta.discourse.org/t/undefined-local-variable-or-method-e-during-phpbb3-migration/48861
 2016-08-19 13:18:27 +02:00
Robin Ward
72d7c055f4 Lots of ugprades to the Disqus importer script 2016-08-17 15:10:46 -04:00
Gerhard Schlager
37a78752e9 phpBB3 importer: fixes for internal links and guest users (#4350) 
* phpBB3 importer: ignore query parameters in internal links

* phpBB3 importer: Don't fail if guest usernames differ only by case

* phpBB3 importer: Import username as name for guest users
 2016-07-27 22:57:33 +02:00
Gerhard Schlager
fccc0c93c5 Store the original username during import (#4349) 2016-07-27 22:30:15 +02:00
Régis Hanol
56d2626187 support 'read_restricted' categories in importers 2016-07-27 18:38:23 +02:00
Arpit Jalan
1c4bc154c9 add SimplePress import script 2016-07-14 19:07:25 +05:30
Arpit Jalan
529528f122 add Drupal JSON import script 2016-07-14 18:56:54 +05:30
Gerhard Schlager
c145e747b6 A few small changes to the phpBB3 importer (#4321) 
* Reconnect to phpBB3 database on connection loss

* Map geek smiley to :nerd: emoji in phpBB3 importer

* Import PMs to yourself from phpBB3

* Allow empty table prefix in phpBB3 importer
 2016-07-11 23:59:15 +02:00
Arpit Jalan
201d344a2d improve vBulletin import script 2016-07-10 14:51:31 +05:30