My discourse instance will be making regular automated public backups
of specific categories. It's preferred to be able to directly control
the path and filename of the output, rather than letting discourse
choose for me. This was already mostly supported, a filename parameter
just needed to be passed through the cli app.
- improve perf of test for existing posts
- always use a system guardian when importing posts
- for lithuim importer requery raw (transform is not repeatable)
The English locale files use aliases and anchors in order to prevent
duplicate translations. Transifex doesn't support them.
The script finds those aliases and anchors in the original locale files
and adds them to the locale files pulled from Transifex.
FEATURE: Adds a link to the Howto on meta
FIX: Suppresses multiple warnings about invalid votes for the same topic
FIX: Adds missing table prefixes
FIX: Removes duplicate column from query
- Extensive refactoring of the existing importer
- Configuration of import with settings.yml instead of editing code
- Supports importing from phpBB 3.0.x and 3.1.x
- Imports all attachments (not just the ones embedded with [attachment])
from posts and private messages
- Imports all existing attachments without the need to configure allowed
file extensions or file sizes
- Imports polls
- Imports bookmarks
- Imports sticky topics and (global) announcements as pinned topics
- Imports categories in the original order and sets the content of the
category description topic
- Sets the creation date of category description topics to the creation
date of the first topic in each category
- Imports additional user attributes: last seen date, registration
IP address, website, date of birth, location
- Optionally set the user's name to its username
- Users that didn't activate their account in phpBB3 are imported as
inactive users
- All imported, active users are automatically approved
- Users that were deactivated in phpBB3 get suspended for 200 years
during the import
- Anonymous user can be imported as suspended users instead of the
system user
- Forums of type "link" are not imported as categories anymore
- Internal links to posts get rewritten during the import (previously
only links to topics got rewritten)
- Ordered lists with BBCode [list=a] (which are unsupported in
Discourse) get imported as if they would be [list=1]
- Importing of avatars, attachments, private messages, polls and
bookmarks can be disabled via configuration file
- Optional fixing of private messages for forums that have been upgraded
from phpBB2 prevents the import of duplicate messages and tries to
group related messages into topics
- Table prefix (default: phpbb) is configurable
- Most of phpBB's default smilies are mapped to Emojis and all other
smilies get uploaded and embedded as images. Smiley mappings can be
added or overridden in the settings.yml file.
- Move some methods into their own classes in order to make it easier
to reuse them outside of classes extending the base importer. For
compatibility reasons the old methods are still in the base importer
and delegate to the new objects. The following methods and hashes were
extracted:
- all the lookup maps for existing and imported data
- all the methods used for uploads and attachments
- No need to store failed users and groups. This information wasn't
used anyway.
- Print progress instead of category names when importing categories.
- Allow importers to override if bbcode_to_md should be used (until now
it always used ARGV)
- Allow importers to add additional site settings that automatically get
restored after the importer finishes.
- Show how many posts and messages are imported per minute. This should
help detecting when the import is slowing down and needs to be
restarted.
- Use max_image_width and max_image_height from settings instead of
hard-coded values for uploaded images.
- new hidden site setting 'migrate_to_new_scheme' (defaults to false)
- new rake tasks to toggle migration to new scheme
- FIX: migrate_to_new_scheme also works with CDN
- PERF: improve perf of the DbHelper.remap method
- REFACTOR: UrlHelper is now a class
- There's no need for configuring "trans.<lang>" for each resource.
Using the global "lang_map" in the Transifex client config is enough.
- The Transifex client can rename locales. No need to do this manually.
- Pull all translation files for each locale that has a file named
config/locales/client.<lang>.yml. Up until now the script did only
update existing files, but didn't add missing ones. That is why there
are so many translation files for the poll plugin missing.
This sets the last_emailed_at for imported users to now which postpones the sending of digest emails for the configured amount of time. Therefore, the first digest email gets sent after 1 week (the default value of default_digest_email_frequency).
As suggested in https://meta.discourse.org/t/dont-trigger-digests-on-migration/26345/7
- Allows importing globally pinned topics
- Doesn't restore the original value of a SiteSetting anymore if an importer changed its value
- Allows all file extensions during the import
- Adds the ability to import bookmarks
This changes the content of `@categories_lookup` from `Category` objects
to IDs since the category names aren't needed anymore. The lookup
method has been renamed too.
- FEATURE: Print the time it took for the import to finish
- FEATURE: Allow importing of unactivated users
- FIX: Remove unused variables
- FIX: Accessing an undefined variable is not a good idea
- FEATURE: TopicCreator now supports 'pinned_at' parameter
- FIX: 🐛 FIX TopicQuerySQL to support pinned topic older than 2010
- FIX: 🐛 Properly remove all HTML Entities from Usernames/Titles/Category Names/Groups in vBulletin importer
- FIX: 🐛 Properly handle specific vBulletin BBCode (quotes/mentions)
- FIX: 🐛 Make sure we generate a username from the name of the user instead of a fake email
- FEATURE: Allow for custom timezone in vBulletin importer
- FEATURE: Support for profile pictures/background in vBulletin importer
- FIX: 🐛 merge the categories tree to only 2 levels in vBulletin importer
I had imported a phpbb forum to mybb previously. The importer script
messed up the firstpost ids. In this case most topics cannot be imported
to discourse as the script is not able to determine the topic post.
In case you are infected of this problem just uncomment this mysql
query. It will retrieve the first_post_id manually but it will slow down
the import.
The post_ids in my mybb installation did not be in the right order. Post
with ID 2 was posted after post 5. I think it has something to do with a
previous import from phpbb to mybb.
bbcode-to-md happily ignores all phpBB's lists. The list syntax is
[list][*]item 1
[*]item 2
[/list]
and
[list=1][*]item 1
[*]item 2
[/list]
respectively for [ul] and [ol]s. Luckily, phpBB adds closing tags for [*] items. My workaround simply converts phpBBs lists into bbcode using [ul] and [ol] which then can be converted by the standard bbcode-to-md code.
FIX: history revision can now properly be hidden
FIX: PostRevision serializer is now entirely dynamic to properly handle
hidden revisions
FIX: default history modal to "side by side" view on mobile
FIX: properly hiden which revision has been hidden
UX: inline category/user/wiki/post_type changes with the revision
details
FEATURE: new '/posts/:post_id/revisions/latest' endpoint to retrieve
latest revision
UX: do not show the hide/show revision button on mobile (no room for
them)
UX: remove CSS transitions on the buttons in the history modal
FIX: PostRevisor now handles all the changes that might create new
revisions
FIX: PostRevision.ensure_consistency! was wrong due to off by 1
mistake...
refactored topic's callbacks for better readability
extracted 'PostRevisionGuardian'
This code adds all three avatar types to the import. Uploaded avatars and default gallery avatars are converted, hotlinked are pulled from the remote web site and stored as local. Beware, though: this is currently done during the importer run and can consume loong periods of time if many downloads fail with timeouts.
A minor fix concerns attachments to posts with white space in the real_name, this is handled properly now.
Changed internals so trust levels are referred to with
TrustLevel[1], TrustLevel[2] etc.
This gives us much better flexibility naming trust levels, these names
are meant to be controlled by various communities.
FEATURE: add backtrace when an exception happen (importers)
FIX: post-processing should also happen on first posts (vBulletin
importer)
PERF: faster topic bypass when already imported
After an import has finished the site settings should be reset to their
previous values. For example, since 'disable_emails' was set to false
before any import, it wasn't possible to login after the import
(sending email for password reset didn't work).
PERF: disabled refresh_avatar callback when importing users
PERF: avoid using UsernameSuggester when not needed
FIX: categories wasn't working
FIX: posts from deleted users are now from the system user
