mirror of
https://github.com/discourse/discourse.git
synced 2025-03-04 02:04:10 +08:00
394cd43d77
Scrubbing an ASCII-8BIT string isn't ever going to remove anything, because there's no code point that isn't valid 8-bit ASCII. Since we'd really prefer it if everything were UTF-8 anyway, we'll just assume, for now, that whatever comes out of SimpleRSS is probably UTF-8, and just nuke anything that isn't a valid UTF-8 codepoint. Of course, the *real* bug here is that SimpleRSS [unilaterally converts everything to ASCII-8BIT](https://github.com/cardmagic/simple-rss/issues/15). It's presumably *far* too much to ask that it detects the encoding of the source RSS feed and marks the parsed strings with the correct encoding...