github-mirror/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-23 03:09:00 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
46,436 Commits 190 Branches 500 Tags 938 MiB
64601779f0
Commit Graph

1 Commits

Author SHA1 Message Date
Alan Guo Xiang Tan
930f51e175 FEATURE: Split up text segmentation for Chinese and Japanese. 
* Chinese segmenetation will continue to rely on cppjieba
* Japanese segmentation will use our port of TinySegmenter
* Korean currently does not rely on segmentation which was dropped in c677877e4f
* SiteSetting.search_tokenize_chinese_japanese_korean has been split
into SiteSetting.search_tokenize_chinese and
SiteSetting.search_tokenize_japanese respectively
 2022-02-07 09:21:14 +08:00