jieba-rs
🚀 Help me to become a full-time open-source developer by sponsoring me on GitHub
The Jieba Chinese Word Segmentation Implemented in Rust
Installation
Add it to your Cargo.toml:
[dependencies]
jieba-rs = "0.7"
then you are good to go. If you are using Rust 2015 you have to extern crate jieba_rs to your crate root as well.
Example
use jieba_rs::Jieba;
fn main() {
let jieba = Jieba::new();
let words = jieba.cut("我们中出了一个叛徒", false);
assert_eq!(words, vec!["我们", "中", "出", "了", "一个", "叛徒"]);
}
Enabling Additional Features
default-dictfeature enables embedded dictionary, this features is enabled by defaulttfidffeature enables TF-IDF keywords extractortextrankfeature enables TextRank keywords extractor
[dependencies]
jieba-rs = { version = "0.7", features = ["tfidf", "textrank"] }
Run benchmark
cargo bench --all-features
Benchmark: Compare with cppjieba
jieba-rs bindings
@node-rs/jiebaNodeJS bindingjieba-phpPHP bindingrjieba-pyPython bindingcang-jieChinese tokenizer for tantivytantivy-jiebaAn adapter that bridges between tantivy and jieba-rsjieba-wasmthe WebAssembly binding
License
This work is released under the MIT license. A copy of the license is provided in the LICENSE file.