Package: tidycjk
Type: Package
Title: Tidy Tools for Chinese, Japanese and Korean Text
Version: 0.2.0
Authors@R: 
    c(person(given = "Youzhi",
             family = "Yu",
             role = c("aut", "cre"),
             email = "yuyouzhi666@icloud.com"))
Description: A tidy toolkit for text that is written in Chinese,
        Japanese or Korean. Most text tooling in R assumes that words
        are separated by whitespace, which CJK writing does not use, so
        ordinary summaries of a text column either treat a sentence as
        one undifferentiated blob or split it into isolated characters.
        Word segmentation is a pluggable engine, with dictionary-based
        segmentation for Chinese and Japanese supplied through 'ICU'
        and further engines registrable by the caller, because where a
        word ends is a fact about a language and not about Unicode.
        'tidycjk' classifies characters by Unicode block, reports which
        script and which language a text is written in, measures how
        much of a text is CJK, and turns those measurements into
        tibbles that slot straight into a 'tidyverse' workflow. It also
        measures display width in terminal columns, pads, truncates and
        wraps to a width rather than to a character count, and
        normalises fullwidth and halfwidth forms surgically --
        including composing halfwidth katakana voiced marks into single
        code points -- while offering the Unicode normalisation forms
        separately for when a full 'NFKC' fold is what is wanted.
        Romanisation, conversion between simplified and traditional
        Han, conversion between the two kana syllabaries, and
        decomposition of Hangul syllables into jamo are provided on the
        same vectorised contract. Sentence segmentation, character
        n-grams, punctuation removal by Unicode category and
        locale-aware ordering cover the preprocessing that generic
        tooling gets wrong on CJK, where the sentence terminators are
        ideographic, a code point sort is not an alphabetical one, and
        the POSIX punctuation class is resolved by the C library rather
        than by Unicode. Language detection deliberately returns NA
        rather than guessing when a text is written in Han characters
        only, because Japanese written without kana cannot be
        distinguished from Chinese by script alone. Everything is
        derived from the Unicode specification; the package makes no
        network requests and needs no compiled code of its own.
License: GPL (>= 3)
URL: https://pursuitofdatascience.github.io/tidycjk/,
        https://github.com/PursuitOfDataScience/tidycjk
BugReports: https://github.com/PursuitOfDataScience/tidycjk/issues
Encoding: UTF-8
Language: en-GB
Roxygen: list(markdown = TRUE)
Depends: R (>= 3.5.0)
Imports: dplyr (>= 1.1.0), stringi, tibble, utils
Suggests: knitr, rmarkdown, testthat (>= 3.0.0)
Config/testthat/edition: 3
VignetteBuilder: knitr
RoxygenNote: 7.3.2
Config/pak/sysreqs: libicu-dev
Repository: https://pursuitofdatascience.r-universe.dev
Date/Publication: 2026-10-05 17:36:49 UTC
RemoteUrl: https://github.com/pursuitofdatascience/tidycjk
RemoteRef: HEAD
RemoteSha: 6adddca5cb50ac5bf581f6440f301951a8a043a3
NeedsCompilation: no
Packaged: 2026-10-07 08:28:49 UTC; root
Author: Youzhi Yu [aut, cre]
Maintainer: Youzhi Yu <yuyouzhi666@icloud.com>