R_REDDYX.XYZ
AIPython★ 201⑂ 9 forksMIT

cdcailf/cjkfold

Normalize Similar CJK Glyphs

Folds visually similar CJK glyphs (Kangxi radicals and CJK radical supplements) into a canonical form, enabling PDF‑extracted text to match searchable Unicode.

// KEY FEATURES

  • Maps Kangxi radicals and CJK radical supplements to their base CJK Unicode equivalents.
  • Provides a pure‑Python `fold_cjk` function that normalizes input strings in place.
  • Useful for cleaning PDF‑extracted text where OCR substitutes look‑alike CJK glyphs.
#chinese#cjk#nfkc#nlp#pdf-extraction#rag#retrieval#text-normalization#unicode
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

DEEPSEEK-CODE-WHALE - AI CODING AGENT★ 201CODEX-APP-TRANSFER: API BRIDGE★ 201ONIONCLAW: TOR FOR AI AGENTS★ 201NODE9-PROXY: SHIELD FOR AI AGENTS★ 201REPOPROMPT-CE — SIMPLE AI CODE AGENT★ 201RSSH: Reliable SSH Client★ 201

← FULL CATALOG

cdcailf/cjkfold — Normalize Similar CJK Glyphs | REDDYX AI