# Ambiguous and invisible Unicode references Gitea maintains the concrete list that most closely matches the requested "ambiguous Unicode characters" catalog: - [vendored `ambiguous.json`](https://github.com/go-gitea/gitea/blob/9c715c0f9d12336e026e89be85f9f711c66e7ff6/modules/charset/generate/ambiguous.json) is the generator input; - [generated Go tables](https://github.com/go-gitea/gitea/blob/9c715c0f9d12336e026e89be85f9f711c66e7ff6/modules/charset/ambiguous_gen.go) are easier to browse as code; - [locale selection](https://github.com/go-gitea/gitea/blob/9c715c0f9d12336e026e89be85f9f711c66e7ff6/modules/charset/ambiguous.go) combines a locale-specific table with `_common` at runtime; and - [the generator](https://github.com/go-gitea/gitea/blob/9c715c0f9d12336e026e89be85f9f711c66e7ff6/modules/charset/generate/generate.go) records the upstream source and a separate invisible-rune catalog. At Gitea commit `9c715c0f9d12336e026e89be85f9f711c66e7ff6`, the wrapped JSON contains 16 tables and 2,108 mapping pairs across them. `_common` contains 1,538 pairs; `_default` and the language-specific tables supply the remainder. The array values alternate between a confusable code point and the code point it resembles. Counts across tables are not necessarily unique global mappings. Gitea's generator identifies [`hediet/vscode-unicode-data`](https://github.com/hediet/vscode-unicode-data/blob/main/out/ambiguous.json) as its upstream. For standards-based identifier spoofing, the canonical broader reference is [Unicode UTS #39](https://www.unicode.org/reports/tr39/) and its [`confusables.txt`](https://www.unicode.org/Public/security/latest/confusables.txt) data file. GitHub separately documents a warning for hidden bidirectional text; that protection is about bidi controls rather than a published Gitea-style locale-aware lookalike table. ## Implemented John Slop profile John Slop uses `unicode-security` 0.1.2, whose checked-in UTS #39 tables target Unicode 16.0. The generated `assets/unicode.toml` contains every non-ASCII scalar whose single-character skeleton is non-empty printable ASCII, plus nine reviewed punctuation mappings and 95 compatibility width folds. Width folding takes precedence when a UTS #39 visual skeleton would produce a different target, such as fullwidth `I` mapping to `I` rather than `l`. That is 1,884 mappings in the current profile. Each entry has a stable ID, source, target, rationale, and enabled switch, and the same asset owns the diagnostic metadata. The default `[unicode] confusables = "replace"` policy emits `unicode-ascii-replacement` diagnostics and supplies the same edits to CLI output, `john-slop-fmt`, LSP quick fixes, and LSP formatting. Set the value to `review` for identical findings without edits, or `off` to suppress generated lookalikes. The reviewed punctuation and width-folding mappings remain active under all three settings and can still be disabled individually. This broad mapping is intentionally more aggressive than Gitea's locale-aware display warning. Greek, Cyrillic, mathematical, and other characters may be legitimate text, so projects handling multilingual content should prefer `review` unless ASCII normalization is their explicit policy. `unicode-invisible-control` uses the invisible ranges generated by Gitea at the pinned commit above, including bidi embedding/override and isolate controls. Ordinary space, tab, line feed, and the carriage return in a CRLF pair are not reported. Each other match receives an exact UTF-8 source span that adapters convert normally. Controls never feed the formatter or LSP quick fixes because silent deletion has no generally safe meaning. Regenerate the checked-in policy deterministically with: ```sh cargo run -p john-slop-core --example generate_unicode_catalog \ > crates/john-slop-core/assets/unicode.toml ``` An exhaustive test walks all Unicode scalar values and rejects mapping drift. Future profile updates must pin their upstream Unicode/data version, retain provenance and licensing, test ordinary multilingual near misses, and renew the hash-bound default-policy sign-off.