#81021 (Unicode NFC Hangul)

/home/claude/golang_go_81021/go/src · Tue 25 Aug 2026 08:34:16 · 2 messages
499f4eb1-6fd9-4190-be20-22ac8f0f6e51

You Tue 25 Aug 2026 08:08:16

Triage and debug go.dev/issue/81021.

Assistant Tue 25 Aug 2026 08:08:18

I'll start by looking at the issue and the working directory.

Ran tools (49 Bash, 1 ToolSearch, 3 WebFetch, 1 SendUserFile)
Bash ls /home/claude/golang_go_81021 && echo "---" && git log --oneline -3 2>&1 | head
command
ls /home/claude/golang_go_81021 && echo "---" && git log --oneline -3 2>&1 | head
description
List working directory contents
Result
env.sh
go
---
fatal: not a git repository (or any of the parent directories): .git
ToolSearch select:WebFetch
query
select:WebFetch
max_results
3
Bash cat /home/claude/golang_go_81021/env.sh; echo "---"; cd /home/claude/golang_go_81021/go && git log --oneline -3
command
cat /home/claude/golang_go_81021/env.sh; echo "---"; cd /home/claude/golang_go_81021/go && git log --oneline -3
description
Inspect env.sh and go repo
Result
export WORK=/home/claude/golang_go_81021
export GOROOT=$WORK/go
export GOPATH=$WORK
export GOBIN=$WORK/bin
export PATH=$GOBIN:/home/claude/.local/bin:/home/claude/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
---
13b477edc7 runtime: fix flaky refill detection in TestScanAllocIssue77573
72755de33c os, internal/poll: use splice for copies involving pipes
3e6ad2cd0a cmd/compile: always do rewriteCondSelectIntoMath for powers of two
WebFetch https://go.dev/issue/81021
url
https://go.dev/issue/81021
prompt
Reproduce the full issue text: title, body, all comments, labels, status, and any code snippets.
Result
REDIRECT DETECTED: The URL redirects to a location that was not fetched automatically.

Original URL: https://go.dev/issue/81021
Redirect URL (from the server's Location header — server-supplied, not verified): https://github.com/golang/go/issues/81021
Status: 302 Found

To complete your request, I need to fetch content from the redirected URL. Please use WebFetch again with these parameters:
- url: "https://github.com/golang/go/issues/81021"
- prompt: "Reproduce the full issue text: title, body, all comments, labels, status, and any code snippets."
Bash gh issue view 81021 --repo golang/go --comments 2>&1 | head -200
command
gh issue view 81021 --repo golang/go --comments 2>&1 | head -200
description
Fetch issue via gh
Result
/bin/bash: line 1: gh: command not found
WebFetch https://github.com/golang/go/issues/81021
url
https://github.com/golang/go/issues/81021
prompt
Reproduce the full issue text verbatim: title, body, all comments, labels, status, and any code snippets.
Result
# Issue Summary: NFC Unicode Normalization Bug with Hangul Syllables

**Title:** "x/text/unicode/norm: NFC drops a canonical composition when a Hangul syllable precedes it (U+113C5, Unicode 17 tables) #81021"

**Status:** Open

**Reporter:** tannevaled (opened Aug 22, 2026)

## Problem Description

The NFC normalization function in `golang.org/x/text` incorrectly handles the Unicode character U+113C5 (TULU-TIGALARI VOWEL SIGN OO) when preceded by a Hangul syllable. 

**Expected behavior:** The character should recompose from its canonical decomposition (U+113C2 U+113C2) in all contexts.

**Actual behavior:** The composition fails when a Hangul syllable precedes it. For example:
- `NFC(U+AC00 U+113C5)` produces `U+AC00 U+113C2 U+113C2` (incorrect)
- Instead of producing `U+AC00 U+113C5` (correct)

## Root Cause

The issue stems from the `reorderBuffer.compose()` function's Hangul handling path. When `isJamoVT` detects any Jamo character, it routes the entire segment to `combineHangul()`, which processes characters without attempting non-Hangul compositions. This prevents the recomposition of U+113C2 + U+113C2 → U+113C5.

## Context

- Affects Go 1.27.0+ (when Unicode 17 tables are selected)
- Not caught by existing tests in the normalization test suite
- Independent of issue #81001
Bash ls src/vendor/golang.org/x/text/unicode/norm/ 2>/dev/null; echo "---"; find / -path /proc -prune -o -name "composition.g...
command
ls src/vendor/golang.org/x/text/unicode/norm/ 2>/dev/null; echo "---"; find / -path /proc -prune -o -name "composition.go" -path "*norm*" -print 2>/dev/null
description
Locate norm package
Result
composition.go
forminfo.go
input.go
iter.go
normalize.go
readwriter.go
tables15.0.0.go
tables17.0.0.go
transform.go
trie.go
---
/home/claude/golang_go_80855/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/sdk/go1.26.0/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80855/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.27rc3.linux-amd64/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/sonnet/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.linux-amd64/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_79754/pkg/mod/golang.org/x/text@v0.40.0/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.27rc3.linux-amd64/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/sonnet/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.linux-amd64/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_79754/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.7.linux-amd64/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/x/text@v0.40.0/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition.go
/home/claude/golang_go_79754/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.7.linux-amd64/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.13.linux-amd64/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.6.linux-amd64/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.7.linux-amd64/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.13.linux-amd64/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.7.linux-amd64/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/sdk/go1.26.0/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80746/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/pkg/mod/golang.org/toolchain@v0.0.1-go1.26.6.linux-amd64/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80746/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_68111/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_68111/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_78576/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_78576/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/go1/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/issue9/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/issue9/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80654/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_78877/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80883/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_78438/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80654/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_79754/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_80883/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_79754/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_81021/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_78438/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_78877/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_33564/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/golang_go_33564/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/home/claude/go1/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
/usr/share/go-1.23/src/vendor/golang.org/x/text/unicode/norm/composition.go
/usr/share/go-1.22/src/vendor/golang.org/x/text/unicode/norm/composition.go
/usr/share/go-1.23/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
Bash cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && ls -la && head -20 tables17.0.0.go 2>/de...
command
cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && ls -la && head -20 tables17.0.0.go 2>/dev/null; echo "=== cmd/vendor ==="; ls /home/claude/golang_go_81021/go/src/cmd/vendor/golang.org/x/text/unicode/norm/
description
Inspect vendored norm package
Result
total 864
drwxrwxr-x 2 claude claude   4096 Aug 25 08:05 .
drwxrwxr-x 4 claude claude   4096 Aug 25 08:05 ..
-rw-rw-r-- 1 claude claude  14452 Aug 25 08:05 composition.go
-rw-rw-r-- 1 claude claude   9422 Aug 25 08:05 forminfo.go
-rw-rw-r-- 1 claude claude   2070 Aug 25 08:05 input.go
-rw-rw-r-- 1 claude claude  11219 Aug 25 08:05 iter.go
-rw-rw-r-- 1 claude claude  15293 Aug 25 08:05 normalize.go
-rw-rw-r-- 1 claude claude   2957 Aug 25 08:05 readwriter.go
-rw-rw-r-- 1 claude claude 395028 Aug 25 08:05 tables15.0.0.go
-rw-rw-r-- 1 claude claude 402801 Aug 25 08:05 tables17.0.0.go
-rw-rw-r-- 1 claude claude   2422 Aug 25 08:05 transform.go
-rw-rw-r-- 1 claude claude   1233 Aug 25 08:05 trie.go
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.

//go:build go1.27

package norm

import "sync"

const (
	// Version is the Unicode edition from which the tables are derived.
	Version = "17.0.0"

	// MaxTransformChunkSize indicates the maximum number of bytes that Transform
	// may need to write atomically for any Form. Making a destination buffer at
	// least this size ensures that Transform can always make progress and that
	// the user does not need to grow the buffer on an ErrShortDst.
	MaxTransformChunkSize = 35 + maxNonStarters*4
)

var ccc = [56]uint8{
=== cmd/vendor ===
composition.go
forminfo.go
input.go
iter.go
normalize.go
readwriter.go
tables15.0.0.go
tables17.0.0.go
transform.go
trie.go
Bash cd /home/claude/golang_go_81021 && source env.sh && which go; go version; ls bin 2>/dev/null; ls go/bin 2>/dev/null
command
cd /home/claude/golang_go_81021 && source env.sh && which go; go version; ls bin 2>/dev/null; ls go/bin 2>/dev/null
description
Check go toolchain
Result
/usr/bin/go
go version go1.22.2 linux/amd64
go
gofmt
Bash ./go/bin/go version
command
./go/bin/go version
description
Check built GOROOT go version
Result
go version go1.28-devel_13b477edc7 Thu Aug 20 11:16:25 2026 -0400 linux/amd64
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
mkdir -p $S/normtest/norm
cp /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm/*.go $S/normtest/norm/
cd $S/normtest
cat > go.mod <<'EOF'
module normtest

go 1.27
EOF
cat > main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/norm"
)

func dump(name string, r []rune) {
	s := string(r)
	out := []rune(norm.NFC.String(s))
	fmt.Printf("%s: in=%U -> NFC=%U\n", name, r, out)
}

func main() {
	fmt.Println("Version:", norm.Version)
	dump("bare", []rune{0x113C2, 0x113C2})
	dump("after A", []rune{'A', 0x113C2, 0x113C2})
	dump("after hangul syl", []rune{0xAC00, 0x113C2, 0x113C2})
	dump("after hangul LV jamo", []rune{0x1100, 0x1161, 0x113C2, 0x113C2})
	dump("precomposed after hangul", []rune{0xAC00, 0x113C5})
	dump("precomposed bare", []rune{0x113C5})
	// NFD then NFC roundtrip
	for _, s := range []string{"가\U000113C5", "\U000113C5"} {
		d := norm.NFD.String(s)
		c := norm.NFC.String(d)
		fmt.Printf("roundtrip %U: NFD=%U NFC=%U ok=%v\n", []rune(s), []rune(d), []rune(c), c == s)
	}
}
EOF
sed -i 's|^//go:build go1.27$|//go:build go1.27|' norm/tables17.0.0.go
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run . 2>&1 | head -40
description
Reproduce the NFC bug in scratch module
Result
norm/normalize.go:15:2: no required module provides package golang.org/x/text/transform; to add it:
	go get golang.org/x/text/transform
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cp -r /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/transform ./transform && sed -i 's|"golang.org/x/text/transform"|"normtest/transform"|' norm/*.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run . 2>&1 | head -40
description
Add transform dep and run
Result
Version: 17.0.0
bare: in=[U+113C2 U+113C2] -> NFC=[U+113C5]
after A: in=[U+0041 U+113C2 U+113C2] -> NFC=[U+0041 U+113C5]
after hangul syl: in=[U+AC00 U+113C2 U+113C2] -> NFC=[U+AC00 U+113C2 U+113C2]
after hangul LV jamo: in=[U+1100 U+1161 U+113C2 U+113C2] -> NFC=[U+AC00 U+113C2 U+113C2]
precomposed after hangul: in=[U+AC00 U+113C5] -> NFC=[U+AC00 U+113C2 U+113C2]
precomposed bare: in=[U+113C5] -> NFC=[U+113C5]
roundtrip [U+AC00 U+113C5]: NFD=[U+1100 U+1161 U+113C2 U+113C2] NFC=[U+AC00 U+113C2 U+113C2] ok=false
roundtrip [U+113C5]: NFD=[U+113C2 U+113C2] NFC=[U+113C5] ok=true
Shell cwd was reset to /home/claude/golang_go_81021
Bash cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && grep -n "isJamoVT\|combineHangul\|func (...
command
cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && grep -n "isJamoVT\|combineHangul\|func (rb \*reorderBuffer) compose\|Hangul\|hangul" composition.go forminfo.go normalize.go | head -60
description
Find Hangul handling
Result
forminfo.go:156:	// TODO: create the decomposition for Hangul?
composition.go:219:	if rune := src.hangul(i); rune != 0 {
composition.go:220:		rb.decomposeHangul(rune)
composition.go:235:	if rune := src.hangul(i); rune != 0 {
composition.go:236:		rb.decomposeHangul(rune)
composition.go:277:// appendRune inserts a rune at the end of the buffer. It is used for Hangul.
composition.go:286:// assignRune sets a rune at position pos. It is used for Hangul and recomposition.
composition.go:293:// runeAt returns the rune at position n. It is used for Hangul and recomposition.
composition.go:301:// It is used for Hangul and recomposition.
composition.go:307:// For Hangul we combine algorithmically, instead of using tables.
composition.go:309:	hangulBase  = 0xAC00 // UTF-8(hangulBase) -> EA B0 80
composition.go:310:	hangulBase0 = 0xEA
composition.go:311:	hangulBase1 = 0xB0
composition.go:312:	hangulBase2 = 0x80
composition.go:314:	hangulEnd  = hangulBase + jamoLVTCount // UTF-8(0xD7A4) -> ED 9E A4
composition.go:315:	hangulEnd0 = 0xED
composition.go:316:	hangulEnd1 = 0x9E
composition.go:317:	hangulEnd2 = 0xA4
composition.go:334:const hangulUTF8Size = 3
composition.go:336:func isHangul(b []byte) bool {
composition.go:337:	if len(b) < hangulUTF8Size {
composition.go:341:	if b0 < hangulBase0 {
composition.go:346:	case b0 == hangulBase0:
composition.go:347:		return b1 >= hangulBase1
composition.go:348:	case b0 < hangulEnd0:
composition.go:350:	case b0 > hangulEnd0:
composition.go:352:	case b1 < hangulEnd1:
composition.go:355:	return b1 == hangulEnd1 && b[2] < hangulEnd2
composition.go:358:func isHangulString(b string) bool {
composition.go:359:	if len(b) < hangulUTF8Size {
composition.go:363:	if b0 < hangulBase0 {
composition.go:368:	case b0 == hangulBase0:
composition.go:369:		return b1 >= hangulBase1
composition.go:370:	case b0 < hangulEnd0:
composition.go:372:	case b0 > hangulEnd0:
composition.go:374:	case b1 < hangulEnd1:
composition.go:377:	return b1 == hangulEnd1 && b[2] < hangulEnd2
composition.go:381:func isJamoVT(b []byte) bool {
composition.go:386:func isHangulWithoutJamoT(b []byte) bool {
composition.go:388:	c -= hangulBase
composition.go:392:// decomposeHangul writes the decomposed Hangul to buf and returns the number
composition.go:394:func decomposeHangul(buf []byte, r rune) int {
composition.go:396:	r -= hangulBase
composition.go:408:// decomposeHangul algorithmically decomposes a Hangul rune into
composition.go:410:// See https://unicode.org/reports/tr15/#Hangul for details on decomposing Hangul.
composition.go:411:func (rb *reorderBuffer) decomposeHangul(r rune) {
composition.go:412:	r -= hangulBase
composition.go:422:// combineHangul algorithmically combines Jamo character components into Hangul.
composition.go:423:// See https://unicode.org/reports/tr15/#Hangul for details on combining Hangul.
composition.go:424:func (rb *reorderBuffer) combineHangul(s, i, k int) {
composition.go:438:			l := rb.runeAt(s) // also used to compare to hangulBase
composition.go:444:				rb.assignRune(s, hangulBase+
composition.go:446:			case hangulBase <= l && l < hangulEnd &&
composition.go:448:				((l-hangulBase)%jamoTCount) == 0:
composition.go:462:// handle alternations between Hangul and non-Hangul characters correctly.
composition.go:463:func (rb *reorderBuffer) compose() {
composition.go:480:		if isJamoVT(rb.bytesAt(i)) {
composition.go:481:			// Redo from start in Hangul mode. Necessary to support
composition.go:483:			rb.combineHangul(s, i, k)
Bash sed -n '300,520p' composition.go
command
sed -n '300,520p' composition.go
description
Read compose and Hangul code
Result
// bytesAt returns the UTF-8 encoding of the rune at position n.
// It is used for Hangul and recomposition.
func (rb *reorderBuffer) bytesAt(n int) []byte {
	inf := rb.rune[n]
	return rb.byte[inf.pos : int(inf.pos)+int(inf.size)]
}

// For Hangul we combine algorithmically, instead of using tables.
const (
	hangulBase  = 0xAC00 // UTF-8(hangulBase) -> EA B0 80
	hangulBase0 = 0xEA
	hangulBase1 = 0xB0
	hangulBase2 = 0x80

	hangulEnd  = hangulBase + jamoLVTCount // UTF-8(0xD7A4) -> ED 9E A4
	hangulEnd0 = 0xED
	hangulEnd1 = 0x9E
	hangulEnd2 = 0xA4

	jamoLBase  = 0x1100 // UTF-8(jamoLBase) -> E1 84 00
	jamoLBase0 = 0xE1
	jamoLBase1 = 0x84
	jamoLEnd   = 0x1113
	jamoVBase  = 0x1161
	jamoVEnd   = 0x1176
	jamoTBase  = 0x11A7
	jamoTEnd   = 0x11C3

	jamoTCount   = 28
	jamoVCount   = 21
	jamoVTCount  = 21 * 28
	jamoLVTCount = 19 * 21 * 28
)

const hangulUTF8Size = 3

func isHangul(b []byte) bool {
	if len(b) < hangulUTF8Size {
		return false
	}
	b0 := b[0]
	if b0 < hangulBase0 {
		return false
	}
	b1 := b[1]
	switch {
	case b0 == hangulBase0:
		return b1 >= hangulBase1
	case b0 < hangulEnd0:
		return true
	case b0 > hangulEnd0:
		return false
	case b1 < hangulEnd1:
		return true
	}
	return b1 == hangulEnd1 && b[2] < hangulEnd2
}

func isHangulString(b string) bool {
	if len(b) < hangulUTF8Size {
		return false
	}
	b0 := b[0]
	if b0 < hangulBase0 {
		return false
	}
	b1 := b[1]
	switch {
	case b0 == hangulBase0:
		return b1 >= hangulBase1
	case b0 < hangulEnd0:
		return true
	case b0 > hangulEnd0:
		return false
	case b1 < hangulEnd1:
		return true
	}
	return b1 == hangulEnd1 && b[2] < hangulEnd2
}

// Caller must ensure len(b) >= 2.
func isJamoVT(b []byte) bool {
	// True if (rune & 0xff00) == jamoLBase
	return b[0] == jamoLBase0 && (b[1]&0xFC) == jamoLBase1
}

func isHangulWithoutJamoT(b []byte) bool {
	c, _ := utf8.DecodeRune(b)
	c -= hangulBase
	return c < jamoLVTCount && c%jamoTCount == 0
}

// decomposeHangul writes the decomposed Hangul to buf and returns the number
// of bytes written.  len(buf) should be at least 9.
func decomposeHangul(buf []byte, r rune) int {
	const JamoUTF8Len = 3
	r -= hangulBase
	x := r % jamoTCount
	r /= jamoTCount
	utf8.EncodeRune(buf, jamoLBase+r/jamoVCount)
	utf8.EncodeRune(buf[JamoUTF8Len:], jamoVBase+r%jamoVCount)
	if x != 0 {
		utf8.EncodeRune(buf[2*JamoUTF8Len:], jamoTBase+x)
		return 3 * JamoUTF8Len
	}
	return 2 * JamoUTF8Len
}

// decomposeHangul algorithmically decomposes a Hangul rune into
// its Jamo components.
// See https://unicode.org/reports/tr15/#Hangul for details on decomposing Hangul.
func (rb *reorderBuffer) decomposeHangul(r rune) {
	r -= hangulBase
	x := r % jamoTCount
	r /= jamoTCount
	rb.appendRune(jamoLBase + r/jamoVCount)
	rb.appendRune(jamoVBase + r%jamoVCount)
	if x != 0 {
		rb.appendRune(jamoTBase + x)
	}
}

// combineHangul algorithmically combines Jamo character components into Hangul.
// See https://unicode.org/reports/tr15/#Hangul for details on combining Hangul.
func (rb *reorderBuffer) combineHangul(s, i, k int) {
	b := rb.rune[:]
	bn := rb.nrune
	for ; i < bn; i++ {
		cccB := b[k-1].ccc
		cccC := b[i].ccc
		if cccB == 0 {
			s = k - 1
		}
		if s != k-1 && cccB >= cccC {
			// b[i] is blocked by greater-equal cccX below it
			b[k] = b[i]
			k++
		} else {
			l := rb.runeAt(s) // also used to compare to hangulBase
			v := rb.runeAt(i) // also used to compare to jamoT
			switch {
			case jamoLBase <= l && l < jamoLEnd &&
				jamoVBase <= v && v < jamoVEnd:
				// 11xx plus 116x to LV
				rb.assignRune(s, hangulBase+
					(l-jamoLBase)*jamoVTCount+(v-jamoVBase)*jamoTCount)
			case hangulBase <= l && l < hangulEnd &&
				jamoTBase < v && v < jamoTEnd &&
				((l-hangulBase)%jamoTCount) == 0:
				// ACxx plus 11Ax to LVT
				rb.assignRune(s, l+v-jamoTBase)
			default:
				b[k] = b[i]
				k++
			}
		}
	}
	rb.nrune = k
}

// compose recombines the runes in the buffer.
// It should only be used to recompose a single segment, as it will not
// handle alternations between Hangul and non-Hangul characters correctly.
func (rb *reorderBuffer) compose() {
	// Lazily load the map used by the combine func below, but do
	// it outside of the loop.
	recompMapOnce.Do(buildRecompMap)

	// UAX #15, section X5 , including Corrigendum #5
	// "In any character sequence beginning with starter S, a character C is
	//  blocked from S if and only if there is some character B between S
	//  and C, and either B is a starter or it has the same or higher
	//  combining class as C."
	bn := rb.nrune
	if bn == 0 {
		return
	}
	k := 1
	b := rb.rune[:]
	for s, i := 0, 1; i < bn; i++ {
		if isJamoVT(rb.bytesAt(i)) {
			// Redo from start in Hangul mode. Necessary to support
			// U+320E..U+321E in NFKC mode.
			rb.combineHangul(s, i, k)
			return
		}
		ii := b[i]
		// We can only use combineForward as a filter if we later
		// get the info for the combined character. This is more
		// expensive than using the filter. Using combinesBackward()
		// is safe.
		if ii.combinesBackward() {
			cccB := b[k-1].ccc
			cccC := ii.ccc
			blocked := false // b[i] blocked by starter or greater or equal CCC?
			if cccB == 0 {
				s = k - 1
			} else {
				blocked = s != k-1 && cccB >= cccC
			}
			if !blocked {
				combined := combine(rb.runeAt(s), rb.runeAt(i))
				if combined != 0 {
					rb.assignRune(s, combined)
					continue
				}
			}
		}
		b[k] = b[i]
		k++
	}
	rb.nrune = k
}
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/dbg_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
)

func TestDbg(t *testing.T) {
	recompMapOnce.Do(buildRecompMap)
	for _, r := range []rune{0x113C2, 0x113C5, 0x1161, 0xAC00} {
		p := NFC.properties([]byte(string(r)))
		fmt.Printf("%U ccc=%d combinesBackward=%v combinesForward=%v isInert=%v\n", r, p.ccc, p.combinesBackward(), p.combinesForward(), p.isInert())
	}
	fmt.Println("recompMap size:", len(recompMap))
	// count how many entries whose decomposition breaks after a Hangul syllable
	bad := 0
	var examples []rune
	for key, val := range recompMap {
		a := rune(key >> 16)
		b := rune(key & 0xffff)
		_ = a
		_ = b
		src := "가" + NFD.String(string(val))
		want := "가" + string(val)
		if got := NFC.String(src); got != want {
			bad++
			if len(examples) < 20 {
				examples = append(examples, val)
			}
		}
	}
	fmt.Printf("Version %s: %d/%d recomposition targets fail after Hangul syllable\n", Version, bad, len(recompMap))
	for _, e := range examples {
		fmt.Printf("  %U (NFD=%U)\n", e, []rune(NFD.String(string(e))))
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestDbg -v 2>&1 | head -40
description
Instrument rune properties and scan recompMap
Result
# normtest/norm [normtest/norm.test]
norm/dbg_test.go:11:12: NFC.properties undefined (type Form has no field or method properties, but does have method Properties)
FAIL	normtest/norm [build failed]
FAIL
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && sed -i 's/NFC.properties(/NFC.Properties(/' norm/dbg_test.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestDbg -v 2>&1 | head -40
description
Rerun debug test
Result
=== RUN   TestDbg
U+113C2 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+113C5 ccc=0 combinesBackward=true combinesForward=false isInert=false
U+1161 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+AC00 ccc=0 combinesBackward=false combinesForward=true isInert=false
recompMap size: 961
Version 17.0.0: 12/961 recomposition targets fail after Hangul syllable
  U+16126 (NFD=[U+1611E U+1611E U+1611F])
  U+16121 (NFD=[U+1611E U+1611E])
  U+16127 (NFD=[U+1611E U+16129 U+1611F])
  U+113C5 (NFD=[U+113C2 U+113C2])
  U+16125 (NFD=[U+1611E U+16120])
  U+16128 (NFD=[U+1611E U+1611E U+16120])
  U+113C8 (NFD=[U+113C2 U+113C9])
  U+16123 (NFD=[U+1611E U+1611F])
  U+113C7 (NFD=[U+113C2 U+113B8])
  U+16122 (NFD=[U+1611E U+16129])
  U+16124 (NFD=[U+16129 U+1611F])
  U+16D68 (NFD=[U+16D67 U+16D67])
--- PASS: TestDbg (0.00s)
PASS
ok  	normtest/norm	0.004s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && sed -i 's/^go 1.27$/go 1.21/' go.mod && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestDbg -v 2>&1 | head -30; sed -i 's/^go 1.21$/go 1.27/' go.mod
description
Test with Unicode 15 tables
Result
=== RUN   TestDbg
U+113C2 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+113C5 ccc=0 combinesBackward=true combinesForward=false isInert=false
U+1161 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+AC00 ccc=0 combinesBackward=false combinesForward=true isInert=false
recompMap size: 961
Version 17.0.0: 12/961 recomposition targets fail after Hangul syllable
  U+16D68 (NFD=[U+16D67 U+16D67])
  U+16123 (NFD=[U+1611E U+1611F])
  U+16127 (NFD=[U+1611E U+16129 U+1611F])
  U+16124 (NFD=[U+16129 U+1611F])
  U+113C5 (NFD=[U+113C2 U+113C2])
  U+16128 (NFD=[U+1611E U+1611E U+16120])
  U+16125 (NFD=[U+1611E U+16120])
  U+113C7 (NFD=[U+113C2 U+113B8])
  U+113C8 (NFD=[U+113C2 U+113C9])
  U+16121 (NFD=[U+1611E U+1611E])
  U+16126 (NFD=[U+1611E U+1611E U+1611F])
  U+16122 (NFD=[U+1611E U+16129])
--- PASS: TestDbg (0.00s)
PASS
ok  	normtest/norm	0.004s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cp -r norm norm15 && rm norm15/tables17.0.0.go && sed -i 's|^//go:build !go1.27$|//go:build ignoreme_never|;s|^//go:build !go1\.27|//go:build |' norm15/tables15.0.0.go && head -5 norm15/tables15.0.0.go && sed -i 's/^package norm$/package norm15/' norm15/*.go && sed -i 's|normtest/norm"|normtest/norm15"|' norm15/*.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm15/ -run TestDbg -v 2>&1 | tail -25
description
Build a Unicode 15 variant and test
Result
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.

//go:build ignoreme_never

package norm
# normtest/norm15
found packages norm15 (composition.go) and norm (normalize.go) in /tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/normtest/norm15
FAIL	normtest/norm15 [setup failed]
FAIL
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest/norm15 && sed -i '1,10s|^//go:build ignoreme_never$||' tables15.0.0.go && sed -i 's/^package norm$/package norm15/' *.go && head -4 tables15.0.0.go && cd .. && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm15/ -run TestDbg -v 2>&1 | tail -25
description
Fix build tag and rerun U15 test
Result
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.



# normtest/norm15
found packages norm15 (composition.go) and norm (normalize.go) in /tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/normtest/norm15
FAIL	normtest/norm15 [setup failed]
FAIL
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest/norm15 && grep -n "^package" *.go
description
Check package clauses
Result
dbg_test.go:1:package norm15
forminfo.go:5:package norm15
input.go:5:package norm15
transform.go:5:package norm15
composition.go:5:package norm15
readwriter.go:5:package norm15
iter.go:5:package norm15
trie.go:5:package norm15
normalize.go:10:package norm // import "golang.org/x/text/unicode/norm"
tables15.0.0.go:5:package norm15
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && sed -i 's|^package norm // import.*|package norm15|' norm15/normalize.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm15/ -run TestDbg -v 2>&1 | tail -25
description
Run U15 variant test
Result
=== RUN   TestDbg
U+113C2 ccc=0 combinesBackward=false combinesForward=false isInert=true
U+113C5 ccc=0 combinesBackward=false combinesForward=false isInert=true
U+1161 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+AC00 ccc=0 combinesBackward=false combinesForward=true isInert=false
recompMap size: 941
Version 15.0.0: 0/941 recomposition targets fail after Hangul syllable
--- PASS: TestDbg (0.00s)
PASS
ok  	normtest/norm15	0.002s
Shell cwd was reset to /home/claude/golang_go_81021
Bash cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && sed -n '277,300p' composition.go
command
cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && sed -n '277,300p' composition.go
description
Read assignRune/runeAt
Result
// appendRune inserts a rune at the end of the buffer. It is used for Hangul.
func (rb *reorderBuffer) appendRune(r rune) {
	bn := rb.nbyte
	sz := utf8.EncodeRune(rb.byte[bn:], rune(r))
	rb.nbyte += utf8.UTFMax
	rb.rune[rb.nrune] = Properties{pos: bn, size: uint8(sz)}
	rb.nrune++
}

// assignRune sets a rune at position pos. It is used for Hangul and recomposition.
func (rb *reorderBuffer) assignRune(pos int, r rune) {
	bn := rb.rune[pos].pos
	sz := utf8.EncodeRune(rb.byte[bn:], rune(r))
	rb.rune[pos] = Properties{pos: bn, size: uint8(sz)}
}

// runeAt returns the rune at position n. It is used for Hangul and recomposition.
func (rb *reorderBuffer) runeAt(n int) rune {
	inf := rb.rune[n]
	r, _ := utf8.DecodeRune(rb.byte[inf.pos : inf.pos+inf.size])
	return r
}

// bytesAt returns the UTF-8 encoding of the rune at position n.
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest/norm && python3 - <<'PY'
import re
p='composition.go'
s=open(p).read()
old="""			default:
				b[k] = b[i]
				k++
			}"""
new="""			default:
				// Not a Hangul composition. The segment may still
				// contain regular canonical compositions, such as
				// combining marks following a Hangul syllable, so
				// fall back to the composition table.
				if b[i].combinesBackward() {
					if combined := combine(l, v); combined != 0 {
						rb.assignRune(s, combined)
						continue
					}
				}
				b[k] = b[i]
				k++
			}"""
assert s.count(old)==1
open(p,'w').write(s.replace(old,new))
PY
cd $S/normtest && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestDbg -v 2>&1 | tail -20 && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run . 2>&1
description
Apply candidate fix and test
Result
=== RUN   TestDbg
U+113C2 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+113C5 ccc=0 combinesBackward=true combinesForward=false isInert=false
U+1161 ccc=0 combinesBackward=true combinesForward=true isInert=false
U+AC00 ccc=0 combinesBackward=false combinesForward=true isInert=false
recompMap size: 961
Version 17.0.0: 0/961 recomposition targets fail after Hangul syllable
--- PASS: TestDbg (0.00s)
PASS
ok  	normtest/norm	0.002s
Version: 17.0.0
bare: in=[U+113C2 U+113C2] -> NFC=[U+113C5]
after A: in=[U+0041 U+113C2 U+113C2] -> NFC=[U+0041 U+113C5]
after hangul syl: in=[U+AC00 U+113C2 U+113C2] -> NFC=[U+AC00 U+113C5]
after hangul LV jamo: in=[U+1100 U+1161 U+113C2 U+113C2] -> NFC=[U+AC00 U+113C5]
precomposed after hangul: in=[U+AC00 U+113C5] -> NFC=[U+AC00 U+113C5]
precomposed bare: in=[U+113C5] -> NFC=[U+113C5]
roundtrip [U+AC00 U+113C5]: NFD=[U+1100 U+1161 U+113C2 U+113C2] NFC=[U+AC00 U+113C5] ok=true
roundtrip [U+113C5]: NFD=[U+113C2 U+113C2] NFC=[U+113C5] ok=true
Shell cwd was reset to /home/claude/golang_go_81021
Bash ls /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/ && grep -c "combinesBackward" /home/claude/pkg/mod/golan...
command
ls /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/ && grep -c "combinesBackward" /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition.go; diff /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition.go /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm/composition.go && echo IDENTICAL
description
Check upstream x/text norm package
Result
composition.go
composition_test.go
data15.0.0_test.go
data17.0.0_test.go
example_iter_test.go
example_test.go
forminfo.go
forminfo_test.go
input.go
iter.go
iter_test.go
maketables.go
normalize.go
normalize_test.go
readwriter.go
readwriter_test.go
tables15.0.0.go
tables17.0.0.go
transform.go
transform_test.go
triegen.go
trie.go
ucd_test.go
2
IDENTICAL
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
rm -rf $S/xtext && cp -r /home/claude/pkg/mod/golang.org/x/text@v0.41.0 $S/xtext && chmod -R u+w $S/xtext && cd $S/xtext && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ 2>&1 | tail -20
description
Run upstream x/text norm tests baseline
Result
ok  	golang.org/x/text/unicode/norm	2.918s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/xtext && cp $S/normtest/norm/composition.go unicode/norm/composition.go && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ 2>&1 | tail -20 && echo "=== verbose list ===" && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ -v 2>&1 | grep -E "^(---|===|ok|FAIL|PASS|\s+---)" | grep -v "^=== RUN\|^=== PAUSE\|^=== CONT" | head -60
description
Apply fix to x/text copy and run full norm tests
Result
ok  	golang.org/x/text/unicode/norm	1.534s
=== verbose list ===
--- PASS: TestFlush (0.00s)
--- PASS: TestInsert (0.00s)
--- PASS: TestDecomposition (0.00s)
--- PASS: TestComposition (0.00s)
--- PASS: TestIterNext (0.03s)
    --- PASS: TestIterNext/NFC/0 (0.00s)
        --- PASS: TestIterNext/NFC/0/fn (0.00s)
        --- PASS: TestIterNext/NFC/0/NFD (0.00s)
        --- PASS: TestIterNext/NFC/0/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/0/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/1 (0.00s)
        --- PASS: TestIterNext/NFC/1/fn (0.00s)
        --- PASS: TestIterNext/NFC/1/NFD (0.00s)
        --- PASS: TestIterNext/NFC/1/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/1/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/2 (0.00s)
        --- PASS: TestIterNext/NFC/2/fn (0.00s)
        --- PASS: TestIterNext/NFC/2/NFD (0.00s)
        --- PASS: TestIterNext/NFC/2/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/2/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/3 (0.00s)
        --- PASS: TestIterNext/NFC/3/fn (0.00s)
        --- PASS: TestIterNext/NFC/3/NFD (0.00s)
        --- PASS: TestIterNext/NFC/3/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/3/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/4 (0.00s)
        --- PASS: TestIterNext/NFC/4/fn (0.00s)
        --- PASS: TestIterNext/NFC/4/NFD (0.00s)
        --- PASS: TestIterNext/NFC/4/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/4/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/5 (0.00s)
        --- PASS: TestIterNext/NFC/5/fn (0.00s)
        --- PASS: TestIterNext/NFC/5/NFD (0.00s)
        --- PASS: TestIterNext/NFC/5/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/5/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/6 (0.00s)
        --- PASS: TestIterNext/NFC/6/fn (0.00s)
        --- PASS: TestIterNext/NFC/6/NFD (0.00s)
        --- PASS: TestIterNext/NFC/6/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/6/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/7 (0.00s)
        --- PASS: TestIterNext/NFC/7/fn (0.00s)
        --- PASS: TestIterNext/NFC/7/NFD (0.00s)
        --- PASS: TestIterNext/NFC/7/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/7/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/8 (0.00s)
        --- PASS: TestIterNext/NFC/8/fn (0.00s)
        --- PASS: TestIterNext/NFC/8/NFD (0.00s)
        --- PASS: TestIterNext/NFC/8/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/8/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/9 (0.00s)
        --- PASS: TestIterNext/NFC/9/fn (0.00s)
        --- PASS: TestIterNext/NFC/9/NFD (0.00s)
        --- PASS: TestIterNext/NFC/9/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/9/NFKD (0.00s)
    --- PASS: TestIterNext/NFC/10 (0.00s)
        --- PASS: TestIterNext/NFC/10/fn (0.00s)
        --- PASS: TestIterNext/NFC/10/NFD (0.00s)
        --- PASS: TestIterNext/NFC/10/NFKC (0.00s)
        --- PASS: TestIterNext/NFC/10/NFKD (0.00s)
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/xtext/unicode/norm && grep -n "func Test" *_test.go | head -40; echo "=== ucd ==="; sed -n '1,40p' ucd_test.go; echo "=== conformance ==="; grep -rn "NormalizationTest\|Conformance\|conformance" *_test.go ../../internal/testtext/*.go 2>/dev/null | head -20
description
Survey norm tests and conformance harness
Result
composition_test.go:46:func TestFlush(t *testing.T) {
composition_test.go:92:func TestInsert(t *testing.T) {
composition_test.go:111:func TestDecomposition(t *testing.T) {
composition_test.go:128:func TestComposition(t *testing.T) {
forminfo_test.go:11:func TestProperties(t *testing.T) {
iter_test.go:32:func TestIterNext(t *testing.T) {
iter_test.go:83:func TestIterSegmentation(t *testing.T) {
readwriter_test.go:29:func TestReader(t *testing.T) {
readwriter_test.go:51:func TestWriter(t *testing.T) {
ucd_test.go:220:func TestCharacterByCharacter(t *testing.T) {
ucd_test.go:246:func TestStandardTests(t *testing.T) {
ucd_test.go:258:func TestPerformance(t *testing.T) {
transform_test.go:14:func TestTransform(t *testing.T) {
transform_test.go:103:func TestTransformNorm(t *testing.T) {
normalize_test.go:155:func TestDecomposeSegment(t *testing.T) {
normalize_test.go:190:func TestFirstBoundary(t *testing.T) {
normalize_test.go:195:func TestNextBoundary(t *testing.T) {
normalize_test.go:312:func TestDecomposeToLastBoundary(t *testing.T) {
normalize_test.go:373:func TestLastBoundary(t *testing.T) {
normalize_test.go:488:func TestSpan(t *testing.T) {
normalize_test.go:566:func TestIsNormal(t *testing.T) {
normalize_test.go:579:func TestIsNormalString(t *testing.T) {
normalize_test.go:903:func TestAppend(t *testing.T) {
normalize_test.go:909:func TestAppendString(t *testing.T) {
normalize_test.go:915:func TestBytes(t *testing.T) {
normalize_test.go:924:func TestString(t *testing.T) {
normalize_test.go:956:func TestLinking(t *testing.T) {
=== ucd ===
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package norm

import (
	"bufio"
	"bytes"
	"fmt"
	"regexp"
	"runtime"
	"strconv"
	"strings"
	"sync"
	"testing"
	"time"
	"unicode/utf8"

	"golang.org/x/text/internal/gen"
	"golang.org/x/text/internal/testtext"
)

var once sync.Once

func skipShort(t *testing.T) {
	testtext.SkipIfNotLong(t)

	once.Do(func() { loadTestData(t) })
}

// This regression test runs the test set in NormalizationTest.txt
// (taken from https://www.unicode.org/Public/<unicode.Version>/ucd/).
//
// NormalizationTest.txt has form:
// @Part0 # Specific cases
// #
// 1E0A;1E0A;0044 0307;1E0A;0044 0307; # (Ḋ; Ḋ; D◌̇; Ḋ; D◌̇; ) LATIN CAPITAL LETTER D WITH DOT ABOVE
// 1E0C;1E0C;0044 0323;1E0C;0044 0323; # (Ḍ; Ḍ; D◌̣; Ḍ; D◌̣; ) LATIN CAPITAL LETTER D WITH DOT BELOW
//
=== conformance ===
ucd_test.go:32:// This regression test runs the test set in NormalizationTest.txt
ucd_test.go:35:// NormalizationTest.txt has form:
ucd_test.go:78:// Holds data from NormalizationTest.txt
ucd_test.go:107:// Load the data form NormalizationTest.txt
ucd_test.go:109:	f := gen.OpenUCDFile("NormalizationTest.txt")
ucd_test.go:205:func doConformanceTests(t *testing.T, tc *Test, partn int) {
ucd_test.go:241:			doConformanceTests(t, &tests[i], 1)
ucd_test.go:250:			doConformanceTests(t, &test, j)
normalize_test.go:1224:// https://unicode.org/Public/UNIDATA/NormalizationTest.txt
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && rm -rf normold && mkdir normold && cp /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm/*.go normold/ && sed -i 's|"golang.org/x/text/transform"|"normtest/transform"|' normold/*.go && sed -i 's|^package norm // import.*|package normold|;s/^package norm$/package normold/' normold/*.go && cat > cmp.go <<'EOF'
//go:build ignore
EOF
rm cmp.go
mkdir -p cmp && cat > cmp/main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/norm"
	"normtest/normold"
)

var prefixes = []string{
	"", "A", "Á",
	"가",             // LV syllable
	"각",             // LVT syllable
	"가",       // L V
	"각", // L V T
	"ᄀ", "ᅡ", "ᆨ",
	"㈎", // parenthesized hangul (NFKC)
	"힣",
}

type form struct {
	name string
	n    norm.Form
	o    normold.Form
}

func main() {
	forms := []form{
		{"NFC", norm.NFC, normold.NFC},
		{"NFKC", norm.NFKC, normold.NFKC},
		{"NFD", norm.NFD, normold.NFD},
		{"NFKD", norm.NFKD, normold.NFKD},
	}
	diffs, badNew, badOld := 0, 0, 0
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		c := string(r)
		for _, p := range prefixes {
			for _, f := range forms {
				// input in decomposed form
				in := p + norm.NFD.String(c)
				gn, go_ := f.n.String(in), f.o.String(in)
				if gn != go_ {
					diffs++
					if diffs <= 40 {
						fmt.Printf("DIFF %s prefix=%U rune=%U: old=%U new=%U\n", f.name, []rune(p), r, []rune(go_), []rune(gn))
					}
				}
				// invariant: normalizing the decomposed form must equal
				// normalizing the original form
				ref := p + c
				if f.n.String(in) != f.n.String(ref) {
					badNew++
					if badNew <= 20 {
						fmt.Printf("NEW-INCONSISTENT %s prefix=%U rune=%U: fromNFD=%U fromRaw=%U\n", f.name, []rune(p), r, []rune(f.n.String(in)), []rune(f.n.String(ref)))
					}
				}
				if f.o.String(in) != f.o.String(ref) {
					badOld++
				}
				// idempotence
				if x := f.n.String(f.n.String(in)); x != gn {
					fmt.Printf("NOT-IDEMPOTENT %s prefix=%U rune=%U\n", f.name, []rune(p), r)
				}
			}
		}
	}
	fmt.Printf("diffs=%d newInconsistent=%d oldInconsistent=%d\n", diffs, badNew, badOld)
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp 2>&1 | tail -60
description
Exhaustive old-vs-new differential test
Result
DIFF NFC prefix=[U+AC00] rune=U+113C5: old=[U+AC00 U+113C2 U+113C2] new=[U+AC00 U+113C5]
DIFF NFKC prefix=[U+AC00] rune=U+113C5: old=[U+AC00 U+113C2 U+113C2] new=[U+AC00 U+113C5]
DIFF NFC prefix=[U+AC01] rune=U+113C5: old=[U+AC01 U+113C2 U+113C2] new=[U+AC01 U+113C5]
DIFF NFKC prefix=[U+AC01] rune=U+113C5: old=[U+AC01 U+113C2 U+113C2] new=[U+AC01 U+113C5]
DIFF NFC prefix=[U+1100 U+1161] rune=U+113C5: old=[U+AC00 U+113C2 U+113C2] new=[U+AC00 U+113C5]
DIFF NFKC prefix=[U+1100 U+1161] rune=U+113C5: old=[U+AC00 U+113C2 U+113C2] new=[U+AC00 U+113C5]
DIFF NFC prefix=[U+1100 U+1161 U+11A8] rune=U+113C5: old=[U+AC01 U+113C2 U+113C2] new=[U+AC01 U+113C5]
DIFF NFKC prefix=[U+1100 U+1161 U+11A8] rune=U+113C5: old=[U+AC01 U+113C2 U+113C2] new=[U+AC01 U+113C5]
DIFF NFC prefix=[U+D7A3] rune=U+113C5: old=[U+D7A3 U+113C2 U+113C2] new=[U+D7A3 U+113C5]
DIFF NFKC prefix=[U+D7A3] rune=U+113C5: old=[U+D7A3 U+113C2 U+113C2] new=[U+D7A3 U+113C5]
DIFF NFC prefix=[U+AC00] rune=U+113C7: old=[U+AC00 U+113C2 U+113B8] new=[U+AC00 U+113C7]
DIFF NFKC prefix=[U+AC00] rune=U+113C7: old=[U+AC00 U+113C2 U+113B8] new=[U+AC00 U+113C7]
DIFF NFC prefix=[U+AC01] rune=U+113C7: old=[U+AC01 U+113C2 U+113B8] new=[U+AC01 U+113C7]
DIFF NFKC prefix=[U+AC01] rune=U+113C7: old=[U+AC01 U+113C2 U+113B8] new=[U+AC01 U+113C7]
DIFF NFC prefix=[U+1100 U+1161] rune=U+113C7: old=[U+AC00 U+113C2 U+113B8] new=[U+AC00 U+113C7]
DIFF NFKC prefix=[U+1100 U+1161] rune=U+113C7: old=[U+AC00 U+113C2 U+113B8] new=[U+AC00 U+113C7]
DIFF NFC prefix=[U+1100 U+1161 U+11A8] rune=U+113C7: old=[U+AC01 U+113C2 U+113B8] new=[U+AC01 U+113C7]
DIFF NFKC prefix=[U+1100 U+1161 U+11A8] rune=U+113C7: old=[U+AC01 U+113C2 U+113B8] new=[U+AC01 U+113C7]
DIFF NFC prefix=[U+D7A3] rune=U+113C7: old=[U+D7A3 U+113C2 U+113B8] new=[U+D7A3 U+113C7]
DIFF NFKC prefix=[U+D7A3] rune=U+113C7: old=[U+D7A3 U+113C2 U+113B8] new=[U+D7A3 U+113C7]
DIFF NFC prefix=[U+AC00] rune=U+113C8: old=[U+AC00 U+113C2 U+113C9] new=[U+AC00 U+113C8]
DIFF NFKC prefix=[U+AC00] rune=U+113C8: old=[U+AC00 U+113C2 U+113C9] new=[U+AC00 U+113C8]
DIFF NFC prefix=[U+AC01] rune=U+113C8: old=[U+AC01 U+113C2 U+113C9] new=[U+AC01 U+113C8]
DIFF NFKC prefix=[U+AC01] rune=U+113C8: old=[U+AC01 U+113C2 U+113C9] new=[U+AC01 U+113C8]
DIFF NFC prefix=[U+1100 U+1161] rune=U+113C8: old=[U+AC00 U+113C2 U+113C9] new=[U+AC00 U+113C8]
DIFF NFKC prefix=[U+1100 U+1161] rune=U+113C8: old=[U+AC00 U+113C2 U+113C9] new=[U+AC00 U+113C8]
DIFF NFC prefix=[U+1100 U+1161 U+11A8] rune=U+113C8: old=[U+AC01 U+113C2 U+113C9] new=[U+AC01 U+113C8]
DIFF NFKC prefix=[U+1100 U+1161 U+11A8] rune=U+113C8: old=[U+AC01 U+113C2 U+113C9] new=[U+AC01 U+113C8]
DIFF NFC prefix=[U+D7A3] rune=U+113C8: old=[U+D7A3 U+113C2 U+113C9] new=[U+D7A3 U+113C8]
DIFF NFKC prefix=[U+D7A3] rune=U+113C8: old=[U+D7A3 U+113C2 U+113C9] new=[U+D7A3 U+113C8]
DIFF NFC prefix=[U+AC00] rune=U+16121: old=[U+AC00 U+1611E U+1611E] new=[U+AC00 U+16121]
DIFF NFKC prefix=[U+AC00] rune=U+16121: old=[U+AC00 U+1611E U+1611E] new=[U+AC00 U+16121]
DIFF NFC prefix=[U+AC01] rune=U+16121: old=[U+AC01 U+1611E U+1611E] new=[U+AC01 U+16121]
DIFF NFKC prefix=[U+AC01] rune=U+16121: old=[U+AC01 U+1611E U+1611E] new=[U+AC01 U+16121]
DIFF NFC prefix=[U+1100 U+1161] rune=U+16121: old=[U+AC00 U+1611E U+1611E] new=[U+AC00 U+16121]
DIFF NFKC prefix=[U+1100 U+1161] rune=U+16121: old=[U+AC00 U+1611E U+1611E] new=[U+AC00 U+16121]
DIFF NFC prefix=[U+1100 U+1161 U+11A8] rune=U+16121: old=[U+AC01 U+1611E U+1611E] new=[U+AC01 U+16121]
DIFF NFKC prefix=[U+1100 U+1161 U+11A8] rune=U+16121: old=[U+AC01 U+1611E U+1611E] new=[U+AC01 U+16121]
DIFF NFC prefix=[U+D7A3] rune=U+16121: old=[U+D7A3 U+1611E U+1611E] new=[U+D7A3 U+16121]
DIFF NFKC prefix=[U+D7A3] rune=U+16121: old=[U+D7A3 U+1611E U+1611E] new=[U+D7A3 U+16121]
diffs=120 newInconsistent=0 oldInconsistent=0
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && mkdir -p cmp2 && cat > cmp2/main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/norm"
	"normtest/normold"
)

func main() {
	var set []rune
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		p := norm.NFC.Properties([]byte(string(r)))
		if p.CCC() != 0 || p.String() != "" || !p.BoundaryBefore() {
			set = append(set, r)
		}
	}
	fmt.Println("interesting runes:", len(set))
	prefixes := []string{"", "가", "각", "가", "A"}
	diffs := 0
	seen := map[string]bool{}
	for _, a := range set {
		for _, b := range set {
			for _, p := range prefixes {
				in := p + string(a) + string(b)
				for _, f := range []struct {
					n norm.Form
					o normold.Form
					s string
				}{{norm.NFC, normold.NFC, "NFC"}, {norm.NFKC, normold.NFKC, "NFKC"}} {
					gn, go_ := f.n.String(in), f.o.String(in)
					if gn != go_ {
						diffs++
						key := fmt.Sprintf("%s %U %U -> %U vs %U", f.s, a, b, []rune(go_), []rune(gn))
						if !seen[key] && len(seen) < 30 {
							seen[key] = true
							fmt.Printf("DIFF %s prefix=%U in=%U old=%U new=%U\n", f.s, []rune(p), []rune(in), []rune(go_), []rune(gn))
						}
					}
					// NFC output must itself be stable and canonically minimal:
					if x := f.n.String(gn); x != gn {
						fmt.Printf("NOT-IDEMPOTENT %s in=%U\n", f.s, []rune(in))
					}
				}
			}
		}
	}
	fmt.Println("total diffs:", diffs)
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp2 2>&1 | tail -45
description
Pairwise differential test over combining/composing runes
Result
# normtest/cmp2
cmp2/main.go:16:24: p.String undefined (type norm.Properties has no field or method String)
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && sed -i 's|p.CCC() != 0 \|\| p.String() != "" \|\| !p.BoundaryBefore()|p.CCC() != 0 \|\| len(p.Decomposition()) != 0 \|\| !p.BoundaryBefore()|' cmp2/main.go && grep -n "CCC()" cmp2/main.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp2 2>&1 | tail -45
description
Fix predicate and run pairwise differential test
Result
16:		if p.CCC() != 0 || len(p.Decomposition()) != 0 || !p.BoundaryBefore() {
interesting runes: 3126
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+113C5] old=[U+AC00 U+0300 U+113C2 U+113C2] new=[U+AC00 U+0300 U+113C5]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+113C5] old=[U+AC00 U+0300 U+113C2 U+113C2] new=[U+AC00 U+0300 U+113C5]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+113C5] old=[U+AC01 U+0300 U+113C2 U+113C2] new=[U+AC01 U+0300 U+113C5]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+113C5] old=[U+AC01 U+0300 U+113C2 U+113C2] new=[U+AC01 U+0300 U+113C5]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+113C7] old=[U+AC00 U+0300 U+113C2 U+113B8] new=[U+AC00 U+0300 U+113C7]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+113C7] old=[U+AC00 U+0300 U+113C2 U+113B8] new=[U+AC00 U+0300 U+113C7]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+113C7] old=[U+AC01 U+0300 U+113C2 U+113B8] new=[U+AC01 U+0300 U+113C7]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+113C7] old=[U+AC01 U+0300 U+113C2 U+113B8] new=[U+AC01 U+0300 U+113C7]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+113C8] old=[U+AC00 U+0300 U+113C2 U+113C9] new=[U+AC00 U+0300 U+113C8]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+113C8] old=[U+AC00 U+0300 U+113C2 U+113C9] new=[U+AC00 U+0300 U+113C8]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+113C8] old=[U+AC01 U+0300 U+113C2 U+113C9] new=[U+AC01 U+0300 U+113C8]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+113C8] old=[U+AC01 U+0300 U+113C2 U+113C9] new=[U+AC01 U+0300 U+113C8]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+16121] old=[U+AC00 U+0300 U+1611E U+1611E] new=[U+AC00 U+0300 U+16121]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+16121] old=[U+AC00 U+0300 U+1611E U+1611E] new=[U+AC00 U+0300 U+16121]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+16121] old=[U+AC01 U+0300 U+1611E U+1611E] new=[U+AC01 U+0300 U+16121]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+16121] old=[U+AC01 U+0300 U+1611E U+1611E] new=[U+AC01 U+0300 U+16121]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+16122] old=[U+AC00 U+0300 U+1611E U+16129] new=[U+AC00 U+0300 U+16122]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+16122] old=[U+AC00 U+0300 U+1611E U+16129] new=[U+AC00 U+0300 U+16122]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+16122] old=[U+AC01 U+0300 U+1611E U+16129] new=[U+AC01 U+0300 U+16122]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+16122] old=[U+AC01 U+0300 U+1611E U+16129] new=[U+AC01 U+0300 U+16122]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+16123] old=[U+AC00 U+0300 U+1611E U+1611F] new=[U+AC00 U+0300 U+16123]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+16123] old=[U+AC00 U+0300 U+1611E U+1611F] new=[U+AC00 U+0300 U+16123]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+16123] old=[U+AC01 U+0300 U+1611E U+1611F] new=[U+AC01 U+0300 U+16123]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+16123] old=[U+AC01 U+0300 U+1611E U+1611F] new=[U+AC01 U+0300 U+16123]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+16124] old=[U+AC00 U+0300 U+16129 U+1611F] new=[U+AC00 U+0300 U+16124]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+16124] old=[U+AC00 U+0300 U+16129 U+1611F] new=[U+AC00 U+0300 U+16124]
DIFF NFC prefix=[U+AC01] in=[U+AC01 U+0300 U+16124] old=[U+AC01 U+0300 U+16129 U+1611F] new=[U+AC01 U+0300 U+16124]
DIFF NFKC prefix=[U+AC01] in=[U+AC01 U+0300 U+16124] old=[U+AC01 U+0300 U+16129 U+1611F] new=[U+AC01 U+0300 U+16124]
DIFF NFC prefix=[U+AC00] in=[U+AC00 U+0300 U+16125] old=[U+AC00 U+0300 U+1611E U+16120] new=[U+AC00 U+0300 U+16125]
DIFF NFKC prefix=[U+AC00] in=[U+AC00 U+0300 U+16125] old=[U+AC00 U+0300 U+1611E U+16120] new=[U+AC00 U+0300 U+16125]
total diffs: 302022
Shell cwd was reset to /home/claude/golang_go_81021
Bash cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && grep -n "recompMap\|func combine" formin...
command
cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && grep -n "recompMap\|func combine" forminfo.go composition.go normalize.go | head -20; echo ---; grep -n "func combine" -A 12 forminfo.go
description
Look at combine and recompMap
Result
composition.go:466:	recompMapOnce.Do(buildRecompMap)
forminfo.go:195:	recompMap = make(map[uint32]rune, len(recompMapPacked)/8)
forminfo.go:197:	for i := 0; i < len(recompMapPacked); i += 8 {
forminfo.go:198:		copy(buf[:], recompMapPacked[i:i+8])
forminfo.go:201:		recompMap[key] = rune(val)
forminfo.go:215:// recompMapOnce.Do(buildRecompMap) sometime before this is called.
forminfo.go:216:func combine(a, b rune) rune {
forminfo.go:218:	if recompMap == nil {
forminfo.go:221:	return recompMap[key]
---
216:func combine(a, b rune) rune {
217-	key := uint32(uint16(a))<<16 + uint32(uint16(b))
218-	if recompMap == nil {
219-		panic("caller error") // see func comment
220-	}
221-	return recompMap[key]
222-}
223-
224-func lookupInfoNFC(b input, i int) Properties {
225-	v, sz := b.charinfoNFC(i)
226-	return compInfo(v, sz)
227-}
228-
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/dbg2_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
)

func TestRecompCollisions(t *testing.T) {
	recompMapOnce.Do(buildRecompMap)
	fmt.Printf("packed entries=%d map entries=%d\n", len(recompMapPacked)/8, len(recompMap))
	// Also: does any composition target exceed BMP in either operand?
	supp := 0
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		d := NFD.String(string(r))
		if d == string(r) {
			continue
		}
		rs := []rune(d)
		if len(rs) >= 2 && (rs[0] > 0xFFFF || rs[1] > 0xFFFF) {
			supp++
		}
	}
	fmt.Println("decompositions with supplementary operands:", supp)
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestRecompCollisions -v 2>&1 | head
description
Check recompMap key collisions
Result
=== RUN   TestRecompCollisions
packed entries=961 map entries=961
decompositions with supplementary operands: 46
--- PASS: TestRecompCollisions (0.04s)
PASS
ok  	normtest/norm	0.038s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/ref_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
	"unicode/utf8"
)

// refNFC is a straightforward, table-driven implementation of the
// canonical composition algorithm from UAX #15 (D117), used as an
// oracle for the optimized implementation in this package.
func refNFC(s string) string {
	recompMapOnce.Do(buildRecompMap)
	return refCompose(NFD.String(s))
}

func refNFKC(s string) string {
	recompMapOnce.Do(buildRecompMap)
	return refCompose(NFKD.String(s))
}

func refCCC(r rune) uint8 {
	return NFC.Properties([]byte(string(r))).CCC()
}

func refHangulCompose(l, c rune) rune {
	const (
		sbase = 0xAC00
		lbase = 0x1100
		vbase = 0x1161
		tbase = 0x11A7
		lcnt  = 19
		vcnt  = 21
		tcnt  = 28
		ncnt  = vcnt * tcnt
		scnt  = lcnt * ncnt
	)
	if lbase <= l && l < lbase+lcnt && vbase <= c && c < vbase+vcnt {
		return sbase + ((l-lbase)*vcnt+(c-vbase))*tcnt
	}
	if sbase <= l && l < sbase+scnt && (l-sbase)%tcnt == 0 && tbase < c && c < tbase+tcnt {
		return l + (c - tbase)
	}
	return 0
}

func refCompose(d string) string {
	r := []rune(d)
	if len(r) == 0 {
		return d
	}
	out := make([]rune, 0, len(r))
	// index in out of the last starter, or -1
	last := -1
	for i := 0; i < len(r); i++ {
		c := r[i]
		if last >= 0 {
			// blocked if there is a char between the starter and c
			// that is a starter or has ccc >= ccc(c)
			blocked := false
			cccC := refCCC(c)
			// chars between out[last] and c are out[last+1:]
			if len(out) > last+1 {
				prev := refCCC(out[len(out)-1])
				if prev == 0 || prev >= cccC {
					blocked = true
				}
			} else if cccC == 0 {
				// adjacent starter: allowed only if the pair composes
			}
			if !blocked {
				l := out[last]
				var comp rune
				if h := refHangulCompose(l, c); h != 0 {
					comp = h
				} else {
					comp = combine(l, c)
				}
				if comp != 0 {
					out[last] = comp
					continue
				}
			}
		}
		out = append(out, c)
		if refCCC(c) == 0 {
			last = len(out) - 1
		}
	}
	return string(out)
}

func TestAgainstReference(t *testing.T) {
	var interesting []rune
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		p := NFC.Properties([]byte(string(r)))
		if p.CCC() != 0 || len(p.Decomposition()) != 0 || !p.BoundaryBefore() {
			interesting = append(interesting, r)
		}
	}
	t.Logf("interesting runes: %d", len(interesting))
	prefixes := []string{"", "A", "가", "각", "가", "각", "㈀", "㈎"}
	bad := 0
	buf := make([]byte, 0, 64)
	check := func(in string) {
		if got, want := NFC.String(in), refNFC(in); got != want {
			bad++
			if bad <= 20 {
				t.Errorf("NFC(%U) = %U, reference %U", []rune(in), []rune(got), []rune(want))
			}
		}
		if got, want := NFKC.String(in), refNFKC(in); got != want {
			bad++
			if bad <= 20 {
				t.Errorf("NFKC(%U) = %U, reference %U", []rune(in), []rune(got), []rune(want))
			}
		}
	}
	for _, a := range interesting {
		for _, b := range interesting {
			for _, p := range prefixes {
				buf = buf[:0]
				buf = append(buf, p...)
				buf = utf8.AppendRune(buf, a)
				buf = utf8.AppendRune(buf, b)
				check(string(buf))
			}
		}
	}
	fmt.Println("mismatches:", bad)
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestAgainstReference -v 2>&1 | tail -30
description
Validate fixed impl against reference NFC/NFKC oracle
Result
Exit code 143
Command timed out after 2m 0s
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && rm norm/ref_test.go && cat > cmp2/main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/norm"
	"normtest/normold"
)

func main() {
	var set []rune
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		p := norm.NFC.Properties([]byte(string(r)))
		if p.CCC() != 0 || len(p.Decomposition()) != 0 || !p.BoundaryBefore() {
			set = append(set, r)
		}
	}
	fmt.Println("interesting runes:", len(set))
	prefixes := []string{"", "가", "각", "가", "각", "A", "㈎"}
	diffs, notEquiv, notShorter := 0, 0, 0
	changed := map[rune]bool{}
	for _, a := range set {
		for _, b := range set {
			for _, p := range prefixes {
				in := p + string(a) + string(b)
				for _, f := range []struct {
					n norm.Form
					o normold.Form
					s string
				}{{norm.NFC, normold.NFC, "NFC"}, {norm.NFKC, normold.NFKC, "NFKC"}} {
					gn, go_ := f.n.String(in), f.o.String(in)
					if gn == go_ {
						continue
					}
					diffs++
					// (a) canonical equivalence preserved
					if norm.NFD.String(gn) != norm.NFD.String(go_) {
						notEquiv++
						if notEquiv <= 10 {
							fmt.Printf("NOT-EQUIVALENT %s in=%U old=%U new=%U\n", f.s, []rune(in), []rune(go_), []rune(gn))
						}
					}
					// (b) new output is strictly more composed
					if len([]rune(gn)) >= len([]rune(go_)) {
						notShorter++
						if notShorter <= 10 {
							fmt.Printf("NOT-MORE-COMPOSED %s in=%U old=%U new=%U\n", f.s, []rune(in), []rune(go_), []rune(gn))
						}
					}
					for _, r := range gn {
						if r != a && r != b {
							changed[r] = true
						}
					}
				}
			}
		}
	}
	fmt.Printf("diffs=%d notEquiv=%d notShorter=%d\n", diffs, notEquiv, notShorter)
	fmt.Print("newly produced runes: ")
	for r := range changed {
		fmt.Printf("%U ", r)
	}
	fmt.Println()
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp2 2>&1 | tail -20
description
Classify all behavioral diffs
Result
interesting runes: 3126
diffs=403464 notEquiv=0 notShorter=0
newly produced runes: U+5ED2 U+23D1E U+7DC7 U+6D77 U+03CC U+9F8D U+98EF U+05D2 U+4FBB U+6094 U+34B9 U+5FF5 U+60D8 U+83DC U+8AFE U+03B9 U+2054B U+7FFA U+881F U+9DFA U+6599 U+6DF9 U+7A40 U+2300A U+8D1B U+7409 U+66C6 U+753B U+7D5B U+5922 U+6081 U+681F U+8353 U+518D U+5D6E U+250F3 U+8CAB U+911B U+91B4 U+51F5 U+5D7C U+63E4 U+72C0 U+704A U+4995 U+AC09 U+52B3 U+8AA0 U+0020 U+52F5 U+9756 U+25C80 U+980B U+633D U+AC1B U+038F U+84EE U+6BAE U+8336 U+7949 U+6469 U+264DA U+985E U+7948 U+5145 U+05D4 U+59EC U+70AD U+4AB2 U+516D U+7BC0 U+1D1BA U+6B21 U+0313 U+09A2 U+6FFE U+9234 U+7AEE U+732A U+5716 U+5C22 U+9723 U+9304 U+88E1 U+8667 U+278AE U+AC1A U+0B3C U+7DA0 U+9CFD U+5375 U+8606 U+68A8 U+63C4 U+05DB U+3D96 U+88D7 U+559D U+6148 U+03CE U+7669 U+26D6B U+20804 U+67F3 U+24FB8 U+771F U+83CA U+8AAA U+7469 U+9686 U+5349 U+5962 U+40E3 U+5951 U+52D2 U+0300 U+7E41 U+113C9 U+4FBF U+795D U+5A66 U+5BFF U+25133 U+8AF8 U+2002 U+916A U+9675 U+50DA U+5140 U+7956 U+05D3 U+5246 U+7F85 U+964B U+9DB4 U+8941 U+6700 U+6ADB U+23A8D U+3D33 U+AC19 U+02B9 U+0389 U+8CC2 U+54F6 U+2A20E U+7565 U+722B U+76F4 U+6368 U+26247 U+27CA8 U+8DBC U+9094 U+0301 U+5317 U+6DCB U+67FA U+6A13 U+962E U+53F1 U+7485 U+46C7 U+9E1E U+6E2F U+6200 U+7406 U+7235 U+8323 U+0916 U+091C U+7740 U+440B U+8363 U+8860 U+16125 U+0F90 U+6B79 U+61A4 U+8060 U+113B8 U+0B22 U+523B U+49E6 U+614C U+62D4 U+3FFC U+0388 U+99F1 U+792A U+5C3F U+502B U+7A81 U+8DF0 U+4A76 U+2003 U+73DE U+6E9C U+28D77 U+7210 U+578B U+3AE4 U+7D00 U+273CA U+0917 U+8449 U+649A U+6717 U+9E9F U+7CD6 U+6C4E U+80B2 U+34BB U+113C8 U+9199 U+5002 U+23AFA U+25F86 U+6AD3 U+585A U+77A7 U+22B0C U+05B7 U+5154 U+7145 U+79EB U+79CA U+5B28 U+6756 U+8CC8 U+3B19 U+2921A U+AC02 U+786B U+541D U+96A3 U+96B7 U+7039 U+AC00 U+88C2 U+669C U+0F40 U+70D9 U+8201 U+25AA7 U+8986 U+771E U+5EEC U+674E U+233D5 U+6D34 U+64C4 U+5717 U+6477 U+2569A U+76CA U+05E0 U+83AD U+2A600 U+16121 U+52FA U+5651 U+23F8E U+8650 U+29145 U+81E8 U+4FAE U+6168 U+20A2C U+7F95 U+4EE4 U+84FC U+7891 U+55AB U+5832 U+2597C U+9D67 U+5587 U+5E74 U+7422 U+5197 U+828B U+0F80 U+58F7 U+5195 U+96C3 U+9F16 U+53E5 U+6578 U+85FA U+5272 U+86A9 U+45D7 U+0F5B U+967C U+38C7 U+3F1B U+285ED U+AC11 U+9E97 U+56F9 U+4E26 U+2219F U+92D7 U+91D1 U+91CC U+7CE8 U+8103 U+58D8 U+934A U+9C57 U+43D9 U+745C U+753E U+24FA1 U+5584 U+6753 U+8B01 U+AC08 U+797F U+807E U+8F26 U+79AE U+83BD U+27F2F U+6A02 U+9732 U+78FB U+1D170 U+0915 U+721B U+7C60 U+6674 U+05B4 U+1D172 U+5BE7 U+5674 U+226D4 U+647E U+2A105 U+8AD2 U+4F8B U+16124 U+4ECC U+86E2 U+03AC U+05B8 U+05EA U+82B3 U+8669 U+05DC U+05E7 U+5B08 U+6721 U+23CBC U+908F U+6012 U+9928 U+5099 U+58F2 U+0029 U+9F9C U+62CF U+8D08 U+16127 U+038E U+5EC9 U+7D10 U+8AED U+7CE3 U+AC0F U+85CD U+5207 U+5C64 U+617A U+675E U+7280 U+3EAC U+7489 U+9748 U+4EC0 U+6649 U+83E7 U+0F74 U+6C88 U+65E3 U+719C U+24C36 U+4227 U+5D19 U+3B49 U+6D78 U+62C9 U+57CE U+964D U+52E4 U+57F4 U+6E6E U+2633E U+0F4C U+51AC U+580D U+4096 U+05E9 U+9678 U+5168 U+614E U+219C8 U+8FB0 U+5305 U+55B3 U+43D5 U+540F U+214E4 U+5E28 U+927C U+16128 U+637B U+96F6 U+96E2 U+8B8A U+382F U+6350 U+98FC U+05D9 U+5406 U+4561 U+88FA U+78CA U+798D U+59D8 U+AC15 U+3009 U+58AC U+7524 U+7E02 U+51CC U+5973 U+7C92 U+5EB6 U+621B U+80AD U+865C U+71CE U+9038 U+6383 U+412F U+95B7 U+0A1C U+0FB5 U+05D8 U+54A2 U+24263 U+05C2 U+2063A U+99A7 U+7149 U+5563 U+7A4F U+7559 U+5BF3 U+AC01 U+8ABF U+51A4 U+591A U+96F7 U+8204 U+456B U+788C U+58DF U+8005 U+05E1 U+73A5 U+9EBB U+55E2 U+05D1 U+7A1C U+63C5 U+5C6E U+7F72 U+6EDB U+5D43 U+5FCD U+8779 U+00B4 U+81ED U+5553 U+55C2 U+0FB2 U+00C5 U+8279 U+AC05 U+AC16 U+5373 U+7BC6 U+4301 U+87E1 U+2940A U+7570 U+4EAE U+671B U+9F8E U+5BD8 U+9415 U+0338 U+6F0F U+7DF4 U+05E4 U+216EA U+238A7 U+863F U+62FE U+7B20 U+5EFE U+28BFA U+6DEA U+8352 U+4018 U+5192 U+4202 U+4D56 U+00B7 U+7506 U+6EBA U+761F U+5EB0 U+3EB8 U+7262 U+8612 U+831D U+1D165 U+0A32 U+9B12 U+4E3D U+3781 U+5C60 U+69EA U+250F2 U+5DB2 U+0F42 U+66B4 U+3862 U+6D1B U+5169 U+5599 U+291DF U+81D8 U+24814 U+6DDA U+5FA9 U+6613 U+05C1 U+7D63 U+8AD6 U+83F1 U+677B U+1D157 U+688E U+6914 U+6D6A U+7DBE U+3515 U+793E U+AC04 U+0385 U+8728 U+9238 U+98E2 U+320E U+58B3 U+24C92 U+25044 U+29B30 U+1E61 U+6852 U+6F6E U+8CC1 U+5ED3 U+798E U+0F56 U+0FAB U+0060 U+6ED1 U+7387 U+95AD U+7F79 U+774A U+73B2 U+641C U+5831 U+6B9F U+26523 U+4BCE U+25CD0 U+349E U+393A U+8001 U+654F U+5ED9 U+5D6B U+9C40 U+03AE U+622E U+05DA U+82B1 U+585E U+6691 U+0A3C U+716E U+7C7B U+5BC3 U+649D U+9F3B U+03CD U+6FEB U+5C62 U+96E3 U+5203 U+535A U+61B2 U+3C4E U+24608 U+8786 U+51DE U+537D U+25BAB U+8863 U+8C55 U+9829 U+9929 U+113C5 U+8F62 U+55C0 U+5840 U+72AF U+38E3 U+5D50 U+52A3 U+5167 U+3008 U+843D U+6D16 U+97E0 U+AC14 U+3B9D U+62FC U+7950 U+0F72 U+6CCC U+5954 U+7AB1 U+737A U+16122 U+6CE5 U+793C U+1D16F U+0342 U+90CE U+2335F U+AC0D U+71D0 U+5906 U+216A8 U+3C18 U+63A0 U+798F U+53EB U+AC13 U+8C48 U+5BEE U+6108 U+267B5 U+4F80 U+452B U+9F05 U+806F U+5E69 U+03AF U+5DBA U+AC12 U+5510 U+4008 U+5606 U+9F43 U+34DF U+22183 U+707D U+42A0 U+29496 U+AC0B U+6BBA U+6D1E U+4039 U+7D2F U+05BC U+2541D U+0F51 U+5442 U+6F23 U+507A U+6B72 U+7077 U+61F6 U+22844 U+701B U+876B U+6F22 U+5F6B U+2872E U+092F U+0314 U+88F8 U+7F9A U+5277 U+2A392 U+256C5 U+233C3 U+0308 U+66F8 U+26B3C U+AC0A U+7CBE U+05B9 U+21D0B U+8964 U+808B U+65E2 U+4CED U+0FA1 U+7ACB U+05E8 U+3A2E U+6B54 U+6210 U+7CE7 U+5668 U+5944 U+5F62 U+26228 U+5AB5 U+2339C U+76E7 U+9E7F U+8FB6 U+76DB U+243AB U+45F9 U+09BC U+05D0 U+5F22 U+6D69 U+25626 U+829D U+2A0CE U+AC10 U+AC18 U+5448 U+36EE U+60C7 U+61DE U+8564 U+26F2C U+8F2A U+75E2 U+6885 U+22331 U+8801 U+4635 U+5217 U+8B58 U+50E7 U+6C67 U+27667 U+5B85 U+232B8 U+9F0F U+66F4 U+6D41 U+514D U+8B80 U+05E6 U+5CC0 U+236A3 U+24735 U+16126 U+0A17 U+88CF U+05D6 U+7701 U+9072 U+1D158 U+5F53 U+5E3D U+656C U+7E45 U+958B U+537F U+5FAD U+78CC U+22BF1 U+0F9C U+90DE U+6075 U+5180 U+69A3 U+6D3E U+8F38 U+6CCD U+813E U+52C9 U+03A9 U+97FF U+455D U+270D2 U+618E U+0A16 U+90FD U+05E3 U+7099 U+7965 U+9EFE U+AC06 U+898B U+5B3E U+93F9 U+0386 U+53C3 U+88DE U+16D68 U+09AF U+7070 U+53CA U+7250 U+6817 U+038C U+5ECA U+AC07 U+20B63 U+72FC U+5A62 U+05BF U+6422 U+907C U+242EE U+6234 U+50CF U+826F U+6144 U+5F69 U+1D1B9 U+53DF U+549E U+7CD2 U+8DCB U+092B U+58EE U+03B0 U+862D U+5A1B U+3B08 U+20525 U+58A8 U+5555 U+387C U+5F9A U+8070 U+0921 U+0922 U+7C3E U+5C65 U+7498 U+8F3B U+1D16E U+5468 U+AC17 U+6B04 U+62D3 U+8457 U+05D5 U+5164 U+391C U+4CF8 U+8910 U+6556 U+5A27 U+46BE U+6B77 U+7642 U+7FBD U+6785 U+2A291 U+09A1 U+701E U+83EF U+9276 U+267A7 U+99FE U+03AD U+4046 U+61F2 U+4E41 U+62B1 U+6CBF U+87BA U+4F86 U+6452 U+7471 U+784E U+7A4A U+8ECA U+4E0D U+6881 U+91CF U+601C U+884C U+25249 U+285D2 U+9B6F U+6E1A U+97DB U+8516 U+208DE U+90F1 U+0390 U+7295 U+262D9 U+870E U+AC03 U+0FB7 U+65C5 U+21DE6 U+5351 U+3A6C U+27966 U+16129 U+51B7 U+82E5 U+4E86 U+761D U+27ED3 U+2051C U+74CA U+99C2 U+AC0E U+7E37 U+5438 U+21B18 U+83CC U+0041 U+113C7 U+61AF U+4E82 U+6EC7 U+738B U+8ED4 U+4A6E U+004B U+7D22 U+5F8B U+093C U+6190 U+5229 U+5DE2 U+23F5E U+0028 U+4E39 U+7375 U+7E09 U+4334 U+23393 U+4CCE U+0F92 U+60E1 U+0A38 U+8B39 U+4F60 U+5177 U+7610 U+26CD5 U+4B33 U+AC0C U+9ECE U+4E38 U+26FB1 U+51C9 U+5C06 U+6FC6 U+446B U+52DE U+5F04 U+82E6 U+6160 U+36FC U+261DA U+625D U+25119 U+8688 U+295B6 U+0FB3 U+5948 U+9818 U+5EA6 U+8996 U+05DE U+84F1 U+05F2 U+20122 U+70C8 U+8CD3 U+8ACB U+5DFD U+2346D U+265A8 U+26C36 U+03AB U+2ADD U+54BD U+533F U+52C7 U+1D171 U+21DE4 U+7228 U+9023 U+983B U+7447 U+0A2B U+0B21 U+8046 U+6688 U+523A U+8F9E U+92D8 U+51B5 U+2284A U+5DE1 U+5EB3 U+8291 U+23ED1 U+84F3 U+16123 U+6797 U+8218 U+6ECB U+7F7A U+9EF9 U+9A6A U+96B8 U+5306 U+0F71 U+63A9 U+6AA8 U+6D85 U+8DEF U+51DC U+038A U+5C8D U+5FF9 U+6942 U+8D77 U+529B U+778B U+9111 U+0FA6 U+795E U+7F3E U+5FD7 U+7BC9 U+82BD U+003B U+4E32 U+5289 U+4359 U+6BBB U+83C9
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/ref_test.go <<'EOF'
package norm

import (
	"strings"
	"testing"
)

// refNFC/refNFKC are straightforward implementations of the canonical
// composition algorithm of UAX #15 (D117), used as an oracle for the
// optimized implementation in this package.

var cccCache = map[rune]uint8{}

func refCCC(r rune) uint8 {
	if c, ok := cccCache[r]; ok {
		return c
	}
	c := NFC.Properties([]byte(string(r))).CCC()
	cccCache[r] = c
	return c
}

func refHangulCompose(l, c rune) rune {
	const (
		sbase, lbase, vbase, tbase       = 0xAC00, 0x1100, 0x1161, 0x11A7
		lcnt, vcnt, tcnt                 = 19, 21, 28
		ncnt                             = vcnt * tcnt
		scnt                             = lcnt * ncnt
	)
	if lbase <= l && l < lbase+lcnt && vbase <= c && c < vbase+vcnt {
		return sbase + ((l-lbase)*vcnt+(c-vbase))*tcnt
	}
	if sbase <= l && l < sbase+scnt && (l-sbase)%tcnt == 0 && tbase < c && c < tbase+tcnt {
		return l + (c - tbase)
	}
	return 0
}

func refCompose(d string) string {
	recompMapOnce.Do(buildRecompMap)
	r := []rune(d)
	out := make([]rune, 0, len(r))
	last := -1 // index in out of the last starter
	for _, c := range r {
		if last >= 0 {
			cccC := refCCC(c)
			blocked := false
			if len(out) > last+1 {
				prev := refCCC(out[len(out)-1])
				blocked = prev == 0 || prev >= cccC
			}
			if !blocked {
				comp := refHangulCompose(out[last], c)
				if comp == 0 {
					comp = combine(out[last], c)
				}
				if comp != 0 {
					out[last] = comp
					continue
				}
			}
		}
		out = append(out, c)
		if refCCC(c) == 0 {
			last = len(out) - 1
		}
	}
	return string(out)
}

func refNFC(s string) string  { return refCompose(NFD.String(s)) }
func refNFKC(s string) string { return refCompose(NFKD.String(s)) }

func TestAgainstReference(t *testing.T) {
	var all []rune
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		p := NFC.Properties([]byte(string(r)))
		if p.CCC() != 0 || len(p.Decomposition()) != 0 || !p.BoundaryBefore() {
			all = append(all, r)
		}
	}
	t.Logf("interesting runes: %d", len(all))

	// Runes that exercise the Hangul path plus the Unicode 17 starter
	// compositions and a few ordinary combining marks.
	lead := []rune{
		0x1100, 0x1101, 0x1112, 0x1113, // jamo L (and one past the end)
		0x1161, 0x1162, 0x1175, 0x1176, // jamo V (and one past the end)
		0x11A7, 0x11A8, 0x11C2, 0x11C3, // jamo T (and the boundaries)
		0xAC00, 0xAC01, 0xD7A3, 0xD7A4, // Hangul syllables and boundary
		0x320E, 0x321E, 0x3200, 0x3260, // circled/parenthesized Hangul (NFKC)
		0x0041, 0x00C0, 0x0300, 0x0301, 0x0316, 0x05B0,
		0x113C2, 0x113B8, 0x113C5, 0x113C9, 0x1611E, 0x1611F,
		0x16120, 0x16121, 0x16129, 0x16D67, 0x16D68,
		0x0344, 0x0F73, 0x1E14, 0x212B,
	}
	prefixes := []string{"", "A", "가", "각", "ᄀ", "가", "̀"}

	fail := 0
	check := func(in string) {
		if got, want := NFC.String(in), refNFC(in); got != want {
			fail++
			if fail <= 20 {
				t.Errorf("NFC(%U) = %U, want %U", []rune(in), []rune(got), []rune(want))
			}
		}
		if got, want := NFKC.String(in), refNFKC(in); got != want {
			fail++
			if fail <= 20 {
				t.Errorf("NFKC(%U) = %U, want %U", []rune(in), []rune(got), []rune(want))
			}
		}
	}
	// lead x all interesting runes, under several prefixes
	var sb strings.Builder
	for _, a := range lead {
		for _, b := range all {
			for _, p := range prefixes {
				sb.Reset()
				sb.WriteString(p)
				sb.WriteRune(a)
				sb.WriteRune(b)
				check(sb.String())
			}
		}
	}
	// exhaustive triples over the curated set
	for _, a := range lead {
		for _, b := range lead {
			for _, c := range lead {
				for _, p := range prefixes {
					sb.Reset()
					sb.WriteString(p)
					sb.WriteRune(a)
					sb.WriteRune(b)
					sb.WriteRune(c)
					check(sb.String())
				}
			}
		}
	}
	t.Logf("mismatches: %d", fail)
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestAgainstReference -v -timeout 20m 2>&1 | tail -30
description
Run reference-oracle conformance test on fixed impl
Result
=== RUN   TestAgainstReference
    ref_test.go:85: interesting runes: 3126
    ref_test.go:107: NFC([U+0300 U+11F41]) = [U+1F43], want [U+11F41 U+0300]
    ref_test.go:113: NFKC([U+0300 U+11F41]) = [U+1F43], want [U+11F41 U+0300]
    ref_test.go:107: NFC([U+0300 U+0300 U+11F41]) = [U+1F43 U+0300], want [U+11F41 U+0300 U+0300]
    ref_test.go:113: NFKC([U+0300 U+0300 U+11F41]) = [U+1F43 U+0300], want [U+11F41 U+0300 U+0300]
    ref_test.go:107: NFC([U+0301 U+11F41]) = [U+1F45], want [U+11F41 U+0301]
    ref_test.go:113: NFKC([U+0301 U+11F41]) = [U+1F45], want [U+11F41 U+0301]
    ref_test.go:107: NFC([U+0300 U+0301 U+11F41]) = [U+1F43 U+0301], want [U+11F41 U+0300 U+0301]
    ref_test.go:113: NFKC([U+0300 U+0301 U+11F41]) = [U+1F43 U+0301], want [U+11F41 U+0300 U+0301]
    ref_test.go:107: NFC([U+0300 U+0316 U+11F41]) = [U+1F43 U+0316], want [U+11F41 U+0316 U+0300]
    ref_test.go:113: NFKC([U+0300 U+0316 U+11F41]) = [U+1F43 U+0316], want [U+11F41 U+0316 U+0300]
    ref_test.go:107: NFC([U+0300 U+05B0 U+11F41]) = [U+1F43 U+05B0], want [U+11F41 U+05B0 U+0300]
    ref_test.go:113: NFKC([U+0300 U+05B0 U+11F41]) = [U+1F43 U+05B0], want [U+11F41 U+05B0 U+0300]
    ref_test.go:107: NFC([U+0300 U+0344 U+11F41]) = [U+1F43 U+0308 U+0301], want [U+11F41 U+0300 U+0308 U+0301]
    ref_test.go:113: NFKC([U+0300 U+0344 U+11F41]) = [U+1F43 U+0308 U+0301], want [U+11F41 U+0300 U+0308 U+0301]
    ref_test.go:107: NFC([U+0300 U+0F73 U+11F41]) = [U+1F43 U+0F71 U+0F72], want [U+11F41 U+0F71 U+0F72 U+0300]
    ref_test.go:113: NFKC([U+0300 U+0F73 U+11F41]) = [U+1F43 U+0F71 U+0F72], want [U+11F41 U+0F71 U+0F72 U+0300]
    ref_test.go:107: NFC([U+0041 U+113C2 U+0300 U+0316]) = [U+00C0 U+113C2 U+0316], want [U+0041 U+113C2 U+0316 U+0300]
    ref_test.go:113: NFKC([U+0041 U+113C2 U+0300 U+0316]) = [U+00C0 U+113C2 U+0316], want [U+0041 U+113C2 U+0316 U+0300]
    ref_test.go:107: NFC([U+0041 U+113C2 U+0300 U+05B0]) = [U+00C0 U+113C2 U+05B0], want [U+0041 U+113C2 U+05B0 U+0300]
    ref_test.go:113: NFKC([U+0041 U+113C2 U+0300 U+05B0]) = [U+00C0 U+113C2 U+05B0], want [U+0041 U+113C2 U+05B0 U+0300]
    ref_test.go:145: mismatches: 304
--- FAIL: TestAgainstReference (3.23s)
FAIL
FAIL	normtest/norm	3.238s
FAIL
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && mkdir -p cmp3 && cat > cmp3/main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/norm"
	"normtest/normold"
)

func main() {
	cases := []string{
		"̀\U00011F41",
		"́\U00011F41",
		"A\U000113C2̖̀",
		"A\U000113C2̀",
		"À\U000113C2",
	}
	for _, in := range cases {
		fmt.Printf("in=%U\n  NFD =%U\n  new =%U\n  old =%U\n", []rune(in),
			[]rune(norm.NFD.String(in)), []rune(norm.NFC.String(in)), []rune(normold.NFC.String(in)))
	}
	for _, r := range []rune{0x11F41, 0x0316, 0x0300, 0x113C2} {
		p := norm.NFC.Properties([]byte(string(r)))
		fmt.Printf("%U ccc=%d cb=%v boundaryBefore=%v\n", r, p.CCC(), p.String() == "", p.BoundaryBefore())
	}
}
EOF
sed -i 's/p.String() == ""/true/' cmp3/main.go
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp3
description
Compare specific mismatch cases old vs new
Result
in=[U+0300 U+11F41]
  NFD =[U+11F41 U+0300]
  new =[U+1F43]
  old =[U+1F43]
in=[U+0301 U+11F41]
  NFD =[U+11F41 U+0301]
  new =[U+1F45]
  old =[U+1F45]
in=[U+0041 U+113C2 U+0300 U+0316]
  NFD =[U+0041 U+113C2 U+0316 U+0300]
  new =[U+00C0 U+113C2 U+0316]
  old =[U+00C0 U+113C2 U+0316]
in=[U+0041 U+113C2 U+0300]
  NFD =[U+0041 U+113C2 U+0300]
  new =[U+0041 U+113C2 U+0300]
  old =[U+0041 U+113C2 U+0300]
in=[U+0041 U+0300 U+113C2]
  NFD =[U+0041 U+0300 U+113C2]
  new =[U+00C0 U+113C2]
  old =[U+00C0 U+113C2]
U+11F41 ccc=9 cb=true boundaryBefore=false
U+0316 ccc=220 cb=true boundaryBefore=false
U+0300 ccc=230 cb=true boundaryBefore=false
U+113C2 ccc=0 cb=true boundaryBefore=false
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/dbg3_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
)

func TestTraceCompose(t *testing.T) {
	recompMapOnce.Do(buildRecompMap)
	fmt.Printf("combine(0x41,0x113C2)=%U\n", combine(0x41, 0x113C2))
	fmt.Printf("combine(0x113C2,0x316)=%U\n", combine(0x113C2, 0x316))
	fmt.Printf("combine(0x113C2,0x300)=%U\n", combine(0x113C2, 0x300))
	fmt.Printf("combine(0x41,0x300)=%U\n", combine(0x41, 0x300))
	fmt.Printf("combine(0x11F41,0x300)=%U\n", combine(0x11F41, 0x300))
	for _, r := range []rune{0x41, 0x113C2, 0x316, 0x300} {
		p := lookupInfoNFC(inputString(string(r)), 0)
		fmt.Printf("%U info ccc=%d cb=%v cf=%v\n", r, p.ccc, p.combinesBackward(), p.combinesForward())
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestTraceCompose -v 2>&1 | head -20
description
Trace combine lookups
Result
=== RUN   TestTraceCompose
combine(0x41,0x113C2)=U+0000
combine(0x113C2,0x316)=U+0000
combine(0x113C2,0x300)=U+0000
combine(0x41,0x300)=U+00C0
combine(0x11F41,0x300)=U+1F43
U+0041 info ccc=0 cb=false cf=true
U+113C2 info ccc=0 cb=true cf=true
U+0316 info ccc=46 cb=false cf=false
U+0300 info ccc=51 cb=true cf=false
--- PASS: TestTraceCompose (0.00s)
PASS
ok  	normtest/norm	0.005s
Shell cwd was reset to /home/claude/golang_go_81021
WebFetch https://github.com/golang/go/issues/81001
url
https://github.com/golang/go/issues/81001
prompt
Reproduce the full issue text verbatim: title, body, all comments, labels, status, and any code snippets.
Result
# Issue Summary: Unicode Normalization Bug in Go 1.27

**Title:** x/text/unicode/norm: NFC/NFKC composes across a ccc=0 character that combines backward (24-33 of 35 affected, not go1.27-specific) #81001

**Status:** Open

**Reporter:** tannevaled (opened August 21, 2026)

## Problem Description

The `golang.org/x/text` package (v0.41.0) exhibits incorrect Unicode normalization behavior when built with Go 1.27.0, but works correctly with Go 1.26.4. The issue stems from a conditional build that selects Unicode 17.0.0 tables only for Go 1.27+.

## Technical Details

When normalizing the input sequence `U+16D68 U+FB01 U+113C2 U+0300 U+0316` using NFKC:

**Expected output:** "f096b5a86669f0918f82cc96cc80" (U+16D68 f i U+113C2 U+0316 U+0300)

**Actual output (Go 1.27):** "f096b5a866c3acf0918f82cc96" - a codepoint is lost

The composition incorrectly occurs across `U+113C2` (which has combining class 0), violating UAX #15 rule D115 that prevents composition when a ccc-0 character intervenes.

## Key Observations

- The `PropertiesString` function correctly reports "ccc(U+113C2): 0"
- Yet the composition step violates the rule that should apply based on this value
- Ruby 4.0.6 (Unicode 17.0.0) produces the expected output
- The trigger appears narrow and reproducible

## Workarounds Evaluated (Insufficient)

Composing from NFKD or segmenting at ccc-0 boundaries creates other regressions, particularly with Hangul syllable composition.
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/dbg4_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
)

// scan for combine() key collisions: pairs (a,b) for which combine returns a
// rune whose canonical decomposition is not [a,b].
func TestCombineCollisions(t *testing.T) {
	recompMapOnce.Do(buildRecompMap)
	// collect all second operands appearing in real decompositions
	second := map[rune]bool{}
	first := map[rune]bool{}
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		d := []rune(NFD.String(string(r)))
		if len(d) >= 2 {
			first[d[0]] = true
			second[d[1]] = true
		}
	}
	fmt.Printf("distinct first operands=%d second operands=%d\n", len(first), len(second))
	bogus := 0
	var ex []string
	for a := rune(0); a <= 0x10FFFF; a++ {
		if a >= 0xD800 && a <= 0xDFFF {
			continue
		}
		for b := range second {
			c := combine(a, b)
			if c == 0 {
				continue
			}
			d := []rune(NFD.String(string(c)))
			if !(len(d) == 2 && d[0] == a && d[1] == b) {
				bogus++
				if len(ex) < 25 {
					ex = append(ex, fmt.Sprintf("combine(%U,%U)=%U but NFD(%U)=%U", a, b, c, c, d))
				}
			}
		}
	}
	fmt.Printf("bogus combine results: %d\n", bogus)
	for _, e := range ex {
		fmt.Println("  " + e)
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestCombineCollisions -v -timeout 20m 2>&1 | head -40
description
Quantify combine() key collisions
Result
=== RUN   TestCombineCollisions
distinct first operands=336 second operands=110
bogus combine results: 15632
  combine(U+00C2,U+0300)=U+1EA6 but NFD(U+1EA6)=[U+0041 U+0302 U+0300]
  combine(U+00C2,U+0301)=U+1EA4 but NFD(U+1EA4)=[U+0041 U+0302 U+0301]
  combine(U+00C2,U+0309)=U+1EA8 but NFD(U+1EA8)=[U+0041 U+0302 U+0309]
  combine(U+00C2,U+0303)=U+1EAA but NFD(U+1EAA)=[U+0041 U+0302 U+0303]
  combine(U+00C4,U+0304)=U+01DE but NFD(U+01DE)=[U+0041 U+0308 U+0304]
  combine(U+00C5,U+0301)=U+01FA but NFD(U+01FA)=[U+0041 U+030A U+0301]
  combine(U+00C7,U+0301)=U+1E08 but NFD(U+1E08)=[U+0043 U+0327 U+0301]
  combine(U+00CA,U+0301)=U+1EBE but NFD(U+1EBE)=[U+0045 U+0302 U+0301]
  combine(U+00CA,U+0309)=U+1EC2 but NFD(U+1EC2)=[U+0045 U+0302 U+0309]
  combine(U+00CA,U+0303)=U+1EC4 but NFD(U+1EC4)=[U+0045 U+0302 U+0303]
  combine(U+00CA,U+0300)=U+1EC0 but NFD(U+1EC0)=[U+0045 U+0302 U+0300]
  combine(U+00CF,U+0301)=U+1E2E but NFD(U+1E2E)=[U+0049 U+0308 U+0301]
  combine(U+00D4,U+0301)=U+1ED0 but NFD(U+1ED0)=[U+004F U+0302 U+0301]
  combine(U+00D4,U+0309)=U+1ED4 but NFD(U+1ED4)=[U+004F U+0302 U+0309]
  combine(U+00D4,U+0303)=U+1ED6 but NFD(U+1ED6)=[U+004F U+0302 U+0303]
  combine(U+00D4,U+0300)=U+1ED2 but NFD(U+1ED2)=[U+004F U+0302 U+0300]
  combine(U+00D5,U+0308)=U+1E4E but NFD(U+1E4E)=[U+004F U+0303 U+0308]
  combine(U+00D5,U+0304)=U+022C but NFD(U+022C)=[U+004F U+0303 U+0304]
  combine(U+00D5,U+0301)=U+1E4C but NFD(U+1E4C)=[U+004F U+0303 U+0301]
  combine(U+00D6,U+0304)=U+022A but NFD(U+022A)=[U+004F U+0308 U+0304]
  combine(U+00DC,U+0300)=U+01DB but NFD(U+01DB)=[U+0055 U+0308 U+0300]
  combine(U+00DC,U+0304)=U+01D5 but NFD(U+01D5)=[U+0055 U+0308 U+0304]
  combine(U+00DC,U+0301)=U+01D7 but NFD(U+01D7)=[U+0055 U+0308 U+0301]
  combine(U+00DC,U+030C)=U+01D9 but NFD(U+01D9)=[U+0055 U+0308 U+030C]
  combine(U+00E2,U+0303)=U+1EAB but NFD(U+1EAB)=[U+0061 U+0302 U+0303]
--- PASS: TestCombineCollisions (1.68s)
PASS
ok  	normtest/norm	1.682s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/dbg4_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
)

// oneStepDecomp returns the canonical (single-step) decomposition of r.
func oneStepDecomp(r rune) []rune {
	return []rune(string(NFD.Properties([]byte(string(r))).Decomposition()))
}

func TestCombineCollisions(t *testing.T) {
	recompMapOnce.Do(buildRecompMap)
	second := map[rune]bool{}
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		if d := oneStepDecomp(r); len(d) == 2 {
			second[d[1]] = true
		}
	}
	fmt.Printf("distinct second operands: %d\n", len(second))
	bogus, bogusSupp := 0, 0
	var ex []string
	for a := rune(0); a <= 0x10FFFF; a++ {
		if a >= 0xD800 && a <= 0xDFFF {
			continue
		}
		for b := range second {
			c := combine(a, b)
			if c == 0 {
				continue
			}
			d := oneStepDecomp(c)
			if len(d) == 2 && d[0] == a && d[1] == b {
				continue
			}
			bogus++
			if a > 0xFFFF || b > 0xFFFF {
				bogusSupp++
				if len(ex) < 30 {
					ex = append(ex, fmt.Sprintf("combine(%U,%U)=%U (real decomp of %U is %U)", a, b, c, c, d))
				}
			}
		}
	}
	fmt.Printf("bogus=%d, of which involving supplementary operands=%d\n", bogus, bogusSupp)
	for _, e := range ex {
		fmt.Println("  " + e)
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestCombineCollisions -v -timeout 20m 2>&1 | head -45
description
Refine collision scan using single-step decomposition
Result
=== RUN   TestCombineCollisions
distinct second operands: 89
bogus=15632, of which involving supplementary operands=15378
  combine(U+1099,U+110BA)=U+1109A (real decomp of U+1109A is [U+11099 U+110BA])
  combine(U+109B,U+110BA)=U+1109C (real decomp of U+1109C is [U+1109B U+110BA])
  combine(U+10A5,U+110BA)=U+110AB (real decomp of U+110AB is [U+110A5 U+110BA])
  combine(U+1131,U+11127)=U+1112E (real decomp of U+1112E is [U+11131 U+11127])
  combine(U+1132,U+11127)=U+1112F (real decomp of U+1112F is [U+11132 U+11127])
  combine(U+1347,U+11357)=U+1134C (real decomp of U+1134C is [U+11347 U+11357])
  combine(U+1347,U+1133E)=U+1134B (real decomp of U+1134B is [U+11347 U+1133E])
  combine(U+1382,U+113C9)=U+11383 (real decomp of U+11383 is [U+11382 U+113C9])
  combine(U+1384,U+113BB)=U+11385 (real decomp of U+11385 is [U+11384 U+113BB])
  combine(U+138B,U+113C2)=U+1138E (real decomp of U+1138E is [U+1138B U+113C2])
  combine(U+1390,U+113C9)=U+11391 (real decomp of U+11391 is [U+11390 U+113C9])
  combine(U+13C2,U+113C2)=U+113C5 (real decomp of U+113C5 is [U+113C2 U+113C2])
  combine(U+13C2,U+113B8)=U+113C7 (real decomp of U+113C7 is [U+113C2 U+113B8])
  combine(U+13C2,U+113C9)=U+113C8 (real decomp of U+113C8 is [U+113C2 U+113C9])
  combine(U+14B9,U+114B0)=U+114BC (real decomp of U+114BC is [U+114B9 U+114B0])
  combine(U+14B9,U+114BA)=U+114BB (real decomp of U+114BB is [U+114B9 U+114BA])
  combine(U+14B9,U+114BD)=U+114BE (real decomp of U+114BE is [U+114B9 U+114BD])
  combine(U+15B8,U+115AF)=U+115BA (real decomp of U+115BA is [U+115B8 U+115AF])
  combine(U+15B9,U+115AF)=U+115BB (real decomp of U+115BB is [U+115B9 U+115AF])
  combine(U+1935,U+11930)=U+11938 (real decomp of U+11938 is [U+11935 U+11930])
  combine(U+611E,U+1611E)=U+16121 (real decomp of U+16121 is [U+1611E U+1611E])
  combine(U+611E,U+1611F)=U+16123 (real decomp of U+16123 is [U+1611E U+1611F])
  combine(U+611E,U+16120)=U+16125 (real decomp of U+16125 is [U+1611E U+16120])
  combine(U+611E,U+16129)=U+16122 (real decomp of U+16122 is [U+1611E U+16129])
  combine(U+6121,U+1611F)=U+16126 (real decomp of U+16126 is [U+1611E U+1611E U+1611F])
  combine(U+6121,U+16120)=U+16128 (real decomp of U+16128 is [U+1611E U+1611E U+16120])
  combine(U+6122,U+1611F)=U+16127 (real decomp of U+16127 is [U+1611E U+16129 U+1611F])
  combine(U+6129,U+1611F)=U+16124 (real decomp of U+16124 is [U+16129 U+1611F])
  combine(U+6D63,U+16D67)=U+16D69 (real decomp of U+16D69 is [U+16D63 U+16D67])
  combine(U+6D67,U+16D67)=U+16D68 (real decomp of U+16D68 is [U+16D67 U+16D67])
--- PASS: TestCombineCollisions (1.45s)
PASS
ok  	normtest/norm	1.450s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > norm/dbg5_test.go <<'EOF'
package norm

import (
	"fmt"
	"testing"
	"unicode/utf8"
)

// NFC must preserve canonical equivalence: NFD(NFC(x)) == NFD(x).
func TestCanonicalEquivalencePairs(t *testing.T) {
	second := map[rune]bool{}
	for r := rune(0); r <= 0x10FFFF; r++ {
		if r >= 0xD800 && r <= 0xDFFF {
			continue
		}
		if d := []rune(string(NFD.Properties([]byte(string(r))).Decomposition())); len(d) == 2 {
			second[d[1]] = true
		}
	}
	var secs []rune
	for r := range second {
		secs = append(secs, r)
	}
	bad := 0
	var ex []string
	buf := make([]byte, 0, 16)
	for a := rune(0); a <= 0x10FFFF; a++ {
		if a >= 0xD800 && a <= 0xDFFF {
			continue
		}
		for _, b := range secs {
			buf = buf[:0]
			buf = utf8.AppendRune(buf, a)
			buf = utf8.AppendRune(buf, b)
			in := string(buf)
			got := NFC.String(in)
			if NFD.String(got) != NFD.String(in) {
				bad++
				if len(ex) < 25 {
					ex = append(ex, fmt.Sprintf("NFC(%U) = %U — not canonically equivalent", []rune(in), []rune(got)))
				}
			}
		}
	}
	fmt.Printf("non-equivalent NFC results over 2-rune inputs: %d\n", bad)
	for _, e := range ex {
		fmt.Println("  " + e)
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm/ -run TestCanonicalEquivalencePairs -v -timeout 20m 2>&1 | head -35
description
End-to-end canonical-equivalence scan over rune pairs
Result
=== RUN   TestCanonicalEquivalencePairs
non-equivalent NFC results over 2-rune inputs: 15379
  NFC([U+05D2 U+0307]) = [U+105C9] — not canonically equivalent
  NFC([U+05DA U+0307]) = [U+105E4] — not canonically equivalent
  NFC([U+1099 U+110BA]) = [U+1109A] — not canonically equivalent
  NFC([U+109B U+110BA]) = [U+1109C] — not canonically equivalent
  NFC([U+10A5 U+110BA]) = [U+110AB] — not canonically equivalent
  NFC([U+1131 U+11127]) = [U+1112E] — not canonically equivalent
  NFC([U+1132 U+11127]) = [U+1112F] — not canonically equivalent
  NFC([U+1347 U+11357]) = [U+1134C] — not canonically equivalent
  NFC([U+1347 U+1133E]) = [U+1134B] — not canonically equivalent
  NFC([U+1382 U+113C9]) = [U+11383] — not canonically equivalent
  NFC([U+1384 U+113BB]) = [U+11385] — not canonically equivalent
  NFC([U+138B U+113C2]) = [U+1138E] — not canonically equivalent
  NFC([U+1390 U+113C9]) = [U+11391] — not canonically equivalent
  NFC([U+13C2 U+113C9]) = [U+113C8] — not canonically equivalent
  NFC([U+13C2 U+113C2]) = [U+113C5] — not canonically equivalent
  NFC([U+13C2 U+113B8]) = [U+113C7] — not canonically equivalent
  NFC([U+14B9 U+114BD]) = [U+114BE] — not canonically equivalent
  NFC([U+14B9 U+114BA]) = [U+114BB] — not canonically equivalent
  NFC([U+14B9 U+114B0]) = [U+114BC] — not canonically equivalent
  NFC([U+15B8 U+115AF]) = [U+115BA] — not canonically equivalent
  NFC([U+15B9 U+115AF]) = [U+115BB] — not canonically equivalent
  NFC([U+1935 U+11930]) = [U+11938] — not canonically equivalent
  NFC([U+611E U+16129]) = [U+16122] — not canonically equivalent
  NFC([U+611E U+16120]) = [U+16125] — not canonically equivalent
  NFC([U+611E U+1611E]) = [U+16121] — not canonically equivalent
--- PASS: TestCanonicalEquivalencePairs (43.98s)
PASS
ok  	normtest/norm	43.981s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cp norm/dbg5_test.go norm15/ && sed -i 's/^package norm$/package norm15/' norm15/dbg5_test.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./norm15/ -run TestCanonicalEquivalencePairs -v -timeout 20m 2>&1 | head -12
mkdir -p cmp4 && cat > cmp4/main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/normold"
)

func main() {
	fmt.Println("unmodified vendored x/text (Unicode", normold.Version+"):")
	for _, in := range []string{"ג̇", "ך̇", "Ꮒ\U000113C2", "̀\U00011F41"} {
		fmt.Printf("  NFC(%U) = %U\n", []rune(in), []rune(normold.NFC.String(in)))
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp4
description
Check U15 tables and unmodified code for the aliasing bug
Result
=== RUN   TestCanonicalEquivalencePairs
non-equivalent NFC results over 2-rune inputs: 15056
  NFC([U+1099 U+110BA]) = [U+1109A] — not canonically equivalent
  NFC([U+109B U+110BA]) = [U+1109C] — not canonically equivalent
  NFC([U+10A5 U+110BA]) = [U+110AB] — not canonically equivalent
  NFC([U+1131 U+11127]) = [U+1112E] — not canonically equivalent
  NFC([U+1132 U+11127]) = [U+1112F] — not canonically equivalent
  NFC([U+1347 U+11357]) = [U+1134C] — not canonically equivalent
  NFC([U+1347 U+1133E]) = [U+1134B] — not canonically equivalent
  NFC([U+14B9 U+114B0]) = [U+114BC] — not canonically equivalent
  NFC([U+14B9 U+114BD]) = [U+114BE] — not canonically equivalent
  NFC([U+14B9 U+114BA]) = [U+114BB] — not canonically equivalent
unmodified vendored x/text (Unicode 17.0.0):
  NFC([U+05D2 U+0307]) = [U+105C9]
  NFC([U+05DA U+0307]) = [U+105E4]
  NFC([U+13C2 U+113C2]) = [U+113C5]
  NFC([U+0300 U+11F41]) = [U+1F43]
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/normtest && cat > cmp4/main.go <<'EOF'
package main

import (
	"fmt"
	"normtest/norm"
	"normtest/normold"
)

func main() {
	cases := []string{
		"\U00016D68fi\U000113C2̖̀", // issue #81001 repro
		"가\U000113C2̖̀",           // Hangul analogue
		"가\U000113C5",                       // issue #81021 repro
		"가\U000113C2\U000113C2",
	}
	for _, in := range cases {
		fmt.Printf("in   =%U\n  NFKC old=%U\n  NFKC new=%U\n  NFC  old=%U\n  NFC  new=%U\n",
			[]rune(in),
			[]rune(normold.NFKC.String(in)), []rune(norm.NFKC.String(in)),
			[]rune(normold.NFC.String(in)), []rune(norm.NFC.String(in)))
	}
}
EOF
GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go run ./cmp4
description
Check 81001 repro is unaffected by the 81021 fix
Result
in   =[U+16D68 U+FB01 U+113C2 U+0300 U+0316]
  NFKC old=[U+16D68 U+0066 U+00EC U+113C2 U+0316]
  NFKC new=[U+16D68 U+0066 U+00EC U+113C2 U+0316]
  NFC  old=[U+16D68 U+FB01 U+113C2 U+0316 U+0300]
  NFC  new=[U+16D68 U+FB01 U+113C2 U+0316 U+0300]
in   =[U+AC00 U+113C2 U+0300 U+0316]
  NFKC old=[U+AC00 U+113C2 U+0316 U+0300]
  NFKC new=[U+AC00 U+113C2 U+0316 U+0300]
  NFC  old=[U+AC00 U+113C2 U+0316 U+0300]
  NFC  new=[U+AC00 U+113C2 U+0316 U+0300]
in   =[U+AC00 U+113C5]
  NFKC old=[U+AC00 U+113C2 U+113C2]
  NFKC new=[U+AC00 U+113C5]
  NFC  old=[U+AC00 U+113C2 U+113C2]
  NFC  new=[U+AC00 U+113C5]
in   =[U+AC00 U+113C2 U+113C2]
  NFKC old=[U+AC00 U+113C2 U+113C2]
  NFKC new=[U+AC00 U+113C5]
  NFC  old=[U+AC00 U+113C2 U+113C2]
  NFC  new=[U+AC00 U+113C5]
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
for d in src/vendor src/cmd/vendor; do
  f=/home/claude/golang_go_81021/go/$d/golang.org/x/text/unicode/norm/composition.go
  python3 - "$f" <<'PY'
import sys
p=sys.argv[1]
s=open(p).read()
old="""			default:
				b[k] = b[i]
				k++
			}"""
new="""			default:
				// Not a Hangul composition, but the segment may
				// still hold a regular canonical composition, as
				// the characters following a Hangul syllable are
				// not necessarily Hangul themselves. Fall back to
				// the composition table, as compose does.
				if b[i].combinesBackward() {
					if combined := combine(l, v); combined != 0 {
						rb.assignRune(s, combined)
						continue
					}
				}
				b[k] = b[i]
				k++
			}"""
assert s.count(old)==1, p
open(p,'w').write(s.replace(old,new))
print("patched", p)
PY
done
cd /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm && sed -n '424,470p' composition.go
description
Apply fix to both vendored copies in the Go tree
Result
patched /home/claude/golang_go_81021/go/src/vendor/golang.org/x/text/unicode/norm/composition.go
patched /home/claude/golang_go_81021/go/src/cmd/vendor/golang.org/x/text/unicode/norm/composition.go
func (rb *reorderBuffer) combineHangul(s, i, k int) {
	b := rb.rune[:]
	bn := rb.nrune
	for ; i < bn; i++ {
		cccB := b[k-1].ccc
		cccC := b[i].ccc
		if cccB == 0 {
			s = k - 1
		}
		if s != k-1 && cccB >= cccC {
			// b[i] is blocked by greater-equal cccX below it
			b[k] = b[i]
			k++
		} else {
			l := rb.runeAt(s) // also used to compare to hangulBase
			v := rb.runeAt(i) // also used to compare to jamoT
			switch {
			case jamoLBase <= l && l < jamoLEnd &&
				jamoVBase <= v && v < jamoVEnd:
				// 11xx plus 116x to LV
				rb.assignRune(s, hangulBase+
					(l-jamoLBase)*jamoVTCount+(v-jamoVBase)*jamoTCount)
			case hangulBase <= l && l < hangulEnd &&
				jamoTBase < v && v < jamoTEnd &&
				((l-hangulBase)%jamoTCount) == 0:
				// ACxx plus 11Ax to LVT
				rb.assignRune(s, l+v-jamoTBase)
			default:
				// Not a Hangul composition, but the segment may
				// still hold a regular canonical composition, as
				// the characters following a Hangul syllable are
				// not necessarily Hangul themselves. Fall back to
				// the composition table, as compose does.
				if b[i].combinesBackward() {
					if combined := combine(l, v); combined != 0 {
						rb.assignRune(s, combined)
						continue
					}
				}
				b[k] = b[i]
				k++
			}
		}
	}
	rb.nrune = k
}
Bash cd /home/claude/golang_go_81021/go/src && source ../../env.sh && GOTOOLCHAIN=local ../bin/go build std cmd 2>&1 | head -...
command
cd /home/claude/golang_go_81021/go/src && source ../../env.sh && GOTOOLCHAIN=local ../bin/go build std cmd 2>&1 | head -20 && echo BUILD_OK && GOTOOLCHAIN=local ../bin/go test -count=1 net net/http/httpguts vendor/golang.org/x/net/idna golang.org/x/text/unicode/norm 2>&1 | tail -20
description
Build std/cmd and run dependent tests
Result
BUILD_OK
# net/http/httpguts
package net/http/httpguts is not in std (/home/claude/golang_go_81021/go/src/net/http/httpguts)
FAIL	net/http/httpguts [setup failed]
ok  	net	29.318s
?   	vendor/golang.org/x/net/idna	[no test files]
?   	golang.org/x/text/unicode/norm	[no test files]
FAIL
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
sed -n '110,160p' $S/xtext/unicode/norm/composition_test.go; echo "=== normalize_test excerpt ==="; sed -n '1215,1260p' $S/xtext/unicode/norm/normalize_test.go
description
Look at existing test style
Result
func TestDecomposition(t *testing.T) {
	runTests(t, "TestDecompositionNFD", NFD, decompositionNFDTest)
	runTests(t, "TestDecompositionNFKD", NFKD, decompositionNFKDTest)
}

var compositionTest = []TestCase{
	{[]rune{0x41, 0x300}, []rune{0xC0}},
	{[]rune{0x41, 0x316}, []rune{0x41, 0x316}},
	{[]rune{0x41, 0x300, 0x35D}, []rune{0xC0, 0x35D}},
	{[]rune{0x41, 0x316, 0x300}, []rune{0xC0, 0x316}},
	// blocking starter
	{[]rune{0x41, 0x316, 0x40, 0x300}, []rune{0x41, 0x316, 0x40, 0x300}},
	{[]rune{0x1100, 0x1161}, []rune{0xAC00}},
	// parenthesized Hangul, alternate between ASCII and Hangul.
	{[]rune{0x28, 0x1100, 0x1161, 0x29}, []rune{0x28, 0xAC00, 0x29}},
}

func TestComposition(t *testing.T) {
	runTests(t, "TestComposition", NFC, compositionTest)
}
=== normalize_test excerpt ===
	doTextBenchmark(b, txt_cn)
}
func BenchmarkOverflow(b *testing.B) {
	doTextBenchmark(b, overflow)
}

var overflow = string(bytes.Repeat([]byte("\u035D"), 4096)) + "\u035B"

// Tests sampled from the Canonical ordering tests (Part 2) of
// https://unicode.org/Public/UNIDATA/NormalizationTest.txt
const txt_canon = `\u0061\u0315\u0300\u05AE\u0300\u0062 \u0061\u0300\u0315\u0300\u05AE\u0062
\u0061\u0302\u0315\u0300\u05AE\u0062 \u0061\u0307\u0315\u0300\u05AE\u0062
\u0061\u0315\u0300\u05AE\u030A\u0062 \u0061\u059A\u0316\u302A\u031C\u0062
\u0061\u032E\u059A\u0316\u302A\u0062 \u0061\u0338\u093C\u0334\u0062 
\u0061\u059A\u0316\u302A\u0339       \u0061\u0341\u0315\u0300\u05AE\u0062
\u0061\u0348\u059A\u0316\u302A\u0062 \u0061\u0361\u0345\u035D\u035C\u0062
\u0061\u0366\u0315\u0300\u05AE\u0062 \u0061\u0315\u0300\u05AE\u0486\u0062
\u0061\u05A4\u059A\u0316\u302A\u0062 \u0061\u0315\u0300\u05AE\u0613\u0062
\u0061\u0315\u0300\u05AE\u0615\u0062 \u0061\u0617\u0315\u0300\u05AE\u0062
\u0061\u0619\u0618\u064D\u064E\u0062 \u0061\u0315\u0300\u05AE\u0654\u0062
\u0061\u0315\u0300\u05AE\u06DC\u0062 \u0061\u0733\u0315\u0300\u05AE\u0062
\u0061\u0744\u059A\u0316\u302A\u0062 \u0061\u0315\u0300\u05AE\u0745\u0062
\u0061\u09CD\u05B0\u094D\u3099\u0062 \u0061\u0E38\u0E48\u0E38\u0C56\u0062
\u0061\u0EB8\u0E48\u0E38\u0E49\u0062 \u0061\u0F72\u0F71\u0EC8\u0F71\u0062
\u0061\u1039\u05B0\u094D\u3099\u0062 \u0061\u05B0\u094D\u3099\u1A60\u0062
\u0061\u3099\u093C\u0334\u1BE6\u0062 \u0061\u3099\u093C\u0334\u1C37\u0062
\u0061\u1CD9\u059A\u0316\u302A\u0062 \u0061\u2DED\u0315\u0300\u05AE\u0062
\u0061\u2DEF\u0315\u0300\u05AE\u0062 \u0061\u302D\u302E\u059A\u0316\u0062`

// Taken from http://creativecommons.org/licenses/by-sa/3.0/vn/
const txt_vn = `Với các điều kiện sau: Ghi nhận công của tác giả. 
Nếu bạn sử dụng, chuyển đổi, hoặc xây dựng dự án từ 
nội dung được chia sẻ này, bạn phải áp dụng giấy phép này hoặc 
một giấy phép khác có các điều khoản tương tự như giấy phép này
cho dự án của bạn. Hiểu rằng: Miễn — Bất kỳ các điều kiện nào
trên đây cũng có thể được miễn bỏ nếu bạn được sự cho phép của
người sở hữu bản quyền. Phạm vi công chúng — Khi tác phẩm hoặc
bất kỳ chương nào của tác phẩm đã trong vùng dành cho công
chúng theo quy định của pháp luật thì tình trạng của nó không 
bị ảnh hưởng bởi giấy phép trong bất kỳ trường hợp nào.`

// Taken from http://creativecommons.org/licenses/by-sa/1.0/deed.ru
const txt_ru = `При обязательном соблюдении следующих условий:
Attribution — Вы должны атрибутировать произведение (указывать
автора и источник) в порядке, предусмотренном автором или
лицензиаром (но только так, чтобы никоим образом не подразумевалось,
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
head -20 $S/xtext/unicode/norm/data17.0.0_test.go; echo ...; grep -n "^var \|^func \|^)" $S/xtext/unicode/norm/data17.0.0_test.go | head -20; echo "=== runTests ==="; sed -n '1,45p' $S/xtext/unicode/norm/composition_test.go
description
Inspect version-specific test data files
Result
// Code generated by running "go generate" in golang.org/x/text. DO NOT EDIT.

//go:build go1.27

package norm

const (
	Yes = iota
	No
	Maybe
)

type formData struct {
	qc              uint8
	combinesForward bool
	decomposition   string
}

type runeData struct {
	r      rune
...
11:)
27:func f(qc uint8, cf bool, dec string) [2]formData {
31:func g(qc, qck uint8, cf, cfk bool, d, dk string) [2]formData {
35:var testData = []runeData{
=== runTests ===
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package norm

import "testing"

// TestCase is used for most tests.
type TestCase struct {
	in  []rune
	out []rune
}

func runTests(t *testing.T, name string, fm Form, tests []TestCase) {
	rb := reorderBuffer{}
	rb.init(fm, nil)
	for i, test := range tests {
		rb.setFlusher(nil, appendFlush)
		for j, rune := range test.in {
			b := []byte(string(rune))
			src := inputBytes(b)
			info := rb.f.info(src, 0)
			if j == 0 {
				rb.ss.first(info)
			} else {
				rb.ss.next(info)
			}
			if rb.insertFlush(src, 0, info) < 0 {
				t.Errorf("%s:%d: insert failed for rune %d", name, i, j)
			}
		}
		rb.doFlush()
		was := string(rb.out)
		want := string(test.out)
		if len(was) != len(want) {
			t.Errorf("%s:%d: length = %d; want %d", name, i, len(was), len(want))
		}
		if was != want {
			k, pfx := pidx(was, want)
			t.Errorf("%s:%d: \nwas  %s%+q; \nwant %s%+q", name, i, pfx, was[k:], pfx, want[k:])
		}
	}
}
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cat > $S/hangul_test_snippet.go <<'EOF'
// Issue #81021: a Hangul syllable at the start of a segment must not
// disable regular canonical composition for the rest of the segment.
// The reorderBuffer switches to Hangul mode as soon as it sees a Jamo,
// so any composition further along in the same segment used to be lost.
func TestCompositionAfterHangul(t *testing.T) {
	prefixes := []string{
		"가",       // Jamo L V
		"각", // Jamo L V T
		"가",             // Hangul syllable LV
		"각",             // Hangul syllable LVT
		"힣",             // last Hangul syllable
	}
	for _, p := range prefixes {
		for r := rune(0); r <= unicode.MaxRune; r++ {
			c := string(r)
			if !utf8.ValidString(c) || NFC.String(c) != c {
				continue // not a composed character
			}
			d := NFD.String(c)
			if d == c {
				continue // nothing to recompose
			}
			// Skip characters that may legitimately merge with the
			// Hangul prefix rather than recompose among themselves.
			if f, _ := utf8.DecodeRuneInString(d); f >= 0x1100 && f <= 0x11ff ||
				f >= hangulBase && f < hangulEnd {
				continue
			}
			in, want := p+d, NFC.String(p)+c
			if got := NFC.String(in); got != want {
				t.Errorf("NFC(%+q) = %+q; want %+q", in, got, want)
			}
			if got := NFKC.String(in); got != want {
				t.Errorf("NFKC(%+q) = %+q; want %+q", in, got, want)
			}
		}
	}
}
EOF
cd $S/xtext/unicode/norm && python3 - <<'PY'
p='composition_test.go'
s=open(p).read()
s=s.replace('import "testing"','import (\n\t"testing"\n\t"unicode"\n\t"unicode/utf8"\n)')
s=s.rstrip()+"\n\n"+open('/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/hangul_test_snippet.go').read()
open(p,'w').write(s)
PY
cd $S/xtext && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ -run 'TestCompositionAfterHangul|TestComposition|TestDecomposition' -v 2>&1 | tail -15
description
Add regression test to x/text and run against fixed code
Result
    composition_test.go:169: NFKC("\ud7a3\u03d2\u0301") = "\ud7a3\u038e"; want "\ud7a3\u03d3"
    composition_test.go:169: NFKC("\ud7a3\u03d2\u0308") = "\ud7a3\u03ab"; want "\ud7a3\u03d4"
    composition_test.go:169: NFKC("\ud7a3\u017f\u0307") = "\ud7a3\u1e61"; want "\ud7a3\u1e9b"
    composition_test.go:169: NFKC("\ud7a3\u00a8\u0342") = "\ud7a3 \u0308\u0342"; want "\ud7a3\u1fc1"
    composition_test.go:169: NFKC("\ud7a3\u1fbf\u0300") = "\ud7a3 \u0313\u0300"; want "\ud7a3\u1fcd"
    composition_test.go:169: NFKC("\ud7a3\u1fbf\u0301") = "\ud7a3 \u0313\u0301"; want "\ud7a3\u1fce"
    composition_test.go:169: NFKC("\ud7a3\u1fbf\u0342") = "\ud7a3 \u0313\u0342"; want "\ud7a3\u1fcf"
    composition_test.go:169: NFKC("\ud7a3\u1ffe\u0300") = "\ud7a3 \u0314\u0300"; want "\ud7a3\u1fdd"
    composition_test.go:169: NFKC("\ud7a3\u1ffe\u0301") = "\ud7a3 \u0314\u0301"; want "\ud7a3\u1fde"
    composition_test.go:169: NFKC("\ud7a3\u1ffe\u0342") = "\ud7a3 \u0314\u0342"; want "\ud7a3\u1fdf"
    composition_test.go:169: NFKC("\ud7a3\u00a8\u0300") = "\ud7a3 \u0308\u0300"; want "\ud7a3\u1fed"
--- FAIL: TestCompositionAfterHangul (0.35s)
FAIL
FAIL	golang.org/x/text/unicode/norm	0.355s
FAIL
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cat > $S/hangul_test_snippet.go <<'EOF'
// Issue #81021: a Hangul syllable at the start of a segment must not
// disable regular canonical composition for the rest of the segment.
// The reorderBuffer switches to Hangul mode as soon as it sees a Jamo,
// and used to drop every non-Hangul composition after it.
func TestCompositionAfterHangul(t *testing.T) {
	prefixes := []string{
		"가",       // Jamo L V
		"각", // Jamo L V T
		"가",             // Hangul syllable LV
		"각",             // Hangul syllable LVT
		"힣",             // last Hangul syllable
	}
	forms := []struct {
		name     string
		c, d     Form
	}{
		{"NFC", NFC, NFD},
		{"NFKC", NFKC, NFKD},
	}
	for _, p := range prefixes {
		for _, f := range forms {
			for r := rune(0); r <= unicode.MaxRune; r++ {
				s := string(r)
				if !utf8.ValidString(s) {
					continue
				}
				want := f.c.String(s)
				in := f.d.String(s)
				if in == want {
					continue // nothing to recompose
				}
				// Skip characters that may legitimately merge with the
				// Hangul prefix rather than recompose among themselves.
				if c, _ := utf8.DecodeRuneInString(in); c >= jamoLBase && c <= 0x11ff ||
					c >= hangulBase && c < hangulEnd {
					continue
				}
				in, want = p+in, f.c.String(p)+want
				if got := f.c.String(in); got != want {
					t.Errorf("%s(%+q) = %+q; want %+q", f.name, in, got, want)
				}
			}
		}
	}
}
EOF
cd $S/xtext/unicode/norm && python3 - <<'PY'
p='composition_test.go'
s=open(p).read()
i=s.index('// Issue #81021')
s=s[:i]+open('/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/hangul_test_snippet.go').read()
open(p,'w').write(s)
PY
cd $S/xtext && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ -run 'TestCompositionAfterHangul' -v 2>&1 | tail -15
description
Rework and rerun regression test
Result
=== RUN   TestCompositionAfterHangul
--- PASS: TestCompositionAfterHangul (0.69s)
PASS
ok  	golang.org/x/text/unicode/norm	0.692s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/xtext && cp unicode/norm/composition.go /tmp/fixed.go && cp /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition.go unicode/norm/composition.go && echo "--- WITHOUT FIX ---" && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ -run 'TestCompositionAfterHangul' 2>&1 | head -12 && cp /tmp/fixed.go unicode/norm/composition.go && echo "--- WITH FIX (full suite) ---" && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ 2>&1 | tail -5
description
Confirm test fails without fix, passes with
Result
--- WITHOUT FIX ---
--- FAIL: TestCompositionAfterHangul (0.73s)
    composition_test.go:175: NFC("\uac00\U000113c2\U000113c2") = "\uac00\U000113c2\U000113c2"; want "\uac00\U000113c5"
    composition_test.go:175: NFC("\uac00\U000113c2\U000113b8") = "\uac00\U000113c2\U000113b8"; want "\uac00\U000113c7"
    composition_test.go:175: NFC("\uac00\U000113c2\U000113c9") = "\uac00\U000113c2\U000113c9"; want "\uac00\U000113c8"
    composition_test.go:175: NFC("\uac00\U0001611e\U0001611e") = "\uac00\U0001611e\U0001611e"; want "\uac00\U00016121"
    composition_test.go:175: NFC("\uac00\U0001611e\U00016129") = "\uac00\U0001611e\U00016129"; want "\uac00\U00016122"
    composition_test.go:175: NFC("\uac00\U0001611e\U0001611f") = "\uac00\U0001611e\U0001611f"; want "\uac00\U00016123"
    composition_test.go:175: NFC("\uac00\U00016129\U0001611f") = "\uac00\U00016129\U0001611f"; want "\uac00\U00016124"
    composition_test.go:175: NFC("\uac00\U0001611e\U00016120") = "\uac00\U0001611e\U00016120"; want "\uac00\U00016125"
    composition_test.go:175: NFC("\uac00\U0001611e\U0001611e\U0001611f") = "\uac00\U0001611e\U0001611e\U0001611f"; want "\uac00\U00016126"
    composition_test.go:175: NFC("\uac00\U0001611e\U00016129\U0001611f") = "\uac00\U0001611e\U00016129\U0001611f"; want "\uac00\U00016127"
    composition_test.go:175: NFC("\uac00\U0001611e\U0001611e\U00016120") = "\uac00\U0001611e\U0001611e\U00016120"; want "\uac00\U00016128"
--- WITH FIX (full suite) ---
ok  	golang.org/x/text/unicode/norm	2.054s
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/xtext/unicode/norm && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/gofmt -l . ; U=/home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm
diff -u $U/composition.go composition.go > $S/81021-xtext.patch; diff -u $U/composition_test.go composition_test.go >> $S/81021-xtext.patch; cat $S/81021-xtext.patch
description
Generate the x/text patch
Result
composition_test.go
--- /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition.go	2026-08-19 10:51:41.929559908 -0400
+++ composition.go	2026-08-25 08:29:40.310769401 -0400
@@ -449,6 +449,16 @@
 				// ACxx plus 11Ax to LVT
 				rb.assignRune(s, l+v-jamoTBase)
 			default:
+				// Not a Hangul composition. The segment may still
+				// contain regular canonical compositions, such as
+				// combining marks following a Hangul syllable, so
+				// fall back to the composition table.
+				if b[i].combinesBackward() {
+					if combined := combine(l, v); combined != 0 {
+						rb.assignRune(s, combined)
+						continue
+					}
+				}
 				b[k] = b[i]
 				k++
 			}
--- /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition_test.go	2026-08-19 10:51:41.929650830 -0400
+++ composition_test.go	2026-08-25 08:29:28.958448288 -0400
@@ -4,7 +4,11 @@
 
 package norm
 
-import "testing"
+import (
+	"testing"
+	"unicode"
+	"unicode/utf8"
+)
 
 // TestCase is used for most tests.
 type TestCase struct {
@@ -128,3 +132,49 @@
 func TestComposition(t *testing.T) {
 	runTests(t, "TestComposition", NFC, compositionTest)
 }
+
+// Issue #81021: a Hangul syllable at the start of a segment must not
+// disable regular canonical composition for the rest of the segment.
+// The reorderBuffer switches to Hangul mode as soon as it sees a Jamo,
+// and used to drop every non-Hangul composition after it.
+func TestCompositionAfterHangul(t *testing.T) {
+	prefixes := []string{
+		"가",       // Jamo L V
+		"각", // Jamo L V T
+		"가",             // Hangul syllable LV
+		"각",             // Hangul syllable LVT
+		"힣",             // last Hangul syllable
+	}
+	forms := []struct {
+		name     string
+		c, d     Form
+	}{
+		{"NFC", NFC, NFD},
+		{"NFKC", NFKC, NFKD},
+	}
+	for _, p := range prefixes {
+		for _, f := range forms {
+			for r := rune(0); r <= unicode.MaxRune; r++ {
+				s := string(r)
+				if !utf8.ValidString(s) {
+					continue
+				}
+				want := f.c.String(s)
+				in := f.d.String(s)
+				if in == want {
+					continue // nothing to recompose
+				}
+				// Skip characters that may legitimately merge with the
+				// Hangul prefix rather than recompose among themselves.
+				if c, _ := utf8.DecodeRuneInString(in); c >= jamoLBase && c <= 0x11ff ||
+					c >= hangulBase && c < hangulEnd {
+					continue
+				}
+				in, want = p+in, f.c.String(p)+want
+				if got := f.c.String(in); got != want {
+					t.Errorf("%s(%+q) = %+q; want %+q", f.name, in, got, want)
+				}
+			}
+		}
+	}
+}
Shell cwd was reset to /home/claude/golang_go_81021
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cd $S/xtext/unicode/norm && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/gofmt -w composition_test.go && GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/gofmt -l .
# sync the exact comment wording into the Go tree copies
for d in src/vendor src/cmd/vendor; do
  cp $S/xtext/unicode/norm/composition.go /home/claude/golang_go_81021/go/$d/golang.org/x/text/unicode/norm/composition.go
done
U=/home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm
diff -u $U/composition.go composition.go > $S/81021-xtext.patch
diff -u $U/composition_test.go composition_test.go >> $S/81021-xtext.patch
cd $S/xtext && GOFLAGS=-mod=mod GOTOOLCHAIN=local /home/claude/golang_go_81021/go/bin/go test ./unicode/norm/ 2>&1 | tail -3
cd /home/claude/golang_go_81021/go/src && GOTOOLCHAIN=local ../bin/go build std cmd && echo GO_TREE_BUILD_OK && GOTOOLCHAIN=local ../bin/go test -count=1 net 2>&1 | tail -3
description
Format, sync into Go tree, rebuild and retest
Result
ok  	golang.org/x/text/unicode/norm	2.075s
GO_TREE_BUILD_OK
ok  	net	28.976s
Bash S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad ...
command
S=/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad
cp $S/81021-xtext.patch $S/../ 2>/dev/null; ls -la $S/81021-xtext.patch; sed -n '1,20p' $S/81021-xtext.patch
description
Confirm patch file
Result
-rw-rw-r-- 1 claude claude 2480 Aug 25 08:30 /tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/81021-xtext.patch
--- /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition.go	2026-08-19 10:51:41.929559908 -0400
+++ composition.go	2026-08-25 08:29:40.310769401 -0400
@@ -449,6 +449,16 @@
 				// ACxx plus 11Ax to LVT
 				rb.assignRune(s, l+v-jamoTBase)
 			default:
+				// Not a Hangul composition. The segment may still
+				// contain regular canonical compositions, such as
+				// combining marks following a Hangul syllable, so
+				// fall back to the composition table.
+				if b[i].combinesBackward() {
+					if combined := combine(l, v); combined != 0 {
+						rb.assignRune(s, combined)
+						continue
+					}
+				}
 				b[k] = b[i]
 				k++
 			}
--- /home/claude/pkg/mod/golang.org/x/text@v0.41.0/unicode/norm/composition_test.go	2026-08-19 10:51:41.929650830 -0400
SendUserFile
caption
Fix + regression test for #81021, against golang.org/x/text@v0.41.0 (also applied to both vendored copies in the local GOROOT).
files
[
	"/tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/81021-xtext.patch"
]
status
normal
Result
1 file delivered to user.
  /tmp/claude-1003/-home-claude-golang-go-81021/499f4eb1-6fd9-4190-be20-22ac8f0f6e51/scratchpad/81021-xtext.patch → file_uuid: 583f2e0e-178e-4424-b792-5a0893d329e7

go.dev/issue/81021 — triage

Confirmed, valid, and the reporter's root-cause analysis is correct. Reproduced against the tree at 13b477edc7 with the Unicode 17 tables:

NFC(U+AC00 U+113C5) = U+AC00 U+113C2 U+113C2   (want U+AC00 U+113C5)
NFC(U+113C5)        = U+113C5                   (correct)

Mechanism

reorderBuffer.compose (composition.go:480) bails to combineHangul the moment it sees a Jamo:

if isJamoVT(rb.bytesAt(i)) {
	rb.combineHangul(s, i, k)
	return
}

combineHangul only knows the two algorithmic Hangul rules; its default: arm copies the rune through without ever consulting combine(). So once a segment starts with a Hangul syllable (which NFD-expands to L V [T], hitting isJamoVT at index 1), every table-driven composition later in that segment is silently dropped.

Why it surfaced only now

U+113C2 has ccc=0 and combinesBackward=true, so it has no boundary before it and gets pulled into the preceding Hangul syllable's segment. Unicode 17 is the first version to add canonical compositions of starter + starter of this shape. Scanning the recomposition map: 12 of 961 targets are affected under Unicode 17, 0 of 941 under Unicode 15 — Tulu-Tigalari (U+113C5/C7/C8), Gurung Khema (U+16121–U+16128), Kirat Rai (U+16D68). So it's a latent bug in combineHangul, newly reachable, and Go 1.27+ only (matching the //go:build go1.27 table selection).

Fix

Fall back to the composition table in combineHangul's default: arm, mirroring what compose does. Patch attached (also applied to src/vendor and src/cmd/vendor in the local tree).

Verification:

  • All 12 affected targets now compose correctly after every Hangul prefix form (LV, LVT, L V, L V T).
  • Differential old-vs-new over 3,126 "interesting" runes × pairs × 7 prefixes × {NFC, NFKC}: 403,464 outputs change, 0 break canonical equivalence (NFD(new) == NFD(old)) and 0 are less composed. Nothing changes outside the intended cases.
  • Full upstream x/text/unicode/norm suite passes; go build std cmd and go test net pass.
  • The new test fails on unpatched v0.41.0 and passes with the fix.

The real fix belongs upstream in golang.org/x/text and then gets vendored into the Go tree.

Two adjacent bugs found while debugging (not fixed here)

  1. #81001 is a distinct bug and this fix does not address it. compose only refreshes s (last-starter index) inside the ii.combinesBackward() branch, so a non-combining mark after a ccc=0 backward-combining starter leaves s stale and composition happens across the intervening starter. combineHangul updates s unconditionally each iteration, which is why the Hangul analogue is already correct — and why this fix inherits no #81001 behavior.

  2. combine() truncates both operands to 16 bits (forminfo.go:216: uint32(uint16(a))<<16 + uint32(uint16(b))), so supplementary-plane runes alias BMP table entries. This is not a Unicode 17 regression — it's long-standing and reachable from plain BMP text:

    NFC(U+05D2 U+0307) = U+105C9     // Hebrew gimel + dot above → a Garay letter
    NFC(U+0300 U+11F41) = U+1F43     // Kawi char destroyed, Greek letter invented
    

    Exhaustive two-rune scan: 15,379 inputs under Unicode 17 (15,056 under Unicode 15) where NFC output is not canonically equivalent to its input. Fixing it needs a wider key in the generated recompMapPacked, i.e. a maketables.go change. This looks unreported and is arguably more severe than #81021 — worth filing separately if you'd like.