How Hard Is a Chinese Text? Measuring Difficulty by HSK Character Coverage

“What level is this text?” sounds like a question with an answer. It has several, depending on what you count. This post uses one of the simplest defensible measures — character coverage — and works it through on eight example passages whose results you can download and check.

I build HSK Studio, so I have a stake in text analysis tools. The method below does not need any of them: it is arithmetic you can do with a word list.

The method

1. Take the passage and keep only the Han (Chinese) characters. Punctuation, digits, Latin letters and emoji count toward total length but not toward coverage.
2. Count occurrences, not unique characters. A character that appears five times counts five times.
3. For each HSK level from 1 to 6, find what share of those occurrences belong to that level or an easier one. That is the cumulative coverage.
4. Pick a threshold. The reference analysis uses 90%: the screening level is the lowest level whose cumulative coverage reaches it.

The word lists are the official HSK 2012 lists, so “HSK 3” here means the 2012 syllabus, not the newer 3.0 standard.

Three worked examples

These come from the Chinese text analysis reference benchmark, a set of eight original diagnostic passages. They are small and constructed; they show how the method behaves, not how typical Chinese text is distributed.

The commute passage is everyday Chinese: coffee, the subway, reading news on a phone. It needs HSK 3 to pass 90%, with 8 of its 51 characters at that level and two (猜 and 篇) above HSK 3. The two characters above the screening level are the ones worth looking up before reading.

The technical passage is about half the length and has a very different profile. Only 44% of its characters are within HSK 1–2, and it takes HSK 5 to reach full coverage because of its specialist vocabulary (protein, gene and machine-learning terms). Two short passages can sit two levels apart.

The repetition example is the one that should make you suspicious of any single number. The text is nine copies of 我 followed by one very rare character, 龘. It reaches 90% at HSK 1 because coverage counts occurrences. The text is not easy; one character in ten is not in any list, and that one is the whole difficulty. Real text rarely looks like this, but long texts with a repeated name or term can drift in the same direction.

Two edge cases that need a rule

A rare character on its own — a single Han character outside the bundled lists — has 0% coverage at every level, so it receives no level rather than “above HSK 6”. A passage with no Han at all also receives no level. Claiming a level for text that has no Chinese in it would be false precision.

Traditional characters. The HSK 2012 lists are simplified. The benchmark’s traditional version of the everyday sentence normalises four characters and then lands on exactly the same coverage as the simplified sentence. If a tool scores traditional text without normalising, it is reporting characters it does not recognise as unknown, not as hard.

What the number does not tell you

• Characters are not words. 学习 is made of two familiar characters and is also a single word you may or may not know. The method does not look at word boundaries.
• Coverage is not comprehension. 96% of characters inside your level leaves about one in twenty-five unfamiliar, and grammar, idiom and topic knowledge are not measured at all.
• Short texts are noisy. A twelve-character sentence moves by about nine percentage points with one character. Use longer samples before you trust the shape.
• 90% is a convention. It is a stated threshold in this analysis, not a research-validated point of comfortable reading. Change the threshold and the levels move.

So use the number to decide whether to try a text, and let the first paragraph decide whether to keep going. For a longer treatment of choosing material, see how to choose Chinese reading material for your HSK level.

Check it yourself

• Paste a passage into the Chinese text analyzer to see its coverage at each level and the characters above your level. It runs in your browser, so nothing is uploaded.
• Download the benchmark as CSV or JSON from the open datasets page. The JSON includes every passage, and the CSV carries a SHA-256 for each so a cited result can be reproduced exactly. Both are CC BY 4.0.
• If your count differs from the published one, send the passage to hello@fulinlabs.com. A discrepancy is more useful to me than agreement.