You are here: Articles --> 2000 --> Feature article: Readability indexes—a debate

Vous êtes ici : Essais --> 2000 --> Feature article: Readability indexes—a debate

Feature article: Readability indexes—a debate

Previously published, in a different form, as: Weber, J.H.; Hart, G.; Jong, S. 2000. Feature article: readability indexes—a debate. The Technical Editors' Eyrie Newsletter, 7 January 2000. [Unpaginated e-mail newsletter, ISSN 1442-8652]

Back in March 1999, Geoff Hart and Steven Jong had a debate on the TECHWR-L list, on the topic of readability indexes (such as the Flesch Index and Gunning's Fog Index).

Geoff started by saying (to someone who asked about a tool for this purpose), "Whatever tool you pick, try this trick to test whether it's any use: take a simple sentence and arrange the words in random order (better still, arrange them maliciously so the sentence makes absolutely no sense, or even says the opposite of what you intended to say). If the software provides a comparable readability index for both versions of the sentence, demand your money back. Good luck finding something that passes this test.

"If you absolutely need a measure of readability based solely on word counts, word lengths, etc., you can almost always use the software's built-in tools. For example, MSWord gives you a total word count, plus the number of paragraphs, lines, and characters. You can generate any index you want using these numbers... though in my opinion, you're still wasting your time. There's almost no correlation between the main readability indexes and actual readability, and there won't be for a good long time to come until someone develops a tool that can parse the content of text in the specific context of a well-defined audience."

To which Steven replied: "This is a sweeping indictment, but it is not true. Readability formulas were originally developed to predict the ability of schoolchildren to comprehend written text. The parameters of the original formulas (starting with the Flesch index) were adjusted heuristically until their predictions matched actual reading-test scores."

Geoff: "In fact, Steve is correct, because I omitted a key word in my original posting: there is almost no causal correlation."

Steven: "Actually, there is a good correlation between readability-index scores and actual readability in certain domains. Textbook publishers and magazine editors still use readability formulas to assess the readability of their products."

Geoff (in another note): "I suggested randomizing the words: 'the dog is white' becomes 'white the is dog', which is hardly readable despite having an identical readability index."

Steven: "Well... It's meaningless, but it's still readable. Perhaps a more interesting example is 'White is the dog,' which is semantically the same and equally readable according to the index, but a little more difficult to read. Why? Because it's a right-to-left branching sentence: the object precedes the verb and the verb precedes the subject... Although sentences can be constructed to branch left, right, or center, they are easiest to understand when they branch left-to-right.

"For this assertion there is experimental verification and theoretical support. Backward-branching sentences tax the reader's short-term memory... This burden is especially true for readers of English as a second language. The link between complex sentence structure and readability is causal: complex sentences are inherently less readable.

"Now, non-straightforward and complex sentences are characteristically longer than straightforward, simple sentences. This is a correlation, not a causal link, but it's a strong one. It takes a good writer to construct a long sentence that is still easy to read and understand."

Geoff: "It's true that well-written short sentences can be more effective than poorly written, convoluted, long sentences. Nobody disputes that, because it's not a fair comparison. What I dispute is the assertion that well-written, well-organized long sentences are inherently less useful than shorter, simpler sentences. In fact, relying on overly short sentences can compromise readability by making the text too choppy and hindering the efficient flow of thought. I'm unaware of any readability index that addresses these issues."

"... I don't know of any language in which readability is independent of meaning; in fact, I define readability as the ease with which the text communicates the author's desired meaning."

Steven: "I disagree—readability is always independent of meaning. I would agree that poor readability obscures meaning, which clearly we don't want. I guess I would define readability the same way you do, though. I can even suggest a worse example for you: readability indices will give the same results if you enter the sentences completely backward; they will give results, though meaningless, for samples in any Romance language."

Geoff: "What do I want? An index that does more than count words and spaces. That's useless, as I hope my two examples have just shown."

Steven: "Actually, most indexes count words and syllables. I think I have shown a correlation between sentence length and readability based on sound theory. What about syllable counts? Reading is a process of decoding symbolic meaning. Adding synonyms, jargon, and Latinates makes the decoding more difficult... It's pretty well established that using ornate language and polysyllabic words obscures meaning; again, it's especially true for foreign readers. This correlates strongly to syllable counts. When Strunk told us to prefer simple words, he knew what he was about.

"So the combination of measuring sentence length and syllable counts has a basis in theory."

Geoff: "There's almost no correlation between the main readability indexes and actual readability, and there won't be for a good long time to come until someone develops a tool that can parse the content of text in the specific context of a well-defined audience."

Steven: "This is important work that I believe has yet to be done. I participated in the study of one such tool, which required measurements of some two dozen characteristics, including passive-voice sentences, sentences with an explicit agent of action, and many others. I believe it was well grounded, but it took me hours to get results for a single document."

Steven (in another note): "I would not want to base any action or decision solely on a readability index... However, readability does have some validity, and could reasonably be considered as part of a larger set of measurements."

Geoff: "And here, I'll conclude by reluctantly agreeing with you. As a red flag for text that is childishly simple, or horridly complex, it can work well enough. But for the vast majority of text, which falls somewhere in between, I consider the indices of so little use that I'd rather pay a good editor to have a read through the manuscript and tell me if it's appropriate for my audience."

Steven: "I would, too; but have you priced an editor lately? The readability index is simple-minded but very quick and easy to use. For working professionals, quick and dirty metrics have advantages.

"Having said all that, I would be the first to man the battlements if someone declared that readability was the only important consideration for my documents. I am just conducting an academic defense of readability in a limited role as part of a larger set of documentation metrics and assessments."

Jean's comment: Several people noted that someone higher up in their company demanded they keep readability metrics, so regardless of the value of those metrics, they use them. No one suggested those metrics should be the only criteria of a quality document, though I'm sure that happens. I certainly know from personal experience that poor writers often defend their writing on the grounds that it "passed the quality tests," which included a readability index. A useful paper on this subject is:

Redish, J.; Selzer, J. 1985. The place of readability formulas in technical communication. Technical Communication 32(4):46–52 [An article that analyses the limits of readability formulas.]

©2004–2024 Geoffrey Hart. All rights reserved.