Login Join IOPS

Our key documents are too difficult

  • Written by:
  • Published on:
  • Categories:
  • Comments:
  • Share:

A well-known readability test is the Flesch–Kincaid Grade Level. It measures the difficulty of a text and gives the result as a U.S. “grade level”: the number of years people who are able to understand the text will have been in school. The average reading level of adult readers is 8th grade.

I have tested our three key documents Mission, Vision, Program and Structure for reading ease. The results are shocking. They have a Flesch–Kincaid Grade Level of 19 or more. That means they are difficult to understand for someone who is not a university graduate.

Other readability tests also come out with very high (which means very bad) scores. The web site Editcentral.com lets you test a text automatically.

Discussion 6 Comments

  • 30th Jul 2012

    Wow. Does 19 mean they've had three years of post-bachelor degree studies? There's a project working on this issue here if you want to check them out: http://www.iopsociety.org/projects/translating-iops.

  • L L 31st Jul 2012

    Hmmm. It's important for us to make IOPS as accessible as possible. The present documents do well to comprehensively explain IOPS, yet perhaps we do need to improve accessibility. Perhaps we should look into making simplified versions of the text and perhaps a spoken version for people with severe dyslexia/visual impairments etc.

  • Oscar Addis 31st Jul 2012

    To test it out I entered in the following works

    1. chapter 1 of Immanuel Kant's work 'fundamental principles of the metaphysics of morals'

    Flesch-Kincaid grade level: 15

    2. The introduction to 'A Contribution to the Critique of Hegel’s Philosophy of Right' by Karl Marx

    Flesch-Kincaid grade level: 12.4

    I then proceeded to only put in paragraphs and the results were as followed

    This passage by Kant received a grade of 18.8
    "In the physical constitution of an organized being, that is, a being adapted suitably to the purposes of life, we assume it as a fundamental principle that no organ for any purpose will be found but what is also the fittest and best adapted for that purpose. Now in a being which has reason and a will, if the proper object of nature were its conservation, its welfare, in a word, its happiness, then nature would have hit upon a very bad arrangement in selecting the reason of the creature to carry out this purpose. For all the actions which the creature has to perform with a view to this purpose, and the whole rule of its conduct, would be far more surely prescribed to it by instinct, and that end would have been attained thereby much more certainly than it ever can be by reason. Should reason have been communicated to this favoured creature over and above, it must only have served it to contemplate the happy constitution of its nature, to admire it, to congratulate itself thereon, and to feel thankful for it to the beneficent cause, but not that it should subject its desires to that weak and delusive guidance and meddle bunglingly with the purpose of nature. In a word, nature would have taken care that reason should not break forth into practical exercise, nor have the presumption, with its weak insight, to think out for itself the plan of happiness, and of the means of attaining it. Nature would not only have taken on herself the choice of the ends, but also of the means, and with wise foresight would have entrusted both to instinct."

    This passage by Marx received a grade level of 13.3
    "Criticism has plucked the imaginary flowers on the chain not in order that man shall continue to bear that chain without fantasy or consolation, but so that he shall throw off the chain and pluck the living flower. The criticism of religion disillusions man, so that he will think, act, and fashion his reality like a man who has discarded his illusions and regained his senses, so that he will move around himself as his own true Sun. Religion is only the illusory Sun which revolves around man as long as he does not revolve around himself."

    It is therefore apparent that the shorter the passage entered the higher the score.

    To see how the test works: http://en.wikipedia.org/wiki/Flesch–Kincaid_readability_test

    The IOPS key documents are far easier to read than Immanuel Kant and Karl Marx so I think there is an issue with this measurement, perhaps we should ask people if they felt it was too complicated rather than relying on the
    Flesch-Kincaid to tell us so.

  • Oscar Addis 31st Jul 2012


    * http://en.wikipedia.org/wiki/Flesch–Kincaid_readability_test

    ** The IOPS key documents are far easier to read than Immanuel Kant and Karl Marx so I think there is an issue with this measurement, perhaps we should ask people if they felt it was too complicated rather than relying on the Flesch-Kincaid test to tell us so.

    • Lambert Meertens 31st Jul 2012

      I don't think it is true that shorter passages tend to result in higher Flesch-Kincaid scores. The score over a sequence of equal-length passages is an average of the separate scores. If these passages come out lower than the longer texts, it means that some other passages have higher scores than the texts from which they are taken.

      Nevertheless, the Flesch-Kincaid test is not ideal. For one thing, it is too sensitive to outliers. It is not hard to write texts that are an incomprehensible mess but get a nice low grade level on the test. Just throw in a simple sentence every now and then, and the score drops dramatically. In the passage from Kant, the last sentence is relatively short and has many one-syllable words. If you leave it out, the score goes up to a whopping 20.4, which I think is more indicative of the difficulty of the text.

      The test also does not take account of jargon, something that our key documents are rife with. This is an extra hurdle for people who are not familiar with the literature on these subjects.

  • 31st Jul 2012

    I agree with Lambert, I think the language of those documents should be easier to understand.

    There seems to be evidence that some will find it hard to understand the texts because of the language. As said, it is difficult for the non-college educated. However, I'd say it is also difficult for foreign speakers of English that don't have access to the documents in their own language. There is a translation effort going on, but I think it's unlikely to help everyone right away. For example, my main language has only a million speakers and is unlikely to be translated soon.

    However, I don't think it is needed to make everything simpler, just the key documents. After all, some of the key documents are a basic summary of a much wider span of ideas. If people want to dig deeper, they can get more information from the blog section or other sources like ZNet. In those sources, having intricate language is fine, as sometimes you need it for complex ideas. However, in key documents it is better if the language is simple and straightforward.

    As such, I think it would be good to have an effort to simplify some of the key documents. Perhaps by using something like the Orwellian standards of English as a guide?