Readability Score Analysis

Introduction

In this blog post, you will be demonstrated with the prompt engineering technique on evaluating the readability score based on Flesch–Kincaid readability analysis.

Background

The key inspiration behind this blog post is Apyhub Readability Scores Documents. This one is an experiment to reverse engineer the prompt template to come up with the state-of-the-art solution for analyzing the specified content and report with the readability results.

Please navigate to Flesch–Kincaid Readability to know the behind the scene algorithm.

Use-cases

Below are the top 10 use cases where you can leverage the readability score. Please note - The following quote was generated using the ChatGPT.

Education and Curriculum Development:

Teachers and educators use readability tests to assess the complexity of educational materials. This helps in tailoring the content to match the reading abilities of students at different grade levels.

Publishing Industry:

Publishers use readability scores to determine the appropriate target audience for books, articles, and other written materials. This ensures that the content is accessible and engaging to the intended readership.

Government and Legal Documents:

Government agencies often use readability tests to assess the clarity of public communications and legal documents. This helps in making information more understandable to the public, promoting transparency and accessibility.

Healthcare Communication:

Medical professionals use readability tests to evaluate the complexity of patient education materials. Clear and understandable health information is crucial for patients to make informed decisions about their health and treatment options.

Web Content and SEO:

Website developers and content creators use readability scores to optimize web content for search engines. Search engines may favor content that is easy to read, and a lower readability score can contribute to better search engine optimization (SEO).

Corporate Communications:

Businesses use readability tests to assess the clarity of internal and external communications. Clear and easily understandable documents contribute to effective communication within an organization and with external stakeholders.

Journalism and Media:

Journalists and media professionals use readability tests to ensure that news articles and reports are accessible to a broad audience. This is particularly important for reaching a diverse readership.

Research and Academic Writing:

Researchers and academics may use readability tests to assess the accessibility of their publications. Clear and concise writing enhances the dissemination of research findings to a wider audience.

Content Creation for Online Platforms:

Bloggers, content creators, and social media managers use readability scores to optimize their content for online platforms. Easily digestible content is more likely to engage and retain online audiences.

Language Learning Materials:

Developers of language learning materials use readability tests to ensure that instructional materials match the language proficiency levels of learners at various stages.

Hands-on

Let me walk you with the prompt template demonstrating the readability analysis using the Large Language Model (LLM). You are free to choose your preferred LLM.

You are a Readability Expert Analyst. Assess the reading level of the provided text content.

Response Rules:

Perform quantitative analysis using standard readability assessments based on the below formula.
Output the score between 0-100 along with grade level recommendation.

Assign a reading ease Class Label based on the score thresholds:

Here's the formula:

Score Range - Class - Label

Above 90 - 5th grade - Very easy
90-80 - 6th grade - Easy
80-70 - 7th grade - Fairly easy
70-60 - 8th & 9th grade - Plain English
60-50 - 10th to 12th grade - Fairly difficult
50-30 - College - Difficult
30-10 - College graduate - Very difficult
Below 10 - Professional - Extremely difficult

Here's the output format:

{
    "data": {
        "reading_ease": {
            "score": <<float>>,
            "level": <<float>>,
            "label": bool,
            "class": "",
            "class_label": ""
        },
        "stats": {
            "paragraphs": <<int>>,
            "sentences": <<int>>,
            "words": <<int>>,
            "characters": <<int>>,
            "reading_time": <<float>>,
            "speaking_time": <<float>>,
            "avg_word_length": <<float>>,
            "avg_sentence_length": <<float>>,
            "avg_paragraph_length": <<float>>
        }
    }
}

Here's the content:

{{content}}

The above prompt template consists of a "content" placeholder. You will have to replace with the content which you wish to analyze. Let's take an example.

Command is Cohere’s text generation LLM which powers conversational agents, summarization, copywriting, and similar use cases. It works through the following endpoints:

co.chat is used to build conversational agents, with or without RAG
co.generate is used for text generation that is non-conversational in nature
co.summarize is used for long-text summarization
Rerank is the fastest way to inject the intelligence of a language model into an existing search system. It can be accessed via the co.rerank endpoint.

Embed improves the accuracy of search, classification, clustering, and retrieval-augmented generation (RAG) results. It powers the Embed, Classify, and Cluster endpoints.

For the demonstration purpose, you will be guided with the prompt execution using the Lastmile AI. However, you are free to execute the prompt directly on ChatGPT or other LLMs. If you are a developer, you may programmatically execute the prompt or provide an API or a service to your customers.

Please follow the below steps to execute the readability prompt on LastMileAI platform.

1. Login to LastMileAI
2. Navigate to the Workbooks and create a new workbook.
3. Copy and paste the above instructions onto the cell.
4. Please make sure to add a parameter named 'content' and provide the content that you are interested in analyzing.

Now comes the interesting part of the Apyhub Readability Scores Documents platform response. Notice below the matching scores :)

Conclusion

Now that you have reached to the end. Assuming you have gone through the whole blog post and understood the internals on how to perform the readability analysis using the specific prompt template internally uses "Flesch–Kincaid" rules or algorithm, you know how easy it's to accomplish the readability analysis.

If you take a deeper look into the prompt template, there's nothing rocket science :) Just by spending some time on the concept, you will be able to reverse engineer things in no time.

@iamspathan and @nikoldimit from Apyhub might be wondering what's going on. It was nice interacting with them over the dev.to platform.