A new approach to Evaluating Digital Learning for Govt Schools

How to evaluate a digital learning for govt schools

If you are a part of a CSR team or belong to an NGO who is either already working or is thinking of introducing digital learning for their target students in government schools, this blog is a must read. We reflect from our experience of interacting with 100+ CSR professionals and heads of prominent NGOs across the country and share what should be the first step in evaluating a digital learning project in a government school.

This is not a very long post and hence won’t take much of your time but a deeper understanding of the thought we are going to build here will go a long way in ensuring success of your investments in digital learning.

For any learning project that is designed, set-up and implemented in a government school, the only way funders evaluate or rather want to evaluate is by knowing how much has the learning of the beneficiaries (which is most cases are the children) improved. And the consideration starts at the discussion table itself. If as an NGO, you can demonstrate the success of your intervention in improving say for example math scores or may be improvement in spoken English, there are more chances of your project getting a go ahead, else it’s a no show.

More so because sometimes the ecosystem expects the outcomes to start reflecting within days or weeks of the intervention.

But with experience, we all are going deeper and understanding the multi-dimensional value and impact that Student Centric Digital Education brings to CSR and therefore evaluate the same looking at the a larger horizon.

Till a few years back, digital learning was just another word for e-learning, although it may not be the most appropriate one. Smart classes with multimedia content were set up in government schools in plenty. The objective was to drive student engagement by showing them animated content and then hope to improve their learning outcomes in various subjects. We have already written blogs earlier on efficacy of smart classes for government schools, so will not delve on it again, but overall smart classes have faced multiple challenges in terms of regular usage, maintenance etc.

There hasn’t been many on-ground case studies that have established any substantial connection between smart classes and learning outcome improvements. There is even not much research material available to showcase any improvement in student engagement in the schools.

With Government of India’s strong push for digital literacy, digital learning has found a new meaning. It has made the idea citizen centric or to put it simply more user centric. From the perspective of a government school, the next wave of technology evolution would be to focus on building student centric digital learning solutions i.e. solutions which transfer the technology from the hands of a teacher into the inquisitive and fearless hands of a child. We wanted to talk about smart classes earlier just to bring out this distinction. Multiple implementation agencies around the country have now started setting up student centric digital learning interventions in various formats. Educational tablets based learning projects is indeed the new wave. What we now need to understand is, how do we evaluate the effectiveness of such interventions and optimise the social ROI.

Many of you might already be experimenting with tablets for government schools with a combination of multimedia videos and a host of play way apps. Before we move to the evaluation part, please consider the numbers below. It will set the context of what we are going to talk about next.

As per 71st NSSO survey on Education 2014, 94% rural households are expected to be digitally illiterate. Till 2019, government has a mandate to reach out to 6 crore individuals and make them digitally literate. This would essentially translate to these individuals being able to send/receive emails, access and use useful apps and do simple bank transactions online. Also, smartphone penetration currently is at 30%, although this number is increasing at a very rapid pace and by 2030, in fact it is estimated that all Indians will have a smartphone and an internet connection.

So if we look at the above numbers and the current scenario in India, it is safe to conclude that a lot of rural individuals and families have not yet seen a touch based device. Considering even if they have, they don’t really understand the capability of this device to transform their and their children’s future by giving them access of the best of digital learning tools available at their fingertips. In most cases, it is used as a medium of entertainment. And with smartphone and internet penetration happening at such a blazing speed, it will be good to establish the worth of these devices as learning devices as well – an idea which can bring about the real transformation in rural areas.

Therefore, while setting up a digital learning project for a government school, adoption by the users is the first yardstick which should be evaluated. The real adoption and skill development happens when the users first happily adopts digital for a purpose, else there are chances of it falling off like a training program or a one time propaganda. Since, government is also emphasising a lot on Digital Saksharta and Digital Literacy Mission, they are also realising that Digital is a means and not an end itself.

Therefore proactive, happy and natural adoption is the first biggest step and impact. Before we confuse you, by adoption we refer to a user becoming comfortable with operating the device, learning to switch it ON/OFF, being able to navigate through the platform, understanding how to use apps and other features on the device.

It has been observed that with the students in government schools, typically complete adoption takes about 3 to 4 months. During these times, children are very edgy moving from one content piece to another spending very little time on any one particular video or a digital book. They are experimenting, there is a lot of initial excitement, curiosity and it makes them very hyper.

We therefore do not see a very structured path which a child takes, nor do we see any structured approach to learning. Should we then consider changing the approach to a more guided one rather than student centric one?

Imagine a 10 year old kid from a rural village who may be seeing a tablet for the first time in his/her life OR even if he/she has seen the device before, they are getting it in their hands for the first time. Imagine the kind of rush that kid will be feeling. Rather than curbing it, the right approach must be to encourage that feeling. Eventually, you will realise that it will lead to natural and proactive adoption. Once that happens, the kid will settle down happily and start to proactively use the learning content with genuine interest in learning.

A device in the hands of the child allows them to use it as per their comfort, learning levels and ability to comprehend. Personalised experience leads to much better engagement and a lot of fun. A child starts connecting with the device and with the content it has.  Regular usage becomes a habit. And that’s when over a period of time, you will start seeing an improvement in learning outcomes – please remember it will still be personalised for every child but nonetheless it will be.

The best thing about student centric digital projects is that after good adoption happens, you get student wise usage and impact data on the devices itself, thereby delivering real time impact information & genuine growth outcomes.

If you are familiar with the concept of theory of change, then you must define the same in two parts for your digital learning projects. We give an indicative blueprint below. You can either use the same or customise as per your project specifications. Here goes:

Theory of Change – Part I: To familiarise students with the idea of using touch based devices for learning purposes. Students will be equipped to operate the device, navigate through the platform and understand different content types. Students will be able to regularly use the devices without any external help.

Theory of Change – Part II: Students will be using the devices to indulge in fun based and engaging learning through various content categories. Students will practise and strengthen their concepts and show can improvement in their learning outcomes.

And as a last step, while you are defining your theory of change, it is imperative to mention the timelines as well. Too long a timeline will make the project unviable and too short a timeline will put a lot of burden on your existing resources at hand and even then you may not get the desired outcome. Using technology and learning are both very natural ideas and should happen that way. You may guide but start influencing it too much and it will all fall apart.

To summarise, to evaluate projects in digital learning for govt schools, improvement in learning outcomes should not be looked at as a first step. Rather the first step can simply be adoption. Understand if your students have started to proactively and happily use the devices regularly. Digital Learning is a long journey but the one that must be taken to transform our government school learning. Good thing is that once you pass the initial phase with patience, results can be fast to come by and multi dimensional. Un imaginal impact and usage of the education recieved happens by the students, once they have proactively and happily adopted your project.

Please do share your experiences of setting up digital learning projects in the comments below. We would love to learn on what worked for you and what did not.

Together we will be able to build a deeper understanding on the most suited digital education our government schools need to transform themselves!

iDream Education

Leave a Reply

Your email address will not be published. Required fields are marked *