“Education is the most powerful weapon which you can use to change the world.”
According to UNESCO, 1 billion children have seen their schools close. Historically, even during war, famine and extreme storms, schools have remained open. As a consequence of the pandemic, young people are relying heavily upon technology to enable remote access to high quality learning. It has accelerated the revolution that was already taking place in EdTech. At Infinite Lambda we have had the privilege of playing a part in this global challenge. Helping one of our US partners Innive Inc, an EdTech organisation working with millions of students, we have contributed to their mission to transform education. In this article, we take a deeper dive into the technical challenges and opportunities that our team discovered and what this could mean for individuals and institutions all over the world.
But first…we caught up with Gautham Sampath, Co-Founder and CEO of Innive Inc, to find out more about their company, their mission and their product.
Hold on to your pencil cases…
Gautham Sampath ITIL,
CEO at Innive Inc
“The COVID-19 pandemic is overwhelming the functioning of education systems worldwide. School districts in the US are now increasingly seeing the use of technology in education as a lifeline. How do you ensure that students have access to high-quality and equitable education? How can schools plan for the upcoming academic year and deliver quality instruction? One answer is to analyze data from new and existing sources. To do that, they have to think about the potential of data and analytics for informed planning and decision-making.”
“To assist school districts in the US, Innive launched K12 360⁰ Remote Learning and Classroom Analytics. This helps teachers, schools and districts understand the technology needed for their different learning models. It assesses what is available, what is required, measures usage and assesses effectiveness of the program by measuring student outcomes. Teachers can measure and monitor student engagement. Districts can track trends over time, empowering all stakeholders to make informed decisions and plan for this unprecedented school year.”
Thank you Gautham!
Now let’s turn our attentions to the team at Infinite Lambda and find out a little more about the project. Over to you Nina…
“We needed to help schools to migrate their classes from real life to online. At a time of a pandemic, that is an immensely important challenge, and a technically very difficult one. It’s been so exciting to work on a project with such big potential for a hugely positive impact. As a company, learning is at our core, and working with Innive has been a fantastic coming together of shared values.”
Nina Anderson / Project Lead & BI Engineer
Schools have had to rapidly adjust to life online. But how do you manage the vast amounts of data that make up a holistic education for a child?
There are so many sources of data, and what is available to the schools will be determined by their resources. Whether it is Google Classroom or Canvas or Zoom and so on; what classes a school offers, which teachers, students, lessons, dates of terms, syllabus, all matters and needs integrating. That way, BI tools at the front end can become useful to governments, school districts, teachers and individual students. They can become automated and personalised to each user account, thereby giving confidence to the end user.
Working closely with our Innive, we have created a blueprint for an integrated learning platform. They are now rolling this out to school districts in the US.
Our solution allows comparison across students, teachers, schools, and districts of how their remote learning programmes are going. Even if different schools are using different technologies to deliver the teaching, it enables platform-agnostic engagement and performance metrics.
Whilst the pandemic has devastated the learning opportunities for so many young people, this project has helped enable millions of children to have access to high quality education. It has given teachers and school districts the ability to learn more about each child, and curate their learning experiences accordingly.
We caught up with Nikolay Vaklinov to find out more behind the headlines from a DevOps perspective.
“In order to access the school district data we had to go through a VPN and then RDP to a Windows VM. This just wasn’t a scalable or efficient solution.”
“Airflow on Cloud Composer didn’t work – the limitations were not compatible with what we needed in terms of python packages and dependencies.”
“We did a direct VPN connection to Google Cloud using a VPC. We tried other solutions such as using Google Cloud VPN, but this didn’t work due to not being able to mask the traffic correctly. Therefore we configured our own instance from scratch.”
“We built our own Airflow on Kubernetes Engine and configured it all from scratch with terraform and helm. We then automated the deployment to AirFlow with GitLab CI/CD.”
“With one command you can spin up the whole infrastructure including the VPN.”
Nikolay Vaklinov, DevOps Engineer
With heads spinning and excitement levels increasing (thanks Nikolay and Nina!), we thought we better give Boris Gaganelov, Data Engineer at Infinite Lambda a chance to throw his hat into the innovation ring.
“Ingesting data from multiple different Education Technologies, ensuring that in most cases they were all in sync with the master education system. A massive limitation was having to ingest the data via an API that is not built for large data exchange.”
“We created an optimised multi-platform flow, which ingested data into Big Query, allowing our database folks to work on the cool stuff and transform the data into a form that is useful to our clients. We also set up flexible synchronization scripts, allowing us to push up-to-date data from the master system to other platforms.”
“The sync from the master education system to Big Query to Google Classroom is so cool.”
Boris Gaganelov, Data Engineer
Every project is an opportunity to learn more, to refine our skills and be better than yesterday. Our Llama Learning is here to pick one thing that we feel has been particularly transformative. We went back to Nina Anderson, and also Zoltán Csonka, Data Engineer at Infinite Lambda.
“As an organisation we’ve been working on ways to automate and standardise our approach to data warehousing and modelling, using a combination of internally developed tools and open-source technology adapted for our purposes. The goals of this approach are twofold: to consistently apply best practices, and to leave ourselves with more time to solve the hard questions. In this case, we would like to highlight DBTVault, a tool for creating automated data vault processes. DBTVault official BigQuery support is under development right now, but we didn’t have time to wait for it, so we ported it. DBTVault code gave us ideas of how to automate a data lake. We therefore generated a data lake, but not with the DBTVault package.”
“In a Project Management capacity, leading a big team with such a variety of expertise has been enlightening. Helping them to prioritise and get to the heart of issues without trying to understand every detail was key for me.”
“I always wanted to certify for Google Cloud and as a result of this project I felt more ready for this. I have recently certified!”
“It was awesome to be working with a massive Ed Tech platform, working with Google, Canvas and more. This is the tech I really want to be working with. The client and the whole team involved was so receptive and keen to innovate.”
Thanks to our partner, our team and all of our technology partners who make all this possible.
If you want to find out more about what we do, life at Infinite Lambda, or how we could help you, your company, your government, your planet or your universe… just drop us a line and we can engineer the data in your world.