122 – Technology in Secondary Education

[00:00:00] Santiago: Hi, and welcome to the IDEMS podcast. I’m Santiago Borio formerly Impact Activation Fellow or Impact Activation Fellow on leave. And I am here with David Stern, one of the co funders of IDEMS. Hi, David.

[00:00:26] David: Hi, Santiago. I loved, how you didn’t know what to call yourself. On leave is an interesting way of framing it. Some of our listeners may remember, Santiago was our Education Lead, he was employee number one at IDEMS, and he then has occasionally got tempted back into teaching because he loves spending time with kids, although he is not so keen on administration within schools. Is that a fair way of framing it?

[00:00:55] Santiago: I wouldn’t say I like spending time with kids; I enjoy interacting with teenagers, adolescents on mathematical issues. And I love the classroom.

[00:01:09] David: Yeah, you miss the classroom when you’re not a teacher and you appreciate not being a teacher when you are a teacher.

[00:01:19] Santiago: That’s fair. And on our last proper episode, because by now our audience might have heard me in the New Year’s Eve episode.

[00:01:33] David: The festive specials.

[00:01:34] Santiago: And it was an interesting episode. But on our last episode we discussed that I was stopping my Impact Activation Fellowship because I was going into school. And of course, as well, I’d like to acknowledge that it was not just because of me missing the classroom, but also because my mental health needed a bit of a change.

And IDEMS has been hugely supportive with that, as always. Thank you to IDEMS for allowing that. And now I am on summer holidays, because I’m of course based in the Southern Hemisphere. And while most of our audience is probably feeling the cold I am nicely warm here in Argentina.

[00:02:20] David: Absolutely. And I think it is also this fact that there was a maternity leave at a very nice school that you’re aware of, which you knew of, which gave you that opportunity to be part of the school environment for a fixed length of time, which I think was also part of the attraction of going back into the classroom.

[00:02:39] Santiago: Yes, and one of the things that I needed for my mental health was routine and structure, and what job gives you more routine and structure than teaching?

[00:02:47] David: What job gives you less routine and structure than IDEMS?

[00:02:53] Santiago: Yeah, that’s a different issue.

On that last episode that we did we mentioned that I was coming back to teaching and we discussed a few aspects of exploring the potential for integrating technology in positive ways. And we focused mostly in two areas. One of them was one of my passions, STACK, and looking for ways to improve the learning of students through formative assessment, mainly through targeted feedback, which STACK specialises on.

And I started slow. I mentioned it a few times. I did a small demo and I finally got approval to run a very small pilot, which is great. So my summer holidays are not going to be so much summer holidays. They’re going to be a bit of planning and implementing new technologies for a school. Which is great because I don’t know what to do with myself when I have two months off with no plans.

[00:04:08] David: Absolutely good. It’s good to keep yourself busy. And just to sort of say what is, of course, the other side of this which has happened is what’s happening in Kenya, which you’ve been very involved in in the past. That team has now taken on a whole new group of interns and they’re building STACK into schools as well alongside PreText and potentially WebWork as well. And so you now have the parallel project in Kenya which is maybe not quite the same age group that you’re working with but it’s still exciting that we’ve got something we’re involved in which is also trying to see this through after a lot of work from yourself and others moving in that direction, preparing for it, and it’s now happening.

[00:04:51] Santiago: Yeah, and I’d like to find out more about that. Probably you should do an episode with Mike on that. Probably we should do an episode on PreText and WebWork, because these are technologies that we haven’t presented to our audience. And they’re very interesting technologies that can really have a positive impact in schooling and education.

[00:05:15] David: Webwork is in some ways an alternative to STACK, which doesn’t have quite the same setup behind it, but because it has Perl as a language behind it, it is extremely powerful in what it can do. Whereas PreText, very briefly, this is a very exciting textbook authoring tool, which is particularly interesting for us because it separates out the authoring from what you would call the production of the end product, or the publishing, I suppose, the publisher role.

[00:05:53] Santiago: As I understand it, it allows you to publish your textbook in different formats that make it more accessible to audiences that have different degrees of access to technology.

[00:06:09] David: Exactly. So this is what I was trying to say is by separating out the authoring from the publishing, it means that the same textbook can then be published in multiple ways. And so you’re able to take a single authored content and publish it on the web, publish it as printed material, publish it as a PDF, in all these different forms, so it’s adapted for different screens, for different ways of consuming it. It’s been very successful with a number of open textbooks in the US, but it’s only very recently that we’ve got to try and integrate this in.

I’ve been trying for years, but it’s only recently that we’ve been able to get partners to sort of engage in this because it is a steep learning curve to be able to take this new technology, this new way of authoring and publishing.

[00:07:01] Santiago: And I would like to go a bit more in depth on that on a separate episode. I want to focus a bit more on myself. Apologies. I’m very excited, my plan is to grab a few topics and author both instructional material and mastery material, two of our types of quizzes I believe we might have done an episode on, I can’t remember off the top of my head.

[00:07:33] David: The five quiz model.

[00:07:34] Santiago: There’s two types of quizzes out of five that we have conceptualised. We can interactively introduce materials introduce concepts, then present questions on that material and let students practice at their own pace repeating questions if they need, and of course questions are randomly generated so they can get multiple variants of the question with different parameters and different values, and that learning through mastery with targeted feedback, it can really be transformative for secondary education.

[00:08:14] David: Absolutely.

[00:08:16] Santiago: I’ll have to carefully select the topics, maybe identify some topics that my students struggle with a bit and see what can be done in that area. It’s going to be an interesting little project to pilot, and of course this school has links with other schools, and if it goes well it has the potential to scale.

And I’d like to thank IDEMS for already informally agreeing to support me on this, at least with the server and the integration to Google Classroom, which is what we use at the school.

[00:08:53] David: It’s really good to see for us, just in terms of the reciprocity of that collaboration, is that we’ve agreed to offer that sort of service in exchange for the fact that the questions you’re authoring will then be part of the open question bank and available to others. They will then potentially be used in Kenya, that’s where we’ve got colleagues who are building similar age group materials, but more generally just available then for anyone else wanting to build from this work.

[00:09:22] Santiago: And of course I want to bring a lot of these ideas to Argentina. So if we do manage to get IDEMS work in Argentina, as we’ve been discussing, and maybe we’ll do an episode on at some point in the future, these materials, as they will be published as Open Educational Resources, they could serve the local society as well, more widely.

If we have a smooth mechanism to integrate these into current technologies that are being used in state schools, there’s no reason why, after translating them, because my questions are going to be in English, it’s a bilingual school, so there will be a bit of additional work, but they could serve social needs here in Argentina, which is something that I find very attractive as well.

[00:10:15] David: Of course, absolutely. And we’ve already discussed the fact that not only is there the potential for some of these ideas to scale in Argentina and to bring that in, but there’s real opportunities to build a as you called it, an AIDEMS, which is an Argentinian IDEMS, which would then potentially build up the capabilities for maybe South to South collaborations and all sorts of other opportunities. It’s an exciting sort of time to be considering these different things.

[00:10:48] Santiago: Yes, and I’m already discussing with a number of people potential projects that could make that a reality. One of them is also related to technology in education, in school education. There’s school databases, which are huge. Our regular listeners will have heard you discuss with Lily about multilevel data. Of course, those databases have layers and layers of complexity. They tend to be very cumbersome and they tend to be very difficult to use. And there’s no, I’m not aware of any open source or even cheap front end that allows you to work with those databases.

And teachers don’t have the time to develop those skills in order to effectively use those databases. And those databases have all sorts of information, from students’ progress, students’ academic progress, to family contacts, to timetable, and being able to locate a particular student at a given time.

They have analytics potentially on individual classes, on all the classes of a particular teacher, all the results for the whole school for a particular subject, loads and loads of interesting things that can be drawn out of it. And that would be a really exciting project to do. And, again, almost universal need.

[00:12:21] David: Absolutely, and this is something which of course, I’ve got sucked into thinking about this in the past in many different contexts. It is a challenging one, and it fits in very well with some of our other work, because not only is the data really complex, but each school is different. And so you really want something which is highly localised, highly tailored, and yet at the same time standardised. It’s that combination of there’s certain things which are just the same for everyone and just need to be there. And there’s other things which is so particular that you need the ability to really localise.

And this comes back to the same challenge we’re finding again and again, where you need standardisation and localisation combined together in complex ways. It’s a problem I’m very interested in. It’s not one I’m taking lightly because it is hard and there’s good organisations who have spent a lifetime trying to crack this one, and it’s a hard one.

We’re mathematicians, we like hard problems, so the harder it is, the more interested I am. But it isn’t one which is just gonna work, we know that. But, as you say, the need is huge.

[00:13:36] Santiago: And it’s not gonna be a six month project, it’s gonna be a long term project, and it needs to be planned thoughtfully. But, what I am, you might completely disagree with me, but the work that we’ve done for the chatbots and for the app builder and so on, I can see how the thinking behind that can really help the customisation and contextualisation of such a system.

[00:14:09] David: No, absolutely. And let me be clear, the underlying fundamentally hard problem is the same underlying fundamentally hard problem that we keep on banging up against, where you need high levels of local customisation alongside expert guided standardisation in ways which interplay with complexity.

And that’s the same problem we keep banging up against again and again, that when you’re wanting initiatives, which aren’t just aiming to be profitable, but are aiming to be socially impactful, you keep hitting this same problem. That if you want to just standardise across every school in the country, it’s one thing. But if you want to enable schools to be able to suit their local context to meet their needs alongside good standardisation which enables you to sort of work across counties, countries, that’s where you get these problems.

[00:15:05] Santiago: Just to give an example. The school that we are discussing with, informally at the moment, their grading system for report cards, yes, report cards still exist and they get printed and sent to the families in paper still. The grading system is quite unusual. It’s a very holistic system whereby what you achieve in paper counts for one of four different grades. The other ones are more related to how you communicate, how you interact, how you participate, how independent you are in your learning.

So the grading system works in a unique way where a combination of all those categories give you your final grade. And you might perform brilliantly in all your assessments and examinations and so on, but you might work completely in isolation. But that’s not what we want to assess. We want to create people who can work collaboratively.

So maybe even if you’re hugely intelligent but need a bit more encouragement or reinforcement on social skills, that affects your grade as well. That is very different to any type of assessment that I’ve done in the UK and it’s very different to the assessments that I’ve done in other schools in Argentina. There are potential criticisms to that assessment system, of course, but there is no perfect assessment system. I like some of the ideas behind it and maybe we can do another episode on that at some point, let’s not get drawn into the Pandora box that I just opened there.

[00:16:58] David: But I think what I can say and what I will say very quickly is that it’s great to me that there is this work happening across these different ways of assessing. And I’m always delighted to find schools that are thinking differently, that are trying to think more holistically and investigating other ways to do it. The fact there are criticisms, of course, it’s hard. These are hard problems, but it’s great to hear about these innovations and these different ways of approaching it, which almost always add value once you sort of learn about them and use them in different ways.

[00:17:36] Santiago: Yeah well, let’s state the obvious criticism that I get often from parents, it’s hugely subjective, and that can lead to disagreements or conflicts. And I have had to meet with parents to explain how the system works and how I decided to give the grades that I gave. But, again, let’s not get drawn in too much into the detail of that.

[00:18:02] David: But this is, I think, a really important point. The sort of value of subjective versus objective measures of assessment. And when one is valuable and when the other is valuable. And what the assessment is being used for. And there’s a real difference between high stakes assessment, which have to be objective because they’re used by external groups in different ways, than ongoing continuous assessment in different ways.

[00:18:27] Santiago: Yeah, like all the standardised testing that is carried out all over the world in so many ways. It’s a novel, interesting system, not perfect but, integrating that or being able to allow for that level of defined distinct individual need of that school and other schools having completely different systems, even the neighboring school having a completely different system. So being able to align those in a single package that will be tough.

[00:19:05] David: Yeah, we don’t need to dig into this, but it is the sort of thing where there are ways to do this, and we are investigating them at the moment, not necessarily for a school based admin system, but in our work in the parenting apps, with a whole range of different components. I won’t spend too long on that.

[00:19:27] Santiago: No. We had another area of technology that we were hoping I would be able to explore a bit in schools, which is the responsible use of AI. AI has been hugely problematic, particularly on internal assessments for the IB diploma. Students using generative AI systems out there to create work.

I haven’t managed to dig into the issue too much because it’s beyond my let’s say, job description. I have observed. I think we agreed that there is a need for education on this because these are tools that are available that if the kids are using them, they need to know how to use them well, and they need to know what is appropriate and they need to know what is not appropriate.

They need to be able to identify, okay, this is my work and this is not my work. They need to know how to use it to, for example, help them with the referencing. If you do your work and you have your sources and you write your sources, putting it through a generative AI to analyse where you’re using the sources helping you reference them properly. That’s a fantastic tool.

And again, it has to be your work, but I have seen some assessments, some essays that were pretty much completely generated by ChatGPT or the like. And I believe quite strongly that it comes down to how we teach the kids on how to use the tools.

[00:21:22] David: Yes and no, of course. Part of the problem, and there’s evidence of this, there’s this wonderful study from Reading, which we keep referencing, where Reading University submitted AI generated scripts to first, second and third year exams and the AI generated scripts outperformed students on the first and second year exams on average.

Now, it’s one thing to actually help students use AI, but the incentives are all wrong. If the assessment can be performed better by AI than by the student, then it’s not just about teaching the student, it’s about changing the assessment. Now this is much harder in the context where you have a standardised assessment for something like the IB.

But the truth is that there is an onus then on them, as well as an onus on the teachers to be able to teach children how to use these systems well. There is also an onus on actually making sure that the incentives are right with the assessments that are given.

[00:22:33] Santiago: Of course, you particularly discussed this with Lily multiple times in episodes.

And the other thing, what is being done in school and in examination boards is trying to use AI detectors. And of course, we have students that have been hard done by that. They’ve done a lot of work, they put a lot of effort into writing an essay that the machine just believes is AI generated.

[00:23:09] David: And this is something where we’ve done another episode with Lily on this. This is a big problem, and there are demonstrated cases where things happen which anyone who knows about how AI works knows will happen. There will be false positives and false negatives, and you need to be able to think about how you’re going to deal with them.

If you just use a tool like that and assume it is correct, it’s not going to work. There are people who slip through the net, and there’s people who will be falsely accused.

[00:23:45] Santiago: And it’s quite a serious issue to be falsely accused of academic dishonesty.

[00:23:50] David: It can totally change your life direction, this is serious stuff. It isn’t something where the tools are going to get better at detecting it. No. The tools are going to get better at not being detected as well. And so, you’ll get more false negatives. If you learn how to use the tools well, you can have a higher chance of being a false negative. If you’re not using the tool, then you might actually end up having a higher and higher chance of actually getting caught up in being a false positive.

These are serious issues. And we keep coming back to saying in many episodes that in education, the answer is not to be focusing on policing it. If you have an assessment where you need to police it for the answer, you probably need to rethink that assessment.

[00:24:39] Santiago: Yes, which of course is a lot more difficult with standardised testing and standardised assessments. Examination boards, international examination boards, they have quite a hard job. Because writing assessments for school level that are of that kind, that cannot be produced by generative AI, it’s quite a difficult thing to do.

[00:25:04] David: It might be impossible for the exam boards at this point in time to get assessment which is at the right level for the students while also not being able to be sort of generated using AI, it might simply not be possible.

[00:25:22] Santiago: Yeah.

[00:25:24] David: And so seeing it as an assessment board problem, I fear for the assessment boards on this. I wouldn’t like to be worrying about this from their side, but that’s different from thinking of it from the school side, and that’s where you’re really at at the moment. And I think the school side, as you say, there’s a lot that can be done about thinking about how to responsibly build the skills of the students to use AI in responsible ways while keeping academic integrity.

[00:25:53] Santiago: Yes, of course. It is something that can improve, should improve, and hopefully will improve with the right thoughtful approach behind it. That’s why I like so much our Responsible AI approaches. But, again, it’s a different context. So I haven’t yet ironed out how to contextualise that to the secondary school education system.

[00:26:22] David: And it’s hard. It’s one of the reasons why, if you remember, when we started working on Responsible AI for education, we got approached by schools as well as universities and I was so much more comfortable working with the universities. The schools problem is hard, it’s harder even than the higher education problem in my mind. But it’s something which I’m glad there are people engaging with. It’s just a really hard problem.

[00:26:50] Santiago: Yeah, and I will keep discussing with management at school on my experiences and my ideas, but I’m very excited that I will be able to bring STACK into the classroom at secondary level, finally, after however many years of wanting to find a way of doing so. All I had to do was get into a school.

[00:27:14] David: Exactly. And here you are I should warn you, the maternity leaves only six months more as far as I understand. So you’ve only got six months in the school to enjoy doing that with the classroom experience itself. But you can carry on afterwards because this is certainly something that I look forward to this being part of what IDEMS does going forward.

[00:27:34] Santiago: Indeed. And other opportunities as well. More experiences bring more opportunities and we observe, as you discussed with Danny in a really interesting episode, how observation is valuable and we observe and we identify needs and those needs tend to cut across different layers of society. It’s exciting times.

[00:27:57] David: Absolutely. And what I’m really happy with is that I’ve always felt that your big role within IDEMS is to be going beyond being an individual educator and working on education more broadly. And to be able to now go back into the classroom, to be able to see this, but with the eyes of someone thinking beyond, it’s great to have this discussion now, and it’s great to see how that’s stimulating your thinking. And you’re gaining experiences, which would be invaluable when you come back out, thinking about applying this a bit more broadly.

[00:28:34] Santiago: Yeah. Maybe in six months time, we’ll have a report on this.

[00:28:41] David: I look forward to it.

[00:28:43] Santiago: Thank you, David.

[00:28:45] David: Thank you.