Now that Magic School has a way for teachers to create chatbots for students like School AI I would love to choose the better one and use that one in the coming school year but I can’t. They are both that good!
In the last several weeks of school, of the 23-24 school year, I created a bunch of Math and Science School AI spaces (chatbots) and Magic Student chatbots. Students had the option to use them to ask questions, get ideas, and check for understanding. Some students used them. I gave students the option for each chatbot experience to choose from School AI or Magic Student and if they could, have someone at their table try a different one so they could compare. There wasn’t a lot of comparing and of the students who used them both experiences seemed rather even to me!
Both School AI and Magic Student provide the teacher with a summary of what students discussed with the chatbot along with a full history of each student’s conversation in a teacher dashboard. Here are two screenshots one of the School AI teacher dashboard of the chatbot space and the other of the Magic School Student chatbot session teacher dashboard.
Both School AI’s spaces and Magic Student chatbot sessions are powerful and can be used in so many ways that I still cannot decide to use one over the other! So I ran both bots through the tests I’ve conducted on other bots this past school year and here are the results as I tried them both side by side.
Here is a School AI spaces chatbot I made to help students write 9-11 articles for a class book we were making with Book Creator:
The School AI chatbot space was pretty helpful and I could have taken it farther to help me re-write and edit the student article that I used. Now here’s Magic School:
Magic School gave the student ideas right off the bat. I stopped my conversation there because I could have used the suggestions to make my article better. I think School AI was more helpful because it guided me by asking questions and only gave me ideas once I got stuck and asked it directly to provide me with more facts.
Then I tried the 5th grade Math problem from the 5th grade Bridges curriculum in Unit 1 on School AI:
This one was strange because I had tried that exact Math problem with this exact same School AI space, as I shared here, and it worked much better than the above interaction and in that example it did not answer the question for the student! This shows how AI can hallucinate and epically fail once in a while!
Now here’s the same Math problem on Magic School:
Note here that the chatbot did the Math for me just like the School AI chatbot did and I explicitly instructed the chatbots to NOT answer the questions for the students!!
THIS! That explanation is what I was expecting and asking my chatbots to do – to give students strategies OTHER than the standard algorithm for multiplying with double digit numbers. Magic School won this one. I instructed my students to ask the chatbot how it got the answer in case it just gave them the answer because I wanted students to use the chatbots more like a tutor than a calculator!!
Now here’s the School AI chatbot I created for my 6th graders with a typical 6th grade Math problem (it says 5th in the title but that should read 6th – my bad).
This one also failed really badly. I will make sure to create new spaces next year and start from scratch because I do not want to see more of the above! Comparing the above conversation to the last time I tested that chatbot also shows how much better the chatbot did last time I asked how to figure out 25% of 60 because it’s easier to divide 60 by 4 than it is to multiply 60 by .25!
Now here’s Magic School with the same Math problem:
Here Magic School’s chatbot did all the work for the student. That is not as powerful a tool as one that asks and guides so I need to do a much better job at prompting these chatbots for this new upcoming school year.
All in all I just can’t choose between them as both provide experiences I am looking to offer my students. Choice is a powerful motivator so I will continue to offer my students the choice between School AI and Magic Student.
