Can AI turn any book into a textbook?

Back in 2014 in the early days of the BC open textbook project I did some research on what makes a textbook a textbook. What are the pedagogical aides in a textbook that make it different than, say, a reference book or a non-fiction book, and what aides are the most helpful for students.

According to the research I did at that time, boldface terms, italicized terms, practice questions, and chapter summary & reviews were the top pedagogical aides that students both used and found most useful in a textbook.

Now, these are not the only things that make a book a textbook. But by including things like bold and italicized terms, chapter summaries, and practice questions you have the tools to begin to turn any book into something that has a similar pedagogical usefulness as a textbook.

I was thinking about this research today while I was reading Ethan Mollick’s blog post What Happens When AI Reads a Book. In his post Mollick asks the question, “So might AI, the technology of the moment, change the way we interact with books?” and then begins to experiment with a generative AI tool called Claude, which is built using GPT 3.5. Why Claude and not ChatGPT? Because Claude is big enough that it can consume an entire book length prompt, which Mollick proceeds to do feeding it a short (29,868 word) book he has authored.

Once he fed his book into Claude, he then proceeds to ask Claude questions about the book. At first these are along the lines of asking for Claude to make editorial suggestions on grammar and structure, but towards the middle of the article Mollick turns his attention to pedagogical applications that begin to resemble the kinds of structures that students find important when working with textbooks.

First, he ask Claude to provide summaries of the key points of the book and finds that Claude does a very good job of summarizing the key points of the book.

Every author knows, and dreads, what most people want AI to do with books – summarize them. They want to distill down pages of thoughtful prose, carefully considered language, and specific phrases into a few pithy points. So, can AI do this? Yes. And surprisingly well. Further, it has enough “sense” of context that you can ask for expansions – tell me the examples and research the book gives to support each point. And it does that, too.

So we have an AI engine that creates very good summaries of a book – the same type of summaries that most textbooks have and that most students use to help them understand content. He then went further to tweak his prompts for summaries to take into account different levels of learners.

Explain the main themes of the book to me at four different levels: first grader, 8th grader, college student, PhD student

Mollick found that this prompt resulted in “good summaries” showing that with a bit of prompt finessing someone could theoretically create summaries that are at different levels of understanding. I say theoretically because here Mollick is relying on Claude to unpack what it means to be a “first grader”, which can be something very different depending on the education system or cultural context you are working in. Still, even with that caveat, it does hint at the role generative AI could have in helping to present the same information to different audiences at a level that makes more sense to them.

Mollick’s next experiment had Claude create sets of questions based on the contents of the book. Remember that practice questions are one of the top pedagogical aides in a textbook. Here Mollick noted that Claude did a good job, but did make a factual mistake that Mollick caught and corrected with additional prompts until he arrived at the correct response. You still need a knowledgeable person to oversee AI output to ensure it is accurate.

Mollick then asked Claude to create case studies based on the book that would require students to use knowledge from the book to complete, and to create a teachers guide to accompany the case studies. According to Mollick, Claude created “an interesting in-class exercise that I could see using.”

With Mollick’s examples you can begin to see how AI can be used to help augment any book to have the kinds of pedagogical tools you generally find in a standard textbook.