Associate Professor Robert Moll and collaborators at the Center for Educational Software Development (formerly CCBIT) have come up with a revised model for online textbooks. The model promises to bring new vitality to computer science education. Called an Evaluator-Integrated Book, or EIB, a textbook in this mold has all of the traditional features of web-based learning, but with one special addition: the book includes exercises that use a real-time, web-based evaluator that scores and checks exercises for correctness.
The most successful EIB so far is an elementary computer programming textbook in Java, called iJava. With this book, students read a few paragraphs, and then answer one or more easy questions, mostly programming, that help them to consolidate reading and thinking in a hands-on way. These learning-by-doing activities are “submitted”—passed to a server-based evaluator—and the server responds with a correctness judgment about the submission in real-time. For incorrect submissions, iJava provides a variety of diagnostics to the student. “The iJava volume, with its 140 embedded exercises, has dramatically reduced the failure rate in our classes over four trial semesters (from a 30-40% final exam failure rate to a 6-12% rate),” says Moll.
A second EIB has also been developed in physical chemistry, which was used for the first time in Fall 2007. Both of these web volumes rely heavily on the OWL (Online Web-based Learning) electronic homework system. This system, with automated grading, was originally developed as a joint collaboration between the UMass Amherst Computer Science and Chemistry Departments in the 1990s, with particular help from then Computer Science chair David Stemple. OWL now has a commercial presence in Chemistry and Computer Science, and OWL-Chemistry is one of the nation’s premiere web-based homework systems, with nearly 100,000 student subscribers.
EIBs represent a tightly integrated marriage between OWL and an online instructional narrative. In addition to the evaluated exercises inside the iJava text, iJava also makes use of the elaborate CS-OWL homework system that was developed several years ago with significant support from the National Science Foundation. Thus, students in an iJava-based course do easy exercises that are embedded in the text narrative, and also do a harder suite of problems, which are separate from the text and which are drawn from the CS-OWL standard homework corpus.
Why is iJava successful? Moll believes that one traditional problem in the sciences is reading. “Students are often lousy readers of technical material,” he says. “They do not know how or when to pay careful attention to a technical narrative. The iJava text is analogous to training wheels for a weak reader.” By providing an easy question every few paragraphs, iJava helps students consolidate concepts while giving hands-on experience with the nuts and bolts of Java programming.
iJava has proved to be successful in other settings as well. It seems to be ideal for distance learning. “Learning Java in isolation is not easy, but the consistent drill of questions in the online book makes a remote class far less daunting,” adds Moll. So far iJava-based distance learning classes have been offered five times, and student success rates are running at about 80%. Moll is also looking into ways of repackaging iJava so that it is appropriate for high school advanced placement (AP) classes.
Moll and graduate student Gordon Anderson have begun to exploit another aspect of EIBs. All interactions between students, the online book, and the associated homework system are logged in a database. With this database in hand, it is possible to analyze in some detail the study habits of students. Anderson has come up with one somewhat surprising early finding, says Moll. It turns out that while most students work through the book material first, and then proceed to the associated OWL homework, a surprising number work backwards—they try the homework first, and then return to the book and its material only later when they find that they have run into trouble. Which is a better method? Anderson’s preliminary finding is that among students who enter an iJava-based course with no previous programming experience, students who work backwards do dramatically worse on exams than forward workers do.
Moll first joined the Computer Science faculty at UMass Amherst in 1973. He holds a B.S. and M.S. in Mathematics from Carnegie Mellon University, and a Ph.D. in Mathematics from the Massachusetts Institute of Technology. Moll is also the department’s Associate Chair for Academic Programs. In addition to the iJava volume, he has collaborated on five paper textbooks on programming and theoretical computer science, and he has co-authored two children’s books. The first three chapters of iJava are online, at: owl.cs.umass.edu/partners/iJava/v1-3.html.