Have you ever wanted to analyze your class discussion forums? You are not alone.
Faculty, staff, and graduate students collaborating in Northwestern’s Experimental Teaching and Learning Analytics at Northwestern (ETLAN) workgroup have been exploring their Canvas discussion forums using Natural Language Processing (NLP) and visualization techniques to gain a broader perspective on forum content and dynamics; the group has also been considering how best to prompt, frame, and generally conduct online discussions to increase student learning.
This new app, developed by Jacob Collins and Philip Burns of Northwestern IT Services and Support's Academic Software Development team, is called Discussion Analytics. It is now available at Northwestern Learning Apps for easy installation in any Canvas course. A special feature of this Northwestern tool for analytics is its inclusion of Natural Language Processing (NLP) results. The complete list of exported fields are shown below, with NLP fields in italics.
This data can be visualized and analyzed. The Discussion Analytics app includes one such visualization: a color-coded word cloud that presents keywords in black and named entities such as people in yellow, places in green, and organizations in purple. The size of the terms in the cloud is determined by their frequency—the terms that appear more often in the discussions show in enlarged font in the visualization (left image below).
Discussion Analytics word cloud and posts containing selected keywords
Clicking on any of the words in the cloud will then display its originating posts (right image above). Other visualization techniques can also be applied to these data.
A couple of examples of how these data have been used for discussion forum analysis and visualization are provided below.
Discussion Board Analytics: Student Use of Keywords and Named Entities
The School of Professional Studies Master of Science in Global Health (MSGH) program used this app to obtain place names mentioned in forum discussion to create a world map of geographic coverage (below).
“In addition to cool data visualizations like the map, first and foremost, the data was really important to get a high-level picture of what students were discussing in the program. The program also provides a glossary of key terms that faculty intend students to adopt and apply throughout their course work. The app was instrumental in confirming that the students were indeed applying the key terms that we want them to learn throughout the program and not just the specific class in which it’s most relevant,” says Jacqueline of their use of the tool.
Countries mentioned in Foundations of Global Health (MSGH 405) Fall15
Online / On-Campus: Analytics Perspective on Medill IMC Online and On-Campus Course Engagement and Outcomes
The Medill School of Journalism, Media, Integrated Marketing and Communications program used the Discussion Analytics tool to measure and compare student engagement behaviors in the same courses that are taught both completely online and face-to-face on campus. While the data set was too small to draw any definitive conclusions, it did show that the students in the courses with the highest grades were the ones who spent the most time on the discussion boards. See Tom Collinger’s (Medill) 2016 Best Practices Forum presentation: Medill’s Learning So Far From Teaching Online – Experiences, Stories and Analytics for details.
IBM Watson plot of forum metric against grade and course evaluation
Discussion Analytics is available now from the Northwestern Learning Apps site for use in any Northwestern Canvas course. For other institutions interested in using this tool with Canvas, the software is available for download at the Northwestern Information Technology, Academic & Research Technologies Software Development Group Project Site.