Traffic light warning system used at Purdue University
This post looks at some of the material available on the Educause web site relating to the use of what they call academic analytics (and we seem to be calling activity data) to support student success.
What Educause http://www.educause.edu/ calls academic analytics is very similar to what we are calling activity data (though clearly the focus is different, as with activity data the focus is on the data while with the analytics it is on the tools, presentation and use, so I guess that I prefer the analytics term). In one report (Academic Analytics: The Uses of Management Information and Technology in Higher Education) they say that academic analytics “describe[s] the intersection of technology, information, management culture, and the application of information to manage the academic enterprise.”
Anyhow over the last few years Educause has produced some very useful material, most of which is available from their Academic Analytics page http://www.educause.edu/Resources/Browse/Academic%20Analytics/16930
Here I will pick out some of the things that might be of interest to you.
7 Things You Should Know About Analytics http://www.educause.edu/ir/library/pdf/ELI7059.pdf Educause produce a series of reports entitles 7 things you should know about x. These are very brief about two sides and include a story / case study, definition and some of the key issues. They can be very useful introduction to those who do not already know about what you are doing, and come from an independent authoritative source.
2011 Horizon Report
The Horizon report is produced annually by Educause and looks at technologies that are going to have an impact over the next year, two-three years and four to five years. Not all technologies make it from important in four to five years to important now (the joys of futurology).
Its part based on survey and part expert discussion and provides a very broad brush overview of the technology. This year one of the areas that they have picked out for the four to five year time frame is learning analytics. Discussed on pp28-30. It provides a two page overview and some examples and further reading.
Signals: Applying Academic Analytics http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/SignalsApplyingAcademicAnalyti/199385 or http://bit.ly/c5Z5Zu This is a fascinating case study from Purdue University, where they say that the use of analytics has improved results, and led those in greatest danger of failing to switch courses earlier. They ran the trial using a control group, so there results have some validity, and courses were sufficiently large for the results to be meaningful.Their statistics include
“Over the succeeding weeks, 55 percent of the students in the red category moved into the moderate risk group (in this case, represented by a C), 24.4 percent actually moved from the red to the green group (in this case, an A or B), and 10.6 percent of the students initially placed in the red group remained there. In the yellow group, 69 percent rose to the green level, while 31 percent stayed in the yellow group”
Although they don’t say how the outcome compares with the control group.
Academic Analytics: The Uses of Management Information and Technology in Higher Education http://www.educause.edu/ers0508, a book discussing analytics in HE. Dated 2005 it still has interesting stuff in it.
Among the things to note is the sources of data people are (were) using in their analytics:
Table 6-3. Information Contained in Data Stores or Warehouses (N = 213)
Student information system
Course management system
Ancillary systems (e.g., housing)
Comparative peer data
Feeder institutions (high schools)
Note that there is no mention of Library systems of any type and the strong emphasis on administrative systems rather than academic systems.
Presentation on analytics: Academic Analytics: A New Tool for a New Era http://www.educause.edu/Resources/AcademicAnalyticsANewToolforaN/162057 the slides themselves are a bit thin, but No 19 is interesting
Results to Date
Typically 10-20% of students receive a message
- Most remained “at risk”
- Still unlikely to take advantage of resources
- Majority were able to leave the “at risk” status
- More likely to take advantage of resources
How the ICCOC Uses Analytics to Increase Student Success
Case study of the use of analytics at Iowa Community College Online Consortium to improve student retention at . http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/HowtheICCOCUsesAnalyticstoIncr/219112 or http://bit.ly/m2HgnZ. Over the period 2005-9 they increased the student success rate from 77% to 85%. What is not clear is how much of the improvement relates to the analytics and how much derives from other work to improve student success.
There is much more and I will post some other summaries of articles that I think will be useful later.