The following is the recommendations that we have submitted to JISC. Your comments would be most welcome by both JISC and us..
This is an informal report outlining the likely recommendations from the Activity Data projects to help JISC to determine future work in the area. This is not intended as a public document, rather to stimulate discussion and lead to a more formal document at a later stage.
There are two things to note at this stage.
- Activity data can serve a wide variety of different functions as exemplified by the range of projects in this programme. However the greatest impact (and return on investment) will be from supporting student success.
- We suggest that the next call explicitly funds other universities to pick up of the techniques and / or software systems that have been developed in this programme in order to see if they are useful beyond the initial institution, and in this process, discover what the issues may be to make effective use of the techniques and / or systems. However, this may not be in accordance with JISC’s standard practice and is not an essential part of the recommendations.
The recommendations appear under the following topic areas:
- Student success
- Student and researcher experience
- Collection management.
“It is a truth universally acknowledged that” early identification of students at risk and timely intervention must lead to greater success. It is believed that some of the patterns of behaviour that can be identified through activity data will indicate students who are at risk and could be supported by early intervention. It has also been demonstrated in work in the US that it can help students in the middle to improve their grades.
- In year 2, JISC should fund research into what is needed to build effective student success dashboards
Work is needed at least in the following areas:
- Determination of the most useful sources of data that can underpin the analytics
- Identification of effective and sub-optimal study patterns that can be found from the above data.
- Design and development of appropriate algorithms to extract this data. We advise that this should include statisticians with experience in relevant areas such as recommender systems.
- Watching what others are doing including in the areas of learning analytics, including VLE developer activity developments.
At this stage it is not clear what the most appropriate solutions are likely to be; therefore, it is recommended that this is an area where we need to “let a thousand flowers bloom”. However, it also means that it is essential that projects collaborate in order to ensure that projects, and the wider community, learn any lessons.
- In year 2 or 3, JISC should pilot some of the systems developed under the current programme:
Student and researcher experience
This area is primarily concerned with using recommender systems to help students and (junior) researchers locate useful material that they might not otherwise find, or would find much harder to discover.
- It is recommended that in year 2, JISC fund additional work in the area of recommender systems for resource discovery.
In particular work is needed in the following areas:
- Investigation of the issues and tradeoffs inherent in developing institutional versus shared services recommender systems. For instance there are likely to be at least some problems associated with recommending resources which are not available locally.
- Investigating and trialling the combination of activity data with rating data. In doing this there need to be acknowledgement that users are very frequently disinclined to provide ratings, and that ways to reduce barriers to participation and increase engagement with rating processes need to be discovered in the context of the system under development and its potential users.
- Investigation and implementation of appropriate algorithms. This should look at existing algorithms in use and their broader applicability. We advise that this should include statisticians with experience in areas such as pattern analysis and recommender systems.
- Some of the systems developed under this programme should be piloted elsewhere.
Activity data provides information on what is actually being used / accessed. The opportunity exists to use data on and how and where resources are being used at a much finer level of granularity than is currently available. Activity data can therefore be used to help inform collection management.
Note that this is an area where shared or open data may be particularly valuable in helping to identify important gaps in a collection.
- It is recommended that in the coming year JISC should fund work to investigate how activity data can support collection management.
In particular work is needed in the following areas:
- Consider how activity data can supplement data that libraries are already obtaining from publishers, through projects such as JUSP.
- Work with UK Research Reserve.
- Assess the potential to include the Open Access publications domain in this work.
- Pilot work from this programme to see if the data that they are using is helpful in this area.
The following are important areas that JISC should pursue.
- It is recommended that JISC continue work on open data for activity data, and in particular investigates appropriate standard formats.
- It is recommended that one or more projects in year 2 should investigate the value of a mixed activity data approach in connection with no-SQL data stores in order to maximise flexibility in the accumulation, aggregation and analysis of activity data and supporting data sets; the US Learning Registry project may be relevant.
- It is recommended that JISC ask appropriate experts (such as Naomi Korn / Charles Oppenheim or JISC Legal) to provide advice on the legal aspects such as privacy and data sharing, similar to Licensing Open Data: A Practical Guide (written for the Discovery project).
The Activity Data Synthesis Project is not in a position to make any recommendation over the use of linked data in this area in the absence of any compelling use.
 Austen J, Pride and prejudice
 In reality – “highly likely” – but that does not fit with the quote
 Arnold K, Signals: Applying Academic Analytics, Educause Quarterly, 2010, Vol 33 No 1 http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/SignalsApplyingAcademicAnalyti/199385 or http://bit.ly/c5Z5Zu