Learning analytics is not a new term for higher education. In fact as far back as 2010 Learning analytics was mentioned in the NMC Horizons report. Last year I had the privilege of attending the LAK conference in Edinburgh in Scotland which is a very established international network of people researching learning analytics. So learning analytics is far from new. But, in my experience, until recently the conversations around student data and learning analytics were limited to a few people within institutions. However in recent months it is a term that is gathering a lot of interest in the every day conversations within higher education institutions. More and more people are seeing the potential and in some cases even realising the benefits of learning analytics. Dr Bart Rientes, Prof Shane Dawson and Prof Dragan Gasevic are people that I follow with great interest. Each of them leading the way in their respective fields, researching different aspects of learning analytics.The National Forum for Teaching & Learning, an organisation supporting T&L in higher education in Ireland has recently launched a project to raise the awareness of Learning Analytics so all of the indicators are that this is an area that is here to stay and will only get bigger in the years to come.
Personally speaking I got involved in learning analytics in DCU nearly 3 years ago examining if there was a correlation between a students engagement with the VLE and their success rates in the module. As you can expect, there is a very strong correlation. But we took it one step forward, using historical data we built algorithms to predict a students success based on their current interactions and we gave this information back to students with the hope of it improving completion rates. More information can be found here. Without spoiling the surprise, while the project was a success this project only gave one piece of the jigsaw.
Several different initiatives emerged following the findings from this project. Each one giving a different piece of the jigsaw.The next series of posts will outline the various learning analytics projects that we have conducted in DCU.
In conclusion learning analytics has proven to be a powerful tool that can help improve the learning experience of the student in so many ways but each bit of data is only one piece of the jigsaw. My word of caution that I would give to anyone interested in learning analytics is measure what you value, don’t value what you measure!
Learning any new software can be difficult at the best of times but if you are not a “digital native” even an apparently simple system can be a huge obstacle. This is most certainly the case with Moodle. If you can’t use it, it won’t matter how good it is, it just won’t be used. In my opinion Moodle has an added complication that as it is an open source product designed by many people to suit a large array of needs it may not always be as simple as you would like it to be.
To tackle this issue we commissioned the development of a plugin for Moodle that gives users a guided tour around each page when they arrive on it for the first time. The guides can be for students or teachers and the plugin is designed in such a way to allow generic tours to be used on any type of course page, no matter how it is laid out or what blocks you have included on your course page.
It has great potential for student orientation with Moodle and also potentially removing a fear for staff if they want to try out new features on Moodle
Hopefully you will find this plugin useful
Here is a short presentation which I gave to staff recently – hopefully explains it a little better.
FYI “Loop” mentioned in this video is the name of our Moodle instance.
Please pass this post onto anyone that you feel would be interested.
Anybody involved in teaching recognizes the benefits of assessment within their courses but equally they curse assessments at the same. Trying to get that balance right over providing enough assessment versus over (or under) assessment; giving detailed feedback versus prompt feedback; offering flexibility yet at the same minimizing opportunity for plagiarism. In DCU we have made several improvements to core Moodle to help improve the Assignment process for everyone involved. Some of these improvements are listed below, with each one discussed in detail in future blog posts
Every staff member and student has a Google Apps for Education account (GAFE). We now have linked Google Calendar with Moodle assignment.If a student has an assignment in Moodle, an “event” is created in their Google calendar. This event contains the assignment title, deadline date, brief description and link directly to the assignment on their Moodle page. This information is dynamic in so far as if a lecturer makes changes to their assignment on Moodle, the students Google Calendar is updated.
Sitting beside every lecturer to explain how to create a quiz in Moodle is just not practical. In addition to creating a suite of instructional videos and user guides we have created a virtual assistant on Moodle. Essentially when a user comes onto a new page in Moodle they are given a “tour” of the page. This tour guides them through key points/blocks on the page with an explanation associated with every point. These tours can be created for every page and every feature on Moodle, we have just decided assessment based features of Loop. This “tours” plugin is available to the Moodle community (https://moodle.org/plugins/local_usertours), all we ask in return is that if you create any tour, that you are willing to share it back to the community via https://moodle.net/mod/data/view.php?id=17
If students have not submitted an assignment Moodle now emails them reminders. A lecturer can choose how many reminders students get in advance of and after the deadline
A student is now presented with their relative grade i.e. how they have performed in a particular assignment relative to their colleagues: are they above or below average, are the close to the lowest or the highest grade
You may have centralised course material e.g. a course on citation and referencing that you want your students to complete as part of your course. Now lecturers can link out to that “other course” via a sub course link. When student click this link they are automatically enrolled in this course and all of the grades from the “other course” are passed back to your course gradebook
We had a really interesting “hot topic” session recently where Dr Tom Farrelly was our guest speaker for the session the title of which “Who Owns My Lectures? An Exploration of Academic Ownership in the Digital Age”
Does the institution or the individual lecturer own the material? It was what I would consider a very healthy discussion and I only wish more lecturers came along on the day to join in. We had lecturers, learning designers, academic developers and management with responsibilities for digital learning in the audience. While we did not come to a resolution on the day, like all of these hot topic sessions the power is in the follow on discussions.
It was through one of these discussions where an interesting TED talk “Embrace the remix” was shared . This talk from Kirby Ferguson to me clarifies that the material we create, the notes we create, the presentations of our knowledge are built on the foundation of other work, books we have read, videos we have watched and discussions that we’ve had.
After watching this video the argument in my opinion changes to – do I have rights over my “performance” of the lecture, like an artist has rights over their art? This question in my opinion is easier to answer – No. If you are paid by your institution to produce the lecture material you do not have rights over it. After all does an actor get paid everything time a film in which they are in is shown. They don’t, they got paid to act the scene, when Star Wars the movie was re-released in the 90’s did the entire cast get paid a second time. I know that there are some exceptions where some actors have particular contracts, but they are the exception not the rule. In a time where it is easy to record a lecture and replay it several times, should the lecturer get paid each time? Actors can make or break a film, just like lecturers can make or break an academic programme. But it is not just the actors, it is the script writers, the set designers, the producers, the directors. It is the entire team working for the studio. In todays’ digital era it is no longer just the lecturer that is involved in the production of learning material, it is the learning technologists, the curriculum design team and all the stakeholders working for the institution.
There are so many different variants of the scenario it is nearly impossible to give a definitive answer but I will say this – the conversation is very timely and one worth having. We should have it sooner rather than later because as technology advances and becomes more ubiquitous it will potentially become more of an issue if not addressed.
What is your opinion? What is the situation in your institution?
For more on NIDL events please visit http://dcu.ie/nidl/events/external.shtml
“Two of the biggest Articulate Guru’s in the business providing a free day long workshop – and would DCU be interested in being a partner when they come to Dublin as part of their road trip?”
To paraphrase from one of my favourite films, it was an offer that I could not refuse.
In 2014 at the inaugural event in DCU, I had the pleasure of hosting Dave Anderson, this time the ante was increased, Tom Kulhman joined Dave on stage for a day long workshop for people interested in using Articulate software. With over 150 participants in attendance, it is evidence that the Articulate community in growing from strength to strength in Ireland. This event was another initiative that strengthens DCU’s relationship with Articulate and it’s community of users. In addition to informative workshops containing very practical tips, we got a sneak preview of how Articulate is improving the viewing of their published content on mobile devices and also their general roadmap for the future. All I can say is exciting times ahead.
We are already looking towards 2016 and what more we can do with the Articulate team and welcome continued conversation with Articulate users in the community – https://community.articulate.com
As promised yesterday this is the follow up to my post on the ECEL 2015 conference. On this occasion I am reflecting on the conference through a different lense – looking at it as someone who organises conferences:
It always of interest to me how others do it – what do they do better than me and what ideas can I take back and implement in my next conference. But I also find it valuable to identify good bits that I do that they don’t. I find it somewhat reassuring and rewarding to an extent. I don’t mean that in a big headed way – it is just good to know that you are doing something right.
The conference website was a great help, despite not being the most aesthetically pleasing – it contained all of the information that was required and it was relatively easy to find it. The correspondence from the organisers was also quite good, with a slight negative being sometimes there was just too much information in their emails but if I’m being honest that is probably me being a little picky. One of the things that I feel would improve the site was a little intro video from each of the mini-track chairs – explaining what their track was going to be about – essentially a little promo video. If they were really ambitious each speaker could post a video like this for their talks – easily done with todays technology
They also had pre-conference workshops – I find these incredibly useful especially if you have people travelling a long distance – it is easier to justify travelling a long way for 3 days instead of two. It also gives you an extra day to network. The downside of course is taking that extra day away from the “day job”. There were two workshops both of which proved valuable to me, for different reasons but definitely worth my while.
During the conference
There were a lot of positives about this conference; most of the talks were very good, the food was fabulous (they had a great gluten free selection), the various stream rooms were all very close to one another. The vast majority of the speakers kept to time and in general the conference ran exactly as scheduled. The conference dinner was amazing – we ate in Hatfield palace, the home of Elizabeth I, and the food was most certainly fit for royalty. A negative though would be the lack of a social element to the conference. I feel the organisers missed an opportunity here – the dinner finished at 10 and with most of the participants staying in one of two hotels, it would have been easy to put on the timetable that participants can network in the hotel bar after dinner. Even by reserving a space in the bar and doing nothing else, it may encourage networking.
The biggest frustration was the poor quality of wifi. If I was the organisers I would be asking the venue for a refund because the quality and reliability for a purpose built conference centre was very poor. Several of the presenters needed the internet to facilitate audience participation and the poor wifi simply prevented that. Luckily enough those who needed videos from Youtube or content from Prezi had back up plans but for an eLearning conference reliable wifi is an absolute must. In my opinion it seriously reduced the amount of conference traffic on twitter as well. As a conference organiser my heart went out to the conference team as all of the hard work that they had put into place was overshadowed by technical problems beyond their control. Rather than end a blog post on a bad note I want to mention two more positives; They had a big screen in the main coffee area which displayed comments, tweets and conference notifications – a really nice idea and something which I will implement in future conferences they had a notice board with the photos and names of all of the presenters displayed by the registration desk – a simple but effective tool to facilitate networking.
All in all I would like to congratulate the organisers on a job well done and I look forward to Prague next year 🙂
All in all I would like to congratulate the organisers on a job well done
Reflections from the conference ECEL 2015, hosted in the University of Hatfield, Hertsforshire.
Nearly every time, any conference that I attend, I curse the conference in the lead up to it. For the simple reason taking time away from the “day job” to attend them is so inconvenient. But in the majority of cases and I glad to say including this one, I come back from the conference glad that I have taken the time to attend. Conferences like these are really important professional development opportunities in my opinion. I’m firmly believe that every manager should facilitate their staff to attend them, and I’m glad to say my manager is very supportive in this respect.
Looking at this conference through two different lenses – a conference organiser and also a conference participant and a manager. This post will concentrate on the latter, a subequent post tomorrow will look at the conference through the lense of a conference organiser.
As a participant, the conference website had all the necessary information and the staff when contacted were fairly quick getting back to you. So inital impressions were very positive and it laid out the path for a promising conference. The venue was a bit awkward to get to and it was split over two venues; pre-conference workshops in the university and the conference itself in a purpose built venue 15 minute walk away in an industrial estate. The university was lovely and very modern – it had a great feel to it and I would have loved if we were there for the three days. The second venue, though nice was very isolated and most certainly did not have the “presence” of a university campus. My talk was on the first day of the conference and I feel that it is always good to speak on the first day because that leaves ample time for networking. The presentation was well received with a lot of positive comments coming both during the presentation and throughout the rest of the conference in various discussions. One or two suggestions for improvements but mostly they were comments of admiration for the work and on more than one occasion Blackboard users saying they would love to change to Moodle. Although this work is essentially platform agnostic because we have access to the core database the raw data is ours to play with and we have no need to involve a big commerical company to do the work on our behalf – hence the VLE envy.
The inevitable questions did arise about ethics but once I explained the student centred approach that we have in place there was no issue. As an improvement for next time I will include more of the ethics information up front. Plus it was great to have the website (predictedanalytics.wordpress.com) for the project as a reference guide for the presentation and I look forward to monitoring the traffic for the site over the next few days :). Making a project website was not something I have done in the past but it is definitely something I will do in future for all projects.
I’ve uploaded the slides on slideshare and included all additional slides that I had prepared in order to answer potential questions after the presentation
The most rewarding part of the conference though was to see the experience of my colleagues. From a managers view point it was great to see @clare_gormley excel during her presentation with @dyagetme. Both gave an excellent presentation on the use of wearable cameras. Their presentation was very well articulated and as a team their presentation styles complemented one another very well. Their presentation generated a lot of positive comments. This was very reassuring to me personally because of the management style that I aim to implement. Where possible I try give my team freedom to work on projects of their choosing once they are aligned with the unit’s overall strategic objectives. The professionalism of both their research and their presentation help vindicate my approach. They led this project from the start and not only did they see it through to completion, this work has spawned other similar projects in other schools and programmes. I have no doubt that they will take on board all of the comments given to them today and take this project to the next level. The slides below will give you an insight into their work.
In conclusion I believe that our unit and indeed DCU was potrayed in a very positive light over the last few days. Yesterday was evidence (as if I needed any) that both of Clare and Patrick have grown in their roles since arriving in DCU and that we’re lucky to have them on board.
The next post will look at the conference through a different lens – looking at as a conference organiser
One of the most refreshing features within Higher Education for me is our willingness to collaborate and share with our counterparts in other higher education institutions. Following an excellent time at the DCU Summit several participants have created a community of practice and agreed to share scripts that they have developed integrating Google Apps with other IT systems e.g. Moodle and / or student record systems.
Our first initiative as a Community is to set up a hangout where we will each give a short presentation ( 4 minutes max) on script that we have developed. If you are interested in joining this community and/or coming along to our hangout online please tweet me via @glynnmark
More and more recently I find myself using Google Docs instead of MS Word. The collaborative features far outweigh the limited functionality that it brings relative to MS Word. But when it comes to writing papers (up until now) I have always went back to the old reliable, using either the plugins from RefWorks or EndNote to help with my referencing. Today at the DCU Summit I was introduced to PaperPile. I’m very impressed and look forward to testing it out over the next few weeks for my 30 day free trial. Then I will start the haggling with the vendors for consortium based pricing. The video below gives a nice introduction to Paperpile but if you need more information contact them on https://twitter.com/pprpile I’d be very surprised if you are not impressed.
A student receives their grades from each assignment along with their feedback. However unless the lecturer posts a list of student grades publicly the student has no idea how they have performed within the class.
We provided a report in Moodle that is visible to the student. The report is available for each module and presents the student with each of the grades for the various assignments within a module
Figure 1 displays grades for 4 different assignments within a particular module for one student. The figures on the left are the grades achieved, the figures on the right are the relative grade. This relative grade is based on a calculation known as the z score.
A Z-Score is a statistical measurement of a score’s relationship to the mean in a group of scores. A Z-score of 0 means the score is the same as the mean i.e. they are average in the class. A Z-score can also be positive or negative, indicating whether it is above or below the mean and by how many standard deviations. If they achieve the lowest grade they will achieve -3 and if they achieve the highest grade their Z-score will be +3. You will note from the results displayed in figure 1 that this student is consistently below average as their Z-score values are all less than zero. You will also note from that the second assignment has a grade of 56 and a Z-score of -0.99 and the fourth assignment, even though achieving a higher grade (59) was actually a poorer result when compared to his classmates (Z-score of -1.66)
How can the Z-score be useful to staff
As a lecturer you can get an overview of your students’ performance across several assignments through the Z-score
As a student’s tutor or course coordinator you can get an overview as to how a student is doing across several modules as displayed in figure 2 for student John Lennon
Figure 2: a Z score displaying a student’ performance across all of their modules
Z-Scores to assist with student retention
A potential project in the future will be integration of the Z-score with the student CRM. So if the student has, for example, 2 or more Z-scores in a programme below -1 a flag can be raised to identify that student as a being at risk of non-completion. A simple intervention could be activated by automatically emailing the student from the course coordinator inviting them to a meeting or simply checking in with the student to ensure everything is alright.
Points to note about the z score
If a Z-Score….
Has a value of 0, it is equal to the group mean.
Is positive, it is above the group mean.
Is negative, it is below the group mean.
Is equal to +1, it is 1 Standard Deviation above the mean.
Is equal to +2, it is 2 Standard Deviations above the mean.
Is equal to -1, it is 1 Standard Deviation below the mean
Is equal to -2, it is 2 Standard Deviations below the mean
Z-Scores are not perfect, they are only meant as a visual indicator of a student’s performance
For more information on a z score please refer to: