29 July 2016

Learning analytics, even student dashboards, are they the wrong way round?

In a recent session discussing a learning analytics project, I seemed to be the only person in the room who was anxious about the whole idea again. I've been this way ever since George Siemens started the Google Group some 10 years ago. That anxiety culminated in a presentation I made to the University Analytics forum in Melbourne in 2012, which I'm sad to say, along with my posts to the forum, has generated little to no response. Is it just me and my tin foil hat, or is there a general reluctance to talk about an elephant in the room with learning analytics?

The best I've seen from the over all movement is a general agreement that it is ethical and progressive to develop analytics as a "student dashboard", that is to say that the effort is first and foremost about collecting data so that the individuals that the data is about can see and reflect on their own patterns, and in relation to the demographic groups that seem relevant to them, presently and historically. The antithesis of this is the collection of data for teachers and administrators to roughly calibrate their behaviouralist experiments - what most learning analytics projects are about.

But in this session recently, it occurred to me that even the projects that describe themselves as "student dashboard" projects, seem to be allowing themselves to be drawn a very long way away from the principles of why they are developing that way. Most of these projects that I have seen seem frustrated by the difficulty of obtaining useful data, and end up narrowing scope in on a single environment like an LMS or a handful of online social platforms, within a single course. They accept that this then renders the project an unscalable proof of concept, and acknowledge that they leave far too much out of a person's wider context to really get any useful insights. Is there another way to try and uncover insights about learning? A way that better fits the principle of student dashboard, and potentially encompasses that wider context that seems impossible to account for?

I'm suggesting a closer affiliation with the field of QuantifiedSelf. Who in the learning analytics world is investigating the large range of mobile applications designed to assist with time management and task completion, for example? It could be that one of these, or a combination of them, offer students an optional way to record and manage their own data, and to even pool in with an online community or collection for comparison and bigger pictures. This seems to be an approach that would inherently deal with many of the ethical concerns of a university gathering data on students - often without even a research ethics application!

It seems to me that this suggestion is to at least qualify the data currently being collected in the more top down approaches, if not control for it. But I suspect it's more than that. With the right additions, the voluntary and guided use of such apps and methods might be the very idea that "student dashboard" projects set out to achieve. The outcomes of projects that take this approach might be a range of suggested apps and guided activities to help participants make the most of logging their lives around learning; how to pool data for comparisons; and how to better design course curriculum to help students manage time and task completion.

A search for "time management" in Google Play reveals quite a few useful candidates to try out, many with data export ability. Learning designers could design weekly time management schedules around a course for example, for participants to run in something like TimeTune to stay on task. We could suggest that participants try using an application like Working Time Management, that tracks the time spent on projects, including communications with people in the project, similar with aTimeLogger, and simple activities where the group compares records. These are just the first few apps available in a search for Time Management.

I've recently started using the application Headspace, a charming application which isn't a life logger at all, though it has some optional features that could be used like that. It's primarily a 10 step course in meditation and mindfulness. It's pretty popular it seems, and interesting as a format for a course. The various tools and techniques for managing time, focus and headspace could conceivably be combined into one, as layers around a course on any topic, where students (if they like) can turn off and on those features, some of which offer guidance in time management, others an opportunity to measure, manage and compare their engagement with topics and projects.

Does anyone know of a learning analytics project that takes this approach? Such an approach would alleviate some of my anxieties about the field and its elephants, especially if they were to dgo as far as to investigate the source of the applications and determine what the companies do with the data collected. 

21 July 2016

Brexit and the urgency of open access and usability

A very interesting perspective on Brexit in relation to open access has been shared by Stevan Harnad in a discussion with Richard Poynder on the Open Access ePrints blog.

But one would have thought that the mature democracies would serve as a civilizing bulwark against that. Yet no, Brexit has shown that the same primitive, sinister, shameful inclinations are alive and well in the United Kingdom (and Trump is rallying them in the US too). 
No, freedom-of-information and open access did not serve as an antidote, as hoped. Disinformation profited more from the power of open media than the truth did. And the proliferation of destructive weapons is only beginning to be exploited by the genetic and cultural heirs of our most barbaric roots.
Perhaps both democracy and liberalism were always doomed; perhaps it was just a matter of time before the law of large numbers, the regression on the mean, would bring out the meanest in us.

The idea that the collapse of the 20thC socialist idea allowed market fundamentalism to grow unchecked, which has inevitably caused base populism and inhumanity to thrive, is a summary that rings true enough to me. But we can't yet know if this populism and inhumanity is leading to - as Harnad would characterise it - an apocalypse of humanity. Perhaps instead what we're seeing is a strange wisdom of the masses, bringing the collapse of 20thC capitalism.

The characterisation of Brexits, Trumpies, Hansonites as racist xenophobes doesn't ring true to me. It may be true that darker elements exist within them (as they do us all), but a more generous characterisation would pay attention to their better arguments, from the more thoughtful voices.

Here's Richard Boyd Barrett speaking about Brexit.

Bringing about the collapse of the EU - or more specifically the super rich and the corporate elites that lobby it, or voting for Trump is largely an expression of disillusionment with the political class and economic elite.

Here's the same argument that Barrett succinctly put out, but from the very people he spoke for - Why we voted leave - voices from northern England.

It is a worry that these arguments and perspectives are so quickly and easily dismissed by the "Progressive Left", and allowed to be characterised as racist, nationalistic and xenophobic.

There have been more hopeful phenomena that I would include in this general movement of resistance. There's the swaths of people that Bernie Sanders appealed to (as likely as it is that Sanders simply contains and ultimately controls opposition). There's the dramatic rise of Jeremy Corbyn, re-energising the socialist principles. There's some justice about to be served via Chilcot and The Killing$ of Tony Blair (to my knowledge, the first feature length documentary to be crowd funded). There's the fresh and progressive ideas of the PirateParty, the disruption by Wikileaks and Occupy, and the various inquiries and possible trails for the ongoing financial fraud and economic mismanagement globally.

How does this relate to open access?

I think the open access movement should focus on these sorts of hopes, and radicalise accordingly.

More than access, we need usability

It is not enough to lobby for educational media, academic research papers and data to be made openly accessible in the formats and customs that they are. Right alongside all that needs to go an active alignment to the issues of the day, and development of novel ideas around popular usability. Not just format usability, but designed usability.

Summaries and takeaways

If we consider the function of an abstract that goes with an academic paper - that it serves as an effective summary to the whole paper, then we should be willing to recognise similar devices in popular media and consider such designs for usability generally. Executive summaries, infographics, synopsis and trailers.

Partnerships with major information highways

Why is it still the exception to the norm, that the multi lingual Wikipedia, the media rich Wikimedia Commons, or the wonders of the Archive and Way Back machine are not entirely in the discourse and workflows of the public service information, research and education sector? Quite the opposite in fact, ignorant and disengaged snobbery prevails toward those bold projects. Why isn't it normal practice for people in those same public service information, research and education services to make bite-sized video abstracts of information and knowledge and distribute them on Youtube and Facebook? Why do they still insist on creating unreliable websites that block the Waybackmachine from archiving them, that will go offline when the funding dries up and have no distribution or communication plan through popular media channels? Why do the so-called professions of instructional and education design still obsess over how to use a learning management system, or how to work within the narrow band of restricted user-pay access, and pay little to no attention to ideas and methods for instruction and education in an open distributed network of society? In that area, the darker professions of advertising and public relations are far more advanced.

Open access is little c conservative

In my opinion, the open access movement has been cobbled with conservatism while the PirateBay, Wikileaks and Aaron Schwartz have been trail blazers. There have been global issues that the open access movement could have been part of - taking relevant openly available information and distributing derivatives with usability in mind. More importantly, this workflow would have been made self evident by now - as getting information in multiple languages on Wikipedia and Youtube is self evident by now. Open access would not only be a matter of course, but creating usable versions of it would be expected as well. But we in the public service information, research and education sector generally seem happy to sit by and let popular debate degenerate into private public relations.

The defeat being discussed by Stevan Harnad and Richard Poynder, should be answered with radicalisation. I thought the very foundation of why we work toward open access and use was to prevent the world that Harnad and Poynder are resigning to. Can we now redouble the effort in linking up the research, information and education sectors with the radical open access and use movement? The justification for it would seem as urgent as ever.

Unless of course the masses turn out to be substantially wiser than the anxious experts give them credit for. In which case, they don't need them.

19 July 2016

Collecting distributed network data for analysis of learning

I recorded a conversation with Kirsty Kitto and Leo Gaggle, the most knowledgable people I know when it comes to #xapi   #LRS  and #learninganalytics  applied in distributed networks (the Internet, or "the wild" as Kirsty likes to call it).

Kirsty leads a national OLT project to the tune of $320000 over 2 years, developing a toolkit (available on GitHub) for collecting data from the wider internet and beginning to make sense of it for specific learning analysis.

The video is fully logged with hyperlinked timecodes and URLs in the description (and below to aid search in this website). I suggest you set youtube to watch at 2x speed and use the time codes to jump around if you need.

Streamed live on Jul 17, 2016
0:00 Kirsty Kitto at QUT and Leo Gaggle at Bright Cookie http://www.brightcookie.com/ thanks for joining us. We're meeting to discuss methods for extracting data from a specified Youtube network, and how we might use that data to analyse the network
0:25 Leo explains why we’re meeting. Coming from a conversation about Experience API (XAPI) and how we might use it to gather data from online networks. Leo suggested we talk with Kirsty Kitto who has been working with XAPI to do such things
1:20 Kirsty explains how her team is using XAPI to extract data from social media networks. It is a project funded by the Office of Learning and Teaching (OLT) that has been running for a little over a year.http://users.on.net/~kirsty.kitto/pro... Twitter, Wordpress, Trello and GitHub. Works with teachers who don’t use Learning Management Systems, but use various social network services instead.
3:35 The of the Connected Learning Analytics Toolkit has been going on at Queensland University of Technology with Kirsty Kitto as the project lead), Mandy Lupton, John Banks, Dann Mallet, Peter Bruza working with students to developing the software. People outside of QUT are giving feedback and advice, including Shane Dawson at University of South Australia http://people.unisa.edu.au/shane.dawson; Simon Buckingham Shum at University of Technology Sydney https://www.uts.edu.au/staff/simon.bu... Abelardo Pardo at University of Sydney https://latte.ee.usyd.edu.au; Dragan Gasevic University of South Australiahttp://people.unisa.edu.au/Dragan.Gas... George Siemens https://en.wikipedia.org/wiki/George_...
4:25 Comments about the difficulties using xAPI, using Linked Data, using the Companion Specification for xAPI, and the challenges of extracting data
7:50 Working toward good mapping of data and the recipes for getting data
8:27 Examples of use Kate Davis at QUT roles out her own Wordpress with Buddypress. She is using the toolkit to report activity, social network, sentiment and classification
11:10 Kirsty’s talk at UNSW about students adjusting imperfect analytics as a learning activityhttps://teaching.unsw.edu.au/can-stud...
14:00 drawing in hashtags from Twitter and discussion about student privacy
17:00 Leo Gaggle talking about the developments at Bright Cookie and some of the difficulties with xAPI, but using it to draw in multiple feeds of data for reporting. Using xAPI to report professional development activities in the corporate environments. Basically using xAPI and LRS to recreate an LMS reporting system, but from more distributed feeds of data.
20:40 Describing the RMIT project to create a learning network in Youtube, where a teacher creates a channel (in this case, “Andrew Robinson Footwear”) and the students create accounts and subscribe to that channel, and each other. The teacher subscribes to the School channels and other teachers. One of our questions is, can we teach youtube to teach? http://dldsc.team/2016/06/17/teaching... Can we establish a connected online proffessional network inside Youtube that can go with a student when they graduate? What sort of data can we extract from that network?
25:10 Kirsty recognises the value of the project. But there are problems with Youtube’s API and the quality of the data, such as view stats. To assess networks, we need all parties subscribed to the toolkit. (After the recording, Leo suggested dropping Google Analytics in on each channel of the network).
30:10 Leo explains how to capture play events in an iFrame embeded into a web page with a player app. Discussion about student participation, and connecting qualitative data gathering.
34:25 Kirsty talks about capturing Youtube comments, as well as “mini LMS” that captures all user’s click streams. Also suggested tracking memes, with a competition between students - who can send their video the furthest, and how do we track that.
37:10 Leigh asks if Leo’s suggestion is essentially a browser plugin or app that runs alongside Youtube
38:38 Leo makes the point that xAPI only helps with the collection of data. The big work is in analysis.
40:00 RMIT is aiming to host a datahack to try and solve analytical challenges. Solid advice on the need to get the data structures thought through before the data hack. Leo points to the recipes that Kirsty’s team has created could be a good starting point. https://github.com/kirstykitto
43:00 Kirsty’s solid advice to watch the wider scene of systems like xAPI to try and ensure the interoperability of data for long term use.