Monat: November 2014

Definitely not for beginners in the field of analytics. That was my first impression of week 5 and still is my impression. The videos were way too fast for me in speaking and content, but nevertheless, I’ll try to note what I got out of them, hopefully it’s correct. In this blog post I’ll also cover my experiences with the „week 5 activity“.

What’s the use of prediction modeling? Sometimes to predict the future, sometimes to make interferences about the present. There could be automated decisions by software or informing teachers so that they can do something. Starting point: There is something you want to predict – that’s called „Label“ (= predicted variable).

a) Regression = numerical (how much of a video a student will watch, what will be the student’s score,…)
In order to build a model, you obtain a dataset where you already know the answer (= training label). There are other variables (= features, predictor variables) which are used to predict the label. Regression means that you determine which features in which combination can predict the label’s value. In order to interpret the weight of the features, transformation is necessary.
One way of regression is linear regression (often more accurate than complex models particularly when you cross-validate). Another kind of regression would be regression trees (either with linear equations at each of the leaves of the tree or as non-linear regression trees).

b) Classification = set of categories (correct/wrong, will drop out/won’t drop out,…)
You get the labels from survey data, field observations, school records etc. With each label there are some features, which could be used to predict the label. A classifier is to determine which features in which combination can predict the label. Software like RapidMiner, Weka etc. have a lot of classification algorithms, but is is hard to say which work best in a certain context. Educational data has lots of systematic noise, so the advice is to use conservative classifiers and find simple models. From experience, considered as not so useful for educational data are „Support Vector Machines“, „Genetic Algorithms“ and „Neural Networks“.
For educational data there are „step regression“ (for binary decisions like will the student drop out y/n via linear regression function and rounding to 0 or 1), „logistic regression“ (for binary decisions like will the student drop out y/n via finding out the frequency of a specific value of the dependent variable, relatively conservative) , „J48/C4.5 Decision Trees“ (good in dealing with interaction effects, can handle numerical and categorical predictor variables, relatively conservative, good when the same result (drop out y/n) can be arrived in different ways), „JRip Decision Rules“ (set of if/then rules – many algorithms, decision trees created), „K* Instance-Based Classifiers“ (predicting the data point from a neighboring data point, good for very divergent data without easy patterns, you need the whole dataset) – and many other algorithms.

 

Week 5 Activity – Assignment: Problems and surprises

* Walk-Through
That wasn’t so easy to do, because the walk-through (which should explain the handling of the RapidMiner software) was in Flash and didn’t have a „back“ button – in my opinion, a simple pdf file would have been more helpful to get the context of the different steps. Thankfully, in the meantime we got a doc-file with the content. Another problem for me was that the software obviously expected that users would use drag&drop in order to add an operator and not „return“ like I did and which worked for some operators – but not all of them.

* External resource
The actual assignment, which was in a math tutor system, wasn’t available until you answered one question correctly. It took me a long time to find this correct answer, as I didn’t expect that I had to check the example csv file in Excel in order to compare if RapidMiner listed the correct attribute types when importing the csv. As RapidMiner identified some field attributes as binominal instead of polynominal I got a wrong result for kappa.
Finally, I had the correct answer and saw the start page of the math tutor system with a login window. I tried my edx login, but got no result as obviously it wasn’t intended that I got to this page. Not until I changed my Browser settings to accept cookies from ALL 3rd-parties, I saw the first question. These questions were hard to answer without background in statistics and I was relieved that there were some helpful infos in the discussion forum.

I answered a question regarding kappa without the field „studentid“, then about kappa without some other fields, conducted analysis with „Naive Bayes“, „W-JRip“ and got stuck with „Logistic Regression“ as my PC needed a long time to come to no result because I lost my patience after 50 minutes and stopped the process. I don’t think it was intended to last for such a long time and therefore tried a different approach with a subset in the operator „Nominal to Numerical“ which  didn’t work either. As the questions in the math tutor are not numbered, I don’t know exactly where I am at the moment (somewhere in the middle I think – going back is not possible and going forward is not possible until I give the correct answer).

Also frustrating was that in my opinion one question included a double negative and I thought for some time if I should say yes for fields to exclude or say yes for fields to include – My choice was nearly 100 % wrong (so maybe my answers weren’t so wrong at all…).

I have to admit, that I would have stopped working for the MOOC if this topic would habe been in week 1… What I got out of this week is that predictions are very difficult even with software which does a lot of the job. When you don’t understand what you are doing, you are getting very wrong results – therefore a deeper engagement with statistics is required. At the beginning of the week, I nearly didn’t understand anything, but after my practice in RapidMiner I have the feeling, that in some examples I actually understood what I was doing and that’s enough for me at the moment.

 

* Working with RapidMiner to get the kappa and therefore access to the Math Tutor assignment

Meanwhile I’m very good in the steps which are necessary to get the kappa with W-J48 because I had a lot of attempts until I got it right…. I’ll never forget binominal and polynominal… So, maybe this 720p video helps somebody who is still trying. And on a foggy November weekend with a heavy bad cold (therefore no audio), I had time for this 🙂

Upon the recommendation of a friend, this week, I read the book „The Circle“ from Dave Eggers, which had quite an effect on me – especially when taking a MOOC with the topic SNA at the same time… Therefore and because I had a heavy workload in my job, I postponed engaging with DALMOOC from the evenings to this weekend. I was really interested in this week’s hangouts and watched the recordings (http://www.youtube.com/watch?v=GUUaP39VpLI and http://www.youtube.com/watch?v=ziM0EvN9n0o) which were fascinating and brought to my attention the level of „that we as participants are object of study as well“. Personally, I find it a little bit scary that my social media and learning activity is under such scrutiny. Certainly I knew before the course that it would be the case – learning analytics naturally means monitoring and drawing conclusions – but it’s different when it’s for real. However, at this moment, I don’t know yet enough of what is possible via Learning Analytics and what is commonly in use. Maybe it’s inevitable and in five years or so it will be quite normal, but I think we as educators really have to think about which tools we are using and how we do that – We have to be very, very careful with our learner data at our universities.

SNA Case Studies
This week, we got an insight in some SNA case studies and the educational constructs which were used (e.g. learning design, sense of community, creative potential, social presence, academic performance, distributed MOOC pedagogy). Thankfully the full text was available and I’ll keep these examples in mind for further reading later on. SNA can help to detect patterns of interactions in online learning environments and instructors could start intervening depending on the intended learning design (e.g. from an instructor-centered network to an „equal distribution of student distributions“). Sense of community means to which extent learners feel that they belong to a community and on the other hand benefit from participating in the community (getting information, getting feedback, also relevant for student retention in universities). The aspects creative potential („network brokers are associated with achievement and creativity“), social presence („teaching presence facilitates the development of network centrality by guiding students to establish social presence“) and academic performance („those students who were central in cross-class networks had best academic performance“) were also covered in studies, but I just watched the intro-videos.
In order to learn more about cMOOCs (where social media is an integral part), this article about Twitter use in CCK11 (socio-technical approach: nodes are persons and hashtags) is very interesting: http://www.sfu.ca/~dgasevic/papers_shared/bjet2014_cmoocs.pdf

Unfortunately I don’t have the time for further experiments with Gephi (which I would really have liked to do because I value that we are doing something concrete with tools) and week 5 with different topics is near.

My goal with these blog posts is to summarize and reflect a little bit about things/content I’ve learned – my blog seems to be a good way to keep this for later on after the MOOC.

Week 3 is about an introduction to Social Network Analysis (SNA) and insights how social processes unfold. „SNA aims to understand the determinants, structure, and consequences of relationships between actors“ (Source http://www.lifescied.org/content/13/2/167.full.pdf+html) SNA is multidisciplinary (not only sociology and statistics) and main analysis methods are density, centrality and modularity types of analysis. We’ll do some analysis with test data and again visualization, this time with Gephi. The interesting thing will be what’s the use of SNA for learning (I’m not there yet).

Networks consist of actors (=nodes) and relationships/connections which can represent friendship, advice, hindrance, communication. In a spreadsheet nodes and relationships would be represented in rows (and weight via adding as many rows). Data can be collected by self-reports, interviews, collection from social networks (who is following whom on Twitter etc.) and special tools which collect data from LMS (activity in online discussion boards,..) and later on be analyzed in tools like Gephi. As these networks are seldom static, you have to decide on a time frame when collecting data. Also important: Anonymizing data, obtaining consent (which may lead to incomplete networks), ethics

 

Network Measures

In my understanding this very informative YouTube video from Dragan Gasevic http://www.youtube.com/watch?v=Gq-4ErYLuLA  lists network measures as follows:

a) Measures which are measuring the entire network:

* Diameter = „a measure which is determining the longest distance between any pair of two nodes in the network“

* Density = „is determining the potential of the entire network to talk to each other“ (how many connections of all the possible connections are actually happening)

b) Measures which are measuring the potential of individual nodes in a network:

* Concept of Centrality: (The meaning of centrality is dependent on the kind of different metric which is used)

** Degree centrality =  A very often used measure which „indicates the total number of connections for each actor in a network“

*** In-Degree centrality = Pointing to an actor / „how many other nodes are directly trying to establish communication or are talking  to a particular node“  (popularity, prestige)

*** Out-Degree centrality = Pointing away from an actor / „outgoing connections, may mean how many emails someone sent, generosity in conversations with others“  (gregariousness)

** Betweenness centrality = „measure which indicates the ease of connection with anybody else in the network but in particular to try to connect all these potentially small subclusters of the nodes“ (network broker)

** Closeness centrality = „used to measure the ease or the shortest distance of a node to anybody else in the network (indicates how quickly you can get to anybody else in the network, not useful for networks with many actors with no ties or groups with no connection to other groups)“

It also can be interesting to think about network modularity, e.g. smaller subgroups that are closely connected to each other (modules=communities). It is relevant for later use of modularity algorithms to identify the „giant component“ and use it as a filter.

 

About Gephi
Installation was easy but when I played around with the test data we were given, I even didn’t find the function to zoom in so I watched this YouTube video which was very helpful to get an overview of how to use Gephi (17 min very well spent): http://youtu.be/L0C_D68E1Q0

As I haven’t done anything like this before, I reduced my tests with the example blog dataset of week 6 to the „Average Degree“ and tried to find something useful. My results are in the attached pdf-file: w3-gephi-2

I’m looking forward to week 4 – maybe the Hangout times of day will be a little bit more convenient for Middle Europe again. And I still have to try Bazaar (I really want to do that), but again, this week, I had no time for that.