Accomplishments: This week, I had a student come into the Empirical studies lab and participate in our Stack Overflow study. I have two more students scheduled to come in next week. This is vital for fresh analysis as we continue with machine learning.
I presented the rough draft of my OCWIC 2017 poster about the results of the first half of our data analysis to Jenna, Dr. Sharif and Dr. Lazar. Ali calculated tag accuracy scores (relevant tags compared to distractor tags chosen by participants), and scores were split up into three categories: Simple, average and complex. Our results showed that simple level tag scores were 97.46%, average: 89.76%, and complex: 87.04%. From these results, we could conclude that users spent average time considering all 10 tags after viewing c++ post contents. After reviewing fixations, saccades and fixation durations, we split the data down further by participant experience level, novice v.s. non-novice. Novices were all those with 1 year or less programming experience and non-novices consisted of those with 3 years or more. We concluded that non-novice developers performed better than novices in tag assignment and depended more on code, and novices were less accurate in tag assignment as they assigned less tags on average.
Weekly goal: My goal is to get at least 6 students in the department to participate in the study. If I can get at least 1 student a week, my goal will be met. My next move is to finish the poster and present these findings at OCWIC. Then, I will continue to train data into Apache Spark.
Long term goal: Create a successful tag prediction system using the tag scores of these individuals.
No comments:
Post a Comment