This week I concluded the data collection phase. I was able to capture a varying range of people in regards to C/C++ experience. All the participants were YSU students, the majors that participated were Computer Science and Electrical Engineering. While I was hoping to get other majors from the CSIS department to compare gaze-data I think having 2 majors will be enough to compare. The process went smoothly, I was able to keep all the collected gaze-data and I also learned a lot from even moderating. After just visually analyzing (i.e. looking over gaze-data representations, no tools) I can already determine a few trends. For example, those who have less experience with C/C++ use the title and question text more to assign tags, especially for the more complicated tasks, versus using the code. It also seems that those with more C/C++ experience were better able to assign tags that apply more to the question solution versus obvious things found directly in the text/code, this was expected. I plan to incorporate simple observations like this, as I think they are useful in interpreting how tags were selected. In this upcoming week I will do the following:
1. Analyze data as a whole - use Tobii to look into fixation count, duration count, and time to first fixation. I hope to compare how people considered oracle (positive) tags vs the distractors (negative) in coming to their tag selections.
2. Compare data from different levels of experience - consider how people came to the correct/incorrect conclusions based on their experience levels and try to determine common trends.
Furthermore, I want to use the gathered data to determine keywords that should award higher weights to suggested tags. I think this is something that will be helpful, especially in the future when applying the machine learning algorithms.
No comments:
Post a Comment