Work Accomplished:
This week, I continued to gather more participants for our study. Overall, we have 13 people who have completed the tasks, and we are looking to get at least 20. I practiced my lightning talk that I will be presenting on Friday at the OCWIC conference. I've been preparing my presentation on the first half of our research study for OCWIC as well. I have also printed out copies of my resume to bring to the conference for the review workshop.
Weekly Goal: Keep gathering more computer science students for the study.
Future Goal: Once all participants have been gathered, stream data into Apache Spark.
Thursday, February 23, 2017
Week 21: 2/14/2017 - 2/22/2017
Work Accomplished
This past week I helped edit Alyssa's lightning talk slide show for our poster presentation at OCWiC 2017 this weekend. Alyssa will be presenting the poster and slide about our Stack Overflow study for this year's CREU project (I will be presenting another poster/slide, see previous blog post). I also made final edits, created an author bio for myself, and wrote an acknowledgements paragraph for the paper (On Predicting Developer Expertise from Eye Gazes for Bug Fix Tasks) that was accepted to The Honors College at YSU's Academic Journal called The Emperor. This paper describes the analysis and results of the prediction experiments we performed in last year's CREU project. Finally, I was able to re-run the ABB eye-tracking data on the new fixation filter producing much more accurate fixation results.
Goal
Weekly goal(s) - In the next week, I will be presenting a poster at OCWiC 2017 and transitioning from the ABB sequential analyses project back to the Stack Overflow project Alyssa has been working on. I plan to aide her and Dr. Lazar in selecting and running the correct data analyses for the data Ali collected last semester. I hope to learn more about the inner workings of the data analyses we select and their appropriate uses.
Long-term goal(s) - Continue working on the ABB sequential analyses project goal (see previous blog post), but outside of CREU working time. For the Stack Overflow project, I plan to select and perform data analyses appropriate for answering the following research questions:
- To what degree do programmers focus on key words that extraction techniques generate?
- To what degree do the top n keywords from our approach and the standard approach match our Oracle generated keywords?
- What are the best machine learning algorithms (informed by eye gaze) that can be successfully used to make predictions?
Outcome(s)
- Alyssa's lightning talk for OCWiC 2017 was edited and submitted
- ABB eye-tracking data re-run on the new fixation filter; much more accurate results
- Final submission for publication of On Predicting Developer Expertise from Eye Gazes for Bug Fix Tasks in YSU's Honors College journal, The Emperor
Wednesday, February 15, 2017
Week 20: 2/8/2017 - 2/15/2017
Accomplishments:
This week, I gathered three more students for our eye tracking study. I finished my poster and lightning talk for the upcoming OCWIC conference, and I uploaded all of the participant data into a file. I will be analyzing this data to look at any trends that I find regarding tag scores of novices v.s. non-novices. My only concern is that I will not have enough non-novice data to compare against, but it should be fine for now.
Weekly goal: I would like to get one more participant for my data, particularly someone who is a non-novice.
Future goal: I need to run my data through Apache Spark, and give my lightning talk on February 24.
This week, I gathered three more students for our eye tracking study. I finished my poster and lightning talk for the upcoming OCWIC conference, and I uploaded all of the participant data into a file. I will be analyzing this data to look at any trends that I find regarding tag scores of novices v.s. non-novices. My only concern is that I will not have enough non-novice data to compare against, but it should be fine for now.
Weekly goal: I would like to get one more participant for my data, particularly someone who is a non-novice.
Future goal: I need to run my data through Apache Spark, and give my lightning talk on February 24.
Week 20: 2/8/2017 - 2/14/2017
Work Accomplished
This past week I created a lightning talk slide show for my poster presentation at OCWiC 2017 (coming up on the 24th of February). I will be presenting the same poster I presented at Tapia (last semester) about the machine learning expertise prediction we did in last year's CREU project. We found that eye-tracking data on software developers solving bugs provide us with a feasible way to predict developer expertise using machine learning algorithms. I also traveled to the University of Notre Dame on a graduate school visit for their Ph.D. program in Computer Science. From this trip I learned that I want to do research in the area of source code summarization and generating source code from the English language.
I did not have time to re-run the ABB eye-tracking data and add support for method call and method use differentiation in our sequential analysis of this data. These goals will roll over to next week.
Goal
Weekly goal(s) - In the next week, I plan to re-run the ABB eye-tracking data we collected on the newly updated fixation filter in iTrace and include support in our sequential analysis for differentiating between a method call and a method use.
Long-term goal(s) - To perform many more (roughly 8) sequential analyses on the ABB eye-tracking data after adjustments to the fixation filter and STS data creation R scripts. This will be done to determine variance in expert and novice eye gaze patterns during bug fixes.
Outcome(s)
Goal
Weekly goal(s) - In the next week, I plan to re-run the ABB eye-tracking data we collected on the newly updated fixation filter in iTrace and include support in our sequential analysis for differentiating between a method call and a method use.
Long-term goal(s) - To perform many more (roughly 8) sequential analyses on the ABB eye-tracking data after adjustments to the fixation filter and STS data creation R scripts. This will be done to determine variance in expert and novice eye gaze patterns during bug fixes.
Outcome(s)
- Lightning talk slide show for OCWiC created
- Graduate school visit to the University of Notre Dame; decided my area of study in graduate school
Friday, February 10, 2017
Week 19: 2/1/2017 - 2/8/2017
Work Accomplished
This past week I wrote the Results section of the paper we are submitting to the Mining Software Repositories conference about mining eye-tracking data for software tasks. I finished generating all the entropy, turbulence, and similarity metrics results for all tasks. After including these results nicely in the Results section, I added bullet points of information in the Discussion and Future Work sections (Dr. Sharif and Dr. Lazar will be finishing the Introduction/Related Work and the Conclusion/Discussion/Future Work sections). I went back and made minor editing changes to the sections I wrote. I also presented a more mathematical intense variation of the similarity metrics work at the 2017 Nebraska Conference for Undergraduate Women in Mathematics on Saturday.
Goal
Weekly goal(s) - In the next week, I plan to re-run the ABB eye-tracking data we collected on the newly updated fixation filter in iTrace (some bugs were fixed that may change our results) and include support in our sequential analysis for differientiating between a method call and a method use (which are distinct source code elements). I am also going on a graduate school visit to the University of Notre Dame. This may cause some of my weekly goals to roll over to the next week.
Long-term goal(s) - To perform many more (roughly 8) sequential analyses on the ABB eye-tracking data after adjustments to the fixation filter and STS data creation R scripts.
Outcome(s)
This past week I wrote the Results section of the paper we are submitting to the Mining Software Repositories conference about mining eye-tracking data for software tasks. I finished generating all the entropy, turbulence, and similarity metrics results for all tasks. After including these results nicely in the Results section, I added bullet points of information in the Discussion and Future Work sections (Dr. Sharif and Dr. Lazar will be finishing the Introduction/Related Work and the Conclusion/Discussion/Future Work sections). I went back and made minor editing changes to the sections I wrote. I also presented a more mathematical intense variation of the similarity metrics work at the 2017 Nebraska Conference for Undergraduate Women in Mathematics on Saturday.
Goal
Weekly goal(s) - In the next week, I plan to re-run the ABB eye-tracking data we collected on the newly updated fixation filter in iTrace (some bugs were fixed that may change our results) and include support in our sequential analysis for differientiating between a method call and a method use (which are distinct source code elements). I am also going on a graduate school visit to the University of Notre Dame. This may cause some of my weekly goals to roll over to the next week.
Long-term goal(s) - To perform many more (roughly 8) sequential analyses on the ABB eye-tracking data after adjustments to the fixation filter and STS data creation R scripts.
Outcome(s)
- Results for entropy, turbulence, and similarity metrics computations on our eye-tracking data
- Completed Results section and final touches of my portions of the MSR paper
- Presentation given at NCUWM
Wednesday, February 8, 2017
Week 19: 2/1/2017 - 2/8/2017
Accomplishments:
This week, I gathered more participants for our Stack Overflow study. I'm at 5 members so far and with more scheduled next week so I will be able to meet my goal of at least 6. I finished my poster about the results of our eye tracking study regarding participant tag scores based on novices v.s. non-novices. I also finished the lightning talk PowerPoint to accompany my poster. After I receive the last participant, I will be ready to start the data analysis.
Goals:
Weekly Goal: Gather participants
Long Term Goal: Use the participant data gathered to conduct another analysis and train a machine algorithm to create a tag prediction system for Stack Overflow. We may or may not look into how to automatically predict keywords in the text as well.
I am also working on an Operating Systems project with a few other students in the department, and it is going to take up a lot of my time. Practicing the Unix commands will help me with Apache Spark so I'm excited to be able to do these simultaneously.
This week, I gathered more participants for our Stack Overflow study. I'm at 5 members so far and with more scheduled next week so I will be able to meet my goal of at least 6. I finished my poster about the results of our eye tracking study regarding participant tag scores based on novices v.s. non-novices. I also finished the lightning talk PowerPoint to accompany my poster. After I receive the last participant, I will be ready to start the data analysis.
Goals:
Weekly Goal: Gather participants
Long Term Goal: Use the participant data gathered to conduct another analysis and train a machine algorithm to create a tag prediction system for Stack Overflow. We may or may not look into how to automatically predict keywords in the text as well.
I am also working on an Operating Systems project with a few other students in the department, and it is going to take up a lot of my time. Practicing the Unix commands will help me with Apache Spark so I'm excited to be able to do these simultaneously.
Thursday, February 2, 2017
Week 18: 1/25/2017 - 2/1/2017
Accomplishments: This week, I had a student come into the Empirical studies lab and participate in our Stack Overflow study. I have two more students scheduled to come in next week. This is vital for fresh analysis as we continue with machine learning.
I presented the rough draft of my OCWIC 2017 poster about the results of the first half of our data analysis to Jenna, Dr. Sharif and Dr. Lazar. Ali calculated tag accuracy scores (relevant tags compared to distractor tags chosen by participants), and scores were split up into three categories: Simple, average and complex. Our results showed that simple level tag scores were 97.46%, average: 89.76%, and complex: 87.04%. From these results, we could conclude that users spent average time considering all 10 tags after viewing c++ post contents. After reviewing fixations, saccades and fixation durations, we split the data down further by participant experience level, novice v.s. non-novice. Novices were all those with 1 year or less programming experience and non-novices consisted of those with 3 years or more. We concluded that non-novice developers performed better than novices in tag assignment and depended more on code, and novices were less accurate in tag assignment as they assigned less tags on average.
Weekly goal: My goal is to get at least 6 students in the department to participate in the study. If I can get at least 1 student a week, my goal will be met. My next move is to finish the poster and present these findings at OCWIC. Then, I will continue to train data into Apache Spark.
Long term goal: Create a successful tag prediction system using the tag scores of these individuals.
I presented the rough draft of my OCWIC 2017 poster about the results of the first half of our data analysis to Jenna, Dr. Sharif and Dr. Lazar. Ali calculated tag accuracy scores (relevant tags compared to distractor tags chosen by participants), and scores were split up into three categories: Simple, average and complex. Our results showed that simple level tag scores were 97.46%, average: 89.76%, and complex: 87.04%. From these results, we could conclude that users spent average time considering all 10 tags after viewing c++ post contents. After reviewing fixations, saccades and fixation durations, we split the data down further by participant experience level, novice v.s. non-novice. Novices were all those with 1 year or less programming experience and non-novices consisted of those with 3 years or more. We concluded that non-novice developers performed better than novices in tag assignment and depended more on code, and novices were less accurate in tag assignment as they assigned less tags on average.
Weekly goal: My goal is to get at least 6 students in the department to participate in the study. If I can get at least 1 student a week, my goal will be met. My next move is to finish the poster and present these findings at OCWIC. Then, I will continue to train data into Apache Spark.
Long term goal: Create a successful tag prediction system using the tag scores of these individuals.
Week 18: 1/25/2017 - 2/1/2017
Work Accomplished
This past week I wrote the Analysis and Data sections of the paper we are submitting to the Mining Software Repositories conference about mining eye-tracking data for software tasks (an extension of last year's CREU project). I wrote about the within sequence turbulence and entropy analyses I am still gathering results for, and I wrote about the sequence similarity k-means clustering analyses I am also still gather results for. I also prepared and practiced a presentation for the 2017 Nebraska Conference for Undergraduate Women in Mathematics.
Goal
Weekly goal - In the next week, I plan to finish gathering results for the sequential analyses I am performing and write the Results and Discussion sections of the paper. I am also leaving for NCUWM 2017 this weekend to present the sequence similarity analyses.
Long-term goal - To perform many more (roughly 8) sequential analyses on the ABB eye-tracking data we collected three years ago. From this we hope to gain insights into developer gaze patterns and differences among novice and expert programmers. This analysis considers the order in which developers looked at source code elements, so in this way it is different from last year's machine learning analyses.
Outcome(s)
This past week I wrote the Analysis and Data sections of the paper we are submitting to the Mining Software Repositories conference about mining eye-tracking data for software tasks (an extension of last year's CREU project). I wrote about the within sequence turbulence and entropy analyses I am still gathering results for, and I wrote about the sequence similarity k-means clustering analyses I am also still gather results for. I also prepared and practiced a presentation for the 2017 Nebraska Conference for Undergraduate Women in Mathematics.
Goal
Weekly goal - In the next week, I plan to finish gathering results for the sequential analyses I am performing and write the Results and Discussion sections of the paper. I am also leaving for NCUWM 2017 this weekend to present the sequence similarity analyses.
Long-term goal - To perform many more (roughly 8) sequential analyses on the ABB eye-tracking data we collected three years ago. From this we hope to gain insights into developer gaze patterns and differences among novice and expert programmers. This analysis considers the order in which developers looked at source code elements, so in this way it is different from last year's machine learning analyses.
Outcome(s)
- Large portions of our MSR paper are complete; we are on time to submit by Feb. 10
- R scripts implemented to perform entropy and turbulence sequential analyses using TraMineR
- Partial implementation of R scripts to perform the similarity analyses using TraMineR
- Learned more about the mathematics of the analysis techniques for my presentation
Subscribe to:
Posts (Atom)