AG350

Overview

At Duke University in 2019, most instructors in electrical engineering and computer science departments were forced to individually run student submissions one-at-a-time and then report grades by whatever means they choose -- and this process can take weeks for each assignment. AG350 is an autograding web app used by students and teaching faculty in Duke’s Electrical Engineering and Computer Science departments aimed at entirely streamlining this process.The app allows users to submit assignments, evaluate code using built-in tests, and assess grades.

During the 2019 spring semester at Duke, I was the lead UI & UX designer for the AG350 team. I worked directly with the CEO to redesign aspects of the existing product, as well as carried out exploratory user research, user flow design and wireframing for new product features.

The challenge(s)

‍

When I joined the team, AG350 was still in early stages. The product allowed students to submit and build their assignments (in other words, run their code). But an “instructor view” of the product that allowed faculty to evaluate class assignments, and an interface for exploring test data to see what errors occurred in the submissions were still just goals in the product roadmap. Beyond designing the features to address these issues, my overall focus with this project was answering: How can we reduce confusion in the routine submission and grading experiences of students and teaching assistants? (In other words, how could I help users easily answer questions like “How did my students do overall on this assignment?” or “How can I improve my code?”)

So I was really focused on 2 challenges:

Faculty need a streamlined, transparent way to assess student progress so that they can reduce hours spent grading and have more time to teach.

Students need a simple, time-efficient way to build assignments and evaluate course progress so that they can spend more time on extracurriculars, their social life, other homework, etc.

I worked with the team to create solutions that were clean, accessible, and intuitive.

Research

I began the design process by gathering information on users, their pain points, product functionality, and so on. The foundation of knowledge I formed from this stage allowed me to give a face to the users and note some key issues to address. Both the AG350 team and I highly prioritized accessible design, and they collaborated with me to find users with unique accessibility needs.

‍

My research methods included:

• Several rounds of user interviews with students and TAs to learn about user behavior.
• Open ended conversations focused on understanding the larger systems AG350 fits into with colleagues who have been students and/or TAs.
• A 5 question survey focused on understanding how AG350 is used now and any shortcomings.

User personas

Key findings

• TA’s want to view aggregate grade data and stats. Having an overview can help summarize grade trends.

• TA’s cannot view assignments and grades for all students from a central page. The current process requires several steps and visiting each student’s Github page.
• TA’s can run thousands of tests and can’t tell students which tests they failed. There is no way to parse data.

• Students are confused about what defines an “unsuccessful” build. There is no feedback explaining if the build failed to run versus if it ran and had errors.

• When a build is in progress, a user would have to refresh the page to see an update. It’s a guessing game to see how long it takes.

Defining goals for success

My conversations with the AG350 team allowed me to establish goals that were prioritized for the product’s success. And based on the learnings from my research, I added more goals to give my process a meaningful direction and further define what success would look like:

1. The design is accessible and removes points of exclusion to reach a diverse audience → there’s a focus on being high in contrast, RG color blind friendly, & screen reader friendly.
2. Grade data is visually displayed to students in non-stressful and clear way.
3. Instructors can understand meaningful information concerning student grade data in seconds.
4. Instructors are able to search through assignment data and find sources of error.

Ideation and initial wireframes

My initial wireframes were pretty exploratory. At first I designed an assignment overview page that displayed test results with either green bars and “Passed” or red bars and “Failed”. But discussion with the team led me to learn that test cases can be complex and are better accompanied by numeric scores. Additionally, with the goal to display data in a way that both did not stress out students and was accessible, we decided to scrap red & green visuals. These would emphasize shortcomings and would be poorly suited for RG color blind folks.

Journey map

I mapped out a user journey to illustrate how users would likely engage with AG350. This helped give vision to the project and pinpoint common user tasks.