Khan Academy SAT review design

I interned on the SAT team at Khan Academy in 2016. Khan Academy partnered with the CollegeBoard to provide free, personalized SAT prep. My first project at KA was to design the most frequently-requested feature — enabling students to review their previous answers.

Designing the review experience

Design goal

When I joined, students could do practice SAT questions and take practice exams, but the review mechanism was quite limited. For some practices, students could never review what they did. For other practices, students may review their answers right when they finish the practices, but once students leave the practices, they can’t go back.

I really wanted to review and resolve all the problems that I solved because I didn't really get problems so I wanted to check for my understanding. However I couldn't go back…

— SAT student

Thus the goal of my first project is to make sure students can review their past answers anytime, when:

• they’ve exited the practice, and
• they’re finishing the practice.

Brainstorming and sketching out ideas

I spent the first week interviewing tutors to understand how to help students get the most out of review, collecting inspiration, and brainstorming ideas with the team.

After brainstorming, I continued by sketching out some of the most promising ideas to distill them down and also to get feedback from the team. This phase was pretty short and I transitioned to digital mocks once I figured out the major directions.

Where does history live?

Then I developed the most promising directions into digital versions and went through design reviews with the team. It became clear that there were two pieces in the project: figuring out a central place to access the history, and putting together a flow for students to step through the practices once again.

To review their answers, students need a place to access them after they leave the practice. The first decision to make is where the history lives:

• inline history info under Practice
• as a separate "Review" page
• at the end of practicing the same skill

I decided to go with storing the history in a separate Review page. The goal of the practice page is to practice, and the goal of the end-of-practice page is to celebrate the completion of the practice and strengthen the learnings, so review should be subordinated there. The idea was also confirmed in the student validation sessions that review is a separate step from practice.

Exploring representation of questions

Once I settled down with creating a Review page, I made 2 decisions: 1) sorting the questions by date of practice since this is a history page; and 2) grouping the questions into tasks to make the review experience consistent with practice. It was unclear, though, what was the best way to represent the questions. I experimented with a few options including:

• question titles
• checkmark / cross boxes from SAT navigation
• time bar charts
• number of correct / incorrect questions.

I went with the last one because the other ones are showing more information than needed for "choosing the tasks to review". Also, the last option has the least visual clutter and effectively leads students to use the call to action — reviewing the practice.

Pushing the pixels

Then I iterated with the visuals of the idea, using vertical grid to achieve the best visual balance.

Putting together the flow

The second part of this project is to put together a flow for reviewing the practices. The final solution is quite simple; the experience of reviewing questions is very similar with practicing them, regardless of where the students are accessing the review.

Validating with students

I made a prototype with Marvel and tested with students. The design was well received; one major finding was that students would like to review questions within one subject in a row. In the final product, I made the subject filters more prominent to address this finding. I also tweaked the copy a bit to make the navigation more intuitive.

Improving SAT

I also worked on a bunch of SAT design improvements.

Rethinking the action links: hint · video · report a mistake

In the footer of practices, we had action links along with the question navigation. It was pretty visually cluttered. I wanted to design a cleaner solution, and make it future-proof, where each question would have its own action links. So I separated the action links from the navigation in the footer, moved them next to each question title, and made visual improvements.

Rethinking question practice navigation

I also updated the navigation of question practice following some of the patterns we settled in the Review project. The challenge was to take into account all the cases across the entire product. In SAT, we have 4 types of tasks × 2 subjects × 2 modes = 16 plus cases. Depending on these parameters, the navigation could vary. Specifically, the questions I answered were:

• Can students skip to the next question?
• Can students submit the answers with unanswered questions?
• What should students do after finishing a task?

Shipping design

I really liked that at KA, I got to see a lot of my design works shipped by our awesome developers. All the design work mentioned above was shipped in June. Since there were lots of variations and nuances in the design, I used Google Slides to detail out the flows and changes, and Zeplin for specs. This helped a lot with minimizing the hand-off work. I also learned to take into account the various application states (e.g. blank states, loading states).

Other internship highlights

Aside from SAT, I spent a lot of time researching and designing a new mobile product. Outside of SAT and this new product, I also got to work on a bunch of other things.

404 page

KA content generator Sketch plugin

KA content editor WYSIWYG concept

Sensei and teacher badges

KA in China tech exploration

In this hackathon project, a team of us explored the tech feasibility of making KA faster in China. It seemed to work! Super excited for the progress.

Thank you

Whoo, this is a really long post, and this is only ~half of my internship! I’m really grateful that I got to work on so many things, and that everyone I worked with was genuinely kind and talented. S/O to Kitt, Tabitha, Anju, May-Li and the rest of SAT team for making this internship a fun and rewarding experience.

For mandarin readers: I also wrote a post on Zhihu about this internship. 🐼