An assessment tracking tool for teachers in Project-Based Learning (PBL) classes.
Design for Education
iPad App
Client: Winchester Thurston High School
Duration: 3 Months | Sep.2018 - Dec.2018
Team: Angie Wang, Greg Bunyea, Zeqian Peng (UX Researcher, Product Designer)
Client Background: Winchester Thurston School (WT), an independent school in Pittsburgh, is in a unique position to facilitate Project-Based Learning (PBL) courses. City as Our Campus is an initiative that brings community-based learning to provide authentic learning experiences for students from diverse backgrounds and academic interests.
Our solution was presented to the principal and faculties in WT high school and was highly recognized.
To better understand students’ experience, expectations, instructors’ pain points and also the interaction between students and teachers, we conducted 3 contextual inquiries with PBL “experts”, 2 stakeholder interviews with the director of City as Our Campus program and an instructor from “Research Science” class, and one class observation.
What is the current experience for students in PBL classes?
Contextual Inquiries
with 3 PBL "expert" students
In order to learn students’ experience, expectation and behavior in PBL classes, we targeted at 3 PBL experts and conducted contextual inquiries with them to observe their work style and inquire information. We created user profile boards, used affinity diagram and identity model (see them here) to synthesize our interview data and find some common mentality of expert students:
What are the current challenges in implementing PBL classes?
Stakeholder Interviews
with the director of the program and a PBL class teacher
Interview 1
Adam Nye,
Director of City as Our Campus
We discussed the implementation challenges of PBL in formal education:
Interview 2
Graig Marx,
Instructor of "Research Science" class
We learned a lot about the instructor’s role in Research Science, which is incredibly central. In order to foster a more open PBL environment which is highly attracted to students, the instructor has no syllabus, seldom use of rubrics, and few assessments without specific criteria. Thus he has difficulty determining the final grades for students due to lack of evidence, problems anticipating which student groups will get farther in the design process, insufficient formal feedback and no systematic process for evaluating student learning.
How do teachers and students interact during PBL classes?
Class Observation
in "Research Science" class
To understand the atmosphere in PBL classes, the way student groups work together, interactions between the teacher and students, and also how teacher observes, documents students' behavior, we conducted a class observation in the "Research Science" class taught by Mr. Graig Marx. To analyze the data we observed in class, we created a simple graph to manifest what we observed:
Observations
What we gained from the user study...
Open Nature
vs
Structure
One of the core values of PBL classes is the open nature of it: students can select their project topic, most of the design decisions are open-ended, there is no fixed schedule etc. Through our user study, we found that the tension between open nature vs structure is a common case and cause for most of the assessment problems:
Hard to Assess
Students' Competencies
Insufficient
Evidence Tracking
Hard to Locate Evidence
Uneven Attention to Groups
We learned that the open nature of the class makes it hard for the instructor to use traditional structured pedagogy to do assessment. In order to help teachers in this class, how can we better facilitate such open-ended project-based class to make it somehow more structured but also not to break the balance and openness of the class?
How did we generate and convey our ideas?
Brainstorming and Storyboarding
To address the problems identified in the first phase, we conducted brainstorm session to create How Might We (HMW) notes, to better understand current challenges in the class and also what are the possible ways to solve them. After that, we discussed and voted to narrow down our potential solutions into 15 scenarios and made them into storyboards.
How did we validate users' needs and consolidate our solution?
Speed-Dating
After creating storyboards, we conducted Speed Dating sessions with instructors to validate whether the needs depicted in the storyboards are real needs of them. The validated needs and potential solutions are listed as follow:
A New Way (Sliders)
to assess students’ competencies
Paper Trail
to document students’ performance
Bi-weekly Student Report
to help students reflect
Calendar-based Evidence Tracking
to help trace back past notes
Final Grade Suggestion
based on students' performance
Setting Milestones
to add more structure to the class
Consolidating ideas into a journey!
User Journey Map
According to the feedback we got from the Speed Dating session, we synthesized potential needs and feature into a more integrated solution:
Based on what we learned from user study, speed dating session and the final user journey map, the idea of making a native iPad app to replace the notepad had come to a stage that can be prototype to better validate the solution. In order to do that, we used both paper and digital prototype technique to conduct two filed tests with the instructor.
How did we make our ideas come true and test them?
Paper Prototyping & Wizard of Oz Testing
In order to test our idea in a fast way, we used pen and paper to build our prototype and conducted Wizard of Oz testing to get feedback:
Field Test 1
What we learned from the first round of field test:
Digital Prototyping & Scenario Walkthrough
According to the feedback we received from the first field test, we put our design into a digital format including a digital wireframe and a dynamic inVision prototype. To test on how the instructor would use the product in his daily scenarios, we designed some situations for the field test 2 to observe how the app can be helpful and what are the things that need to be improved.
Field Test 2
During field test two, we asked Mr. Marx, as the participant, used our digital prototype to test some of the scenarios we created for him, including:
Scenarios
After the scenario walkthrough, we validated most of the features we ideated but also found following elements to be improved:
After user study, ideation sessions and user testing, we selected several core features to focus on, while other needs are still be potentially considered in future development. The final design was presented to the principal and faculties in the WT high school and was highly appreciated for our right-on-spot, light-weighted and user-friendly design.
Smart Group Order
Slider to Assess Compentencies
Multimedia Note Taking and Sharing
Evidence Tracking Calendar
Throughout the whole process of designing the product, we all learned a lot about the challenges in PBL education and how schools are making efforts to cast impact on the community. After the project, we also reflected on ourselves about what did and did not go well and what is the future work as our post mortem.
What went right?
What could be better?
