Stephanie R. Haley, User Experience Research Lead

Nathan Magyar, User Experience Designer

Rebecca M. Quintana, PhD, Learning Experience Design Lead
@rebquintana

Yuanru Tan, Learning Experience Designer and Accessibility Coordinator
@YuanruTan

At the Center for Academic Innovation, Learning Experience Designers (LXDs) and User Experience Designers (UXDs) collaborate, exchange resources, and combine methods in order to best support the design and development of our online learning experiences and software projects.

One of these software projects, Gamut, is a suite of online learning tools for use in the university’s residential courses and online learning experiences. Included in this suite of tools is a web application called the Gallery Tool, which provides learners with a digital space to share written and visual artifacts with each other as well as provide feedback on them. In a previous blog post, Evolving the Learner Persona Creation Process at Academic Innovation, we discussed how LXDs leveraged the UX method of personas to inform the course design process. As a follow up to that conversation, we would like to present how LXDs and UXDs joined forces to create the Gallery Tool, and highlight how the UX team learned from LXDs during the experience.

Background

Online course delivery platforms such as Coursera, edX, and FutureLearn offer a variety of features that allow design teams to create rich, educational environments. However, these capabilities are not always well-suited for constructing an experience that will best support the learning goals of a specific activity. Instructional teams must often work within the constraints of the platform features available to them (Head, 2017; Najafi, Rolheiser, Harrison, & Håklev, 2015).

While developing a specific online course (Storytelling for Social Change), LXDs observed and identified limitations learners face when interacting with online course delivery platforms. These included inefficient information exchanges within asynchronous peer feedback systems (i.e., on peer-graded assignments) and online discussion forums that present major usability challenges. Ideally, a digital solution to this problem would integrate with any online learning platform, allow learners to share work and receive timely feedback from their peers, and structure the peer feedback process so as to elicit high-quality interactions.

To further explore this potential solution, our LXD team collaborated with the Gamut team to develop the Gallery Tool, a tool developed through our software design and development process and informed by the learning needs and related pedagogy identified by the LXDs.

The Process

After the LXDs proposed their solution to these limitations, the Gamut team held informational interviews with LXDs to frame and scope the problem and collect feature ideas. These conversations also allowed the UXDs to better understand LXDs’ pedagogical design needs.

Next, LXDs identified two MOOCs which would guide the design of the tool, one focused on writing (Storytelling for Social Change) and the other on programming (Python Basics). These courses contained assignments that were well-suited for peer sharing and feedback and allowed us to explore how the tool might work in a range of subject matters. Meanwhile, the UXD mapped the tool’s features onto corresponding proposed pages and shared these groupings with the LXDs for feedback.

From there, the UXD iterated through design solutions, starting with paper-and-pencil sketches and ending with a realistic mock-up (see Figure 1). The UXD then drafted a usability test protocol for the high-fidelity mock-ups in order to obtain feedback on usability and learning goal. Our UX Researcher ran usability tests with 15 undergraduate learners and summarized her findings: overall, learners thought the tool was easy to use. Suggested improvements included different wording for certain page titles (i.e., using “All Submissions” as the page title for the list of submissions instead of “Browse”) , and more advanced search options (i.e., being able to search by author name).

After six weeks of rapid development, the Gallery Tool was complete.

Reflections

The results of this collaboration between LXDs and UXDs did not stop with the Gallery Tool. UXDs gained valuable insights from LXDs about designing for educational software for use with the development of future online learning tools. Below are three key takeaways.

Optimize, then prioritize

Traditionally, UX design is focused on making interfaces more intuitive and pleasant to use. As a part of this process, UXDs must consider the kinds of experiences and interactions users are accustomed to having with similar systems. They must then work to incorporate those aspects into their own design so that the system is as familiar to users as possible. When designing the submissions page in the Gallery Tool, the UX team initially adhered to the conventions of other popular content-driven sites, like Facebook, and designed the page to list submissions in reverse chronological order; most recent items were at the top of the page, and the oldest, at the bottom.

Although LXDs agreed this decision optimized the page’s UX, as it conformed to users’ expectations, they also suggested it didn’t necessarily meet the goals of the learning experience. The Gallery Tool, after all, aimed to facilitate the feedback process and promote learner discussion evenly across all submissions. The initial content order did not support these goals. Instead, it actually encouraged conversation on the most recent items at the top of the page. To resolve this challenge, LXDs and UXDs decided to order content from most to least in need of feedback, which they calculated using a simple algorithm: submissions that had gone the longest without receiving any views or written comments floated to the top of the page, while those that had received attention fell toward the bottom.

Thus, the UX team learned the importance of considering the pedagogical goals of an educational software tool from the LXDs as well the value in prioritizing those aims after sufficiently optimizing a tool’s UX.

Can I customize that?

As a whole, Gamut strives to create educational web applications that are usable across a variety of different disciplines and contexts. To ensure such flexibility, the team consistently seeks a diverse combination of initial use cases. In designing the Gallery Tool, UXDs and software developers looked to LXDs to provide valuable insight about places where the tool would benefit from certain configurable features. For example, thanks to LXDs, the Gallery Tool now allows instructors to create up to two custom lists of meta tags to which learners then apply to their submissions during the upload process (Figure 2) and subsequently use as filters while exploring others’ work.

Without LXDs representing instructors’ perspective of the tool, the UX team may not have known how important it was to leave room for such customizations or which aspects of the tool needed to be configurable.

The art of scaffolding

UXDs typically aim to design an interface so that it requires as little explanatory text as possible. However, in the case of the Gallery Tool, the feedback-focused nature of the tool posed unusual challenges to users and the UX team alike. During the design process LXDs pointed out that, for some users, giving feedback to others is an unfamiliar experience. Knowing what observations to make and how to communicate them requires skill and experience. The challenge for the UX team then became to find a way to support users who were less experienced with giving feedback.

LXDs suggested adding a small section near the feedback form that relayed a handful of tips for providing good, quality feedback — such as “Be as specific and detailed as possible” and “Avoid overly negative language.” After implementing this idea, the UX team learned that sometimes additional scaffolding is necessary so that users can successfully use the interface as well as have the desired learning experience.

Conclusion

The Center for Academic Innovation is a highly collaborative environment in which individuals from different disciplines come together in pursuit of creating the best learning experiences possible. This is exciting not only because it leads to high-quality educational experiences and technologies, but also because it inspires new ways of working together and learning from each other. The Gamut Gallery Tool is just one example of this process at work. At the time of writing, 1,696 submissions have been generated across seven different instances of the Gallery Tool (six online course instances and one residential instance). Both LXDs and UXDs have grown significantly in their professional knowledge and skills by sharing and leveraging each other’s expertise and continue to find ways to build on this collaborative, cross-disciplinary approach.

References

Head, K. J. (2017). Disrupt This!: MOOCs and the Promises of Technology. Lebanon, NH: University Press of New England.

Najafi, H., Rolheiser, C., Harrison, L., & Håklev, S. (2015). University of Toronto instructors’ experiences with developing MOOCs. The International Review of Research in Open and Distributed Learning, 16(3).