Practicing for the Pressure of Performance with Piano Lab Pro

Woman in blue blouse shown in profile sitting at a piano with an XR headset on.
Aya Higuchi Hagelthorn, lecturer of piano pedagogy at the School of Music, Theatre & Dance, sits at a Piano with Piano Lab Pro shown in her XR headset. Charlyn David | Center for Academic Innovation

XR sets the stage to help students prepare for the big night

Sean Corp, Associate Director of Communications

Piano students can practice a difficult passage until their fingers know every note. Walking out from the wings and onto the stage of Hill Auditorium is different. The silence can seem to fill the room. That moment, when you walk to the piano bench, take your seat, and put your fingers on the keys, is almost impossible to prepare for. 

It’s not just the silence, of course. There is the quickening pulse, the distant cough, the squeaking of a chair, the chirp or rumble of a smartphone. The sound of a large concert hall can unsettle even a seasoned musician.

Piano Lab Pro, an extended reality immersive experience, gives University of Michigan students a way to rehearse those moments before a performance. Developed through a partnership between the Center for Academic Innovation, the School of Music, Theatre & Dance, and the Institute for Social Research, the immersive application recreates performance settings in which students can practice navigating the pressures of playing for an audience.

Aya Higuchi Hagelthorn already had plenty of experience helping her students prepare for the anxiety of performance, and she wasn’t afraid to use unconventional means. Hagelthorn, lecturer of piano pedagogy, director of collegiate class piano, and coordinator of the Piano Pedagogy Laboratory Program at the U-M School of Music, Theatre & Dance, blared random YouTube videos in the background. She asked students to run up and down stairs to raise their heart rates right before playing a piece. 

The hardest part to prepare for, though, was the size and scale of stepping into a performance hall for the first time. The 3,500-seat Hill Auditorium doesn’t exactly accept walk-ins who want to rehearse. That lack of access and familiarity creates a barrier that can keep students from delivering a note-perfect performance. This often means they can’t fully showcase the abilities they’ve worked so hard to develop. 

“Too often, students leave an exam feeling that their nerves kept them from showing what they can do. An exam should reflect their true skill,” said Hagelthorn, lecturer of piano pedagogy and director of collegiate class piano at SMTD.

If routine visits to practice on the Hill Auditorium stage were off the table, Hagelthorn and the center devised a way to bring Hill to the students. 

Woman in dress stands at center of frame with arms at hips on the stage of a theater.
Aya Higuchi Hagelthorn stands on the stage at Hill Auditorium. She worked closely with the XR team at the Center for Academic Innovation and other partners to recreate the visuals and acoustics of Hill in a virtual environment for Piano Lab Pro. Charlyn David | Center for Academic Innovation

Bringing a hard-to-access stage into reach

Piano Lab Pro recreated U-M’s Britton Recital Hall and Hill Auditorium, allowing students to experience those settings through a mixed reality headset.

During the team’s work at Hill Auditorium, Eric Schreffler, the center’s XR developer lead, explained that the aim was to capture both the appearance and acoustic properties of the space.

“So that when they come out here for the first real time, they’re not overwhelmed by the experience, and they can focus on their actual performance,” Schreffler said.

The project became CAI’s first application built for Apple Vision Pro. Its contributors included staff and student fellows working in XR development, environmental art, user experience, cognitive science, performing arts technology, learning experience design, audio engineering, and project management.

Man holds camera up high while pipes adorn the wall of Hill Auditorium.
Ray Majewski, XR environmental artist at the center, takes hundreds of photos at Hill Auditorium, which will serve as part of the reference material as it is built in a virtual environment. Charlyn David | Center for Academic Innovation

Making a virtual stage feel consequential

The team worked to replicate Britton Recital Hall, where many piano juries and student performances take place, as well as Hill Auditorium.

Making a concert hall available in a classroom began with a decidedly physical task: placing tape markers across the floor, walls, and stage of Hill Auditorium.

The team used photogrammetry, a process that combines overlapping photographs taken from different positions to reconstruct a space in three dimensions. Rather than build every architectural detail by hand, developers could capture the hall’s geometry, colors, and surface details through thousands of images.

There was one problem – asking computer software to correctly interpret and connect the thousands of images was challenged by the repeating patterns on Hill’s stage floor and walls. It made it difficult for the software to distinguish one section from another. 

The center’s XR team solved this problem by randomly positioning tape to create recognizable reference points. The reference points worked like the edges of a puzzle piece, allowing each photo in the database to be more easily identified and connected to its neighbors. Handwritten codes on stage markers gave the developers another way to match images if the software needed help.

Schreffler described passes of roughly 600 to 1,000 photographs, with the camera moving between shots to capture changes in perspective. In total, the team took 5,500 images of Hill Auditorium and 2,500 photos of Britton Recital Hall. Developers then loaded the photographs into a computer for a preliminary reconstruction. That helped reveal any missing areas or details that needed to be captured.

While there were myriad technological challenges to solve, the team ensured they always focused their energy on what would most benefit the students. 

“We had to think carefully about what a student would see and hear from the piano bench,” said Moeezo Saleem, who led the project as a senior XR developer at the center. “Our staff, Student Fellows, and partners brought different skills to the project, but we kept coming back to the same question: How does this detail help a student practice for the real experience?

“That meant not only recreating the visual and audio details of the performance Halls, but also seamlessly blending those virtual spaces with the physical environment so students could play a real class piano while experiencing what it feels like to perform on a grand piano in Britton or Hill Auditorium.”

A group of women position equipment and a piano bench, with microphones on each side on the stage of Hill Auditorium.
Hagelthorn and the design team work to get the bench and several microphones in place to appropriately capture the acoustics a player would experience while playing piano at Hill Auditorium. Charlyn David | Center for Academic Innovation

Recreating what a performer hears

But a convincing performance environment requires more than a view from the piano bench. The size and shape of the room impact what a musician hears, too.

Anıl Çamcı, associate professor of performing arts technology at SMTD, and his graduate students captured the acoustic properties of both venues. Among their methods: popping balloons in different locations to record how sound moved through each space. Those properties could then be applied to piano audio.

“When you go on the stage in the Hill Auditorium, the grandeur of the space is amazing,” Çamcı said. “The audience is daunting, but one of the most critical aspects of that experience is that all of a sudden, the auditorium expands your instrument sonically.”

The team also worked on audience sounds, including coughing, laughter, and a ringing cellphone. All of this combined to help musicians become familiar with the distractions often found in performance spaces. 

Woman in blue shirt wears XR headset at table while another woman observes her.
Moeezo Saleem, a senior XR developer and lead on the Piano Lab Pro project at the center, assists Hagelthorn in putting on an XR headset. Charlyn David | Center for Academic Innovation

Connecting the experience to evidence

Creating a realistic performance environment is one part of the project. Understanding what happens to students within it is another.

Researchers from ISR’s Biosocial Methods Collaborative designed a pilot using physiological sensors, surveys, and student reflections to examine how performers respond to pressure. Students reviewed recordings of their performances and described what they had been feeling at particular moments.

“The sensors really were picking up and corroborating what the musician was saying,” said Alicia Carmichael, research process director at the collaborative. “When they said they felt nervous, the signal really did spike up, and so their physiology was corroborating their story.”

That work gives the partnership a way to investigate the experience beyond whether a virtual concert hall looks or sounds convincing. Researchers can examine students’ responses, while faculty consider how those insights might inform instruction.

After the final note

The team has more research to do as it makes Piano Lab Pro available to more students. She is currently incorporating the experience into her Fall courses across SMTD’s piano sections. This will include a companion module called “Performing Under Pressure,” and some students will also complete guided sessions in the Apple Vision Pro headset.  

Hagelthorn said she is looking toward the moments that follow a real performance: when students leave the piano knowing they have accomplished something that once felt beyond their reach.

“When a student walks off stage, and you see the joy and pride on their face, it’s hard not to feel it yourself,” she said. “They’ve discovered what they can do. You help however you can, but that accomplishment belongs to them. 

“That’s what I hope this experience makes possible for more students — a confidence they’ve earned that they can carry into whatever comes next.” 

Recent Posts

Practicing for the Pressure of Performance with Piano Lab Pro

Practicing for the Pressure of Performance with Piano Lab Pro

Read More
Center-Supported AI Projects Give Students Safe Space to Learn

Center-Supported AI Projects Give Students Safe Space to Learn

Read More
UT Austin AI leader will discuss how AI tutors engage students in deeper learning

UT Austin AI leader will discuss how AI tutors engage students in deeper learning

Read More