Some people may scoff at the idea of crystal balls offering a window into the future, but according to Sidney Fels, there is one that just might show us the future of P2P communication. Reece Webb reports.
Sidney Fels, professor of electrical and computer engineering at the University of British Columbia in Vancouver, Canada, has played an instrumental role in developing Crystal, a 3D spherical display that uses multiple projectors to display images on a spherical surface.
Fels has a strong background in research, working at the University of British Columbia for over 20 years as a professor, with previous experience as a researcher at the AJTR Institute in Kyoto, Japan.
He also has a background in computer science, psychology and interactive arts that helped to shape the research into the spherical display.
For Fels, developing a first of its kind display has been a long term project stretching back over two decades.
He explained “It’s been an ongoing project since the late 90’s. The original idea revolved around looking into a box, not a sphere. Dr. Ian Stavness, a professor at the University of Saskatchewan and I worked on building a cubic version.”
The cubic display was made in different sizes, but the biggest issue with a cubic display was the seams of the cube.
Researchers investigated how large the seams could be made before they had an effect on the perception of content displayed on the screen.
Fels said “We were investigating how big those seams could get and the logical step was to have a display that has no seams that would be a spherical screen.”
The transition from cube to sphere occurred five years ago and presented unique geometric challenges that had not been overcome before. A cube was simpler to build and didn’t need accurate calibration. However, calibration for a sphere proved challenging, with existing solutions using a single projector.
Fels encountered limitations with a single projector format, saying “A single projector solution that fills the sphere gives you low resolution and the best you’re going to get is an HD type projector or a SuperHD, which get very expensive.
“You still can’t get uniform density because you need a fisheye type lens to spread the image around, so you start to lose quality because of that.”
These technical limitations convinced the team of a need for a multi-projector solution that could be scaled.
The team later worked out how to optimise the calibration to allow multiple projectors to fill the sphere, allowing a high resolution, high brightness, low-cost spherical display.
Fels and his team were not the first to attempt to create a spherical display, but none had succeeded in making the display operational in high resolution with multiple projectors.
Fels worked with collaborators in Brazil, adapting their multi-projector cave system, using projectors that were calibrated together on a planar surface.
But Fels encountered limitations with this approach when applied on the sphere.
“You can’t get the image to go right to the edge, everything has to be in the centre of the sphere, so we needed a different approach.”
“We had to think in terms of spherical coordinates and calibration along the surface, not making assumptions about the curvature and using calibration to compensate for a camera-projector pairing so that it could figure out the topology, allowing us to do the depth, mapping and blending calculations.”
Crystal is comprised of a hollow plastic sphere with a hole cut into the bottom. Beneath the sphere is a camera and multiple Asus projectors on tripods which project through the hole to the inside surface of the sphere.
Each projector projects calibration markers on to the surface, allowing the camera to pick them up.
The Asus projectors feature no special lenses and are arranged by hand to fill the screen and overlap.
“The projectors don’t have any external syncs, so you can’t synchronise them with hardware. We used bespoke software and a synchronisation circuit that we built to make sure the left image and the right image are all synchronised.”
For Fels, the greatest challenge came in getting the projection mapping to the surface of the sphere, figuring out the mathematics to allow the curvature of the display to be arbitrary. To do this, the team used the camera-projector pairings with calibration projection, determining where the surface is in the free space, locating exactly where each pixel would fall on the surface.
Some VR/AR experts feel that shared AR/VR experiences could overhaul the P2P experience, driving the industry closer to 3D communication. Fels believes that spherical displays could help drive that transformation.
“Our perspective on VR worlds and some of the AR experiences is that they’re not that effective for the P2P experience, because of the lack of eye contact and lack of co-location experiences. With VR; if we’re in the same room you’ve got all this gear on, so you can’t have human-human contact very easily.
“In the spherical display, because it’s a mixed reality, it is in the real world. When you point at something inside the virtual scene, I know what you’re pointing at, I know what you’re looking at.
“The ability to have that experience of eye contact and deictic gestures, allows us to cooperate much more easily. We think there’s a big future for that.”