Imagine a video call where the person on the other side of the screen feels real, almost as if they are really present in front of you. A 3D real-life version of themselves and yourself in a video call, instead of the flat two-dimension look we are used to at the moment.
The user converted into a 3D Hologram of themselves is not a new idea. It is also one that big tech companies have been pitching fervently for a while now. Google’s Project Starline is the latest attempt at this. More recently, Microsoft showed its own take on mixed reality with its Mesh platform.
Google showcased a glimpse of its Project Starline at the ongoing I/O conference. “Several years ago, we kicked off a project to use technology to explore what’s possible. We call it Project Starline. It builds on the different areas of computer science I spoke about today. It relies on custom-built hardware and highly-specialised equipment. It is early and currently available in a few of our offices, but we thought it would be fun to give an early look at experiencing it for the first time,” Sundar Pichai said during the I/O keynote.
In a blogpost about Project Starline, Google is very sparse when it comes to details, which is understandable given this is still a developing field. It only states that Project Starline “combines advances in hardware and software” to enable friends, families and coworkers to feel together, even when they are far apart, be it different cities or countries.
Further, Google says it is using “research in computer vision, machine learning, spatial audio and real-time compression” to make these realistic 3D holograms possible. Google says it has “developed a breakthrough light field display system that creates a sense of volume and depth that can be experienced without the need for additional glasses or headsets.”
So unlike a Microsoft HoloLens, Project Starline, if it becomes common, would work without the need for any extra AR or VR headsets. Further, the system relies on a 3D image of the user, which is then compressed in real-time and then sent to a 3D display to give the effect that the person is really sitting opposite you.
Of course, our regular smartphone and laptop displays won’t support this kind of 3D effect, and for Project Starline to succeed, it will need special equipment as it was made clear by Pichai and the brief demonstration.
But a detailed report on Wired by Lauren Goode gives a glimpse into how Project Starline, which has reportedly been in the making for nearly five years, would work. The report describes the high-tech ‘video booths’ where these 3D holographic calls are possible, and as expected they are packed with sensors and cameras.
Nor has Google revealed how much it will cost to build one of these specialised booths, adds the report. But it appears that Google is using WebRTC, the same open-source infrastructure that powers Google Meet, for the same, notes Wired. It is also not clear what kind of internet speeds such a system would require in order to work without any glitches given that it will transmitting 3D holographic images, which will likely be very heavy.
According to Google, Project Starline is currently available in just a few of its offices. The company says they are using “custom-built hardware and highly specialised equipment.” The post adds, “Our goal is to make this technology more affordable and accessible, including bringing some of these technical advancements into our suite of communication products.”
It also looks like Google is keen on rolling out the technology beyond its offices, though it has “already spent thousands of hours testing Project Starline” in-house and has connected employees “between the Bay Area, New York and Seattle.”
“We’ve also been conducting demos with select enterprise partners in areas like healthcare and media to get early feedback on the technology and its applications. We’re planning trial deployments with enterprise partners later this year,” notes the blog post.