Outlook for user interface and experience

Speech, gesture and mobile devices are among the tools that AV pros can use to provide more intuitive control system interfaces and experiences. But as Tim Kridel explains, each one has as much peril as promise.

How does thing work? It’s the last question a control system vendor or its reseller wants to hear, but it’s often the first.

Speech and gesture recognition are among a host of new and emerging technologies available to make the control user interface and experience (UI and UX) more intuitive. The catch is that those technologies also have the potential to make the UI and UX even more bewildering. So for vendors and integrators that can get them right, those technologies are new market-differentiation opportunities. 

For example, when an AV system doesn’t prompt users to call support half the time, or it doesn’t require a tech in every meeting to run things, the operating expenses are much lower. Those savings could sway a customer to choose that system over rival products, especially for clients whose dozens or hundreds of meeting rooms compound the bottom-line benefit.

“Speech is a definite trend in control,” says Eric Olson, business development manager for Almo Professional A/V, a US-based distributor. “But more generally, it’s intelligent and invisible automation. Customers want a series of events to happen without the need for user input at all. 

“For instance, reserving a conference room for a video call would mean the room system would be powered up, lights adjusted, connection made, etc. The meeting can start immediately without the need to touch a screen or even speak to the system.” 

Speak up

“Speech recognition” is a catchall term that covers a wide variety of applications, including voice-controlled virtual assistants such as Apple’s Siri. Another is voice biometrics, which some banks have added to their interactive voice response (IVR) telephone systems to authenticate callers just by their voice, thus eliminating the hassle of passwords and Pins. 

In pro AV and elsewhere, speech recognition is an attractive UI option largely because virtually everyone has a voice and already knows how to use it. Indeed, control’s use of it isn’t new.

“There were people taking an AMX touchpanel and embedding their own voice control five to seven years ago,” says Shaun Robinson, Harman Professional Solutions director of CEG solutions and marketing.

Those hacks are now being replaced by vendor-provided solutions, some for the enterprise market and others for the residential space. Two respective examples are Harman’s partnership with IBM Watson and Crestron home systems that now can be controlled using Amazon’s Alexa.

“Harman/AMX is using artificial intelligence through IBM’s Watson to control room settings via voice commands,” says Almo’s Olson. “There is a plan to implement this in the hospitality vertical first to make simple changes to one’s hotel room like climate control, shading, display functions, lighting, etc.”

Why the sudden surge of pro AV solutions? One reason is because virtual assistants such as Amazon Echo, Google Home and Apple’s Siri are steadily making more people comfortable with the concept of using their voice to control devices. It’s yet another example of how people’s experiences as consumers set their expectations for what’s possible and preferable at work. 

“Speech will definitely move into meeting and collaboration places,” says Michael Jarl Christensen, Neets CEO and co-owner. “But looking back on previous tech trends, it’s driven by the consumer market being first-mover, and once [the] technology is accepted, then it moves in different solutions into the commercial market.”

Another reason is why speech-controlled consumer devices are suddenly so common: because speech-recognition technology has become sophisticated enough to provide a consistently good user experience, even when a person has a cold or a thick accent. For example, speech vendors that target the IVR market frequently say their platforms can understand well over 90% of what callers say. That high accuracy makes pro AV vendors more willing to embed speech recognition in their control and other products.

“Now it’s to the point that the reliability is there to be able to offer a solution that your brand can stand behind,” Robinson says. “We’re very, very focused on speech recognition and voice control. We see that as a key way for delivering an intuitive experience.”

Speak naturally

In business IVRs, today’s speech recognition technology enables callers to use everyday terms instead of only industry jargon—a capability often referred to as natural language understanding (NLU). In control, NLU could mean the system understands, for example, to lower the shades when a person instead says to “drop” them.   

Meanwhile, the voice biometrics used in IVRs could be adapted to control to recognise and authenticate meeting participants. Pair that with NLU, and one potential scenario is a presenter simply saying, “Start my video,” and the system knowing which one to pull down from the cloud because it’s already verified her identity.  

A call to customer support typically involves just one person, while a household device such as Amazon Echo deals with maybe three or four family members. A business meeting typically has a much larger number of people, which means the control system needs to be sophisticated enough not only to discern between each presenter, but also understand whose commands to follow at a particular moment.  

To meet that requirement, pro AV could leverage speech technologies used in other verticals. For example, some speech-to-text platforms now can distinguish between multiple participants on a conference call for accurate attribution in the transcript. A control vendor could license the underlying technology to help its system identify each meeting participant. Or it could expand its use of voice biometrics for distinguishing between people in a room. 

Either way, there’s no shortage of speech technology vendors looking for licensing and partnership opportunities in pro AV.  

“All of us have been approached by a lot of those companies to consider their services or technologies,” says Rainer Stiehl, Extron Electronics vice president of marketing for Europe.

What does that gesture mean?

Besides a mouth, most meeting participants have hands and arms, too. That installed base is why some control vendors are exploring gesture UIs. However, gesture is trickier to design and use.  

“If there are many LCD displays that look similar in a conference room, and I’m not sure which ones to touch, how do I know which gestures are supported on each?” Stiehl says.

Gesture also is an example of how technologies that are viable in the consumer world aren’t always  practical at work, too. It’s one thing to wave that new TV remote to see which movement changes channels, or poke and pinch on a new phone. It’s another to do that in the heat of an important presentation. 

“On a public device, how much time do I have between my meeting start and my presentation to figure out whether I use a gesture to swipe for volume or an up-down button?”

“On my personal device, I have plenty of time to experiment,” Stiehl says. “On a public device, how much time do I have between my meeting start and my presentation to figure out whether I use a gesture to swipe for volume or an up-down button?”

A similar example is tablets. When the iPad debuted in April 2010, many people immediately speculated that it would decimate the market for touchpanels. If many companies provided their employees with iPads, the reasoning went, then they could just as easily control AV gear, lighting and other systems from those devices. 

Instead, iPads and other tablets have turned out to be complementary rather than competitors. 

“We found that tablets became great augmenters to solutions that already had a dedicated touchpanel,” Robinson says. “We have installations where they use only iPads. It’s really a customer choice.

“But we haven’t seen a lot of degradation in touchpanel sales, that’s for sure. People still value a dedicated device that they know will work.” 

Choose your UI

Tablets, as well as smartphones, could play another type of role going forward. In EMEA and the rest of the world, there’s a demographic shift underway where millennials and other younger people are starting to outnumber older workers. Different demographics have different comfort levels with technology, which is one reason why some clients still request simple, pushbutton-style control UIs. 

“We see that primarily in the education space,” Robinson says. “I think it has a lot to do with the workforce demographic, where maybe some people prefer the simplicity of a button push.”

Almo also sees healthy demand for pushbutton UIs. “It’s most often because of lower cost,” Olson says.

In the Mærsk Tower, a university project covered by InAVate in the Jan/Feb 2017 edition of the magazine, Neets pushbutton control panels were deployed in smaller classrooms where the complexity of the equipment didn’t warrant a more sophisticated system and teaching staff favoured the straightforward operation. In the more technology rich spaces Crestron control panels were provided.

“Despite the growing number of mobile devices, we still experience growth in keypad control systems and solutions,” Christensen says. “It’s simply easy and intuitive for users, and that’s exactly what we focus on. We believe in making life easy for presenters, whether that’s by physical keypad buttons equal to a fixed remote control, touchpanel-based solutions or even automatic detecting display controllers.”

The demographic shift likely means pushbutton systems will become even less common, although some vendors say they will never entirely disappear from the market. 

“There were people taking an AMX touchpanel and embedding their own voice control five to seven years ago.” 

“We’re getting more requests now [where clients say] ‘Yes, I know it would be cheaper to put a keypad here, but we want a touch interface because our users are demanding and expecting that because they grew up with mobile devices,’” Robinson says. “Are keypads and pushbutton interfaces going away? No, there’s definitely going to be tons of applications for those.” 

That means until all of the older demographics have retired, many companies will need control systems capable of meeting the needs and wants of all employees. There are at least two ways that tablets and smartphones could strike that balance.

Younger workers could download control apps to their mobile devices, thus meeting their preference for tile and touch UIs, while older workers could use pushbutton panels. Or the control system could use technologies such as Bluetooth, Wi-Fi or nearfield communications (NFC) to connect to each employee’s mobile device as she enters a meeting room so it knows which type of interface she prefers. 

Then as each participant goes to present, the system then would know to configure the room’s touchpanel with a tile, touch or even gesture UI, or a pushbutton one, albeit with virtual rather than physical keys. For employees who are so low tech that they don’t carry a smartphone or tablet, the latter scenario could get their preference from the electronic ID badge they probably already wear.

For control vendors and their integrator partners, one challenge with leveraging smartphones and tablets is that those devices are constantly changing. For example, their operating system (OS) versions—and thus capabilities—change dramatically in just a couple of years, so control systems will have to be updated to support the new OS. But some AV pros believe that these and other challenges won’t prevent mobile devices from playing an increasingly bigger role in control.

“It’s likely that this function, which exists today, will become more common and in higher demand,” says Almo’s Olson. “Future-proofing any aspect of a system is always a challenge for integrators and can rarely be 100% ensured. However, the risk of later incompatibility can definitely be mitigated by designing around standard protocols like Bluetooth and NFC rather than the proprietary technologies like AirPlay.”

Article Categories