How Arup future-proofed an AV vision for NATO
Arup was instrumental in helping set and realise a vision for NATO’s new Brussels headquarters that works to automate and streamline the processes and activities of the alliance. Anna Mitchell reports.
Consider the immense challenge of designing integrated and advanced AV systems for one of the most ambitious public building projects embarked on in recent years. A project under global scrutiny, a project that spans 250,000 sq m, and a project that has to work for one of the world’s most recognised alliances for many years to come.
Now imagine doing that around a decade before the systems will be installed and most of the technologies you will have to use aren’t even invented.
That was the immense task that fell to consultancy Arup when it took on the design of AV systems for the new NATO (North Atlantic Treaty Organisation) headquarters in Brussels.
“We had to get an enormous crystal ball out,” begins Jim Maynard, principle consultant at Arup.
If you go back to 2007 when Arup came on board, you find yourself in an analogue landscape, with the promise of a digital future. Of course the team had to design a digital system, but they weren’t sure of the technologies they could rely on becoming available. Signing NDAs with key manufacturers was a start, but the Arup team also took on the design of many of the systems when off-the-shelf technologies weren’t available or even on the horizon.
Back in 2007 Arup had started conversations with NATO, and principally the deputy secretary general of the alliance at that time, who had a vision for the future and how the new headquarters would fit into that. While the vision was primarily from NATO staff, complexity was added due to the fact that every member country was a stakeholder in the project.
“At that time everybody flew to NATO meetings and the meetings were very formal – even if they were ad-hoc type meetings,” explains Graham Naylor-Smith, associate director of the digital team at Arup.
“The vision was to introduce technology that would reduce costs, particularly travel costs, and to facilitate the dissemination of information during meetings with a goal to reduce the amount of paperwork. We also wanted to automate as much functionality as possible.”
Arup quickly pinpointed that the agendas published for meetings at NATO contained a lot of potentially useful information. The question was, according to Naylor-Smith, “how could the agenda information be used to automate the room booking process, and then aspects of the meeting itself?”
These early discussions were led by Naylor-Smith, before Maynard came on board to lead system design.
The scope of the project is huge. The headquarters houses 29 embassies and 4,500 staff.
In the conference centre alone there are 11 committee rooms seating between 20 and 40 delegates.
Each delegate can have up to three support staff so the largest rooms house 160 participants. Add to that remote connections for spill over rooms.
Committee rooms are joined by six conference rooms with between 40 and 300 seats, including the 280 seat main Council Room.
There is a conference room in theatre style and interpreter offices. All facilities are linked with multipurpose areas. There is also a press area and theatre, a staff centre including swimming pool and leisure facilities, broadcast studios, radio studios and server and communications rooms.
After system design and specification, Arup remained on the project in the capacity of technical advisor throughout the build and systems integration. Given the intricacies of design, and how much of the systems were crafted by Arup from the ground up, it was vital they were on hand to ensure the integrity of the specification was adhered to.
TAVN was the delivery integrator.
The automated camera system was a case in point.
“We approached this from an ergonomics perspective,” says Naylor-Smith.
Pods in the centre of circular conference tables contain cameras and LCD panels, each positioned to cover five participants. This provides a front view of the person talking and offers “near eye contact” with local and remote participants. “On air” indicators show which cameras are live.
“When the mic goes live, the camera trains on the person talking and that’s setup on a standard human footprint,” explains Maynard.
“If you have someone particularly tall or particularly short, or is sat back at the table, the technician can adjust the camera and the system creates a temporary preset.
They won’t have to adjust it again. The next time the room booking system shows there’s a new meeting, the system will return to the default position. The output is almost guaranteed to give you a perfect head and shoulders shot to the interpreters, out to the broadcasters and for the archive recording and video streaming.”
If the best camera for the shot is not available – for instance it may already be required – another suitable camera will be selected. Monitoring the switching frequency of the cameras required means that the system can track back to a full room shot when necessary to avoid switching too quickly between speakers.
“Availability of technicians can be limited,” continues Maynard. “So we wanted to do as much as we could with the technology to minimise the amount of intervention that is needed.
“We trialled the system at NATO in 2010 using analogue equipment. It was almost a theatrical set up required but we were able to communicate to the senior staff on the NATO side what we were trying to achieve and to test if it was something they wanted. We got a very positive response.”
At the new headquarters, the system knows the seated delegates because they have to log in with a staff identification card.
The room booking system also provides details such as start, finish and break times; the meeting agenda; and speakers and attendees lists.
This information is also used to reserve the most suitable available space, the right number of interpreters and any breakout rooms that might be required.
The AV system will monitor that usage too. If there’s nothing booked in a room for 30 minutes and no user activity on the touch panel for 30 minutes, or due to start, the system goes into standby. If there is no change after another 30 minutes the system fully powers down.
Arup also had to design a bespoke system for video streaming around the NATO headquarters.
As a side note, if NATO wish to stream meetings out to the internet in the future, that functionality is already included in the system.
“One of the things we had to get particularly clever on was how we dealt with multiple languages,” says Maynard.
“Normally to have a video recording or a file with a language the video piece is enormous compared to the audio piece. If you need to record it in ten languages, you’ll have ten copies of the same video file with one language per file, so the amount of data storage is huge.
“The archive/streaming system we specified ingests both the presentation graphics feed and the camera video feed as high definition steams, and up to 32 languages of audio. So all languages that the interpretation system could output, are streamed in digital format, and married to a single video file, dramatically reducing the overall footprint of data storage while also allowing maximum archive potential,” details Maynard.
It doesn’t stop there.
“Embedded within that camera stream is all of the metadata from the agenda and room booking system,” Maynard continues. “You have the meeting title, live agenda item, sub items, current speaker, at what point and so on. This metadata is also searchable making it possible to find specific points of meetings quickly.
“So now imagine you are on the front end of this as a user via a web-based user interface.
You start a stream of the meeting, you’ve got two adjustable windows; one with camera video, one with the presentation. You’ve got a dropdown list to select any one of 32 languages, the agenda, and you know from the live agenda that the chairman is ticking off on his touchpanel precisely which topic is in discussion.
We also know who is speaking, at what time, which country they represent, etc. This is available on all devices.”
Naylor-Smith jumps in: “Now you may think we’ve just taken standard video streaming a bit further. But when we designed this, it didn’t exist. No one had done it.”
“We also created a platform where each meeting could accommodate remote participants in up to three languages,” says Maynard. “We are using the local interpretation equipment to deal with the local interpretation and interpretation for the far end as well.”
“Making sure the interpreted audio gets to the right place at the right time, that the far end goes to the right place at the right time and so on is complex in terms of audio routing and processing,” elaborates Naylor-Smith. “We needed a many variable matrix to work that out.”
Maynard adds: “Every time an interpreter presses their mic button, control systems are assessing sources and outputs to figure out which routes need to be made. We had to make sure they never end up sending audio back to themselves and that echo cancellation is taken care of.”
Talking of interpreters, Arup placed them at the heart of much of its design. “Interpreters can find it very difficult to flick their eyes from a local picture of someone speaking, and the person actually speaking,” says Naylor-Smith.
“If there is any delay it can cause them lots of problems so we had to consider the end-to-end lag time of the whole system including the camera and the display.”
For this reason, Arup stipulated that the overall camera to display latency could not exceed 100ms.
Maynard adds that to help interpreter comfort they included big projection screens in remote interpreter rooms so they had two distances to focus on when looking at graphics and the speaker.
“We call it differential viewing,” he notes. “Putting interpreters into different rooms saved space on having booths around each meeting room but of course someone sitting at their desk staring at a small LCD screen for 45 minutes is not comfortable. They need to flick between the near and far distances as if they were there in the room themselves.”
By this stage Arup had designed a number of very clever systems, often with little or no existing use cases to base them on. But what really makes the conference centre work in a way that is possibly unparalleled in current facilities of its type is how those systems link together.
It’s hard to simplify the complexity but in an effort to summarise, the room booking system is used to reserve a meeting room.
The information included in the booking goes into the database. At that point the AV system pulls the information it requires from the database. It also communicates to the iBMS system to handle heating, cooling, lighting and so on.
That data also automatically configures the set up for the room, such as the correct audio configuration for the space.
Meanwhile staff will use ID cards to log into the conference system. At this stage each delegate or visitor is known so that data is fed into the conference system.
The automated camera system provides a video of the speaker, which can be combined with graphics of what’s being presented and added to up to 32 channels of audio.
All the metadata from the booking system and the agenda goes into the streaming and archiving system too where it is taken as full bit rate uncompressed video.
It is staggering what has been achieved. This complex integration, coupled with high level meetings, containing high profile participants demanded a high level of resilience to match.
Two AV system control processors operate in a master/ slave pair. If one processor stops responding for 30 seconds, a handover process will start, which fully hands over control to the other processor and instigates a reboot procedure of the first.
In terms of technical specification there were a number of key brands deployed.
At the time Arup was designing its systems, NATO had standardised on Polycom videoconferencing.
When the systems were integrated, Albiral was selected for LCDs, Barco projectors were deployed firing on to dnp screens, Panasonic cameras were chosen, AMX kit was used and all conferencing and interpretation was down to Televic Conference equipment.
A long journey of development culminated when the first NATO summit was held in the new headquarters in July 2017, followed one year later by the first meeting in the main council chamber.
All photo credits to NATO