From protesters at shareholder meetings to suicide bombers at football games, live events are attractive targets. Tim Kridel explores how video surveillance is evolving to thwart those threats.
This month marks two years since a suicide bomber killed 22 people and wounded another 139 as they were leaving an Ariana Grande concert at the UK’s Manchester Arena. Like the November 2015 terrorist attack at Stade de France during a football match, the Manchester bombing highlights the growing need for next-generation video surveillance technology capable of quickly identifying attackers—ideally before they’re able to carry out their plan.
"If somebody pulls a gun, everybody in the crowd runs. You can detect that pattern, but you don’t know that’s an incident. All you know is that some behaviour has changed. But then you can pull up a camera and have an operator look at it." - AJ Frazer, Agent Video Intelligence
In other cases, the threats are motivated by environmental concerns or celebrity obsessions. For example, businesses preparing for shareholder meetings or product-launch road shows could use video surveillance to identify activists sneaking into the venue to disrupt those events. Another is Taylor Swift’s use of facial recognition surveillance to identify stalkers among concertgoers.
The common denominator is that event organisers and venue owners are increasingly looking to upgrade their video surveillance systems.
“We’re getting more interest,” says Mary Haskett, co-founder and CEO of Blink Identity, a US-based company specialising in video surveillance with facial recognition. “There’s growing awareness that they need to do something combined with a sort of caution of ‘What is the right thing to do?’
“So they’re investigating. They’re asking questions. They’re watching what other people are doing to try to figure out the right next step. It’s all over the map in terms of the people we’re talking to. Some people are like: ‘What databases can we tie into? Can we put cameras everywhere?”
AnyVision is another vendor seeing demand.
“Security and surveillance are definitely top of mind for operators of venues where large amounts of people gather,” says Max Constant, AnyVision chief commercial officer. “Concerts, certainly, but also sports facilities, airports, casinos and city surveillance, to name a few.”
Taking surveillance on the road
Arenas, convention centres and other event venues typically have video surveillance systems, but the quality and quantity of coverage varies widely.
“Some [stadiums] have remarkably few cameras,” says AJ Frazer, Agent Video Intelligence vice president of business development. “But 400 is a decent number for a large venue.”
Rental and staging companies could add portable surveillance systems to their solution portfolios to augment a venue’s system. AnyVision is an example of how some vendors are developing portable solutions.
“AnyVision has designed a Tactical Kit, which is our full Better Tomorrow solution in one portable pelican case,” Constant says. “This portable solution offers the full feature suite of our system as if it were on premise, except now you can use on the go.”
Taylor Swift used a portable system on her recent Reputation tour. At venue kiosks, fans who stopped to watch rehearsal videos had their faces scanned and checked against a database of her known stalkers, according to a Rolling Stone story.
When that story broke, a lot of people assumed the vendor was Blink Identity, partly because its investors include Live Nation and partly because it specialises in facial recognition.
“My inbox just exploded,” Haskett says. “I didn’t have a whole lot to say because it wasn’t our system.”
Nevertheless, Blink Identity did a sentiment analysis that included studying the comments on news reports about Taylor Swift’s system. What the company learned is noteworthy for anyone considering selling or buying a facial recognition system for live events, regardless of whether it’s portable or permanently installed.
“It was a very viral story,” Haskett says. “‘Big brother is coming to the concert!’ ‘You’re getting watched!’ All of that.
“But if you actually read the comments, most were either neutral or positive: ‘I’m not happy about discovering after the fact that I was being photographed, but she’s got stalkers, so I get it.’ If you looked just at the headlines, you’d think that there was this really strong, negative public feeling. The reality is that it’s not quite what it seems.”
So many variables, so little time
Portable systems come with a few caveats. One is legal: When using them for multinational tours, it’s critical to understand what each country’s privacy laws allow.
Another is environmental. Permanent installations have the luxury of spending weeks identifying the ideal camera locations and figuring out how to deal with variables such as how ambient lighting changes as the sun waxes and wanes through a stadium concourse. A portable system is severely limited in terms of time and locations.
Blink Identity is an example of how surveillance vendors are working to address those challenges—and in ways that also benefit permanent installations.
“[We do] facial recognition in any lighting condition, which is really important for venues where it can be first thing in the morning at a festival or late at night, indoors or outdoors, [around] lasers, coloured lights, etc.” Haskett says. “Facial recognition traditionally does not perform well in really uncontrolled, ambient light conditions.”
Some event organisers have begun phasing out paper tickets in favour of digital versions that attendees can display on their smartphone at the gate. But there’s been fan backlash due to problems such as mobile network congestion when tens of thousands of other fans try to pull up their electronic tickets at the same time. And even if they all loaded instantaneously, fans still have to wait in line to have them scanned, just like with a paper ticket.
“Those [scans] can take five to seven seconds, which doesn’t sound like much until you’re talking about a large arena with tens of thousands of people,” Haskett says. “All of a sudden, that’s a long time.”
Facial recognition isn’t necessarily faster if attendees still have to queue up to be scanned, one by one. That’s one reason why Live Nation invested in Blink Identity, whose technology can identify ticketholders in half a second even when they’re walking—over 60 people per minute.
To avoid the privacy pushback that Taylor Swift’s system encountered, Blink Identity requires people to opt in by providing a selfie. How many people are willing to do that? Time will tell, but “fast pass” options at theme parks and airports suggest that a lot are willing to share their face and money rather than stand in long lines.
Finally, facial recognition also can be used to identify venue staff and other authorised people, such as media or a band’s road crew. Besides helping with security, that use case also can eliminate some manual processes, such as automatically verifying each employee’s time on task.
Does this stuff really work?
But as sophisticated as facial recognition is becoming, it still can be a tough sale to sceptical event organisers and venue owners. The technology is part of the broader video content analytics (VCA) field, which even vendors admit has often overpromised and underwhelmed.
“It’s come a long way,” Frazer says. “That’s a euphemism for ‘stuff is finally catching up to the promises that analytics providers have made over the last 10 years.’”
So one challenge is convincing people that today’s facial recognition and other VCA technologies deliver on those promises.
“I think these [venue] operators are looking for the technology to actually work: are looking for accuracy and speed, first and foremost,” says AnyVision’s Constant. “We recognised early on that, if we could design an artificial intelligence (AI) algorithm that would take away the burden of having to replace existing sensors or worry about poor lighting conditions or which angle you were pointing your camera, there would be enormous market interest — and an enormous impact in terms of actually solving many core problems.
“That's what our flagship product, Better Tomorrow, does. We use existing customers' cameras to do facial, body and object recognition.”
For large venues, VCA is a matter of when, not if, simply because the more cameras they install, the tougher it is to afford enough people to monitor all of those feeds. VCA can lighten that workload by monitoring for certain patterns and alerting humans when those occur, thus creating situational awareness. The trick is determining which patterns to tell the system to look for.
“If somebody pulls a gun, everybody in the crowd runs,” Frazer says. “You can detect that pattern, but you don’t know that’s an incident. All you know is that some behaviour has changed. But then you can pull up a camera and have an operator look at it.”
VCA also help speed up forensic investigations, such as identifying the person who dropped a bag with a bomb so security and police have a face to look for. Do that fast enough, and it could enable them to catch that person before he gets to an exit. So one capability to look for when comparing VCA solutions is how quickly it can search for certain attributes and then present the possibilities in a format that only a human can winnow down and act on.
“To do that, you have to have a true, metadata-driven search capability,” Frazer says. “[With] thumbnails presented to a security officer, they can scroll through a thousand in 2 minutes and say, ‘Why does that person have a big, bulky jacket at a July concert?’
“We’re nowhere close as an industry to being able to identify that kind of variant, let alone know that it’s odd. The brain is great at that.”
Right place at the right time
To venue owners and other end users, surveillance systems are like insurance policies: big, never-ending expenses that don’t help their bottom line. Hence interest in having those systems do double duty for business intelligence (BI).
One example is studying concession stands. If the lines at some are long, while others are short, it might be time to adjust the mix of food, beverages and merchandise sold at each one. Events and venues lose money when people get so frustrated by the prospect of waiting in a long line that they give up. Lost concession sales can sting hard if, for example, the venue operator’s contract has the home team getting the lion’s share of ticket and parking revenue.
“They’re constantly asking about BI on their concourses to improve efficiency,” Frazer says about his meetings with stadium owners.
Leveraging surveillance for BI isn’t a new concept, of course. Retailers, for example, have been doing that for years. One lesson from that vertical is that a camera ideally positioned to catch shoplifters and pilfering employees often isn’t the right spot to discern age, race, gender and other demographics.
“If you want more accuracy from your BI, you have to put up more cameras,” Frazer says. “There’s a point of diminishing returns with BI, meaning the customer always decides—because of cost—‘how accurate do I want my data?’ They’re not going to put up cameras every 5 feet. There’s a point of diminishing returns, and customers need to understand that.”