“Always have a backup” has been a mantra of radio engineers since the earliest days of broadcasting. Much time and energy has been spent developing disaster recovery plans that outline responses to fires, floods, tornadoes and other cataclysmic events.
But the outbreak of COVID-19 required most station employees, including air staff, to work at home for extended periods of time, a contingency that some engineers might wish they’d spent more time considering.
Lessons learned in the first weeks of the pandemic were discussed in an interesting episode of “This Week in Radio Tech (TWiRT),” hosted by Kirk Harnack, senior systems consultant for The Telos Alliance, and Chris Tobin, IP solutionist. They convened a special edition to talk about “Broadcast Continuity in a Pandemic,” sharing experiences in adapting station workflows and technology.
Codecs and Chromebooks
Geary Morrill is regional director of engineering at Alpha Media USA, an early adopter of the WideOrbit 4.0. He said that platform was being used by those working at home for remote access to the station via iPad and iPhone apps, mainly for recording and voice track activities.
Robbie Green, director of technical operations at Entercom Houston, said his employer created a work-from-home protocol for most staff, although air talent was still in the building. Should it become necessary for them to leave, equipment was set up so they can voice track from home. For the sports staff, he purchased a number of Comrex Opal IP Audio Gateways, as well as refurbished Chromebooks to equip remote kits.
Green said that the cluster’s building includes 600,000 feet of rentable space, of which the station occupies half. If someone working there were to become infected, building management would have to close the building for decontamination, so plans were developed for that eventuality.
A challenge facing many broadcasters, including Green, is how to handle the generation of logs. “Our traffic people have been working remotely for over a week, and program directors can also do logs remotely. We have a secure portal where they can dump everything into our WideOrbit system.”
Tom McGinley, chief engineer at KUFM(FM/TV), engineering manager at Townsquare Media in Missoula, Mont., and Radio World technical advisor, said that a global pandemic occurs about once every hundred years. If this outbreak had happened 20 years ago, he said, broadcasters wouldn’t have had the internet and IP connectivity we have today. The challenge would have been much greater for stations merely to stay on the air.
With no confirmed cases of COVID-19 in his area at the time of the podcast, McGinley said buildings were still accessible for talent. Traffic was being managed off-site through an internet connection. Plans were underway to do live shows remotely via Comrex Access gear, along with RCS NexGen iPush and Remote. McGinley added that the stations were already planning an upgrade to RCS Zetta, which has more flexibility for remote broadcast.
“This is What We Do”
Consultant Gary Kline applauded the efforts of broadcast engineers worldwide. Rising to unexpected challenges and having solutions ready before management knows to ask is “what we do,” he said. Kline praised codec manufacturers whose shipping departments worked overtime in March and April to make sure broadcasters got the tools they needed to stay on the air.
He said that while there has been growing awareness in recent years of the need to prepare for disasters, not all contingencies have been addressed.
“Many stations would have emergency generators as well as backup IP and internet facilities,” he said, describing conversations with clients, “but I would ask ‘What if you have to leave your building?’ and they weren’t so sure about that. Next time, we’ll be so much better prepared for something like this, so there is a silver lining to the story.”
“I tell everyone, it’s just a remote broadcast, only from your home.” — Jim Armstrong
Jim Armstrong, director of eastern U.S. sales at the Telos Alliance, has fielded a lot of questions about accessing equipment such as consoles off-site.
“I tell everyone, it’s just a remote broadcast, only from your home.” One aspect that sometimes gets overlooked is that most AoIP consoles can be operated remotely, and routing switchers can also be controlled off-site.
Several software products are available to fill these needs. There’s third-party software from IP Studios in Paris that runs IP tablet software. He discussed Axia SoftSurface, a program that connects to an engine or console to control the mix bus and faders, and Axia Pathfinder Core Pro, a development tool that allows users to create a virtual Fusion console.
He praised radio engineers for their handling of the situation and joked that a person is not really in radio until they’ve slept at the station, a rite of passage.
Bill Bennett, media solutions manager for ENCO Systems, talked about how stations could access and use their automation playout systems remotely.
For some time, he said, automation has meant servers in the studios plus some form of offsite backup. The cloud has experienced explosive growth over the past 15 years. Engineers have gotten comfortable with the idea of the cloud as a place to store data offsite and as part of their backup plans.
ENCO’s current automation playout system has a web interface, the front of which is HTML5-compliant so it can run on a browser. At the same time, the software is running on a virtual machine in the cloud.
Another bit of software keeps the virtual machine in sync with the studio machine over a VPN line.
An important consideration is keeping viruses that might infect the studio machine from reaching the virtual machine in the cloud. Harnack noted that Paravel Systems’ Rivendell 3.0, the open source automation playout system, is capable of running from the cloud during disasters.
With this type of system in place, all that is necessary in emergencies is to access the virtual machine via a laptop, and route a stream to the transmitter.
Also participating in the conversation were Mike Sprysenski, regional director of engineering at iHeartMedia, and Bryan Waters, chief engineer at Cumulus Media, Atlanta. The podcast can be accessed at thisweekinradiotech.com, or you can watch the full episode below.
Chris Tobin had the last word, talking about understanding workflow solutions. Air talent may be accustomed to working in front of a console and a stack of three audio devices and hotkeys to fire things off. They won’t have those at home and may experience initial stress if everything is different. The goal of the engineer should be to know the workflow of your announcers off-site, and try to make it as similar to the studio environment as possible.
Lightning Round
Harnack asked panelists for words of wisdom that could fit in 30 seconds or less.
Bill Bennett — Use two-factor authentication for network authentication and file access. Yes, it’s more complicated and slows things down, but much more secure.
Gary Kline — Ask yourself who is your backup if you become sick or quarantined. Formally designate someone if you need to.
Geary Morrill — Keep an even keel and be patient with staff as they adapt to the unfamiliar. People will feed off your emotional state.
Jim Armstrong — Have essential spares on the shelf. Remember that you don’t really need it until you need it, and then you really do.
Mike Sprysenski — Remember to take care of yourself as you’re taking care of everybody else.
Bryan Waters — Keep it simple. Give people what they need to work from home, but don’t overwhelm or complicate.
Robbie Green — Create documentation that’s written for the non-technical person. Have someone do a test drive with it before you distribute.
Tom McGinley — Look for the silver linings as this situation winds down. Expect a new level of competence from stations as they revise disaster recovery plans.
To watch the full episode, click here.
The post Broadcast Continuity in a Pandemic appeared first on Radio World.
