One day, a solid disaster recovery plan could be the only thing keeping you in business. Are you ready?
On a Friday afternoon in September 2007, a warehouse in Phoenix stacked high with pallets caught fire. Located next door was the corporate office of Knight Transportation. The news got worse when Cory Staheli, vice president of information systems, realized the company’s data center and backup generator were in a room only 100 feet away from the 3,000-degree fire. Disaster appeared imminent.
After the building was evacuated, Staheli dispatched a team member to check the data backup system. Another team member began to notify technology partners to be on standby throughout the weekend in case any data needed to be rebuilt or restored. Meanwhile, another group went to a separate building to prepare the site for relocation.
When the fire department arrived, firefighters were able to create a wall of water to protect the data center. “The data center never got above 78 degrees,” says Staheli, who knows the situation could have been much worse. “One lesson we learned is that nothing is out of the realm of possibilities.”
If a fire, earthquake, flood or vandalism suddenly wiped out your information and communications systems, every hour of downtime could draw you closer to the brink of financial ruin. Even minor incidents like power outages, sudden hardware failures or disruptions to phone and Internet connections can cost thousands in lost revenue and productivity.
One day, a disaster recovery system might be the only mechanism keeping you in business. Like an insurance policy, disaster recovery is something you pay for that you hope you never use. And like with insurance, you easily could spend many times as much as you really need in order to manage your risk. Fortunately, disaster recovery comes in many shapes and sizes, and is even a standard feature in many of the latest software products.
Redundancy, redundancy
The cost of downtime is directly proportional to a fleet’s size, so it’s not surprising that larger carriers invest major sums to minimize any downtime due to a disaster or systems failure.
Knight Transportation already had started moving toward redundancy before the fire incident, Staheli says. The company already was using a system to mirror its IBM iSeries servers. In the event of a disaster, the time to converge information from one server to the other would take about two hours. “Without mirrored servers, your recovery time could be weeks,” Staheli says. If the company were to transfer data to a second iSeries using backup tapes, the process would have taken 15 to 20 hours, he says.
Currently, should another disaster occur, all of Knight’s other servers – such as its Windows and Linux servers – can be switched over to a secondary system instantaneously with its VMWare software system.
Backups are the most common form of disaster recovery planning. However, manually recording and storing tapes offsite is not foolproof, and it’s the slowest process for restoring lost data. Therefore, many fleets have taken steps to eliminate human intervention, using technologies for redundancy and replication of data critical to their operations and communications systems.
In 2006, Ron Edwards, then chief information officer of Crete Carrier Corp., worked closely with IBM to implement a state-of-the-art high-availability disaster recovery system for IBM’s midrange systems – the iSeries or AS/400. As part of the project, IBM developed a hardware-based data replication solution that utilizes a co-location to support a complete replica of the carrier’s primary IT facility, says Edwards, who is now president and chief executive officer of Intelek Technologies, a software company that develops EDI software and load automation tools for the trucking industry.
The IBM technology that Lincoln, Neb.-based Crete implemented enables the AS/400 and other computer platforms to talk to an external array of disk drives and stay in sync, regardless of whether or not the AS/400 is running. The system continuously performs a complete data replication between the primary facility and a co-location in less than three seconds.
“The results were staggering,” Edwards says. “While the solution was very complex at the time, it’s now much more common, and there are numerous other installations around the world doing the same thing we did.”
Perkins Logistics – a Noblesville, Ind.-based truckload fleet with 350 owner-operators – uses the Internet for redundant connectivity via two T-1 lines bonded together. “If one goes down, the other keeps right on going,” says Dan McKinnon, vice president of information technology. The company’s land phones and faxes also have redundancy through voice-over-IP (VOIP) technology. Perkins’ VOIP vendor, SmoothStone, maintains a redundant facility, McKinnon says.
The computer systems in Perkins’ office have redundancy through a virtualized server environment. The VMWare software used by the company automatically replicates data from the company’s primary servers to a set of secondary servers. If an internal problem forces one of its primary servers down, VMWare automatically transfers this so-called “virtual” server to another physical server onsite. “By making our own facility redundant, we have provided a very stable working environment,” McKinnon says.
All of Perkins’ internal data is replicated automatically to an outside disaster recovery service called BlueLock, which also maintains a redundant facility. “Not only do we have a disaster recovery plan for our facility, but more importantly, our two vendors have redundant facilities in case they go down,” McKinnon says.
Should a disaster force an evacuation of its own office, Perkins could operate as normal from a fortified BlueLock facility 20 minutes away. “We would log onto their PCs, into our network and be ready to start operating with data at full speed,” McKinnon says. Phone and fax service would be provided through its VOIP solution’s “SoftPhone” option, a phone emulation program that uses a PC’s speakers and microphones to send and receive calls.
Where is your software?
Many software vendors now include high availability and disaster recovery services as part of their offerings. Individually, these sophisticated hardware and disaster recovery systems would be beyond financial reach for small and midsize carriers. Increasingly, vendors are able to offer high-end disaster recovery systems to carriers of all sizes by using the software-as-a-service (SaaS) model.
Before the burst of the dot-com bubble, software companies that offered products online were called application service providers (ASPs); today, SaaS has replaced ASP to denote this software model. By 2011, SaaS will account for 35 percent of software deployments, according to Gartner, a technology research and consulting firm. One of the key differences today is that fleets no longer access SaaS through a Web browser; instead, software users connect directly to their provider’s servers. An “emulator” gives the SaaS version the same look and feel as if the software were installed on their own servers.
Like group insurance discounts, an online version allows the vendor to aggregate its user base to share the costs of providing advanced systems for disaster recovery such as hardware and secure data centers. Edwards says Intelek is in the process of switching all of its software products to the SaaS platform. “We hope to have everything done by the end of August,” he says.
Innovative Computing Corp. began offering a fully redundant SaaS version in 1999. ICC initially offered an online version of the Innovative Enterprise System, IES Access, to fleets with fewer than 60 trucks. After some modifications, Innovative began marketing IES Access to fleets with up to 300 trucks. For as low as $22 per truck per month, customers get full software support and managed services that include hot site redundancy, redundant third-party interfaces and system monitoring. For IES Access, ICC has deployed redundant iSeries servers located at SunGard data centers in St. Louis and Nashville, Tenn., says Ernie Betancourt, ICC president and chief executive officer.
Innovative Computing plans hardware and system upgrades to IES Access about every three years. For the next release, scheduled for next summer, ICC may implement IBM technology similar to what Crete is using today, Betancourt says. If this solution is implemented, the architecture should help reduce variable costs for data storage and replication and thus enable ICC to offer IES Access to fleets with more than 300 trucks at a discounted rate, he says.
In December 2008, PCS Software released Express Online, a SaaS version of its client-server Express transportation management software system. The SaaS version accounts for about 75 percent of new business for PCS Software, says Sean VanDyck, sales manager. With Express Online, fleets that otherwise would be delaying capital expenditures due to the economy can sign up for a monthly user-based fee with no contract required. Users can discontinue use at any time with a 30-day notice.
For its client-server customers, PCS Software offers a backup service that transmits a backup file each night to its office from the client’s site. “It has been a lifesaver for a lot of companies,” VanDyck says. If an Express user goes down, PCS Software can restore the company’s data using the backup file from the previous night. In the event of hardware failures or power outages, PCS can restore the file to its online servers so the customer can connect to their software from a separate location.
Enterprise software developer TMW Systems currently offers online versions of its TMT Fleet Maintenance Software commercially. TMW gradually is expanding that service offering to cover other TMW software applications, including TMW Optimization products from the former IDSC office and core enterprise software such as TMWSuite, TruckMate and TL2000, says Monica Truelsch, director of marketing.
For the moment, these other product lines can be considered for SaaS service on a customer-by-customer basis, but this is not a standard delivery option when buying. TMW expects to make an announcement later in the year regarding standard SaaS service availability for a broader selection of its software, the company says.
McLeod Software currently does not offer a SaaS version of its LoadMaster or PowerBroker enterprise software systems for truckload carriers and brokerage companies, respectively, says Robert Brothers, manager of product development. The company recently acquired a SaaS provider in iLENS, a freight management system that brokers, logistics firms and shippers use to manage transportation processes such as carrier bid procurement. iLENS has redundant servers in Dallas and Austin, Texas.
Regardless of whether companies choose to purchase their own disaster recovery systems or leverage SaaS providers to do so, the cost should be considered an investment rather than an expense, Perkins’ McKinnon says. While the technology is expensive, having a bulletproof disaster recovery plan can be a strategic sales tool. Not all carriers can guarantee customers 24/7 uptime for tracking of shipments and other critical information needs, he says.
The flipside of SaaS Online software offers cost benefits – but be aware of the tradeoffs
The software-as-a-service (SaaS) model is simple: A vendor installs and maintains software on its host server and gives its customers access to the program through the Internet. For the end user, the benefits include fixed monthly billing with zero upfront costs for equipment and application licensing. The monthly cost includes the ongoing services provided by the vendor, which include data storage and backup.
In many cases, the software licensing costs work out to be essentially the same as with on-premise installed technologies. But the amount of overhead associated with the software is a fraction of the cost of in-house staff, dedicated server rooms and physical security precautions. “From a balance sheet standpoint, technology can be an ongoing expense rather than a capital outlay with standard depreciation,” says Monica Truelsch, director of marketing for TMW Systems, a provider of enterprise resource planning software for the transportation industry.
The tradeoff for using SaaS versions are flexibility and responsiveness, Truelsch says. Having software in-house and an IT staff to manage it helps businesses stay attuned to user needs and business priorities. An in-house IT staff can see how users perceive technical problems and monitor for any issues to speed up troubleshooting and resolution. Skilled IT staff also can provide quick customization such as reports that help businesses respond to market and customer opportunities.
Another tradeoff is relying on an Internet connection to access business information; the Internet is susceptible to hackers and other unauthorized users. But security doesn’t need to be a tradeoff; vendors can force users to change passwords and encrypt communications.
Innovative Computing Corp. (ICC) includes an IBM product, ClientAccess, as part of its IES Access service, a SaaS version of its Innovative Enterprise Software system. Only PCs that have ClientAccess installed are authorized to login to IES Access, as ClientAccess encrypts communications between users and Innovative’s servers, says Ernie Betancourt, ICC president and chief executive officer.
Dan McKinnon – vice president of information technology for Noblesville, Ind.-based Perkins Logistics – prefers to keep software in-house to have more flexibility, control and responsiveness to business needs. “Like everything that is in-house, I see it every day,” McKinnon says. For SaaS versions, if something needs to be changed or fixed, the provider must be contacted and may charge for support calls. Another consideration is the financial stability of the SaaS provider, especially in today’s economy; a vendor’s financial problems could interfere with access to data.
“That is something that would weigh on the back of my mind more,” McKinnon says. “That is something I might not have thought of a year ago.”
