This article was written by Steve Havert, a seasoned IT pro with a 36 year IT career. Here he discusses some of his client's experiences with ransomware.
Client under attack from Cryptolocker
When I worked for a small IT consulting firm in Seattle, we dreaded getting a phone call from a client facing a CryptoLocker attack. The first one we encountered was with a client who wasn't even ours at the time. Their current IT person was planning to retire, and let us know that he would be transitioning his clients to our firm. He called us late one afternoon and described the message one of his clients was receiving when he attempted to open any image files on their network-attached storage (NAS) device - "Your personal files are encrypted!" He had immediately disconnected the NAS from the network and began researching the problem, but decided he was in over his head.
A colleague and I dropped everything and headed to the client's office. It took us until about 1:00 a.m. to identify the computer that was compromised, identify the corrupted files, and restore backup copies of them. It turned out that the CryptoLocker malware had infected a computer the previous day and had been encrypting files for close to 24 hours. This was an architecture firm, so they had a huge number of JPG, PDF, and CAD drawing files.
A large number of encrypted files had been backed up to the external hard drive that was attached to the NAS that evening. Luckily for us, their current IT consultant swapped out external backup hard drives every other day. We were able to recover clean files from the previous backup drive.
This client was lucky because it had a backup system in place and discovered the problem fairly quickly. It still cost them quite a bit of money, but they were grateful for our prompt response and a successful resolution of the problem.
SynoLocker Hits!
The second experience was, for me, somewhat more anxiety-producing. I was leaving for a few days' vacation when I received a call from a client who reported that he was receiving a message that "all important files on this NAS have been encrypted using strong cryptography." This ransomware variant specifically targeted Synology NAS devices and was called SynoLocker. Within a few minutes of hanging up with him a second client called with the exact same problem.
I called the other two consultants who worked at the firm and asked if they could each handle one of my clients' potential disasters. I hated not being on-site and handling the problems myself, but not enough to cancel my vacation. We had standardized on Synology as a NAS device for clients who were not large enough to require a true Windows server, so I kept my fingers crossed that we wouldn't receive a third or fourth or fifth such call.
Fortunately, Synology technical support was able to provide a decryption key that let us decrypt all the files. (I suspect they paid the ransom to obtain the key, which was universal for all their NAS devices.) It turned out that the problem was with a vulnerability that Synology had fixed in a recent firmware update. Only two of our many clients who were using Synology devices had not updated to the latest firmware. After that experience, we became diligent about updating everyone's NAS firmware as soon as it became available.
No Happy Ending - Ransom Paid
One other experience did not have as happy an ending. I received a call from a company that wasn't a client but that had obtained our name and number from a company that was. The caller served as the company's IT guy while performing his real full-time job. He sounded like a bright guy and understood the problem he was facing. Unfortunately, since his primary job wasn't IT, he sometimes forgot to perform routine IT functions - such as making sure backup was working and changing backup media.
He understood from the company that had referred us that we had been able to recover their data from a ransomware attack. I explained that, yes, we had, but that we did it by restoring from a recent backup. His most recent backup was more than three months old. I told him I'd see what I could do, but I didn't offer much hope. A little while later he called back and said that the owner of the company wanted him to pay the ransom - around $3,500, I believe. He tried to follow the instructions to pay by using the Tor browser but was unable to. I followed the instructions supplied in the ransomware message and was able to get to the payment portal. I emailed him very specific instructions and he paid the ransom.
What can we learn from these experiences? My takeaways are:
- Utilize a high-quality email filtering service that blocks the bad stuff. In both cases of the CryptoLocker instances the culprit was an infected email. Neither client was using an email filter service at the time.
- Make sure you have a sound backup strategy and that you monitor those backups' success. Use products that auto-update and regularly check that they are working as expected.
- The company I worked for established two backup methods for our clients. One was a backup to a set of at least five external hard drives that were swapped daily. The second was an online backup to the cloud; we typically used an Amazon S3 account for storage. Another practice we implemented was to use backup software that generated an email with the status of the daily backup jobs. On our monthly on-site visits, we always performed a restore test to make sure we could actually restore files from both the external hard drive backups and the cloud backups.
- Keep all devices up-to-date with software, firmware, and other updates. Although we have all experienced a Windows Update that caused a system crash or introduced another severe problem, overall I have found that keeping software and firmware up-to-date prevents issues significantly more often than it causes them.
- Educate computer users. When a potential disaster strikes, like the CryptoLocker experiences that my clients encountered, everyone in the company spends the next two or three weeks in a heightened state of paranoid awareness. By a month after the event, everyone except maybe the owner of the company has forgotten about it and reverted to their blissful ignorance of the dangers of opening email attachments and clicking on web links.
- I recommend that my clients hold monthly brown bag lunches where they present information about the latest email scams, phishing technique, and malware, just to keep all employees alert and aware of what they may encounter while surfing the Net or rushing to get through their inboxes as quickly as possible.
About Steve Havert - Steve Havert is an independent IT professional based in Seattle, WA. He has spent his 36-year career working in every facet of IT for large corporations as well as his own IT consulting business in Orange County, CA. He continues to work as a freelance IT consultant while pursuing a second career in photography.