I am very fortunate to have been accepted as a Friend of Red Gate for a second year!
If you’re not familiar with “FoRG”, Friends of Red Gate work with the Red Gate development and product teams to provide feedback on existing products, new tools, and feature enhancements. You can also find us speaking at events and writing articles on Red Gate tools and how they have saved our bacon and allowed us to accomplish our tasks more quickly.
Red Gate is a tremendous organization. Not only do they make some amazing tools and books, but they’re also very community-oriented. They sponsor many functions such as user group meetings and SQL Saturdays, and also put on events of their own such as SQL in the City, which I loved attending and later presenting at. Red Gaters are also incredibly receptive to feedback, yet another reason why their products are just plan awesome.
Thanks so much, Red Gate! I’m looking forward to another great year of being a friend, sharing my thoughts, and learning a ton from other friends and your amazing staff!
Last week Microsoft announced that they will be retiring their TechNet Subscriptions service. This is very disappointing and will profoundly affect myself and many others.
What Is TechNet?
If you’re not familiar with TechNet, it’s a program Microsoft offers that contains “Resources and Tools for IT Professionals”. Most notably it has a subscription service that provides copies of software for development and evaluation purposes. Need to create a test environment with older versions of software? TechNet has you covered. Windows 3.1, FrontPage, Access 2003, it’s all there. (Though sadly I don’t see Microsoft Bob available for download.)
TechNet is also a source for current software (again, for development and evaluation purposes). All the virtual machines I use for my lab environment run software I’ve obtained through TechNet. Back in school I didn’t have the funds to buy SQL Server 2000 or Windows Server 2000, but I have a VM running them now thanks to TechNet.
As a DBA, I shouldn’t even need TechNet because the SQL Server team is kind enough to release a reasonably-priced developer edition for development, testing, and demonstration environments. Unfortunately the Windows Server team never got that memo. I’d gladly pay $50 or $100 for a development/testing license of Windows Server, but there is no such offering. There’s a free 180-day trial available (which you can extend to 240 days) but still the thought of having to rebuild all my VMs every 8 months sounds like a lot of wasted time that I could better spend blogging or writing presentations. $350 ($250 per year to renew) for a TechNet subscription seemed like a fair trade-off: access to all the software I could possibly want and no need to reinstall trialware. It was a great win-win, until I got an email that said, among other things:
“In recent years, we have seen a usage shift from paid to free evaluation experiences and resources. As a result, Microsoft has decided to retire the TechNet Subscriptions service and will discontinue sales on August 31, 2013.”
I’ll be able to renew my subscription for another year, but this time next year I’ll be out of luck. The “replacements” Microsoft recommends are the existing free trial versions (with the reinstalls every 240 days) or of course, subscribing to MSDN. I have a sneaking suspicion that converting TechNet subscribers to MSDN is the real reason for the demise of TechNet Subscriptions. Coming in a close second is probably abuse of licensing privileges. Software provided by TechNet Subscription Service is for testing and evaluation only, but I’m sure plenty of people installed stuff on all their personal or business computers and didn’t think twice. It’s very unfortunate for the rest of us who were using it for its intended purpose.
I’m not going to claim to know all the ins and outs of MSDN. I know that they at least offer all the downloads TechNet does, and I’m guessing there’s many additional features as well, since they seem to charge so much more for it. From what I can see, a “MSDN Operating Systems” subscription goes for $699 ($499 annually after that). But that only includes operating systems. To get SQL Server, I’d need “Visual Studio Professional with MSDN”, which costs $1199 ($799 annually to renew).
For professionals and businesses, MSDN is a great value, but it’s definitely priced out of the range of enthusiasts like myself. TechNet seemed reasonable to me, but my budget can’t handle paying more than twice that for MSDN, so I guess I’ll just be using the trial versions and periodically reinstalling my lab VMs. You just lost my money, Microsoft, and unfortunately I’ll lose a bunch of my time. It’s a shame for both of us.
Again?
Yes, again. This isn’t the first time I’ve associated with an entity named TechNet that has come to an untimely end. Back in high school, some friends and I started a technology club that we named “Technet”, completely oblivious to the fact that Microsoft created their TechNet a year earlier. In addition to discussing computers and internet-related topics, we were also responsible for designing the school’s website. We spent lots of time in Netscape Composer cranking out a site that is now a testament to either how far web design has come in the past 13 years, or how terrible we were at HTML. I’m guessing a little of both. If you want to see what 16-year-old me and my buddies came up with, check it out on The Wayback Machine. Technet met its demise when we graduated and school IT staff took over the site.
Being a DBA and data professional doesn’t mean I always work with SQL Server – sometimes I’m not working with databases at all. We’ve recently acquired some new storage at work (aka Daddy Warbucks bought us a new SAN) and I’ve been charged with moving things to it. Some aspects of this are easier than others and there will be a few more posts coming about that in the future.
For now though, let’s talk about copying files. Lots of files. Server logs, audit logs, things like that. Mostly they were small in size but large in quantity. I ended up with a handful of directories with several thousand files in them that needed to move from SAN A to SAN B. Windows gives us several ways to do this:
Robocopy, the most advanced of the MS copy utilities (though I believe it prefers to be called Murphy)
All of these will do a fine job of copying your files, though Robocopy will probably be the fastest due to its multithreading capabilities. But how do you know they all reached their destination intact? Copy and Xcopy offer the option of verification (both using the /v parameter) but sadly Robocopy does not. I’m not sure if verification is just built-in to Robocopy and can’t be disabled, or if it doesn’t exist at all. Either way I didn’t want to risk errors in moving all this data, so I decided to go the extra mile and use another tool to make sure. It didn’t take me long to find the Microsoft File Checksum Integrity Verifier (“FCIV” for short), a nifty little unsupported command-line utility that does exactly what I was looking for.
FCIV In A Nutshell
Basically, FCIV calculates MD5 or SHA-1 hash values for files and outputs them either to the screen or to an XML file. It can also compare files to those checksums saved in XML and tell you if anything differs or is missing. A demo is worth a lot of words, so let’s see it in action!
Download Microsoft FCIV and extract the executable wherever you like – for this demo I put it in G:\
-wp means we’re saving only the file names in the XML file, not their full path -sha1 specifies to calculate a SHA-1 hash on each file. The default is MD5. -xml means output the checksums to an XML file, in this case the G:\hashdb.xml that follows it.
Let’s open up that XML file and see what it contains:
As you can see it’s very simple, just the file names and a checksum for each. Now let’s make a few changes.
Change the name of the directory the files are in. I changed mine from “demofiles” to “demofiles2”.
Delete fileE.txt
In fileD.txt, delete the line that says “***DELETE THIS LINE***”
Now let’s use FCIV to verify our files against the checksums we captured in the XML file. Change the current directory to demofiles2 (it won’t work unless you do this) and then run
G:\fciv.exe -v -sha1 -xml G:\hashdb.xml
-v means we’re now in verification mode, so it will verify checksums in the current directory against those in the XML file -sha1 again specifies we’re using the SHA-1 hash -xml is the file we’re comparing our calculated checksums against
Here’s the output it produces:
As you can see, FCIV is telling us that the contents of fileD have changed and fileE is missing. It’s really that easy!
Final Thoughts
I think FCIV is a great utility to keep in your toolbox. Some people may argue that checksum verification isn’t necessary – that Windows does it for you behind the scenes. That may be entirely true, but I wasn’t able to find any concrete documentation proving that it does. Then 10 minutes I spent finding this program online and figuring it out is a very small price to pay for some extra peace of mind in knowing that thousands of files made it to their destination intact.
Others may raise the point that both the MD5 and SHA-1 checksums both suffer from collision vulnerabilities and there are better alternatives out there that this application doesn’t support. They’re totally correct, but it’s also important to remember that we’re using these checksums to detect changes, not for cryptography or protecting secrets. Any form of verification is better than none, and for my purposes FCIV has proven to be very helpful.
If you’ve been following my blog, you’ll know that I’ve written quiteafewposts about the wonders of cloud backups and why you should be doing them to protect your precious data and memories. This doesn’t just apply to personal computer use – businesses should be taking advantage of offsite backups as well. Large businesses typically have this under control, but I’ve found that small businesses (especially those without an IT staff) tend to be the most vulnerable. Owners of many small businesses know just enough technology to do what they need, and rarely have the time or desire to keep up with changing times and best practices. While I don’t hold this against anyone, it is rather unfortunate.
A few months ago, a large bridal shop near me burned to the ground. This shop was of particular significance because Michelle ordered her wedding dress from there. We were very fortunate that her dress had not yet been made, so it wasn’t at the shop. Many others weren’t nearly as lucky. The business was a total loss, and with it went thousands of dresses, many of which were being held for upcoming weddings. The fire was on a Wednesday and many brides were scheduled to pick up their dresses that night for weddings taking place that weekend. Some literally arrived to pick up their dress and saw the building in flames. Through the generosity of other bridal shops in the area, I believe everyone was able to find *a* dress for their wedding, though obviously not the one they had ordered.
So what does this have to do with cloud backups? Nothing. The cloud can do many things, but putting out fires and replacing burned wedding dresses is not among them. Since there was plenty of time until our wedding, Michelle decided to wait a few weeks for things to die down and the business to set up a temporary space before calling and seeing what the deal was with her dress. We were assured that the dress order (placed several months earlier) was at the factory and we had nothing to worry about regarding its delivery in time for our wedding. That being said, they also informed us that they had lost absolutely all of their data, and were asking that we please fax or email over any paperwork we had including order forms and receipts.
This was not an issue for us at all, as we’ve been keeping all of our wedding-related info (including scanned forms) in Google Docs and sharing it between our accounts. This has been incredibly helpful, and I’ll have to blog about it all sometime in the future. We were able to email them everything we had within the hour. A little while later I got to thinking about the tremendous loss this business just incurred. Not only did they lose their inventory and their building, but all their customer and order data as well. Anyone whose name was on a list to get called back about something probably never heard from them. We were honest and sent back the forms reflecting how much money we still owed for the dress, but if they truly lost everything (and we didn’t have a conscience) we could have very easily doctored that receipt to say that everything was paid in full.
Once again, the cloud couldn’t have prevented the fire and I feel terrible for all the brides who were thrown a curve ball at the last minute and couldn’t get married in the dress of their dreams, but to me it’s equally sad that all the data loss that followed could have been prevented for a few dollars a month. Disasters like this are exactly what cloud backup can prevent. To all the small business owners out there: Your business is your data – treat your data like your job depends on it!
A while back I did a few posts covering my favorite cloud backup solutions, and one of my favorites was Mozy. That very well may change now that Mozy has announced they are changing their pricing structure. They claim that “the backup market has changed” since 2006 due to people taking more photos and videos than ever before, and even though the majority of their users back up 50GB or less, the few that greatly exceed that number are ruining it for everyone. Gone are Mozy’s days of backing up unlimited data for a flat rate.
Instead of $4.95 per month per computer for unlimited data backup, Mozy is now charging $5.99 per month for 50GB of backup space for 1 computer, or $9.99 per month for 125GB of space shared between up to 3 computers. Need more space? You can add to the $9.99 plan in increments of 20GB for $2 per month, and additional computers can be added for that same monthly rate. As before, there are discounts if you pre-pay for 1 or 2 years.
I wasn’t thrilled about how Mozy is giving its user base a “one-two punch” of raising prices and reducing value, and judging by some of the comments over at the Mozy Community Discussion Boards it looks like I’m not alone. Shouldn’t storage only be getting cheaper with time? I understand that enterprise-class storage isn’t exactly as simple as picking up a bunch of hard drives from your local geek store, but disk space in general is a lot cheaper than it used to be.
In defense of Mozy, they still are cheaper than using cloud storage such as Amazon S3. Mozy’s giving users 125GB for $9.99 a month. As of S3’s rates today, you’d pay $.14 per GB/Month or $17.50 and that’s just for the storage (don’t forget they also charge for transferring the data to them). On the other hand, there are many competitors who still offer unlimited cloud backup for a flat rate, and I’d imagine they are seeing a lot of new business from disgruntled Mozy users. Some of them, such as CrashPlan and Backblaze are even offering a discount for those who switch from Mozy.
Another option for cloud backup that has the potential to be awesome is Google Paid Storage. Their prices are amazing – currently $0.25 per GB per year, and they’ll sell you up to 16TB of space! The downside is that there’s no easy way to back up to this space (sometimes I wonder if this is on purpose). You can utilize their space by uploading files to Google Docs, however that’s neither fast nor convenient. People have been hoping for a Google online storage service (usually referred to as “G-Drive”) for a while now and there’s still no sign of it coming, but online backups would be another great way for them to make a killing.
Even with this recent price-jacking, are online backups still worth it? Absolutely. If your computer is stolen or goes up in smoke you will have no trouble replacing the hardware, installed applications, or music, but what you really need to protect are the things that can’t be replaced such as photos and videos. Things like that are priceless, and the best way to protect them is to keep a copy somewhere far far away, like the cloud. Backing up to an external hard drive is fine, but should your home be burglarized or destroyed by fire that external drive is very likely to be just as missing/destroyed as your computer. The peace of mind that can be had from cloud backups should far outweigh the cost. I think of it as an insurance plan – the premiums fees paid for online backup may be a pain, but are far less than the cost of losing priceless data.
