Backup Policy
 

I'm really at a loss to understand the poor backup policy. Backups should be done daily, not monthly. I've been in the computer business for 30 years and I just wonder what's going to happen when all of us seasoned professionals retire and the teenagers take over.

Let us know when you start up your own Mercedes forum . . .

And when you have such extensive threads and a user base over 10000 users.

It's easy to have a loss to understand when you don't know all the facts! :eek:

What is it about computers geeks? We all seem to believe that we know everything there is to know about computers. Believe me, I'm a network administrator, I know the attitude. When a system isn't yours, don't critique unless asked and then only if one is intimately familiar with the system.

Yup - sysadmins, like engineers, often seem to be lacking in certain social graces. Tactless comments are commonplace, as the average mindset tends to be more geared towards problem solving, and less towards polite human interaction.

However, when you work in these fields, you have to have thick skin. The ability to absorb and evaluate information from peers, ignoring the attitude that so often accompanies it, is a valuable trait.

Sorry, can't go along with that in all cases, and this is one of them. Suggesting a solution without knowing the details is foolish, but pointing out that the current system/procedures are flawed is on target. Losing 30 days of data should never happen, and it's relatively easy to prevent. Intimate knowledge of the system or processes involved is not necessary; whatever is currently in place can be improved.

I'm guessing there are a fair number of folks here with some technical clue, and probably more than a handful of us who make a living doing this kind of work. If a more detailed explanation were provided as to the failure, I'm sure plenty of seasoned sysadmins would be glad to share strategies that might help prevent the problem from recurring.

As the director of a large hospital IT department, I can tell you the Achilles heel of any server is going to be the storage system. In healthcare, because we are regulated by the federal government, with standards such as the Health Insurance Portability and Accountability Act (HIPAA), we are expected to have in place mechanisms that ensure our systems have valid backup media and that it is tested regularly. We had an incident Thursday where we had to restore several files and it worked flawlessly. :D

However, in spite of all this, the unexpected can and usually does occur. :eek: Whether it's a bad backup tape, bad replacement drive, or even a bad RAID controller that takes out multiple drives (we've had all of those things happen by the way), you only have so much control over your environment.

To the folks running this website, I can sympathize . You've got a tough job of keeping these servers up and available 24/7. My only recomendation would be to increase the frequency of your backups. Perhaps once a week? Keep up the good work! :2thumbsup

I see that the search still has volumes of information,,,, all is not lost. Looks like just a little glitch on a privately run site. I, for one, am still indebeted to this site and thank whoever it is that started it and keeps it going. I would be very sad to ever see it go. Pete G.

You guys give me the creeps.

My office is switching from film photography and optical analysis to totally digital. Several of you gave me some excellent advice, especially Lebenz and I think Suginami(?), which really helped guide our plans. So we've spent about $80K on network storage and server upgrades. That was supposed to bridge us for a year or so. Then we get these two damned hurricanes and we have 14 terabytes(!!!) of new imagery that we have to store and serve. WTF am I going to do with that?! We got 3 more (I think) T1 lines to serve the data and we've been buying terabyte bricks like, well, bricks, in order to send and receive data from across the country (and into the next cubicle). Worse, now I get to worry about losing it if I don't plan to spend an enormous amount of money on storage.

OMG.

B

In a business, losing 30 days of data could mean the loss of the business. However there is usually a budgeted expense involved for providing backup services. Although its unfortunate that we lost some good contributions to the MB forums, we shouldn't expect more than what we paid for, which for the great majority of us, is nothing.

IT will become a virtual video game world

Nobody will ever have to leave their house, once the genius who will invent pizza delivered electronically, we will have all we need, tubes to transport fluids, pizza, online porn, and a cathater/bag :) sorry about the last image...

True, although the average nerd-types that run a place like this can do it for next to nothing.

On the side, I run 6 sites, the largest of which probably has more data than is stored here. It has 1/3 the number of posts, but has a huge and very active photo gallery - total size including database and photos is currently about 10GB. My nightly automated job exports and compresses the db, and transfers it along with any updated server files (usually the photos) nightly. Typical time for the job over a 1.5 DSL connection - about 5 minutes.

My hosting provider does backups, just as they do here. But rule #1 when hosting a site - never trust the hosting provider. And in fact, based on the webmaster's post here, that was the primary reason for the data loss - the hosting provider screwed the pooch, an all too common occurence.

Local nightly backups for a site like this are easy and cheap to perform. All it takes is a little knowledge, an old PC, a pair of large drives (Linux software RAID for redundancy), and a broadband connection. Add a DVD writer for additional peace of mind if necessary.

Assuming one has an old PC laying around (a fair assumption for most nerds), total cost is well under $100 for a pair of 80GB drives, toss in another $50 if you want a DVD writer. Linux is free, as are all the tools necessary to perform the automated local backups. You end up with pretty bullet-proof redundancy, as you've got two independant, geographically dispersed backup sets - assuming your hosting provider is doing their part.

Yes, it takes some time and a little tweaking, and some may suggest it's overkill for a community site like this. Ultimately, that decision lies with the one paying the bills...given the valuable info hosted here, I'd suggest it's worth it, but that's not my call.

In the bigger offices I care for, I use 4 different backup strategies, but any or all can and have failed before. Generally this stratagee includes one or more redundant backups for each server. The backup simply makes a daily data copy of everything, which resides on another server. The backup skips files which haven't changed since the last backup. As a result, after the first run, the backup is comparatively fast. In addition, another copy of data kept on external drives which are cycled in and out of the building and used primarily for off-site and disaster recovery related storage. Depending on the nature of the data being stored and the capability of the backup, the task can be very complex. As example, a site such as this has processes which are almost always running. This means that several critical files are not generally able to be backed up. The common solution is to pick a time to shut off all the services related to the forum, run the backup and then turn the services back on. Another but far less common solution is an agreement between the product vendor and backup or OS vendor(s) to make it possible for the backup to start and stop services to permit selected files to be backed up. In turn, the need to take stuff off line for a while puts pressure on the fastest backup media available. For this reason i long ago abandoned all streaming media.

The Lacie big disk genre is one of the best values for high volume & high speed storage: Plus their smaller capacity drives are a bargain.

http://www.lacie.com/products/product.htm?pid=10351

http://www.lacie.com/products/product.htm?pid=10600

http://www.lacie.com/products/product.htm?pid=10458

One thing about storage devices is that you want at least 2 of ‘em. Everyone that hasn’t been bitten assumes a backup will be there indefinitely. As a result, folks put things on one of em to store them indefinitely, but the backup media tends to fail at the worst time, and unless you have 2 you are planning for a nightmare like dead end. Worse, the cost to have a data recovery center pry a TB of stuff off a dead drive is enough to notch up the federal deficit. The cost of recovery makes the cost of a couple of drives thrifty by comparison.

Lastly the hardest part of this process is data verification. Generally I ask 2 or 3 folks a week to name something they've worked on. I'll then see if it made it to the backup. This is even more cumbersome with sites such as this one as the only real way to verify is to restore the content to another computer and test. That kind of task is the typical budget breaker but still far less costly than :eek: rebuilding from scratch.

Edit: Oh yeah, if you want an excellent and dirt cheap backup program look for one called xxcopy. It has crptic instructions but they have a great support site and unmatched flexibility

At your own expense and charge nothing for it as well.

I feel we, as users, are responsible for our own back-ups. If there is a thread you are following, simply save it to your hard drive. Each time I see a thread in a forum that is of particular interest to me, I simply save it. Had I saved any one thread from mshop, I would be glad to supply it to the owner/moderator; however, I looked and I have not save a thread from mshop, SL forum in a long time...

Oh, for those who do not know how... Using IE, for example, open the thread, then click on your file menu and then click on "save as". Designate a folder and save. The html will be saved, along with a folder of the same name. The folder will have all the .gif, jpeg, etc files located in it.