NAS Setup and Tape Backup Strategy

Tape Backup Questions (2/2009)

How are tapes labeled?
I don't want to think about which tape to use. With date of last backup as well as a volume-name and an increasing index number (e.g., user1, admin1, gis1, user2, admin2, gis2, user3, admin3, gis3, user4, etc.).

How often are tapes rotated in/out?
Oldest tape every 12 weeks, before that week's backup. Check out the tape backup/rotation schedule.

How often do we backup?
Every two weeks.

On what schedule are backups run?
M, T, W via cron w/ e-mail notification of fail/success and reminder to change tape. After all 3 backups have run in a week (as appropriate), send e-mail reminder to clean the tape drive (or setup cron task).

How many tapes in rotation?
Six (6) for each partition.

When do we test our backups?
Every 12 weeks.

How do we test our backups?

1. Generate checksum of all files before backup.
2. Time passes.
3. Select a tape (how?) from archive.
4. Restore tape contents to unused partition/area.
5. Compare restored partition file checksums to that backup's saved file checksum list (should be unchanged).

How often do we clean the tape drive?
Cleaning tape says it is drive-dependent, suggested frequency for our drive is once a month (assumed usage?) Clean tape drive every 18 uses. Should map to every 6 weeks, assuming we backup 3 volumes (one volume per tape) every 2 weeks.

Where do we store the archived tapes?

1. Firesafe in CBI
2. Firesafe at NOC
3. Mail to Austin

Maximum times a tape can be used?
Manufacturer says certified 260 times, we'll say 200.

How does the number of times a tape has been used for backup affect that tape's life expectancy?

Tape Rotation Schedule

After some discussion, James and I built a spreadsheet (OpenOffice) outlining our tape backup/rotation schedule: Tape Rotation Schedule (ODS 21KB). The starting "week of" date (upper left) can be changed if we don't want to start counting backup date from the week of 4/13/2009.

The tape rotation schedule is also available as a PDF: Tape Rotation Schedule (PDF 47KB)

Tape Replacement SOP

We backup the user partition every other Tuesday morning (early AM) and the admin partition every other Wednesday morning. It is the responsibility of whoever is on call to change the tapes.

When it is your turn to swap the tapes, please do the following:

1. Write the date of the latest backup/use on the tape label
2. After use, store the backup tapes in the fire safe in James and Niall's office
3. Take the tape set (e.g., both tapes) for the next backup run (see the tape rotation schedule) and put them on top of outhouse in the server rack
4. Load the tape for the next backup in the tape drive (user)

Ideas/Implementation

• Use flash memory available on tape media [James D.?] to store date of last use, number of times used, etc. This information is checked before a backup commences. If it has not been 12 weeks between the last backup on this tape and now, the system does not initiate the backup. It also sends a nasty e-mail.

References

• Our old backup strategy notes here: Old DNR tape backup strategy
• Backing up Linux and other Unix(-like) systems
• Troubleshooting: Dell PowerVault 110T LTO-3 Tape Drive User's Guide

Openfiler Setup

Network Access Configuration

Note: Openfiler uses conary for package management. To update: conary updateall

In /opt/openfiler/etc/httpd/conf/httpd.conf, changed Listen 446 to Listen 444 (apparently 446 is blocked)

What is backed up?

user volume

• /home (/u2)

admin volume

• dnrpub (lighthouse)
• merdian
  • /system (/m0)
  • /var/named
  • /var/log/httpd
• pharos (lighthouse)
  • pharosdb/var (exclude spool and tmp)
  • pharosdb/stations
  • pharosdb/www (exclude img/cache)
  • clerk/var (exclude spool and tmp)
  • clerk/archive
  • pobot/var (exclude spool and tmp)
  • corrections - /u3/pharosdb/var/smcorr.dat
  • /var/log/httpd
  • other /u3/
    • clerk2?
    • field?
    • forecasts?
    • mesoeta?
    • voiceinfo? (I guess this is running on monarch)
  • cbidl?

gis volume