Archive for the Category » backup «

Sunday, August 04th, 2013 | Author:

History

Much had changed since I last mentioned my personal serverit has grown by leaps and bounds (it now has a 7TB md RAID6) and it had recently been rebuilt with Ubuntu Server.

Arch was never a mistake. Arch Linux had already taught me so much about Linux (and will continue to do so on my other desktop). But Arch definitely requires more time and attention than I would like to spend on a server. Ideally I’d prefer to be able to forget about the server for a while until a reminder email saysumthere’s a couple updates you should look at, buddy.

Space isn’t freeand neither is space

The opportunity to migrate to Ubuntu was the fact that I had run out of SATA ports, the ports required to connect hard drives to the rest of the computerthat 7TB RAID array uses a lot of ports! I had even given away my very old 200GB hard disk as it took up one of those ports. I also warned the recipient that the disk’s SMART monitoring indicated it was unreliable. As a temporary workaround to the lack of SATA ports, I had even migrated the server’s OS to a set of four USB sticks in an md RAID1. Crazy. I know. I wasn’t too happy about the speed. I decided to go out and buy a new reliable hard drive and a SATA expansion card to go with it.

The server’s primary Arch partition was using about 7GB of disk. A big chunk of that was a swap file, cached data and otherwise miscellaneous or unnecessary files. Overall the actual size of the OS, including the /home folder, was only about 2GB. This prompted me to look into a super-fast SSD drive, thinking perhaps a smaller one might not be so expensive. It turned out that the cheapest non-SSD drive I could find actually cost more than one of these relatively small SSDs. Yay for me. 🙂

Choice? Woah?!

In choosing the OS, I’d already decided it wouldn’t be Arch. Out of all the other popular distributions, I’m most familiar with Ubuntu and CentOS. Fedora was also a possibilitybut I hadn’t seriously yet considered it for a server. Ubuntu won the round.

The next decision I had to make didn’t occur to me until Ubiquity (Ubuntu’s installation wizard) asked it of me: How to set up the partitions.

I was new to using SSDs in LinuxI’m well aware of the pitfalls of not using them correctly, mostly due to their risk of poor longevity if misused.

I didn’t want to use a dedicated swap partition. I plan on upgrading the server’s motherboard/CPU/memory not too far in the future. Based on that I decided I will put swap into a swap file on the existing md RAID. The swap won’t be particularly fast but its only purpose will be for that rare occasion when something’s gone wrong and the memory isn’t available.

This then left me to give the root path the full 60GB out of an Intel 330 SSD. I considered separating /home but it just seemed a little pointless, given how little was used in the past. I first set up the partition with LVMsomething I’ve recently been doing whenever I set up a Linux box (really, there’s no excuse not to use LVM). When it got to the part where I would configure the filesystem, I clicked the drop-down and instinctively selected ext4. Then I noticed btrfs in the same list. Hang on!!

But a what?

Btrfs (“butter-eff-ess”, “better-eff-ess”, “bee-tree-eff-ess”, or whatever you fancy on the day) is a relatively new filesystem developed in order to bring Linuxfilesystem capabilities back on track with current filesystem tech. The existing King-of-the-Hill filesystem, “ext” (the current version called ext4) is pretty goodbut it is limited, stuck in an old paradigm (think of a brand new F22 Raptor vs. an F4 Phantom with a half-jested attempt at an equivalency upgrade) and is unlikely to be able to compete for very long with newer Enterprise filesystems such as Oracle’s ZFS. Btrfs still has a long way to go and is still considered experimental (depending on who you ask and what features you need). Many consider it to be stable for basic usebut nobody is going to make any guarantees. And, of course, everyone is saying to make and test backups!

Mooooooo

The most fundamental difference between ext and btrfs is that btrfs is aCoWorCopy on Writefilesystem. This means that data is never actually deliberately overwritten by the filesystem’s internals. If you write a change to a file, btrfs will write your changes to a new location on physical media and will update the internal pointers to refer to the new location. Btrfs goes a step further in that those internal pointers (referred to as metadata) are also CoW. Older versions of ext would have simply overwritten the data. Ext4 would use a Journal to ensure that corruption won’t occur should the AC plug be yanked out at the most inopportune moment. The journal results in a similar number of steps required to update data. With an SSD, the underlying hardware operates a similar CoW process no matter what filesystem you’re using. This is because SSD drives cannot actually overwrite datathey have to copy the data (with your changes) to a new location and then erase the old block entirely. An optimisation in this area is that an SSD might not even erase the old block but rather simply make a note to erase the block at a later time when things aren’t so busy. The end result is that SSD drives fit very well with a CoW filesystem and don’t perform as well with non-CoW filesystems.

To make matters interesting, CoW in the filesystem easily goes hand in hand with a feature called deduplication. This allows two (or more) identical blocks of data to be stored using only a single copy, saving space. With CoW, if a deduplicated file is modified, the separate twin won’t be affected as the modified file’s data will have been written to a different physical block.

CoW in turn makes snapshotting relatively easy to implement. When a snapshot is made the system merely records the new snapshot as being a duplication of all data and metadata within the volume. With CoW, when changes are made, the snapshot’s data stays intact, and a consistent view of the filesystem’s status at the time the snapshot was made can be maintained.

A new friend

With the above in mind, especially as Ubuntu has made btrfs available as an install-time option, I figured it would be a good time to dive into btrfs and explore a little. 🙂

Part 2 coming soon

Share
Saturday, February 21st, 2009 | Author:

The relatively new document types Office 2007 has given some web hosts problems when their clients want to offer documents for download. Most often, the documents are being offered by the web server as text/html which is then rendered as a ton of garbage on the web user’s screen.

The best way to resolve this is to add all the MIME types to the server’s main configuration. IIS7 for Windows already has these MIME types set up correctly by default. IIS6 and IIS5 require the MIME types to be added, as might Apache on older installations. For Apache, there is also a workaround for the individual domain owner to add the mime types via Apache’s .htaccess file.

IIS 6 MIME type addition (for the Server Administrator)

Before this can be done, ensure that your server is also set to allow direct metabase editing:

  1. Load IIS Manager: Start -> Run, “inetmgr” -> [OK]
  2. Right click theserverand clickProperties
  3. Within theInternet Information Servicestab (usually the only tab), ensure that theEnable Direct Metabase Editcheckbox is checked.
  4. Click [OK]

Be sure to back up IIS’s configuration (here for IIS5) beforehand. I won’t take any responsibility for an admin breaking his server. I have reason to believe this may also work on IIS5 however I have just as much reason to believe that it might just give lots of errors. If an IIS5 / Windows 2000 admin is willing to test this for me after backing up your configuration please let me know of the results.

Copy the following text into a file named msoff07-addmime.vbs and execute it once from the commandline by typing cscript msoff07-addmime.vbs and pressing Enter. If you run it more than once, the MIME types will be added each time and you will have multiple identical entries:

' This script adds the necessary Office 2007 MIME types to an IIS 6 Server.
' To use this script, just double-click or execute it from a command line.
' Running this script multiple times results in multiple entries in the
' IIS MimeMap so you should not run it more than once.
' Modified from http://msdn.microsoft.com/en-us/library/ms752346.aspx
 
Dim MimeMapObj, MimeMapArray, MimeTypesToAddArray, WshShell, oExec
Const ADS_PROPERTY_UPDATE = 2 
 
' Set the MIME types to be added
MimeTypesToAddArray = Array(".docm", "application/vnd.ms-word.document.macroEnabled.12", _
".docx", "application/vnd.openxmlformats-officedocument.wordprocessingml.document", _
".dotm", "application/vnd.ms-word.template.macroEnabled.12", _
".dotx", "application/vnd.openxmlformats-officedocument.wordprocessingml.template", _
".potm", "application/vnd.ms-powerpoint.template.macroEnabled.12", _
".potx", "application/vnd.openxmlformats-officedocument.presentationml.template", _
".ppam", "application/vnd.ms-powerpoint.addin.macroEnabled.12", _
".ppsm", "application/vnd.ms-powerpoint.slideshow.macroEnabled.12", _
".ppsx", "application/vnd.openxmlformats-officedocument.presentationml.slideshow", _
".pptm", "application/vnd.ms-powerpoint.presentation.macroEnabled.12", _
".pptx", "application/vnd.openxmlformats-officedocument.presentationml.presentation", _
".sldm", "application/vnd.ms-powerpoint.slide.macroEnabled.12", _
".sldx", "application/vnd.openxmlformats-officedocument.presentationml.slide", _
".xlam", "application/vnd.ms-excel.addin.macroEnabled.12", _
".xlsb", "application/vnd.ms-excel.sheet.binary.macroEnabled.12", _
".xlsm", "application/vnd.ms-excel.sheet.macroEnabled.12", _
".xlsx", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", _
".xltm", "application/vnd.ms-excel.template.macroEnabled.12", _
".xltx", "application/vnd.openxmlformats-officedocument.spreadsheetml.template") 
 
' Get the mimemap object
Set MimeMapObj = GetObject("IIS://LocalHost/MimeMap")
 
' Call AddMimeType for every pair of extension/MIME type
For counter = 0 to UBound(MimeTypesToAddArray) Step 2
    AddMimeType MimeTypesToAddArray(counter), MimeTypesToAddArray(counter+1)
Next
 
' Create a Shell object
Set WshShell = CreateObject("WScript.Shell")
 
' Stop and Start the IIS Service
Set oExec = WshShell.Exec("net stop w3svc")
Do While oExec.Status = 0
    WScript.Sleep 100
Loop
 
Set oExec = WshShell.Exec("net start w3svc")
Do While oExec.Status = 0
    WScript.Sleep 100
Loop
 
Set oExec = Nothing
 
' Report status to user
WScript.Echo "Microsoft Office 2007 Document MIME types have been registered."
 
' AddMimeType Sub
Sub AddMimeType (Ext, MType)
 
    ' Get the mappings from the MimeMap property.
    MimeMapArray = MimeMapObj.GetEx("MimeMap") 
 
    ' Add a new mapping.
    i = UBound(MimeMapArray) + 1
    Redim Preserve MimeMapArray(i)
    Set MimeMapArray(i) = CreateObject("MimeMap")
    MimeMapArray(i).Extension = Ext
    MimeMapArray(i).MimeType = MType
    MimeMapObj.PutEx ADS_PROPERTY_UPDATE, "MimeMap", MimeMapArray
    MimeMapObj.SetInfo
 
End Sub

Apache MIME type addition (for the Server Administrator)

Apache stores its MIME types in a file normally located at $installpath/conf/mime.types. See the mod_mime documentation for more on how it works. Arch Linux installs its MIME types at /etc/httpd/conf/mime.types and Parallels Plesk installs it in /usr/local/psa/admin/conf/mime.types. Your distribution might have it in another place, so find your mime.types file by running locate mime.types.

Add the following lines to your mime.types file:

application/vnd.ms-word.document.macroEnabled.12                          docm
application/vnd.openxmlformats-officedocument.wordprocessingml.document   docx
application/vnd.ms-word.template.macroEnabled.12                          dotm
application/vnd.openxmlformats-officedocument.wordprocessingml.template   dotx
application/vnd.ms-powerpoint.template.macroEnabled.12                    potm
application/vnd.openxmlformats-officedocument.presentationml.template     potx
application/vnd.ms-powerpoint.addin.macroEnabled.12                       ppam
application/vnd.ms-powerpoint.slideshow.macroEnabled.12                   ppsm
application/vnd.openxmlformats-officedocument.presentationml.slideshow    ppsx
application/vnd.ms-powerpoint.presentation.macroEnabled.12                pptm
application/vnd.openxmlformats-officedocument.presentationml.presentation pptx
application/vnd.ms-powerpoint.slide.macroEnabled.12                       sldm
application/vnd.openxmlformats-officedocument.presentationml.slide        sldx
application/vnd.ms-excel.addin.macroEnabled.12                            xlam
application/vnd.ms-excel.sheet.binary.macroEnabled.12                     xlsb
application/vnd.ms-excel.sheet.macroEnabled.12                            xlsm
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet         xlsx
application/vnd.ms-excel.template.macroEnabled.12                         xltm
application/vnd.openxmlformats-officedocument.spreadsheetml.template      xltx

Apache MIME type addition (For the domain owner with at least FTP accessusing .htaccess file)

Add the following text to your domain’s .htaccess file, most commonly in an httpdocs/ directory

AddType application/vnd.ms-word.document.macroEnabled.12 docm
AddType application/vnd.openxmlformats-officedocument.wordprocessingml.document docx
AddType application/vnd.ms-word.template.macroEnabled.12 dotm
AddType application/vnd.openxmlformats-officedocument.wordprocessingml.template dotx
AddType application/vnd.ms-powerpoint.template.macroEnabled.12 potm
AddType application/vnd.openxmlformats-officedocument.presentationml.template potx
AddType application/vnd.ms-powerpoint.addin.macroEnabled.12 ppam
AddType application/vnd.ms-powerpoint.slideshow.macroEnabled.12 ppsm
AddType application/vnd.openxmlformats-officedocument.presentationml.slideshow ppsx
AddType application/vnd.ms-powerpoint.presentation.macroEnabled.12 pptm
AddType application/vnd.openxmlformats-officedocument.presentationml.presentation pptx
AddType application/vnd.ms-powerpoint.slide.macroEnabled.12 sldm
AddType application/vnd.openxmlformats-officedocument.presentationml.slide sldx
AddType application/vnd.ms-excel.addin.macroEnabled.12 xlam
AddType application/vnd.ms-excel.sheet.binary.macroEnabled.12 xlsb
AddType application/vnd.ms-excel.sheet.macroEnabled.12 xlsm
AddType application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx
AddType application/vnd.ms-excel.template.macroEnabled.12 xltm
AddType application/vnd.openxmlformats-officedocument.spreadsheetml.template xltx
Share