Tag-Archive for » linux «

Sunday, August 04th, 2013 | Author:

I had a power outage affect my server’s large md RAID array. Rather than let the server as a whole be down while waiting for it to complete an fsck, I had it boot without the large array so I could run the fsck manually.

However, when running it manually I realised I had no way of knowing how far it was and how long it would take to complete. This is especially problematic with such a large array. With a little searching I found the tip of adding the -C parameter when calling fsck. I couldn’t find this in the documentation however: fsckhelp showed no such option.

The option turns out to be ext4-specific, and thus shows a perfectly functional progress bar with a percentage indicator. To find the information, instead offsckhelp” Aŭ “man fsck”, you have to inputfsck.ext4help” Aŭ “man fsck.ext4”. 🙂

Interŝanĝado
Sunday, August 04th, 2013 | Author:

History

Much had changed since I last mentioned my personal server – Ĝi kreskis de saltegoj kaj saltegoj (it now has a 7TB md RAID6) and it had recently been rebuilt with Ubuntu Server.

Arch was never a mistake. Arch Linux had already taught me so much about Linux (Kaj daŭros tiel fari sur mia alia surtabla). But Arch definitely requires more time and attention than I would like to spend on a server. Ideally I’d prefer to be able to forget about the server for a while until a reminder email saysumthere’s a couple updates you should look at, buddy.

Space isn’t freeand neither is space

The opportunity to migrate to Ubuntu was the fact that I had run out of SATA ports, the ports required to connect hard drives to the rest of the computerthat 7TB RAID array uses a lot of ports! I had even given away my very old 200GB hard disk as it took up one of those ports. I also warned the recipient that the disk’s SMART monitoring indicated it was unreliable. Kiel intertempa workaround al la manko de SATAaj havenoj, I had even migrated the server’s OS to a set of four USB sticks in an md RAID1. Freneza. Mi scias. Mi wasn’t tro feliĉa pri la rapido. I decided to go out and buy a new reliable hard drive and a SATA expansion card to go with it.

The server’s primary Arch partition was using about 7GB of disk. A big chunk of that was a swap file, Cached datumo and otherwise miscellaneous or unnecessary files. Entute la reala grandeco de la OS, including the /home folder, Estis nur pri 2GB. This prompted me to look into a super-fast SSD drive, thinking perhaps a smaller one might not be so expensive. It turned out that the cheapest non-SSD drive I could find actually cost Pli Ol unu el ĉi tiuj relative malgranda SSDs. Yay for me. 🙂

Choice? Woah?!

En elektanta la OS, I’d jam decidis ĝin wouldn’t esti Arkaĵo. Ekstere de ĉiuj la aliaj popularaj disdonadoj, I’m most familiar with Ubuntu and CentOS. Fedora was also a possibilitybut I hadn’t seriously yet considered it for a server. Ubuntu won the round.

The next decision I had to make didn’t occur to me until Ubiquity (Ubuntu’s instalaĵa sorĉisto) Demandis ĝin de mi: How to set up the partitions.

I was new to using SSDs in Linux – I’m bone konscia de la enfaliloj de ne uzanta ilin ĝuste, Plejparte pro ilia risko de malriĉa longevity se misuzita.

Mi didn’t deziras uzi diligentan interŝanĝan septon. Mi planas sur altgradiganta la servilon’s motherboard/CPUa/memoro ne tro ege en la estonteco. Bazita sur kiu mi decidis min metos interŝanĝon en interŝanĝa dosiero sur la ekzistanta md TRUDENIRO. The swap won’t be particularly fast but its only purpose will be for that rare occasion when something’s gone wrong and the memory isn’t available.

This then left me to give the root path the full 60GB out of an Intel 330 SSD. Mi konsideris apartiganta /hejmon sed ĝi nur ŝajnita malgranda sencela, Donita kiom malmulte estis uzita en la pasinteco. I first set up the partition with LVMsomething I’ve recently been doing whenever I set up a Linux box (really, there’s no excuse not to use LVM). Kiam ĝi akiris al la parto kie mi formus la filesystem, Mi klakis la falon-malsupren kaj instinkte elektita ext4. Tiam mi rimarkis btrfs en la sama listo. Pendi sur!!

But a what?

Btrfs (“Butero-eff-ess”, “Pli bona-eff-ess”, “Abelo-arbo-eff-ess”, Aŭ kio ajn vi fantazias sur la tago) Estas relative nova filesystem evoluigita por alporti Linukson’ Filesystem kapabloj reen sur vojeto kun fluo filesystem tech. La ekzistanta King-de-la-Monteto filesystem, “Ext” (La nuna versio vokis ext4) Estas sufiĉe bona – but it is limited, stuck in an old paradigm (think of a brand new F22 Raptor vs. an F4 Phantom with a half-jested attempt at an equivalency upgrade) and is unlikely to be able to compete for very long with newer Enterprise filesystems such as Oracle’s ZFS. Btrfs ankoraŭ havas longan vojon iri kaj estas ankoraŭ konsiderita eksperimentan (Dependanta sur kiu vi demandas kaj kio prezentas vin bezono). Multaj konsideri ĝin esti stabila por baza uzo – Sed neniu estas iranta fari ajnajn garantiojn. Kaj, Nature, Ĉiu estas diranta fari kaj testaj rezervoj!

Mooooooo

La plej fundamenta diferenco inter ext kaj btrfs estas ke btrfs estas “BOVINO” Aŭ “Kopii sur Skribi” Filesystem. Tio ĉi signifas ke datumo estas neniam efektive intence anstataŭigita de la filesystem’s internals. Se vi skribas ŝanĝon al dosiero, btrfs will write your changes to a new location on physical media and will update the internal pointers to refer to the new location. Btrfs iras paŝon cetere en kiu tiuj internaj sugestoj (Plu-gvidita al kiel metadata) Estas Ankaŭ BOVINO. Pli malnovaj versioj de ext havus simple anstataŭigitan la datumo. Ext4 uzus Revuon certigi tiun korupton gajnis’t okazas devus la AC ŝtopas esti elretirita ĉe la plej inopportune momento. La revuaj rezultoj en simila nombro de paŝoj postulita ĝisdatigi datumon. With an SSD, the underlying hardware operates a similar CoW process no matter what filesystem you’re using. Tio ĉi estas ĉar SSDaj stiradoj povas ne efektive anstataŭigi datumon – Ili devi kopii la datumon (with your changes) to a new location and then erase the old block entirely. An optimisation in this area is that an SSD might not even erase the old block but rather simply make a note to erase the block at a later time when things aren’t so busy. The end result is that SSD drives fit very well with a CoW filesystem and don’t perform as well with non-CoW filesystems.

Fari aferojn interesanta, BOVINO en la filesystem facile iras manon mane kun ĉefaĵo vokis deduplication. Tio ĉi permesas du (Aŭ pli) Identaj blokoj de datumo esti entenita uzanta nur ununuran kopion, Savanta spacon. Kun BOVINO, Se deduplicated dosiero estas modifita, La aparta ĝemelo gajnis’t esti influita kiel la modifita dosieron’s datumoj estos estinta skribita al malsama fizika bloko.

BOVINO en ĝiradaj faroj Snapshotting Relative facile efektivigi. Kiam momentfoto estas farita la sistemon simple registras la novan momentfoton kiel estanta duplikatado de ĉiuj datumoj kaj metadata ene de la volumo. Kun BOVINO, Kiam ŝanĝoj estas farita, La momentfoto’s datumoj restas sendifektan, Kaj konsekvenca vido de la filesystem’s statuso ĉe la tempo la momentfoto estis farita povas esti daŭrigita.

A new friend

Kun la supra en menso, Precipe kiel Ubuntu faris btrfs havebla kiel instali-tempa elekto, Mi kalkulis ĝin estus bona tempo plonĝi en btrfs kaj esplori malgrandan. 🙂

Part 2 coming soon

Interŝanĝado
Friday, September 11th, 2009 | Author:

Peace in the land of USB

Under a *nix operating system, having multiple partitions on a USB drive isn’t rocket science, it just works. In my case, my USB drive has two partitions because the first partition is a bootable Arch Linux installer.

I have Windows on a desktop at homemostly for gamingand many of my colleagues use it too. Since Windows doesn’t do very well with non-Windows partitions I figured I could create a FAT32 partition on the memory stick after the bootable Arch Linux partition. FAT32 is almost ubiquitous and is usable on every common desktop operating system in the world.

Bleh

Unfortunately it doesn’t work straight off the bat. Apparently, Microsoft in their infinite wisdom decided that memory sticks are supposed to have one (and only one) partition. In reality Windows finds the first partition and then ignores any others that happen to be set up:

Please Format

Err, no, I do not want you to format my Arch Linux installation partition

The trick to getting it working is to fool Windows into thinking the device is not a regular USB memory stick but perhaps a solid-state hard disk which happens to be connected via USB. Yes I know, this is seriously stupid that Windows behaves this way. A solid-state hard disk is just a whopping big (and fast) memory stick after all!

I found a few sources on how to do this however I still had to figure out some things on my own. Specifically, the guides I found either skipped some steps or didn’t provide enough information on where to download the driver package.

This procedure involves manually changing hardware drivers and installingnon-signeddriversnot intended for your hardware”. I know someone is going to break their system and blame me so I say now that I take no responsibility for any damage you may do to your Windows system as a result of this. Read that again. 😛

Instructions

remove the highlighted text

click for larger version

Download and unzip the driver, originally created by Hitachi, here. Open the cfadisk.inf file in notepad (or your favourite plaintext editor), and find the section labeled [cfadisk_device]. Remove the section highlighted on the right:

Minimize (don’t close) the editor and go to your desktop iconsright-click on My Computer and select Properties. Select the hardware tab and then select [Device Manager]:

System Properties

Find the device underDisk drives”, right-click your memory stick and select Properties:

Device Manager

Click the Details tab and in the dropdown box on that page, selectHardware Ids”. Click the first line in the list of Hardware IDs and press Ctrl+C to copy the name:

USB Hardware Ids

Don’t close this dialog, go back to notepad (which was minimised) and paste the hardware ID into where the previous content was removed.

Changes pasted into notepad

Save the file in notepad and go back to the device’s property dialog window. Click theDrivertab and click the [Update Driver…] button. In the windows that pop up, selectNo, not this time”; [NExt] -> “Install from a list or a specific location (Advanced)”; [NExt] -> “Don’t search. I will choose the driver to install.”; [NExt] -> [Have Disk…].

Unsigned Drivers - Click Continue Anyway

Browse to the folder where you have saved the modified cfadisk.inf file. Click [OK]. You will find

there is a Hitachi Microdrive driver listed. Select this and click [NExt]. When the warning

appears, click [Yes]. Another warning will pop up regarding a similar issue (these are theunsignedandnot intended for your hardwarewarnings I mentioned earlier). Click [Continue Anyway]:

At this point I recommend closing all the dialog boxes related to the setup. Finally, remove and re-insert the memory stick into your USB port and you should find that the extra partitions on the stick are accessible. In the worst-case scenario, you might still need to partition the disk however the hard part is over. 🙂

Interŝanĝado
Wednesday, April 22nd, 2009 | Author:

Arch Linux’s installation process is documented on the Arch wiki. I recommend that persons new to Arch try the excellent Beginner’s Guide instead of the Official Arch Linux Install Guide. Though both wiki entries cover similar ground, the Beginner’s Guide gives a lot more relevant information for those new to the system. The Beginner’s Guide is aimed at desktop installation and, as I’m installing a server, I won’t be going through the installation of the graphical environment at all. Assuming that you’re following my installation, assume that I’ve followed the Beginner’s Guide right up to and including the installation of sudo. I installed the ssh daemon afterwards rather than during the initial setup however.

A few small recommendations and notes regarding installation:

  • If you can, consider using a USB memory stick for the installer and keep it handy for future installations.
  • I keep a copy of my localrepositoryof installed applications on my installer memory stick. Once installation is finished I save a bit of download and update time by copying this to the new server’s /var/cache/pacman/pkg/ folder. The repository on my desktop is typically 1.7GB
  • For the rc.conf, South African-appropriate regional settings are:
    LOCALE=en_ZA.utf8
    TIMEZONE=Africa/Johannesburg
  • I’ve set up the network very simply, according to the guide, and will be expanding on the network setup in a later post.
  • As it is for a server, my non-privileged user on the server is only part of 3 groups: wheel (for sudo), storage, and users. A desktop user will likely be in many more groups.

I prefer using an application called yaourt instead of Arch’s default package manager. Yaourt has the exact same usage syntax as pacman except that it supports a few extra options. It is actually a wrapper application in that it, in turn, uses pacman. Importantly, yaourt supports installation of applications from Arch’s AUR. The AUR is a repository of installation scripts built by Arch users for Arch users to easily install applications that are not officially supported by the main Arch repositories. Yaourt can download and install applications from AUR or the main repositories with the same command, treating the AUR asjust another repository”. Pacman unfortunately does not support this.

Again, the installation is covered in the wiki. I recommend the easy route mentioned in the wiki if you’re new at Arch. Its too much too soon to do it the hard way (also mentioned in the wiki entry).

When done, update your system by issuing the single command:

yaourt -Syu

OR

pacman -Syu

and follow the given recommendations.

Interŝanĝado
Saturday, March 28th, 2009 | Author:

Its amazing how much you can do by combining the small yet powerful commands Unix has available.

This little-used command, time, finally became useful today as a way to report the length of time that certain automated operations are running. In my example, I’m timing how long it takes to build the Linux kernel:

$ time rebuild-kernel26
...
couple-thousand-lines-of-scrolling-text
...
==> Finished making: kernel26 2.6.28.8-1 x86_64 (Sat Mar 28 17:19:52 SAST 2009)
real    62m21.994s
user    43m31.846s
sys     6m1.096s

Yup, that took a little over an hour to build. The values are:

  • realthe actual time elapsed while the command was running – 62 minutes
  • userthe amount of userland time the command used – 43 minutes
  • systhe amount of system time the command used – 6 minutes

(I was busy doing other things while this was happening which is why it took 62 minutes for the desktop to do (43+6=) 49 minutes-worth of work)

If you’re using the GNU version (most likely), it also gives you the option of displaying the results in a custom fashion. Mostly, this command could be useful in scripts where you need to report how long a task tookor maybe? just a geeky way to time something random. 😛

Interŝanĝado
Category: linux  | Tags: , , , , , , ,  | Leave a Comment