Thursday, September 17th, 2015 | Author:
  • Part 1 – Introduction – Qhov chaw mus Tej yam yooj yim Queues (Cov ncej no)
  • Part 2 – Nti Nrhiav tsheb – Qhov chaw mus mangle kev cai (Tab tom yuav tuaj TM)
  • Part 3 – Tseem ceeb thiab Txwv – Qhov chaw mus Queue Tsob ntoo (Tab tom yuav tuaj TM)
  • Part 4 – Monitoring Usage – Redefining Queues – Limiting Abusive Devices (Tab tom yuav tuaj TM)
  • Part 5 – ??? Nyiaj ???

Introduction

The first problem one usually comes across after being tasked with improving an Internet connection is that the connection is overutilised. Feem ntau tsis muaj leej twg paub vim li cas, uas, los yog dab tsi yog ua tau rau cov teeb meem – except of course everyone blames the ISP. Tej zaum nws yog ib lub ISP – but typically you can’t prove that without having an alternative connection immediately available. I currently manage or help manage four “qhov chaw / thaj chaw” uas siv QoS los mus tswj lawv Internet connectivity. Ib tug yog kuv chaw ua hauj lwm, two are home connections, and the last one is a slightly variable one – feem ntau yog ib lub tsev kev twb kev txuas tab sis xwb, for a weekend every few months, it becomes a 140-txiv neej (thiab loj hlob) LAN. Kev lom zem. 🙂

MikroTik RouterOS thiab

MikroTik'S RouterOS is very powerful in the right hands. Many other routers support QoS but not with the fine-grain control MikroTik provides. Alternatively you could utilise other Linux-based router OS’s, xws li DD-WRT, Smoothwall, Untangle, thiab thiaj li tawm. Feem ntau ntawm cov feem ntau yuav tsum tau hais tias koj muaj ib tug spare neeg rau zaub mov dag txog los yog ib tug tau tshaj kho vajtse router. Mikrotik muag RouterBoards uas muaj RouterOS builtin – thiab lawv no kuj pheej yig.

Kuv kev nrog routers yog feem nrog Cisco thiab MikroTik – and my experience with QoS is primarily with Faib lub NetEnforcer / NetXplorer tshuab thiab MikroTik. Cov feem nrov MikroTik pab kiag li lawm nyob rau hauv kuv kev (lwm yam tshaj li lawv muaj nplooj siab ntev-ntau wireless pab kiag li lawm) tau lawv rb750 (tshiab version npe “Hex“) thiab rb950-raws li laug cam. Lawv muaj ntau lwm tus neeg available and are relatively inexpensive. In historical comparison with Cisco’s premium devices, Kuv twb tended piav MikroTik tus pab kiag li lawm raws li “90% cov yam ntxwv ntawm 10% tus nqi”. Raws li phau ntawv no yog tswj feem ntau ntawm SME / Tsev siv, inexpensive makes more sense. If you’re looking at getting a MikroTik device, nco ntsoov tias MikroTik routers ua tsis typically include DSL modems, yog li koj uas twb muaj lawm cov khoom yog feem ntau tseem tsim nyog. Nco ntsoov kuj hais tias qhov no yog tsis ib tug nyeem teem tau ib MikroTik ntaus ntawv los ntawm kos. Muaj ntau ntau cov ntawm cov muaj nyob hauv internet rau cov uas twb.

Ziag rau hauv kev xyaum – thawj kauj ruam

Teem caij mus QoS kom, you need to have an idea of a policy that takes into account the following:

  • Lub zuag qhia tag nrho kev twb kev txuas ceev
  • Yuav ua li cas ntau cov neeg siv / pab kiag li lawm yuav tau siv cov kev twb kev txuas
  • The users/devices/services/protocols that should be prioritised for latency and/or throughput

Mus cuag cov saum toj no nyob rau hauv kuv piv txwv, Kuv yuav xav li nram qab no:

  • Lub MikroTik yog teem nrog lub neej ntawd network configuration qhov twg lub zos network yog 192.168.88.0/24 thiab hauv Internet kev twb kev txuas yog muab ntawm PPPoE.
  • Cov kev twb kev txuas ceev yog 10 / 2Mbps (10 Mbps download ceev; 2 Mbps upload ceev)
  • Yuav muaj 5 cov neeg siv uas raws li muaj ntau yam raws li 15 pab kiag li lawm (ntau yam computers / ntsiav tshuaj / txawb tes / WiFi thiab lwm yam)
  • Raug downloads yuav tsum tau plaws nrog ntxiv rau tab sis tsis muaj-qhov tseem ceeb nrog latency
  • Gaming/Skype/Administrative protocols require high priority with both latency and throughput
  • Tsis neeg siv yog cov yuav tsum tau prioritized tshaj lwm leej lwm tus

The first and probably quickest step is to set up what RouterOS refers to as a Tej yam yooj yim Queue.

Kuv twb ua tau ib tug luv luv tsab ntawv uas kuv tau txais kev cawmdim rau kuv MikroTik pab kiag li lawm thiaj teeb tau lub yooj yim queues. Nws yog raws li nram no:

:rau x los ntawm 1 rau 254 ua ={
 /queue yooj yim ntxiv lub npe ="hauv internet-usage- $ x" dst ="pppoe" max-txwv = 1900k / 9500k phiaj ="192.168.88.$x"
}

Yuav ua li cas saum toj no tsis yog txwv lub siab tshaj plaws ceev tej neeg ntaus ntawv yuav siv rau “1900k” (1.9Mb) upload thiab “9500k” (9.5Mb) download.

Sau ntawv:

  • Yog vim li cas vim li cas lub max txwv yog cov nyob 95% of the line’s maximum speed is that this guarantees no single device can fully starve the connection, negatively affecting the other users. With a larger userbase I would enforce this limit further. Piv txwv li, nrog 100 cov neeg siv rau ib tug 20MB kev pab kuv yuav muab qhov no txwv rau 15Mb los yog txawm raws li me ntsis li 1Mb. Qhov no yog nkaus nyob rau yuav ua li cas “tsim” lub cov neeg siv yog thiab, raws li koj xam tawm nyob qhov twg thiab yuav ua li cas npaum li cas tsim txom tshwm sim, you can adjust it appropriately.
  • Lub prefix “hauv internet-pab” nyob rau hauv lub npe parameter yuav yog Mekas. Feem ntau kuv cia li muab cov mus rau xa mus rau lub chaw zov lub npe. Piv txwv li, with premises named “alpha” thiab “beta”, Kuv yuav feem ntau muab tso rau “hauv internet-alpha” thiab “hauv internet-beta”. Qhov no yuav pab nrog instinctively differentiating ntawm qhov chaw.
  • Lub dst parameter muaj “pppoe” nyob rau hauv qhov piv txwv hauv. Qhov no yuav tsum tau pawg nrog lub npe ntawm tus interface that provides the Internet connection.

Xyuas kom koj customize rau hauv tsab ntawv yuav tsum tau tsim nyog rau koj configuration. Txuag lub tsab ntawv mus rau lub MikroTik thiab khiav nws – los yog muab tshuaj txhuam nws ncaj qha mus rau hauv lub MikroTik lub davhlau ya nyob twg rau txim tuag nws.

In my next post I will go over setting up what RouterOS refers to as Mangle kev cai. Cov kev cai pab kom paub tias / faib cov network tsheb nyob rau hauv thiaj li yuav ua finer-grained QoS tau.

Qhia
Category: random  | Leave a Comment
Thursday, September 17th, 2015 | Author:

Tsis pub twg paub, Lub sij hawm, Nyiaj

Kuv tsis nyiam daim debit kev txiav txim. Kuv twb yeej tsis nyiam lub tswv yim uas lwm lub koom haum tau, ntawm yuav, noj yuav luag txhua qhov nyiaj kuv cov nyiaj (zoo … txawm yog muaj). Ib tug colleague taw tes qhia tawm qhov teeb meem nrog MTN yuav tau raug zam tau kuv tau siv ib daim debit thiaj. Tej zaum cov “yooj yim” zoo tshaj yuav yog tsis xws li ib tug phem tshaj plaws.

Kuv kaj lias lub penultimate lo lus nug no yog seb puas los yog tsis koj xav qhov yooj yim thiab yuav tso siab rau tsev (nyob rau hauv cov ntaub ntawv no nrog koj cov nyiaj) – los yog hais tias koj yuav tsis ntseeg lawv thiab txaus siab rau tso uas yooj yim. Nyob rau hauv kuv cov ntaub ntawv, txawm tias kuv tseem nug cov yooj yim, Kuv kawm cov nyuaj txoj kev nrog MTN hais tias nws doubly yuav ua tau yooj yim rau koj kev cob cog rua lub ntiaj teb no txo ​​kom “tej thaj chaw deb kob” raws li txoj cai. Yuav luag txhua leej txhua tus niaj hnub no mus nrog tus kab ntsab zoo tshaj.

Kab ntsab

Nyob rau lwm cov tes, tam sim no ib tug ntev lub sij hawm dhau los, I had a dispute with Ntiaj Chaw qoj qhov twg yooj yim yog ib tug muab ob npaug rau-edged rab ntaj. Kuv qhia lawv ua lag ua luam xyaum mus rau lub Consumer kev tsis txaus siab Commission (txij li thaum rov ncaav li cov National Consumer Commission) and never got feedback from them. The gist of the issue is that Planet Fitness’s sales agent lied to me and a friend in order to get more commission/money out of my pocket.

Kuv yog ib tug Discovery loj hlob neeg uas muab nyiaj pab ntau, xws li txo nqi rau hwm hom – feem ntau noj qab haus huv uas muaj feem ntawm cov hoob kawm, as Discovery is a Medical Aid/Health Insurance provider. Yuav kom muab tso nws tsuas, Discovery yog txaus. Kev loj hlob kev pab them gym memberships uas ntxiv muaj xws li ntiaj chaw qoj. You still have to pay something, ib tug me me token ntawm no yam, rau Discovery, rau lub gym ua tswv cuab. Tab sis, tom qab tag nrho cov, lawv XAV kuv yuav noj qab nyob zoo, so they don’t mind footing the bulk of the bill. Tab sis, Thaj, qhov no txhais tau ntiaj chaw qoj’ kev muag khoom cov tshuaj mas siv tsis tau lub commission!

Yog li ntawd dab tsi no tshwm sim nyob rau hauv? Cov tshwm sim yog tias PF tus muag khoom neeg sawv cev muab kuv ib daim duab rau ib tug inflated “Kev loj hlob raws li” kev ua tswv cuab. Nws dag. Nws ces tau kuv kos npe rau hauv lub dotted line rau ib tug inflated nqi ntawm ib tug “cov” kev ua tswv cuab (yog, nws twb ua tau ntau tshaj li txawm ib txwm ua tswv cuab yuav muaj nqi), ending up about 4 thiab 5 lub sij hawm ntau npaum li lub loj hlob raws li cov tswv cuab.

Epiphanies

Some time in 2011 Kuv thaum kawg wisened mus txog rau lub nqi kuv yuav tsum tau them. Discovery Kuv paub hais tias yuav tsis muaj zoo siab heev txog qhov no fiasco. Kuv hais rau tus thawj tswj ntawm lub gym, thiab kuv twb paub hais tias tus tag nrho daim ntawv cog lus yuav tsum scrapped. Kuv tsis yog ib tug ua nruj ua tsiv rau … tshwj tsis yog tias nws rau kev ua si nawv … nyob rau hauv ib tug Octagon … tab sis tom qab kuv mus ntsib mus rau sis thib 5 ntawd lub Manager nug yog vim li cas lub Debit Orders tseem tshwm sim, nws hais rau kuv nws twb xav tsis thoob Kuv tsis tau coj riam phom nrog kuv rau lub mus ntsib. Tom qab ib tug ob peb ntau mus ntsib, lub Manager twb tau ncaim ntiaj chaw qoj thiab piav rau kuv hais tias tus “daim ntawv cog lus” was between myself and Head Office and that the local gym, Thaj ib tug franchise-style lag luam, tau tsawg los tsis muaj hais txog seb puas los yog tsis yog nws yuav tsum tau muab tso tseg. Yog hais tias lub taub hau ua hauj lwm said tsis muaj, tawv koob hmoov.

By this point I’d lost it. I had my bank put a nres mus rau lub debit kev txiav txim. It was a huge schlep: I had to contact the bank every month because the debit order descriptions would change ever so slightly. It also cost me a little every couple of months to “reinstate” the blocking kev pab cuam. Kuv yuav pab tsis tau tab sis xav tias lub ntsuam xyuas nyiaj system txhawb cov kab zauv tab sis cov neeg ua hauj tsis tas paub yuav ua li cas siv nws.

Technically I’m still waiting on the CCC to get back to me (tsis tau tshwm sim – thiab ntawm hoob kawm lawv twb rov ncaav raws li hais saum toj no ces tus cov ntaub ntawv tej zaum poob los ntawm cov kab nrib pleb). Ntawm cov hoob kawm, los ntawm cov uas taw tes PF kuj xav blacklist kuv tsis them!

Lub Npaj txhij Hero

Ib tug haphazard mention ntawm qhov teeb meem rau Discovery (Kuv xav tias kuv muab lawv hu ua txog ib tug kws kho hniav mus xyuas) tau nyob rau hauv ib tug callback los ntawm ib tug ntawm Discovery lub tshuaj mas siv. Lawv ces nug hais tias kuv mus piav qhia txog qhov teeb meem, nyob rau hauv kev nthuav dav thiab nyob rau hauv kev sau ntawv, kom zoo dua piav qhia los ntawm kuv foundations dab tsi tau tshwm sim tiag tiag. Kuv yuam ua hauj lwm. Nws puv tawm kuv yog txoj cai hais txog lawv tsis ua “zoo siab heev” txog nws. Nyob rau hauv qhov tseeb lawv tiag tiag tsis nyiam nws. About three weeks later, Ntiaj Chaw qoj refunded kuv nyob rau hauv PUV rau tag nrho cov nyiaj uas tau puas tau them nyiaj rau lawv.

Discovery yog Txaus. 🙂

Qhia
Sunday, August 04th, 2013 | Author:

I had a power outage affect my server’s large md RAID array. Rather than let the server as a whole be down while waiting for it to complete an fsck, I had it boot without the large array so I could run the fsck manually.

However, when running it manually I realised I had no way of knowing how far it was and how long it would take to complete. This is especially problematic with such a large array. With a little searching I found the tip of adding the -C parameter when calling fsck. I couldn’t find this in the documentation however: fsckhelp showed no such option.

The option turns out to be ext4-specific, and thus shows a perfectly functional progress bar with a percentage indicator. To find the information, instead offsckhelporman fsck”, you have to inputfsck.ext4helporman fsck.ext4”. 🙂

Qhia
Sunday, August 04th, 2013 | Author:

History

Much had changed since I last mentioned my personal serverit has grown by leaps and bounds (it now has a 7TB md RAID6) and it had recently been rebuilt with Ubuntu Server.

Arch was never a mistake. Arch Linux had already taught me so much about Linux (and will continue to do so on my other desktop). But Arch definitely requires more time and attention than I would like to spend on a server. Ideally I’d prefer to be able to forget about the server for a while until a reminder email saysumthere’s a couple updates you should look at, buddy.

Space isn’t freeand neither is space

The opportunity to migrate to Ubuntu was the fact that I had run out of SATA ports, the ports required to connect hard drives to the rest of the computerthat 7TB RAID array uses a lot of ports! I had even given away my very old 200GB hard disk as it took up one of those ports. I also warned the recipient that the disk’s SMART monitoring indicated it was unreliable. As a temporary workaround to the lack of SATA ports, I had even migrated the server’s OS to a set of four USB sticks in an md RAID1. Crazy. I know. I wasn’t too happy about the speed. I decided to go out and buy a new reliable hard drive and a SATA expansion card to go with it.

The server’s primary Arch partition was using about 7GB of disk. A big chunk of that was a swap file, cached data and otherwise miscellaneous or unnecessary files. Overall the actual size of the OS, including the /home folder, was only about 2GB. This prompted me to look into a super-fast SSD drive, thinking perhaps a smaller one might not be so expensive. It turned out that the cheapest non-SSD drive I could find actually cost more than one of these relatively small SSDs. Yay for me. 🙂

Choice? Woah?!

In choosing the OS, I’d already decided it wouldn’t be Arch. Out of all the other popular distributions, I’m most familiar with Ubuntu and CentOS. Fedora was also a possibilitybut I hadn’t seriously yet considered it for a server. Ubuntu won the round.

The next decision I had to make didn’t occur to me until Ubiquity (Ubuntu’s installation wizard) asked it of me: How to set up the partitions.

I was new to using SSDs in LinuxI’m well aware of the pitfalls of not using them correctly, mostly due to their risk of poor longevity if misused.

I didn’t want to use a dedicated swap partition. I plan on upgrading the server’s motherboard/CPU/memory not too far in the future. Based on that I decided I will put swap into a swap file on the existing md RAID. The swap won’t be particularly fast but its only purpose will be for that rare occasion when something’s gone wrong and the memory isn’t available.

This then left me to give the root path the full 60GB out of an Intel 330 SSD. I considered separating /home but it just seemed a little pointless, given how little was used in the past. I first set up the partition with LVMsomething I’ve recently been doing whenever I set up a Linux box (tiag tiag, there’s no excuse not to use LVM). When it got to the part where I would configure the filesystem, I clicked the drop-down and instinctively selected ext4. Then I noticed btrfs in the same list. Hang on!!

But a what?

Btrfs (“butter-eff-ess”, “better-eff-ess”, “bee-tree-eff-ess”, or whatever you fancy on the day) is a relatively new filesystem developed in order to bring Linuxfilesystem capabilities back on track with current filesystem tech. The existing King-of-the-Hill filesystem, “ext” (the current version called ext4) is pretty goodbut it is limited, stuck in an old paradigm (think of a brand new F22 Raptor vs. an F4 Phantom with a half-jested attempt at an equivalency upgrade) and is unlikely to be able to compete for very long with newer Enterprise filesystems such as Oracle’s ZFS. Btrfs still has a long way to go and is still considered experimental (depending on who you ask and what features you need). Many consider it to be stable for basic usebut nobody is going to make any guarantees. And, ntawm cov hoob kawm, everyone is saying to make and test backups!

Mooooooo

The most fundamental difference between ext and btrfs is that btrfs is aCoWorCopy on Writefilesystem. This means that data is never actually deliberately overwritten by the filesystem’s internals. If you write a change to a file, btrfs will write your changes to a new location on physical media and will update the internal pointers to refer to the new location. Btrfs goes a step further in that those internal pointers (referred to as metadata) are also CoW. Older versions of ext would have simply overwritten the data. Ext4 would use a Journal to ensure that corruption won’t occur should the AC plug be yanked out at the most inopportune moment. The journal results in a similar number of steps required to update data. With an SSD, the underlying hardware operates a similar CoW process no matter what filesystem you’re using. This is because SSD drives cannot actually overwrite datathey have to copy the data (with your changes) to a new location and then erase the old block entirely. An optimisation in this area is that an SSD might not even erase the old block but rather simply make a note to erase the block at a later time when things aren’t so busy. The end result is that SSD drives fit very well with a CoW filesystem and don’t perform as well with non-CoW filesystems.

To make matters interesting, CoW in the filesystem easily goes hand in hand with a feature called deduplication. This allows two (or more) identical blocks of data to be stored using only a single copy, saving space. With CoW, if a deduplicated file is modified, the separate twin won’t be affected as the modified file’s data will have been written to a different physical block.

CoW in turn makes snapshotting relatively easy to implement. When a snapshot is made the system merely records the new snapshot as being a duplication of all data and metadata within the volume. With CoW, when changes are made, the snapshot’s data stays intact, and a consistent view of the filesystem’s status at the time the snapshot was made can be maintained.

A new friend

With the above in mind, especially as Ubuntu has made btrfs available as an install-time option, I figured it would be a good time to dive into btrfs and explore a little. 🙂

Part 2 coming soon

Qhia
Monday, October 29th, 2012 | Author:

It appears that, in infinite wisdom, Google have a security feature that can block an application from accessing or using your google account. I can see how this might be a problem for Google’s users, in particular their GTalk thiab Gmail users. In my case it was Pidgin having an issue with the Jabber kev pab cuam (which is technically part of GTalk). I found the solution after a little digging. I was surprised at how old the issue was and how long this feature has existed!

To unlock the account and get your application online, use Google’s Captcha page here.

Qhia