First, I have 6 2GB USB thumb drives for this test. Unfortunately, 2 of them are slightly smaller than the other 4. As such, I felt that LVM would be a good solution for making sure each filesystem was put on the exact same storage container.
The result? 6 logical volumes exactly the same size, each with 486 PEs with 4MB per PE. Each filesystem was mounted to it’s own directory under /mnt:
aaron@kratos:~ 4149 % df -h /dev/mapper/test-* Filesystem Size Used Avail Use% Mounted on /dev/mapper/test-ext2 1.9G 2.9M 1.8G 1% /mnt/ext2 /dev/mapper/test-ext3 1.9G 35M 1.8G 2% /mnt/ext3 /dev/mapper/test-jfs 1.9G 376K 1.9G 1% /mnt/jfs /dev/mapper/test-reiser 1.9G 33M 1.9G 2% /mnt/reiser /dev/mapper/test-vfat 1.9G 4.0K 1.9G 1% /mnt/vfat /dev/mapper/test-xfs 1.9G 288K 1.9G 1% /mnt/xfs
Next, I needed to populate these filesystems with some data. I ran the following for-loop:
for i in ext2 ext3 vfat xfs jfs reiser; do dd if=/dev/zero of=/mnt/$i/foo.img bs=1024 count=500000 done
Let’s see how they fared:
aaron@kratos:~ 4166 % df -h /dev/mapper/test-* Filesystem Size Used Avail Use% Mounted on /dev/mapper/test-ext2 1.9G 492M 1.3G 28% /mnt/ext2 /dev/mapper/test-ext3 1.9G 524M 1.3G 29% /mnt/ext3 /dev/mapper/test-jfs 1.9G 489M 1.5G 26% /mnt/jfs /dev/mapper/test-reiser 1.9G 521M 1.4G 27% /mnt/reiser /dev/mapper/test-vfat 1.9G 489M 1.5G 26% /mnt/vfat /dev/mapper/test-xfs 1.9G 489M 1.5G 26% /mnt/xfs
VFAT, XFS and JFS all seem to do fairly well on data conservation. Knowing that the FAT filesystem isn’t very robust, or feature-packed, looking at just this data, I would be willing to spend some time further with JFS and XFS. However, to be fair, I’ll give FAT a good look in respect to features.
It is a pity however, that Sun Microsystem’s ZFS is licensed under the CDDL. I would rather enjoy working with that filesystem, I think, as it supports a great set of features. Unfortunately, unless ZFS is ported to the GPL, it’s unlikely that we’ll see it in kernel space, and I’m not really interested in an implementation of it under FUSE.
]]>What I noticed last night, during the install really surprised me. ReiserFS is a screaming filesystem compared to XFS. After doing the install 3 times on 2 identical machines (Dell Precision 490), in every case, ReiserFS was nearly twice as fast completing the install. Didn’t matter how the disk was partitioned either. It is a screaming speed machine. Further, launching applications, and playing with KVM, it was also noticeably faster than it’s XFS competition.
However, XFS did have much better storage management in the way of sqeezing out every last byte on the drive. In fact, on bare bones installs, XFS gave me 10% more storage space than ReiserFS.
Conclusion? If you’re after speed, from what I gathered last night, ReiserFS screams. If you’re looking for maximized disk space, XFS is the clear winner there. Was the disk space gain worth the wait of the install? Yeah, probably. But man, post-install, launching applications with ReiserFS was still noticeably faster than XFS.
I’m curious about these, and will be benchmarking some more in the future. Expect follow-up posts on this topic.
]]> -f or –force
Do what I say, even if it is stupid.
I don’t know why I found that funny, but I thought to myself that more man pages need to be written in this fashion. It sure does brighten up the whole system-administration thing.
]]>One of the issues that has plagued me, but I haven’t bothered to do anything about it until this morning, is networks dropping my TCP connections if there is no activity after a given interval. Currently, I’m in Mountain View, California teaching a Linux course, and the training center network is one such network with dropping inactive TCP connections after 60 seconds. Annoyed (being a heavy SSH user), I began digging in the SSH man page on my machine, and found a way to keep my connection alive.
There are two options for addressing my need: TCPKeepAlive and ServerAliveInterval. Each of those are explained here:
ssh -o TCPKeepAlive=yes user@some.host.com
ssh -o ServerAliveInterval=30 user@some.host.com
If ServerAliveInterval is used in the SSH command, then TCPKeepAlive is not needed, and should be turned off.
Now, in the training centers I visit, giving this option will ensure that my SSH connection stays connected, so I can stay on top of my IRC and MUC. ![]()
I’m running screen locally on my laptop (we’ll refer to it as SCREEN_L), and remotely on my server (we’ll refer to this one as SCREEN_R). For SCREEN_L, there is a ~/.screenrc that I use to show what screen window buffer I’m using via the “hardstatus” directive. My ~/.screenrc is thus:
aaron@kratos:~ 2609 % cat .screenrc
screen -t python 0
screen -t irssi 1
screen -t shell 2
screen -t notify 3
hardstatus alwayslastline
hardstatus string '%{= kG}[ %{G}%H %{g}][%= %{= kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B} %d/%m %{W}%c %{g}]‘
That puts a nice status line at the bottom of my terminal, letting me know what screen buffer I’m in, as well as it’s title. However, I don’t want to run a ~/.screenrc under SCREEN_R. This just means yet another hardstatus line that I really don’t want (I value my pixels). Rather, I would like to be notified what screen window I’m under, if any, in my prompt. After a bit of digging, I came up with the solution.
First, the environment variable $WINDOW is keeping track of this automatically for me. For example:
aaron@kratos:~ 2610 % echo "\"$WINDOW\"" "" aaron@kratos:~ 2611 % screen aaron@kratos:~ 2612 % echo "\"$WINDOW\"" "0"
Perfect! Now, to get it into my prompt. This is easy with adding a little if logic to our ~/.zshrc:
if [ x"$WINDOW" = x ]; then SCREEN="($WINDOW)" else SCREEN="" fi PS1="%n@%m$SCREEN:%~%(?..[%?]) %h %# "
What am I doing here? Well, first off I’m checking to see if the $WINDOW variable is blank. I’m using x”$WINDOW” != x for a simple reason. Zsh is expecting a valid condition after ‘!=’. Unfortunately, an empty string isn’t satisfying it. So, I prepend the letter x to the $WINDOW variable, and test to make sure it doesn’t equal just ‘x’. Second, if $WINDOW equals anything other than ‘x’, a SCREEN variable is defined with the $WINDOW variable contents wrapped in parenthesis. This $SCREEN variable is then added to the prompt.
Now, I add this to my ~/.zshrc for my SCREEN_R, and I can see where I’m at in my screen session remotely without the need for a remote ~/.screenrc. Here’s the results, also showing off exit code:
aaron@kratos:~ 2614 % screen aaron@kratos(0):~ 2615 % ping foo ping: unknown host foo aaron@kratos(0):~[2] 2616 %
Cheers!
]]>As the collection grew, this grew more and more tedious, until I discovered a simple solution to the problem. Rather than keep the movies back-to-back from A to Z, throw in spaces at periodic points through the alphabet to account for growth. In other words, put some space between sections “F” and “G” to allow for 5 DVDs growth in A-F. Put more space between sections “L and “M” to allow growth for 5 DVDs in G-L. Follow this through the remainder of the alphabet, and sanity has once again entered your DVD collection.
Nothing is more annoying than automatic, especially when automatic is random. I don’t like DHCP, as such, all my IP addresses at home are statically assigned. I don’t like automatic transmissions, rather, I prefer to have granular control over my vehicle. Further, I don’t like the automatic window numbering of Irssi. I would much prefer to have all my windows stay in static locations, regardless of whether or not a join or leave a channel. Just because I close a window, doesn’t mean I want all of my windows there after to be renumbered. I’ve been an Irssi user now for nearly 2 years, and I had no idea that static windows existed until today.
So, with this news, suppose you are in 25 channels. You would like to “categorize” your windows in such a manner that is consistent with the existing channels. For example, maybe you are in 6 ubuntu channels, 5 generic support channels, 7 local channels (LUGs, user groups, etc), and 7 private messages. Further, let’s say you have the categorized together as follows:
1: status 2: &bitlee 3-8: ubuntu channels 9-13: generic support channels 14-20: local channels 21-27: private messages
That’s a fairly clean irssi window list. However, what happens if you want to join a new ubuntu channel? Then, by default, it becomes the last window. So, in our example, it would be window 28. Well, now our list is fragmented. I want all the ubuntu channels together. So, I add it to the Ubuntu window group. However, this has now pushed all of my support channels down by one: My new window list would be as follows:
1: status 2: &bitlee 3-9: ubuntu channels 10-14: generic support channels 15-21: local channels 22-28: private messages
If #irssi was at location number 13, it has now been moved to window location 14. This can be annoying when trying to remember what channel number belongs to what channel, especially if I visit the channel often.
Further, what happens when I leave a channel, or more? All the windows are shifted accordingly, getting renumbered yet again. The new window scheme could be as follows:
1: status 2: &bitlee 3-9: ubuntu channels 10-13: generic support channels 14-19: local channels 20-26: private messages
Talk about insanity! Wouldn’t it be nice if I could have space between groups to allow for growth and shrinkage, without affecting the placement of the rest of the windows, just like my DVDs? Wouldn’t it be nice if I could have my windows organized as follows:
1: status 2: &bitlee 3-9: ubuntu channels 21-25: generic support channels 31-37: local channels 41-47: private messages
In other words, there are now “gaps” between my windows, which allows for growth, without affecting the other windows. Further, if I leave a channel, I don’t want it affecting the others. So, the question remains now, how do we set this up? Luckily, I just need to change an Irssi setting, then I can start my window numbering. The first thing I need to do is turn off automatic window renumbering. I can run the following command from within Irssi:
/set windows_auto_renumber off
With that “feature” turned off, I can now focus on sending my windows to their appropriate spots. If ##jabber was in window 10, but I want to send it to window 21, I can issue the following command (assuming I’m already at the ##jabber window):
/window number 21
With “/window number”, I can send any window to any number in my session, thus giving me the ability to create “gaps” between my windows allowing for growth. Further, now when I close a window or leave a channel, it won’t affect the windows following it by renumbering them.
Don’t forget to save your changes to the config file:
/save
Finally, window categorization sanity! That made my day! Further, if you take advantage of saving your layout, then, when joining previously “remembered” channels, it will go in it’s previous position. This can be done with:
/layout save
Nice.
]]>Anyway, I’ve gotten off track. What I wanted to point out, is my wife is a blogger. I help set her blog up about 2 years ago, as she wanted to add it to her domain. It took a bit for her to really catch the blogging bug, but now she’s non-stop. She’s even edited the presentation HTML for her WordPress theme… without knowing HTML. Further, she gets her students involved by adding comments to her posts. She’s out setting the bar on teachers blogging, and joining the ranks of female bloggers everywhere. I think she’s worth subscribing to (she is my wife, after all.
) by adding her feed to your RSS reader.
Keri- I love you, and I’m so very proud of you!! Keep up the amazing work!
]]>Before I begin, however, I want to make something painfully clear. Linux gives you the awesome power to completely break your box. I’m talking to the point where not even a backup would work, and you’re left with the task of reinstalling the operating system. However, on the flip side, Linux gives you the awesome power to manage and administer your box. Intense flexibility and unbelievable portability. Today, if you’re following along with the examples given in this tutorial, it is possible to leave your box in an unbootable state, or in a state of undesired operation. Of course, this is not what we want. So, if you are going to follow along, I would recommend following along in an Ubuntu virtual machine. Otherwise, take extra care to follow the tutorial exactly, and all should be well.
Ok, in my last post, I talked about what a runlevel is and what it does to your box. However, I never really mentioned why you should care, or how to to manage a specific runlevel. I plan on discussing those items here. First, is why you should care about runlevels. I’ve seen the argument that runlevels add nothing to your box, and managing services is trivial, so runlevels just add an extra layer of complexity. I disagree. For me, runlevels add a layer of dependability. The whole idea of runlevels is to maintain a level of predictability with your box. I should be able, at any given time during the day, predict what is running on my box. If not, then I’ve given up a level of security. By not knowing how my box is performing, or what it is executing, I’ve opened a door for potential vulnerability. However, if I can predict the state of my box, regardless of time of day, stress it’s under or number of users logged in, then I’ve maintained a level of security and dependability. As such, I’ve done my job well as a system administrator. Runlevels give me that predictability, as well as flexibility in the state I want my box operating under.
As a review, let’s look at the runlevels in Ubuntu:
0: Halt / Shut down. All processes are stopped, all filesystems are unmounted, and all users taken offline. Power can now be safely removed. 1: Single user mode. Only root allowed login (requires password). All filesystems listed in /etc/fstab are mounted. Limited set of services started. 2-5: Multi-user mode. All filesystems listed in /etc/fstab mounted, graphical environment (if installed) brought up. Full standard operational mode. 6: Reboot. As with runlevel 0, all processes stopped, all filesystems unmounted, all users taken offline, the box rebooted, and taken to the default runlevel.
If you did your homework in the last post, you probably changed your default runlevel in your /etc/event.d/rc-default file. If you changed this to runlevel 6, you probably noticed a problem: your box infinitely rebooted. Unfortunately, I did not give you a way out in the last post. I’ll do that now.
When booting the box, you can bypass the defined default runlevel by passing ’s’ or ‘S’ as a kernel argument in GRUB. When your box is booting, press the [Esc] key before the timeout ends, and you’ll be taken to the GRUB menu. You’ll have the ability to use your arrow keys to highlight an entry. From there, there is a prompt at the bottom of the menu telling you what you can do to that entry that is highlighted. For example, you can press the ‘e’ key to edit that line. Go ahead, and do that to one of the highlighted entries. You’ll notice that it takes you to a new menu, showing an initrd (initial ramdisk, used by the kernel to load some necessary drivers for the system), the kernel itself, and a memory test for your RAM. Select the kernel entry, and hit the ‘e’ key again to edit the kernel. At the end of the line, enter ’s’ or ‘S’, and press enter. Press the ‘b’ key to boot the kernel now, and you will boot into an sulogin prompt. If you enabled the root account, which is disabled by default on an Ubuntu box, you will be prompted for the root password, WHICH IS IN PLAIN TEXT ECHOED TO THE SCREEN (it’s a bug, we’re working on it). If the root account is not enabled, then you’ll be thrown to a root prompt, at which you can edit your /etc/event.d/rc-default file to change the default runlevel to the appropriate entry.
Woah. Wait a minute. You mean, all I have to do is enter an ’s’ at the kernel line in GRUB, and I get root access to the machine?! Yes. Any box that is physically available can be easily compromised, even with harddrive disk encryption. So, considering the physical security of your box is something that you’ll obviously want to take into account as a system administrator. However, we’re off topic a bit, so let’s get back on target.
Runlevels are nothing more than your box running in a specific state. Fortunately, we can change the desired state that we want our box to be running in by adjusting the runlevel. This is done with the update-rc.d command.
At this point, let’s play with a package for this tutorial. I’ve chosen sendmail as our culprit, so if you don’t already have it installed, run the following command below. If you do have sendmail installed, and would rather use another service package, go for it. We’ll use sendmail in this post:
aaron@kratos:~ 1354 % sudo aptitude -R install sendmail Password:
In my last post, sendmail was one of the services on my box that was starting when I enter runlevel 2. Maybe I don’t want sendmail to start, or rather, if it is already started, when I enter runlevel 2, I want to kill it. In other words, I don’t want it running for runlevel 2. How can I make this change?
Well, first, I could just delete the soft link from the runlevel directory /etc/rc2.d/:
aaron@kratos:~ 1355 % sudo rm /etc/rc2.d/S21sendmail Password:
That would definitely keep it from starting when I enter runlevel 2, but what if I wanted to kill it if it was already started from a previous runlevel? Just deleting the soft link won’t do it. I need to turn it into a K-script. Further, deleting and creating soft links in my /etc/rc[0-6].d/ directories by hand is a bit of a pain. This is where the update-rc.d command comes in:
aaron@kratos:~ 1356 % sudo update-rc.d -f sendmail remove Password: Removing any system startup links for /etc/init.d/sendmail ... /etc/rc0.d/K19sendmail /etc/rc1.d/K19sendmail /etc/rc2.d/S21sendmail /etc/rc3.d/S21sendmail /etc/rc4.d/S21sendmail /etc/rc5.d/S21sendmail /etc/rc6.d/K19sendmail aaron@kratos:~ 1357 % sudo update-rc.d sendmail stop 20 0 1 2 3 4 5 6 . Adding system startup for /etc/init.d/sendmail ... /etc/rc0.d/K20sendmail -> ../init.d/sendmail /etc/rc1.d/K20sendmail -> ../init.d/sendmail /etc/rc2.d/K20sendmail -> ../init.d/sendmail /etc/rc3.d/K20sendmail -> ../init.d/sendmail /etc/rc4.d/K20sendmail -> ../init.d/sendmail /etc/rc5.d/K20sendmail -> ../init.d/sendmail /etc/rc6.d/K20sendmail -> ../init.d/sendmail
We’ll get into the syntax a bit later, but these commands, as observed, removed any existing soft links that previously existed, and created new K-scripts for all of my runlevels.
First off, why two separate commands? Well, by Debian policy, no package upgrade will ever overwrite a previous configuration. This includes updating soft links in the runlevel directories. This also ensures persistent changes and allows the system administrator to prevent daemons from launching. So, if soft links already exist, they first need to be removed, then new links created.
Second, you’ll notice that I need to specifically state when I want the service to start and when I want it to stop during the runlevel process. This is intentional, and offers a great deal of flexibility to the system administrator. Remember, that all the soft links either start with an S for starting the service, or a K for stopping (killing) the service. Each soft link is processed in alphanumeric order, so there is a number following the K or S telling the order in which I want the scripts to execute.
There is something to be said about logic behind the order of when you are starting and stopping services. For example, you may want to restore your firewall before enabling your network device. Further, you may want your network brought up before you begin starting services. Ultimately, the choice is up to you.
Let’s continue with managing our runlevels, and learn the syntax of the update-rc.d command. There are four possible ways in which I can use the update-rc.d command. Let’s start with the first- defaults. As our symlinks already exist, we’ll need to remove them:
aaron@kratos:~ 1358 % sudo update-rc.d -f sendmail remove Password: [... snip ...] aaron@kratos:~ 1359 % sudo update-rc.d sendmail defaults [... snip ...]
(I’ll be snipping some output from here on out). As we can see, the defaults for sendmail were to stop it at position 20 for runlevels 0, 1 and 6, and to start it at position 20 for runlevels 2 through 5. Actually, position 20 is the default for any package, regardless. This means apache, postfix, squid, ssh, etc. This may not be what we want. Maybe I want to affect sendmail at a different position.
The 2nd possible syntax of the update-rc.d command to to change that default position to another number for all runlevels. Remember to remove our soft links first, then create new ones:
aaron@kratos:~ 1360 % sudo update-rc.d -f sendmail remove Password: [... snip ...] aaron@kratos:~ 1361 % sudo update-rc.d sendmail defaults 10 [... snip ...]
As we can see, I was able to change the position of stopping and starting sendmail from 20 to 10. This affected all runlevels 0 through 6. However, I may not want to affect all runlevels the same. Maybe I want to treat runlevels 0, 1 and 6 differently than 2 through 5.
Which brings us to our 3rd possible syntax for the update-rc.d command- using the default runlevels, but changing when it’s stopped and started. Again, let’s remove our symlinks, and create new ones:
aaron@kratos:~ 1362 % sudo update-rc.d -f sendmail remove Password: [... snip ...] aaron@kratos:~ 1363 % sudo update-rc.d sendmail defaults 10 21 [... snip ...]
As we can see, the default runlevels 0, 1 and 6 were changed to stop sendmail at position 21, and the default runlevels 2 through 5 were changed to start sendmail at position 10. So, I can change the starting and stopping position using the default runlevels. Cool.
Finally, let’s look at our last usage of the update-rc.d command: full flexibility of when I start and stop the service in each individual runlevel. I bet you haven’t forgotten to remove your soft links before creating new ones, have you:
aaron@kratos:~ 1364 % sudo update-rc.d -f sendmail remove Password: [... snip ...] aaron@kratos:~ 1365 % sudo update-rc.d sendmail start 10 2 3 . start 20 4 5 . stop 19 0 1 6 . [... snip ...]
Here, as we can see, I affected the starting position for sendmail in runlevels 2 and 3 to be 10. For runlevels 4 and 5, I wanted sendmail to be at position 20. Finally, for runlevels 0, 1 and 6, I wanted to kill sendmail at position 19. You’ll also notice that the syntax is fairly straight-forward:
update-rc.d [service] start|stop [position] [runlevel] [runlevel] [runlevel] … [.]
The first argument, as has been consistent up to this point, is the service. Then, I mention whether or not I want to create K-scripts or S-scripts by calling either the start or stop option. Both start and stop have a set of arguments, the first being the position I’m going to affect. Then, following the position, I give a space-separated list of runlevels followed by a ‘.’ (dot) to terminate either that start or stop call.
Let’s look at a couple more examples of this 4th syntax, so we can get a better feel for it:
aaron@kratos:~ 1366 % sudo update-rc.d -f sendmail remove Password: [... snip ...] aaron@kratos:~ 1367 % sudo update-rc.d sendmail start 45 2 3 4 5 . stop 1 0 1 6 . [... snip ...]
The above example starts sendmail at position 45 for runlevels 2 through 5 and stops sendmail at position 1 for runlevels 0, 1 and 6. I should probably point out, that you most likely don’t want to start a service for runlevels 0 and 6, for obvious reasons. One more example:
aaron@kratos:~ 1368 % sudo update-rc.d -f sendmail remove Password: [... snip ...] aaron@kratos:~ 1369 % sudo update-rc.d sendmail stop 0 0 1 2 3 4 5 6 . [... snip ...]
In that example, I set sendmail to stop at position 0 for all runlevels, thus effectively disabling the service, regardless of the state of my box. If I wanted sendmail to start, I would need to call it directly:
aaron@kratos:~ 1370 % sudo /etc/init.d/sendmail start Password: * Starting Mail Transport Agent (MTA) sendmail [ OK ]
One final note about managing runlevels. All this is fine and dandy with the update-rc.d command, however, we’re just removing and creating soft links. We are not affecting the running process at all. In other words, just because I’ve configured sendmail to start in runlevels 2-5 doesn’t mean that I’ve started the service. If sendmail is not running, I will still have to start it by hand. Same goes for creating K-scripts in those directories. The service won’t be stopped because I have created those K-scripts. I’ll still have to stop it by hand. All we’re doing is affecting if it starts or stops when I enter a new runlevel.
There is some philosophy about when to start and stop services, and definitely some logic behind the order. However, the position at when to start and stop these services is entirely up to you. If you find that you’re starting a service too early (or too late), change the soft links. It may be of interest to you to start your X display manager first, before starting services. Now that you’re familiar with the update-rc.d command, you know how to do this (this approach may have some consequences as some services, such as xfs or gdm may provide dependency to X).
At this point, we’ll stop here with this post in the series, and pick up again tomorrow with more service management. If you followed both of these posts, you should be fairly comfortable with runlevels, and managing when to start and stop services in specific runlevels. You’re turning into a system adminstrator.
]]>First off, to understand services, we need to understand runlevels. Runlevels are a way to automatically start and stop services, thus effectively controlling what is running on your box. When you enter a specific runlevel, you are telling your box that you want to stop a specific set of services and/or start a specific set of services. In other words, think of runlevels as categorizing your system. For example, if I enter runlevel 1, I may want to tell my box to go into “single user mode”, thus effectively knocking everyone off except for the root user. Also, I may want to stop all services, including networking, taking my box off the network for troubleshooting reasons. Thus, stop Apache, Squid, SSH, and so on, leaving my box in maintenance mode.
On all Linux and Unix systems, runlevels are treated differently. For example, runlevels in Red Hat have different meaning than runlevels in OpenBSD which have a different meaning than Solaris which finally have a different meaning for Ubuntu. Of course, in this post, I’m only gong to worry about Ubuntu runlevels. If you’re curious about the runlevels on other systems, there is a great page here dscribing the differences.
So, let’s look at the runlevels that affect Ubuntu. They are as follows:
0: Halt / Shut down. All processes are stopped, all filesystems are unmounted, and all users taken offline. Power can now be safely removed. 1: Single user mode. Only root allowed login (requires password). All filesystems listed in /etc/fstab are mounted. Limited set of services started. 2-5: Multi-user mode. All filesystems listed in /etc/fstab mounted, graphical environment (if installed) brought up. Full standard operational mode. 6: Reboot. As with runlevel 0, all processes stopped, all filesystems unmounted, all users taken offline, the box rebooted, and taken to the default runlevel.
Ubuntu does have one extra special runlevel that in processed before entering any of the regular runlevels above. It’s known as ’sulogin’, and it is considered an emergency mode if the system will not boot, or there are other troubles. it is denoted by “S” as it’s runlevel characterization. We will not worry about runlevel S in this post.
Each Unix and Linux operating system has a default runlevel in which it will boot into. If init is the daemon manager for your system, then you will find an /etc/inittab file that will specify your default runlevel. However, Ubuntu has replaced init with Upstart, an event-driven daemon manager. As such, there is no /etc/inittab file. However, there is an /etc/event.d/rc-default file, which as probably guessed, specifies your default runlevel. Let’s take a look at the file:
aaron@kratos:~ 1346 % cat /etc/event.d/rc-default # rc - runlevel compatibility # # This task guesses what the "default runlevel" should be and starts the # appropriate script. start on stopped rcS script runlevel --reboot || true if grep -q -w -- "-s\|single\|S" /proc/cmdline; then telinit S elif [ -r /etc/inittab ]; then RL="$(sed -n -e "/^id:[0-9]*:initdefault:/{s/^id://;s/:.*//;p}" /etc/inittab || true)" if [ -n "$RL" ]; then telinit $RL else telinit 2 fi else telinit 2 fi end script
Parsing through that file, we see that first, it tests to see if we are running in runlevel S, or ’sulogin’. If not, then we have a series of checks to find our default runlevel. The first check is to see if there is in fact an /etc/inittab file existing. If so, we look for the line that says “id:?:initdefault:”, where “?” could be any valid number 0 through 9. If that number exists, it is our default runlevel, otherwise, runlevel 2 will be our default, as none of the checks would pass.
So, as discovered, runlevel 2 is the default runlevel on Ubuntu. Furthermore, runlevels 2 through 5 are all exactly the same by default. Thus, entering runlevel 4 will have no effect on your box if already running in runlevel 2. What’s the point of having so many if they’re the same then? Well, a few reasons. First, if you followed that link above, showing the different runlevels on different systems, 0 through 6 is not uncommon, and many of the Unices and Linuxes take advantage of every one of them. So, I guess you could say they’re present for historical reasons. However, having that many runlevels gives the system administrator the flexibility to manage a specific runlevel at his/her convenience. For example, maybe runlevel 2 should be treated differently than 3 through 5, starting a different set of services than the rest. Maybe I don’t want runlevel 2 to start the graphical mode, but keep it text-based only. This could be useful in troubleshooting my graphical display without it running. You get the idea.
Ok, fine. The next question inevitably asked, is how can I tell what runlevel I’m currently in? Simple, just run the ‘runlevel’ command:
aaron@kratos:~ 1347 % runlevel N 2
“N” tells me that there was “None” for my previous runlevel. In other words, this box was just booted into my current runlevel, which I can see as the 2nd identifier, “2″. This is one thing that you may like or hate about the ‘runlevel’ command, but it only shows you your current runlevel and the previous runlevel. No more, no less. However, generally, this is sufficient.
As such, the next question you are probably asking yourself, is how do I change runlevels? This is done with the ‘telinit’ command. The command tells Upstart, our services manager, what services to stop and start when we enter a specific runlevel. For example, if I wanted to reboot my box from the command line, knowing that runlevel 6 is a reboot, I could enter, as root:
aaron@kratos:~ 1348 % sudo telinit 6 Password:
Ok. So, now we’ve learned what runlevels are, how to check my current running runlevel, and even how to change to a different runlevel. The next concept in this post is to see what runlevels are doing to the system under the hood. In other words, how does the system know that I want to knock everyone off the box except for root in runlevel 1? How does the system know to start the graphical display in runlevels 2 through 5? The answers to these questions lie in simple shell scripts.
Each runlevel has it’s own directory, which contains what we call “K-scripts” and “S-scripts”. These directories reside in /etc. In fact, let’s get back to our terminal, and type the following, noticing the output:
aaron@kratos:~ 1349 % ls -ld /etc/rc*/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc0.d/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc1.d/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc2.d/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc3.d/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc4.d/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc5.d/ drwxr-xr-x 2 root root 4096 2008-02-16 18:06 /etc/rc6.d/ drwxr-xr-x 2 root root 4096 2007-10-15 17:36 /etc/rcS.d/
That’s interesting. I have directories that have the same number as runlevels. Could this be a coincidence? Hardly. When I enter “telinit 4″ on the command line, for example, I’m telling Upstart to enter /etc/rc4.d, and execute the K-scripts and S-scripts found in that directory in a specific order. Same goes for entering “telinit 2″, “telinit 0″ or “telinit 6″. Every directory having a specific set of shell scripts that it’s going to execute. So, let’s dig into one of these directories. Let’s start with /etc/rc0.d/:
aaron@kratos:~ 1350 % ls /etc/rc0.d/ K01gdm K20postfix S15wpa-ifupdown K01usplash K25hwclock.sh S20sendsigs K16avahi-daemon K50alsa-utils S30urandom K16dhcdbd K59mountoverflowtmp S31umountnfs.sh K18consolekit K99laptop-mode S40umountfs K19sendmail README S60umountroot K20apport S01linux-restricted-modules-common S90halt
Here, I can see only 20 scripts and a README file listed. I notice further that I have my K-scripts and S-scripts that I’ve mentioned earlier. In fact, the K-scripts and S-scripts are numbered. This has to do with order in which to execute those scripts. Let’s look at them in detail. First, Upstart is going to execute the K-scripts then it will execute the S-scripts. K-scripts stop a service, while S-scripts start a service. So, the first script it will execute when it enters this directory is “K01gdm”. This script manages my graphical display. In other words, I’m going to kill my X session before doing anything else. Next, I’ll stop usplash followed my avahi-doemon, etc.
Once all the K-scripts are executed, then I execute the S-scripts. Just as with the K-scripts, they are also executed in order. So, the first script I start is “S01-linux-restricted-modules-common”. This script manages my restricted drivers for the kernel, returning nothing more than the exit code for /sbin/lrm-manager, and logging the result. Looking further, I have “S15wpa-ifupdown” which is managing my WPA helper library for wireless devices. We notice further that the last script executed in this directory is “S90halt”, which shuts the box down fully. Each of those scripts, as mentioned before, are just shell scripts, and can be opened with any text editor. If you’re curious about what a script is doing, open it up and read it.
Let’s look at runlevel 2 this time:
aaron@kratos:~ 1351 % ls /etc/rc2.d README S20hotkey-setup S30gdm S05vbesave S20makedev S50ntp S10acpid S20nvidia-kernel S89anacron S10powernowd.early S20postfix S89atd S10sysklogd S20powernowd S89cron S10xserver-xorg-input-wacom S20rsync S98usplash S11klogd S21sendmail S99acpi-support S12dbus S22consolekit S99laptop-mode S12hal S24avahi-daemon S99rc.local S19cupsys S24dhcdbd S99rmnologin S20apport S25bluetooth S99stop-readahead
First off, we may notice that there are no K-scripts in the runlevel 2 directory. In other words, we aren’t interested in stopping any services when we enter runlevel 2. Rather, we are only interested in starting services. Second, we may notice a logical order to the scripts. For example, we start power management right out the gate with S10acpid and S10powernowd.early. Next, we start S10sysklogd, which is our generic logging daemon for logging services. It makes sense to start this daemon before starting specific services, as if they fail to start, I will have a log as to the reason. Also notice that anacron is starting before cron (S89anacron to S89cron). This makes sense, as we want to run any cron jobs that weren’t run before running them again. The other daemons start in their numerical order as specified.
Looking in that directory is a bit deceptive, however. Although we see our K-scripts and S-scripts in each directory, they are actually not separate scripts. They are one script actually residing in /etc/init.d/. Let’s take a look:
aaron@kratos:~ 1352 % ls -l /etc/rc2.d total 4 -rw-r--r-- 1 root root 556 2007-10-04 05:17 README lrwxrwxrwx 1 root root 17 2008-01-09 00:45 S05vbesave -> ../init.d/vbesave lrwxrwxrwx 1 root root 15 2008-01-09 00:45 S10acpid -> ../init.d/acpid lrwxrwxrwx 1 root root 25 2008-01-09 00:45 S10powernowd.early -> ../init.d/powernowd.early lrwxrwxrwx 1 root root 18 2008-01-09 00:45 S10sysklogd -> ../init.d/sysklogd lrwxrwxrwx 1 root root 34 2008-01-09 00:45 S10xserver-xorg-input-wacom -> ../init.d/xserver-xorg-input-wacom lrwxrwxrwx 1 root root 15 2008-01-09 00:45 S11klogd -> ../init.d/klogd lrwxrwxrwx 1 root root 14 2008-01-09 00:45 S12dbus -> ../init.d/dbus lrwxrwxrwx 1 root root 13 2008-01-09 00:45 S12hal -> ../init.d/hal lrwxrwxrwx 1 root root 16 2008-01-09 00:45 S19cupsys -> ../init.d/cupsys lrwxrwxrwx 1 root root 16 2008-01-09 00:45 S20apport -> ../init.d/apport lrwxrwxrwx 1 root root 22 2008-01-09 00:45 S20hotkey-setup -> ../init.d/hotkey-setup lrwxrwxrwx 1 root root 17 2008-01-09 00:45 S20makedev -> ../init.d/makedev lrwxrwxrwx 1 root root 23 2008-01-09 00:45 S20nvidia-kernel -> ../init.d/nvidia-kernel lrwxrwxrwx 1 root root 17 2008-02-16 15:44 S20postfix -> ../init.d/postfix lrwxrwxrwx 1 root root 19 2008-01-09 00:45 S20powernowd -> ../init.d/powernowd lrwxrwxrwx 1 root root 15 2008-01-09 00:45 S20rsync -> ../init.d/rsync lrwxrwxrwx 1 root root 18 2008-02-16 18:06 S21sendmail -> ../init.d/sendmail lrwxrwxrwx 1 root root 20 2008-01-09 00:45 S22consolekit -> ../init.d/consolekit lrwxrwxrwx 1 root root 22 2008-01-09 00:45 S24avahi-daemon -> ../init.d/avahi-daemon lrwxrwxrwx 1 root root 16 2008-01-09 00:45 S24dhcdbd -> ../init.d/dhcdbd lrwxrwxrwx 1 root root 19 2008-01-09 00:45 S25bluetooth -> ../init.d/bluetooth lrwxrwxrwx 1 root root 13 2008-01-09 00:45 S30gdm -> ../init.d/gdm lrwxrwxrwx 1 root root 13 2008-01-12 13:41 S50ntp -> ../init.d/ntp lrwxrwxrwx 1 root root 17 2008-01-09 00:45 S89anacron -> ../init.d/anacron lrwxrwxrwx 1 root root 13 2008-01-09 00:45 S89atd -> ../init.d/atd lrwxrwxrwx 1 root root 14 2008-01-09 00:45 S89cron -> ../init.d/cron lrwxrwxrwx 1 root root 17 2008-01-09 00:45 S98usplash -> ../init.d/usplash lrwxrwxrwx 1 root root 22 2008-01-09 00:45 S99acpi-support -> ../init.d/acpi-support lrwxrwxrwx 1 root root 21 2008-01-09 00:45 S99laptop-mode -> ../init.d/laptop-mode lrwxrwxrwx 1 root root 18 2008-01-09 00:45 S99rc.local -> ../init.d/rc.local lrwxrwxrwx 1 root root 19 2008-01-09 00:45 S99rmnologin -> ../init.d/rmnologin lrwxrwxrwx 1 root root 24 2008-01-09 00:45 S99stop-readahead -> ../init.d/stop-readahead
Ahh. We can see that each script in the /etc/rc2.d/ directory is actually a soft link pointing to a script of the same name in the /etc/init.d/ directory. This is consistent with /etc/rc[0-6].d/ directories as well. Every file in those directories just pointing to a single script residing in /etc/init.d/. This makes it easy to manage runlevels, as I’m sure you could guess. Because there is only one script, I can create symbolic links in each runlevel directory to start and stop that specific service. As mentioned before, if it’s a K-script, it will stop the service in /etc/init.d/, if it’s an S-script, it will start the service in /etc/init.d/. This would be the same as calling the script directly:
aaron@kratos:~ 1353 % sudo /etc/init.d/sendmail start Password: * Starting Mail Transport Agent (MTA) sendmail [ OK ]
The above is exactly the same as S21sendmail in the /etc/rc2.d/ directory. In fact, if you call the script directly, with no arguments, you can see that it contains a list of valid options, where start and stop are the only requirements:
aaron@kratos:~ 1354 % sudo /etc/init.d/sendmail
Password:
Invalid command <>
Usage: /etc/init.d/sendmail <command>
Where <command> is one of the following
start|stop|restart|restart-if-running
reload-if-running|reload|force-reload
newaliases|hoststat|purgestat|mailstats|mailq|runq|control
status|debug|clean
As we can see, the sendmail script has start, stop, restart, restart-if-running, reload-if-running, reload, force-reload, newaliases, hoststat, purgestat, mailstats, mailq, runq, control, status, debug and clean as valid arguments. However, we’ll talk more about the /etc/init.d/ directory in the next post in this series. For now, suffice it to say that the /etc/init.d/ directory contains all my shell scripts that a bunch of soft links point to from /etc/rc[0-6].d/. Specifying a runlevel to enter just means to either call the /etc/init.d/ script with a start option, or with a stop option, pending on whether or not it is an S-script or K-script respectively.
This is a good stopping point before continuing on to the next part of the series: Controlling Runlevels. Hopefully now, you understand the different runlevels, how to enter a runlevel, and what a specific runlevel is doing to your box. We looked at the various commands for identifying what runlevel the box is running in and how to change runlevels. We looked at some shell scripts, seeing what arguments are available when calling a daemon directly, and we found out how to tell what our default runlevel is. In other words, you should be fairly fluent on identifying runlevels now.
For the second post in this series, we’ll talk about how to manage those runlevels, giving us the flexibility to control exactly what is started and stopped when I enter a specific runlevel. We’ll pretty much be spending our entire time on one command, running through lots of examples.
I now have some homework for you to familiarize yourself with the concept of runlevels:
1) Spend some time looking through the various /etc/rc[0-6].d/ directories, identifying K-scripts and S-scripts. Do not make any changes or modify any files. 2) Open up and read the shell scripts found in those directories, and see if you can figure out what is happening with each daemon when that script stops or starts the service. 3) Spend some time changing runlevels with the ‘telinit’ command, and see how it affects your box. 4) Change your default runlevel by changing the /etc/event.d/rc-default file (MAKE A BACKUP BEFORE CHANGING THE FILE). Reboot your box, to see if it worked. 5) If you have access to a Fedora or Red Hat-based distribution, see how the runlevels differ, and see if you can do the previous 4 steps with that distro as well.]]>
Well, it’s time to ditch the legacy IM protocols in favor of Open Standards, such as IRC, XMPP/Jabber, SIP/SIMPLE and others. Of course, this means making a sacrifice to potentially lose friends and family on your roster. However, at the same time, it’s an opportunity to advocate Free Software to your friends and family, and encourage them to register open IM accounts.
May 19th has been named Open Discussion Day. This is the day, if you have not already, to ditch your legacy IM accounts for good. As it sits, you have 11 weeks to advocate to friends and family, letting them know of your decision, and what you can do to help them make the switch as well.
Lastly, I’m looking for some volunteers to help me manage Open Discussion Day this year. I’m looking for folks willing to do a little advocacy (marketing, blogging, websites, wikis, etc). If interested, you can find me in our official channel on Freenode, #opendiscussion.
]]>aaron@kratos:~ 1268 % aptitude show aptitude
Package: aptitude
State: installed
Automatically installed: yes
Version: 0.4.6.1-1ubuntu3
Priority: important
Section: admin
Maintainer: Ubuntu Core Developers
Uncompressed Size: 9449k
Depends: libapt-pkg-libc6.6-6-4.5, libc6 (>= 2.6-1), libgcc1 (>= 1:4.2.1),
libncursesw5 (>= 5.6), libsigc++-2.0-0c2a (>= 2.0.2), libstdc++6 (>=
4.2.1)
Recommends: aptitude-doc-en | aptitude-doc, libparse-debianchangelog-perl
Suggests: tasksel, debtags
Description: terminal-based apt frontend
aptitude is a terminal-based apt frontend with a number of useful features,
including: a mutt-like syntax for matching packages in a flexible manner,
dselect-like persistence of user actions, the ability to retrieve and display
the Debian changelog of most packages, and a command-line mode similar to that
of apt-get.aptitude is also Y2K-compliant, non-fattening, naturally cleansing, and
housebroken.
That’s good. I certainly wouldn’t want it “leaking” all over my computer. ![]()
Supposedly, news has hit the front that Microsoft will be supporting OpenID as a provider, and rumors have it that your GMail account can be used as an OpenID identity. But what about logging into these providers with an existing identity? Here’s the question posed: Can I login to AOL, or create and AOL account, with an already existing OpenID identity? What about LiveJournal? Wordpress? Yahoo!? Blogger? etc.
NOPE.
Like mentioned earlier, the big players are willing to throw their support behind being a provider, but not allowing the creation of new accounts, or signing into existing accounts, with previously setup identities. So, what’s the point then? If everyone has an identity to use, but no one can use it, what’s the point of creating the identity to begin with? I thought OpenID was all about 1 account and 1 password. Yet, I still have to login to Yahoo! with my Yahoo! account. I still have to login to GMail with my GMail account. And I still have to login to AOL with my already existing AOL account. So, it’s a far cry from the single login that OpenID is working so hard to achieve.
My cry to the providers, is if you really want to support OpenID, then allow users access to their account through an already existing OpenID identity. Don’t only become a provider, but show 100% support by giving them the ability to login with their OpenID account. I don’t want 500 OpenID accounts that I can’t use, because my providers won’t allow me login access with them.
]]>I recently discovered that Fedora 9 will be replacing SysVinit with Upstart. To me, this is good news. To others, maybe not so much. So, the question you may be asking is, what is SysVinit and what is upstart?
First, a little background. SysVinit, pronounced “System 5 init” or “Sys 5 init” or just “init” is a the first daemon that runs on a Unix or Linux system. As such, it typically has the PID of 1. Also, every other process also running on the box is a child process of init. What does this mean? It means that if you stop init, you will effectively stop every process on your box, basically inappropriately shutting down.
For many years, SysVinit has been the defacto in managing processes on your box. Not only processes, but also the order of booting and maintaining run levels, among other things. Because it has existed for so long, certainly there is no need to replace it. I mean, if there were issues with SysVinit, we would have discovered them by now, right? One would think.
Actually, developers the world over have noticed the shortcomings of SysVinit, and have made replacements, some of which are already deployed. Common ones include:
So, why the need to replace SysVinit? Let’s look over some common scenarios, and see where SysVinit falls short. All of these scenarios can be found on the Replacementinit page of the Ubuntu wiki:
If you read through that list, hopefully you can see where SysVinit falls short. Because of the dynamic nature of removable hardware, and a very robust kernel, SysVinit no longer meets the needs of our users. We need something more. Something that will speed up the boot process. Something that will launch processes based on events. Thus, the reason for Upstart.
The idea of an event driven system is nothing new. Crond, atd and inetd are all event driven tools that we’ve used on Unix and Linux for years. So, why not start jobs based on listening events, such as plugging in a USB disk, mounting /usr when needed and unmounting when not needed, etc? This is the design of Upstart.
First off, Upstart is 100% backwards compatible with SysVinit scripts. All of your /etc/init.d/ goodness is still available. We want to meet those use cases listed above, without modifying existing init scripts. Further, because Upstart is event driven, we can also replace crond, atd and inetd, while still maintaining backwards compatibility.
Ubuntu has been using Upstart since their 6.10 release. Nearly 18 months later, the reception of Upstart has been largely positive to the point that the Fedora Project is replacing init with Upstart in their Fedora 9 release. This also means it’s very likely that we’ll see it filter it’s way to RHEL 6, which means CentOS and even OEL. Other Linux and Unix variants are also encouraged to drop the aging SysVinit with Upstart.
It just makes sense.
]]>
I was spending some time today on Wikipedia, reading the entry on Ubuntu, and something hit me- something which I’ve known for a long while. I decided to sit down and blog it for those who haven’t really thought about it. Maybe this post will catch you unaware, and maybe not.
Ubuntu is ONE distribution that you can use both on your desktop and in your corporate data center.
Think about this for a second. Ubuntu was released as a fork of Debian unstable in October 2004, and has been releasing a new version every 6 months. Then, in June or 2006, we saw the release of the first long term support (LTS) release. Ubuntu continued to release new versions every 6 months for the desktop user. Then, almost 2 years later, Ubuntu is ready to release their second LTS release in April of this year, codenamed Hardy Heron. See what’s going on here? One distro for two purposes- the desktop and the data center. Let’s contrast this to Red Hat and Novell.
First, I have the utmost respect for Red Hat. It’s a company that has done a lot of things right in the Linux scene. Its developers have brought a lot of solid packages to the Open Source world, and it does many things correct, like run levels and system-config-*. However, you want an “enterprise” (I hate that word) distro for your data center? Better get Red Hat Enterprise Linux. Want more “bleeding edge” software on your workstations? The Fedora Project is the answer there. In other words, you need two separate distros. Same goes with Novell. SUSE Linux Enterprise Server for your data center, and OpenSUSE for your workstation.
I find this rather unfortunate, as neither Fedora nor OpenSUSE are officially supported by Red Hat or Novell respectively. You must purchase their enterprise offerings for support. Contrast this with Canonical, the parent company of Ubuntu, who supports not only the LTS releases, but the regularly 6-month scheduled releases. In other words, one distro for both your “enterprise” needs and your workstation needs, with official full-scale support. There isn’t an “enterprise” release and a community release. Ubuntu IS enterprise. Ubuntu IS community.
I look at my personal situation. The server powering this blog, and many other sites and services, is running the first LTS release, Ubuntu 6.06.2, codenamed Dapper Drake. Come April with the next LTS release, I plan on upgrading the server. Two years later, I will continue with the next LTS upgrade, and so on and so forth. Yet, I am running the exact same distribution, different release, on my 2 laptops and desktop. This makes it easy for me to only have to learn one distribution. It makes it easy to migrate packages, config files and maintain binary compatibility. While there is something to be said for remaining “distro neutral”, there’s also something to be said for learning one distro, and learning it well.
Ubuntu, keep thriving. I’ve been with you since your inception in October 2004, and you’ve got me 100% so far. To me, the philosophy of releases, support and Free Software is dead on. Not to mention, 1 CD, versus 5 or a DVD, is a no-brainer. It’s no wonder you’re leading the Linux world currently. You’re making all the right decisions.
]]>