Shinobu’s Secrets

June 26, 2009

On Sleep

Filed under: Productivity/Perspectives, Rant — Shinobu @ 12:36 am

The wave of interest in recent times of so-called polyphasic sleep has become very pervasive. I’ll have to admit that even I took keen interest in it around the time I started this blog. I’ve read countless articles on sleep, and now, I’ve come to the following conclusion: the problem is not figuring out the bare minimum of hours of sleep you need each day; rather, the real problem is figuring out how you can be more active/productive while awake.

I think most people initially accept the proposition that if you sleep less, you’ll get more done. The common one-liner you’ll read in most articles goes something along the lines of “even if you shave off just 1 hour off of your usual sleeping cycle, this meager saving will net you X extra years of life!

This kind of thinking is plain wrong. Unless you are narcoleptic, sleeping too much is NOT the problem. The real problem is figuring out what you are going to do while you are awake. You can sleep 6 hours, 4, hours, 2 hours even, in a single day. Nobody cares. The only thing you should care about is how you spend your time.

Now, I do understand that for some people, spending more hours out of bed will guarantee more productivity. But this is only true if the pending task is very simple and does not require much decision-making. Very few people in the modern world today share the same working career as child laborers did at the dawn of the Industrial Age. Unless a supervisor directs all of your attention to predefined goals from the minute you wake up to the minute you go to bed, so that you are reduced to a thoughtless worker, sleeping less does NOT mean increased productivity.

Besides, there are far more important problems to tackle than “hacking” your brain to get less sleep:

  • How can I spend more quality time with my family?
  • How can I lose weight/stay healthy?
  • How can I improve my relationship with my partner?
  • How can I improve my mind?
  • How can I conquer my fears?
  • How can I become more virtuous?
  • How can I stop procrastinating? (More to the point: How can I stay interested in what I’m doing? How can I do everything with legendary focus? How can I squeeze out every millisecond of time I spend awake so that they are not wasted?)

None of these important questions involve sleep as a material part of the solution. Sleep, my friend, is irrelevant. All of the questions above instead have the goal of increasing the quality of life.

I’d like to analogize my discussion so far with the game of chess. In chess, as Garry Kasparov describes in his book “How Life Imitates Chess”, there is the concept of board time. Board time is simply measured by the number of moves played. A game with 50 moves played has the same board time as another game with 50 moves played. However, board time is NOT the same as real time (as measured in seconds, not moves): a blitz game of 50 moves may take 10 minutes, whereas a classic game of 50 moves may take upwards of 6 hours. Thus, one can play many more blitz games than a single classic game in the same amount of real time. In chess, it is universally understood that classic games have higher quality, or beauty (more beautiful ideas, flashes of technical brilliance, demonstration of flawless technique) than blitz games. All of the elite chess grandmasters are most proud (and respected in the chess community) for their ideas in classic games.

Now the analogy: let’s say that each day, as we go through it, we are playing chess. As chess players, we derive our greatest joy and triumph (as well as peer respect/recognition) from chess games that have the most beauty in them. Thus, the goal is to create the most chess beauty in a given day. We are looking for excellent games — games where each move had real meaning and thought behind them. The way I see it, those who argue that spending more time awake is a quick and easy way to increase productivity are confusing board time with real time. These people are saying that more board time = higher quality. That is, playing 50 blitz games in 5 hours results in more chess beauty than playing 1 classic game in 5 hours. They are saying that the more time you spend moving the chess pieces with your fingers over the board (the more moves you play), the greater the chess quality. What a ridiculous argument! I don’t care if you play 5, 10, 20 thousand moves of chess — if all you were playing were blitz games! None of them would be worthy of respect and recognition. We should value ourselves not from the objective perspective of X hours spent “alive” in nonsleep time, but rather, from the beauty that we create in the time that we spend — the quality of our time, the feeling of “yeah, today was an excellent day.”

So don’t fool yourself into thinking that sleeping less is an integral part of self-improvement. Sleep, like everything else in life that can be objectively quantified, must be taken in moderation.

June 21, 2009

Arch Linux: Clean up those .pacnew and .pacsave files

Filed under: Linux — Shinobu @ 12:55 am

Once in a while, pacman gives messages about some system config file, and the creation of either a .pacnew or .pacsave file (or sometimes a .pacorig file). The reason why this happens is because your system upgrade results in conflicting configuration files — pacman is smart enough to realize that it would be very dangerous to simply overwrite your configuration files automatically with the new ones from the newer package(s). A good example is GRUB’s menu.lst file — you certainly would NOT want to delete the old (unupgraded) one, since it holds all your valuable data about which partition to boot from, and as well as any other operating systems you use under GRUB. So, how should you take care of them — after all, the newer config files DO often contain newer settings and options.

The answer: install the yaourt package from AUR if you haven’t already. Then, issue the command yaourt -C. This will take care of all those pesky config files via an interactive shell session. Yaourt will let you view the original config file, and the .pacnew or .pacsave variant of it, using a variety of programs — including gvim and vim. I use gvim because my default colorscheme in vim is too dark for displaying the diffs properly. Simply do CTRL + W, W in normal mode to switch between the two windows (copy and paste to quickly merge long sections).

You should get in the habit of doing yaourt -C every time you encounter a message about a .pacnew or .pacsave.

June 7, 2009

Tips For the Novice Linux User (Use the Shell)

Filed under: Linux — Shinobu @ 11:43 pm

This post will be updated periodically (or even edited entirely) to reflect my currently knowledge of Linux. It’s mainly intended for people who’ve just gotten into Linux, and to help them know some essential things/config files about Linux. I am NOT a Linux/Unix expert — I merely present the following material to the best of my knowledge, an experience thus far, of using Linux.

The basic Linux layout for the modern user:

Whether you’re using Gnome, KDE, XFCE, etc, your system looks like this:

System -> X (aka X.org or xorg) server -> User Input/Output

As you can see, the X server acts as a bridge between you and the system — mainly in the form of a (pretty) graphical interface. I’ve worked with Ubuntu, Xubuntu, and Arch Linux, and they all use the 7th TTY (aka “virtual console”) for where the X server displays its out put. (That’s right, when you’re looking at X you are looking at the output on CTRL + ALT + F7; you can do CTRL + ALT + F1 through F6 to see 6 other places where you can log in and use Linux on a purely text-based, non-graphical level!! Try it out!) Some newbies get confused and do not know how to diagnose problems if their X server suddenly stops working (usually due to an upgrade to the latest, barely tested X version), and their computer doesn’t boot into the desktop environment (DE). It is important to realize that X is *not* a requirement to fix most if not all of your problems. As I’ve hinted above, you can switch to a different TTY/virtual console any time to access everything on a text-based level. This is a handy feature, since everything in Linux is text-based at the most fundamental levels.

Some vocabulary first:

Terminal/console (aka command-line interface, or “CLI”): a terminal or console is the text-based, “DOS prompt”-like interface that most Linux users love to use from their Gnome/KDE/other X environment (it emulates the TTY/virtual console described above, but from within the X environment — a useful feature indeed!). The “terminal” application on most desktop environments (DE’s) point to either gnome-terminal, xfce4-terminal, or some other terminal installed with the base installation. If you don’t have a DE, it’s still likely that you use X (I myself use Xmonad, which itself depends on X), in which case xterm will be installed. There are a plethora of terminal emulators you can choose from — I use the 256-color enabled version of rxvt-unicode, since it handles unicode text and window-resizing very well.

Shell: a shell is the program between your TTY/virtual console/terminal application and you.

System -> TTY/virtual console/terminal app -> Shell -> (text-only) user input/output

Most systems use the Bourne Again Shell or “bash” by default. Here again, there is a plethora of alternatives to choose from: zsh, fish, csh, tcsh, ksh, etc. I use zsh by default (it is the “login” shell for my username — i.e., whenever I go to a TTY/virtual console with CTRL + ALT + F1 through F6 and log in, the zsh shell takes over by default), since it has some really painless tab-completion (pressing TAB to cycle through various options) and prompt customization (a prompt is what you see after pressing “enter” — usually the location of your present working directory along with your username, and the name of the machine (aka “hostname”) you are on). Once you start up a shell session, either via a virtual console, or via a terminal emulator from the X environment, you are placed into a directory — usually your home directory (the shorthand is the “~” character). You can move around with “cd ..” to go up 1 parent directory, or “cd [directory name]” to go to that sub-directory.

Command: a command is a word that you type into your shell-of-choice, which invokes some other program or does something to the system to either (1) change the system or (2) produce some output. For example, the “ls” command outputs all the contents of your current directory.

Argument/parameter: an argument is a word that follows a command. E.g., for “echo hi”, echo is the command, and “hi” is the argument. An argument can be a flag (aka “option”), which is usually a hyphen with a letter, like “-r”, or two hyphens followed by a full word like “–option”.

Shell scripts: a shell script is a series of commands, written into a text file, to be used over and over again to save time. It can contain not just commands, but also loops, if/else statements, etc to become even more useful. The first line looks like “#!/bin/sh” in 99% of cases (and you’ll see that /bin/sh is symlinked to /bin/bash). The “#!” is called the shebang (I did not make this up!). The shebang is very important, as it tells Linux which program to use to execute those commands. That’s right, you can write a script in not just the language used for shells, but also Python (the programming language), Ruby, Perl, etc.

And now, some tips:

TIP #1: USE THE SHELL

Most people will say that the shell is “powerful” and that’s the reason why they like to use it instead of some GUI-based solution. While I agree that the shell is very powerful, the real reason why I like to use the shell is because it represents the closest distance between me and the computer. If run from within an X session, the shell essentially removes the graphical layer that separates me from the system. Even running a non-text-based program by invoking it from the shell has an advantage: you can see exactly what the program is thinking, because most programs output additional messages to the shell, but not to the GUI — a prime example is mplayer.

Using the shell, of course, requires some knowledge of basic commands. I already mentioned a couple: cd to change the current directory, and ls to list the contents therein. A really useful command you’ll use throughout your Linux experience is the man command. Type “man [some command/program name]” to access that command/program’s manual. Apparently, all manuals are grouped into 9 different sections — the most notable are section 1 for executable programs or shell commands (such as cd or ls) and 3 for C programming commands. Since cd is a shell command, you can do man 1 ls. However, man is smart enough to accept just man ls, since ls is unique. If a manual page has an ambiguous name in that the same name appears in multiple man sections, you’ll need to type man [section] [command/name] to get what you want. If you’re not sure whether the command you are typing has multiple entries in different sections, you can use the whatis command to tell us all the manpage sections with the given name. You can navigate through man’s output (called the “manpage”) with “j” for scrolling down, “k” for scrolling up, CTRL – D/U for scrolling multiple lines down and up respectively, and PAGEUP and PAGEDOWN to go up and down a whole page. Press “q” to exit. For some manpages, you’ll see a “SEE ALSO” section, e.g., “mandb(8)” or “less(1)”. The number in parentheses is simply the section number. So to go to mandb(8), you must type man 8 mandb. As both man and whatis are themselves commands, you can do “man man” or “whatis whatis” or “whatis man” or “man whatis” to get more information about them.

Note: the output of the man command is actually “piped” into the less program, and what you are actually interfacing with is the less program (i.e., all the hotkeys I described above for “man” are actually the hotkeys for less). All less does is it shows you all the given text, while still allowing you to “scroll back” to earlier lines and back down again, interactively. Because manual pages are often hundreds of lines long, outputting it directly to your shell would mean you’d have to scroll up and down from your shell — not a very pleasant experience, although it is doable. Piping long, multiline text it into less makes the experience more manageable. Another case where less is useful: say you have a directory with 500 subdirectories. Typing ls by itself would be a bit too much. Instead, type ls | less, and now it’s much more manageable. The pipe “|” operator is useful whenever you want to give one program’s output and feed  it to another program.

Here are some other basic shell commands (use man to find out more about these commands):

  • du: to find out how much disk space is being used
  • df: to see how much free disk space is remaining
  • rm: to delete a file (this is permanent — ou cannot ‘undo’ this command, so be sure to make a backup if you want to get it back later); to delete a directory (and thus all its contents), type rm -rf dirname
  • mv: renames a file or directory; in the case of a directory, you can move it to a different location if you provide a nested path (e.g., “mv dir ../dir2″ — this says “go up one directory, and if the directory “dir2″ exists, move dir into dir2, otherwise rename dir into dir2); mv is useful because it is a move/rename operation combined into one command, depending on whether the renamed-to argument conflicts with an existing file/dir name
  • cp: copies files from one location to another (e.g., “cp file file_backup” creates a copy of file, names it as “file_backup”, and places it in the current directory); to copy everything within a directory, do “cp -r” (the -r flag stands for “recursive”)
  • ln -s [original file] [hot clone/ghost name]: creates a symbolic link — symbolic links are very useful: e.g., if you symlink “/bin/copy” to “/bin/cp”, then typing “copy” is exactly the same as typing “cp” for the copy command, and will take all the same flags as the regular cp. I’ve called the second argument a “hot” clone/ghost because modifying the symlink affects the original (except for the “rm” command). (If you’re a C programmer then, suffice it to say that ln -s creates essentially a new pointer to the given object). Another example: if you symlink a name to another directory, then cd-ing to that name will move you into that other directory, but Linux will treat that other directory as being present in the same parent directory as the name provided. I.e., it’s effectively a ghost/clone. Another example: say you are symlink-crazy and want to have a particular file to be present in every directory in the entire system. You can do this by creating a symlink to that particular file for every directory on the system — this would result in creating a ghost image of that file in every directory. You can do whatever with this ghost image, and it will affect the original file. The only exception is the deletion of that ghost image with the “rm” command — it will NOT delete the original file; it will only delete the symlink itself. If you want to see some good examples of heavy, nested ln -s usage, simply go to your /dev directory. Your /dev/cdrom file is most likely a symlink pointing to another file. (For all of my Arch Linux machines, I discovered that /dev/cdrom, /dev/cdrom0, /dev/cdrw, /dev/dvd and /dev/dvdrw ALL point to the single device file /dev/sr0, for the CD/DVD burner.)
  • scp: acts just like cp, but over to a different machine over a network
  • vi or vim [text file]: invokes vi or vim, depending on which one is installed, to edit a file; learning to use vim is a smart investment of time; note that a file in Linux is either text-based or binary. (Text files are the regular files that humans can read (it has words in it), while binary ones are things like picture files (.jpg), music/video files (.mp3, .mkv), and program executables (in Linux, they have no file extension). A text file need not have an extension.)
  • touch [filename]: if the given filename is not present, it creates an empty text file with that name — otherwise, it updates the “last modified” metadata for that file to the current time; using touch is useful if you want to create a series of “placeholder/nameholder” files that you want to update/edit later, but not now.
  • poweroff: turns the computer off
  • reboot: turns the computer off, then back on again
  • ssh: connects you to a different machine as a different user. ssh is arguably one of the most useful commands in the Linux world, as it allows you to remotely access a networked machine’s shell. ssh is helpful because it gives you a cryptographically secure shell connection to modify systems remotely. This means that you can reboot a computer on your home network without physically going to that computer — you can just ssh to it, and then type “reboot”
  • eject: this is here for illustrative purposes — the eject command simply ejects the CD tray of the given optical device (and closes it if it is already open); it’s quite interesting in the same sense as the poweroff and reboot commands, in that it affects via software the hardware orientation of your computer. Isn’t cool to realize that, you can eject the CD tray of a computer on the other side of the world (as long as you can ssh to it)?
  • sudo: this is another command you’ll be using for a long time — it executes whatever that follows it as if it were from the root user (the root user is a “user” that is installed with every Linux installation, and lives in the /root directory, who has the power to change core system files that are off-limits for normal “regular” users). If you forgot to prepend your argument with “sudo”, and Linux complains about permission issues, just do “sudo !!” and the “!!” will act as an alias for the last-typed command. Oftentimes, you’ll encounter statements like “do X as root” or “with root permission, do Y” — all this is saying is that you must run the command as if you were the root user — either by logging in as root, or by using some other method such as invoking the “sudo” command.
  • su -m: this command changes the current user to the root user, while preserving the current shell (otherwise, the default shell of the root user, usually bash, will be used); this command is useful if you want to run a series of commands for modifying/accessing core system files in root-owned directories, like the /etc directory.
  • UP ARROW: this is not really a command, but a hotkey present in all shells: pressing the up arrow key will go back to the previous command you typed; the size of the command history depends on the history size variable of the shell, which can be changed (for zsh, it’s defined in your ~/.zshrc file). The DOWN ARROW is the logical opposite.
  • find: finds files/directories. Here’s a concrete example: “find -maxdepth 1 -name document” searches for a file named “document” in the current directory; if it is found, it outputs the path to the document. If not found, it outputs nothing.

Some things (quirks) about the shell:

  • Everything is case sensitive. And I mean everything. “CD” is different than “cd”. The flag “-f” is different than “-F” (unless the command/program treats them the same way). A file called “file” is different than a file called “FILE” or “File” or “fiLe” or “filE”, etc. You get the idea.
  • The “~” (called “tilde“) character: this character is an alias for your home directory; so if your username is “joe”, then the “~” character, as interpreted by your shell, is the SAME as “/home/joe”. Indeed, typing “~” is a lot easier, and faster, than /home/joe.
  • The “.” (period) character: this character literally means “full path to current directory.” For example, if you type “cp file .”, it will fail because it really means “copy file into the present working directory”, and the name “file” is already present. The “.” character comes into play in two frequent situations, however: (1) when you want to run a program located in the present working directory (see the discussion about “./” below), and (2) when you want to create a symlink in the current directory (e.g., “ln -s /full/path/to/some/file/or/directory .” creates the symlink to some file or directory in the present directory, keeping the same name as the target file/directory.
  • The “/” character: by itself, it means “the highest directory in the system”; e.g., “cd /” will take you to the highest directory that contains /etc, /root, /home, etc. If this character follows a word, it’s simply a (sub)directory delimiter. Combining “.” and “/” gives us “./”, which literally means “full path to current working directory”.
  • The “*” character means “everything in this directory.” This is useful, for example, if you want to delete every file in the current directory, but keeping the current directory. You could just go up 1 directory, and invoke “rm -rf [directory name]“, but that would delete the directory itself as well. If the “*” character is mixed up with letters, it acts as a regular expression that matches any character. For example, “rm *.mp3″ will delete every mp3 file in the current directory. If you want to use the “*” character as an argument to pass to a program, use “\*” instead.
  • The “\” character is the escape character. This is used to disambiguate identical-looking characters that mean different things. For example, say you have 3 files “hello,” “there”, and “hello there”. You want to edit the “hello there” file. Typing “vim hello there” would not work, as vim would interpret “hello” and “there” as two different files; the proper way is “vim hello\ there”. Here, the “\” character is followed by a space, serving to disambiguate the space character to mean “the space between the “hello” and “there” in the “hello there” file, and not the usual meaning of an argument list delimiter. If you have a long file name with a bunch of spaces in between, then typing all those “\” characters might be confusing; in this case, you could just enclose the wordy file with double quotes, so that vim “a filename with a lot of spaces in between” works as expected.
  • Your shell has direct access to so-called “environment variables“. You can invoke the “printenv” program to list those environment variables set in your current shell session by typing “printenv” (”env” does the same thing if no arguments are supplied). Environment variables are simply configuration settings for different aspects of your system, that are used by programs (including shells!) to customize your user experience. Perhaps the most common variable is HOME, which is the path to your home directory. Another common one is PATH, which is simply a list of directories that have executable programs in them (like /bin or /usr/bin); any program that resides in a directory listed by PATH can be executed directly in the shell without specifying the full location of that program. E.g., you can type “/bin/cp” to invoke the cp program, but you can just as easily type “cp”, since the “/bin” directory is defined in PATH. (Note: if you want to invoke a program that is in your current directory, then prepend it with “./” — e.g., if you downloaded a program that is self-contained, then going into that program’s directory and typing “./program_name” will invoke that program; in fact, the use of “./” here is required as it ) Environment variables really start to shine when you use them in shell scripts.

TIP #2: BACKUP YOUR ESSENTIAL SYSTEM FILES

You should always keep your eye on your core system files. The most common ones are:

  • /boot/grub/menu.lst: this holds the file that the GRUB bootloader reads on bootup to boot to the proper operating system; this file is especially important in dual-boot machines that have multiple, different operating systems installed
  • /etc/X11/xorg.conf: this file is read by the X server each time it starts — among other things, here you can customize your keyboard’s locale (if you have an Apple keyboard, for example, you can tell Xorg that you are using such a keyboard, so that X can correctly interpret all the keys on the keyboard), as well as touchpad settings for synaptic touchpad users (how fast the touchpad scrolls, whether touchpad tapping is enabled, etc.)
  • ~/.xinitrc: this file is executed by X once the /etc/X11/xorg.conf contents have successfully loaded, and all commands are executed as the logged-in normal user. Think of the .xinitrc file as a file that contains a list of shell commands that you want executed automatically for you every time you log into your computer. Usually, this file is used to either set environment variables for the entire X session, or to start daemons that require special arguments and settings (e.g., SCIM, urxvtd, xscreensaver, etc). For example, if you want to set your keyboard’s repeat key rate to a certain speed, then you can hav “xset r rate 400 50″ in your ~/.xinitrc file, and each time X is run it will set the key rate to the specified rate. The so-called “rc” files are configuration files, and here “xinitrc” stands for the configuration file for the xinit command (the command that is executed to start the X server — either manually or automatically if you have a DE; startx is a front end to the xinit command).
  • /etc/fstab: this is the file that Linux uses to mount partitions/devices onto your system. The basic format of every line is [device or partition] [mount point directory] [options]. Your device can be, for example, /dev/sr0 for your CD/DVD burner, or a UUID (a hexadecimal-based string that is unique to a partition on a hard drive). The mount point directory simply represents what the mounted device looks like, in terms of directories; e.g., I have a second hard drive partition whose mount point is “/mnt/hd2backup1″ — here, if I go to /mnt, I will see a directory called “hd2backup1″, which contains all the contents of this hard drive partition. For ex-Windows users: a partition in Linux does NOT start with a capital letter followed by a colon, like “C:” or “D:”. Since everything in Linux is a file or directory (a directory being nothing more than a file that points to a collection of other files), even a hard drive partition does not deserve special distinction, and is merely, following my example above, a subdirectory under the /mnt directory. Do realize that the mount point directory is customizable — you can call it anything you want, as long as it exists on the system. For example, I have my /dev/sr0 and /dev/sr1 devices (my 2 CD/DVD burners) mounted to /mnt/media-sr0 and /mnt/media-sr1, respectively, as I find mounting them to the default /media directory a bit confusing (we mount devices to use them — we don’t “media” them).
  • /etc/rc.local: this file is merely a shell script that is run before anything else is run on bootup — even before xorg.conf or ~/.xinitrc is read! The counterpart to this file is /etc/rc.local.shutdown, which is a shell script that executes, appropriately enough, on shutdown.
  • /etc/sudoers: this file contains a list of users who can use the “sudo” prefix in their shell commands to run the command as the root user. It also can contain a list of executables that can be run by the specified normal user, so that the executable is run as root but without the (sometimes annoying) password prompt. This is useful if you run a root-level command often, but find the required password-typing a bit tedious (for example, both the poweroff and reboot commands require root user permission, since we want to avoid the situation where one user to turn off the computer if another user is also logged into the same machine  (Linux is a truly multi-user-concurrency system!) — however, if you are the only user on the system, then you can avoid password prompts by including the poweroff and reboot commands in the /etc/sudoers file). The preferred way to edit the /etc/sudoers file is with the command “visudo” as root.

TIP #3: KNOW YOUR SHELL

Since your time spent at the shell will grow more and more as you become more familiar with Linux, you should try to learn how to make your shell more useful than simply treating it as a box to dump your commands. A good place to start is editing your shell’s configuration file. For zsh, it’s ~/.zshrc. Whether your shell is zsh or any other shell, the most obvious place to start is aliases. An alias is simply a word that acts as a much different (and usually more complicated) full-fledged command. E.g., I have “k” set up as “cd ..”, “kk” aliased to “cd ../..”, “kkk” aliased to “cd ../../..”, etc. It’s a lot easier and faster to type “k” than the more wordy “cd ..”. I also have “discon” aliased to “sudo mount /dev/sr0″ — and since I have the mount command in the sudoers file, I can just type “discon” and it will mount the CD or DVD to the given path (the command here only works without specifying the mount point directory because the mount point directory for /dev/sr0 is defined already in my /etc/fstab file). Similarly, I have “usbon” aliased to “sudo mount -w -t vfat -o uid=shinobu,gid=shinobu /dev/sdc1 /mnt/media-flash” to mount USB thumbdrives easily with just one word. My aliases “discof” and “usbof” unmount the above devices, respectively.

Once you get comfortable with your shell and that big black command-line prompt does not get so intimidating any more, you should consider learning how to write shell scripts. If you find the bash scripting language too arcane in appearance, consider using Python or Ruby instead — both of these languages can execute shell commands easily, with the added benefit of an extremely simple, low-learning-curve syntax. The biggest reason for learning how to write a shell script is this: you can automate anything. For example, the mp3 replay gain scripts I mentioned here owe their usefulness to bash/Python scripting. If you know how to use cron, then you can automate something to happen on every specified interval — be it hours, days, months, even years!

April 12, 2009

Disable touchpad tapping in HP G60 on Linux

Filed under: Computer Hardware, Linux — Shinobu @ 8:20 pm

I scoured all over the internet for a good hour and a half before finding the solution. To disable touchpad tapping for the G60, you need to: (1) enable synaptics from /etc/X11/xorg.conf, and (2) use a very specific, hard-coded device option to point to your touchpad device.

Here is the relevant portion of my xorg.conf for the HP G60 (G60t-200, to be exact — at least according to the sticker on the bottom; NOTE: I’m using the latest Xorg 1.6, but that shouldn’t really make any difference for our purposes here):

Section "ServerLayout"
    Identifier     "X.org Configured"
    Screen      0  "Screen0" 0 0
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice     "SynapticsTouchpad" "SendCoreEvents"
EndSection

Section "Module"
    Load  "glx"
    Load  "dri"
    Load  "dbe"
    Load  "xtrap"
    Load  "extmod"
    Load  "freetype"
    Load  "synaptics"
EndSection

Section "InputDevice"
   Identifier  "SynapticsTouchpad"
    Driver      "synaptics"
    Option        "SendCoreEvents" "true"
    Option        "Device" "/dev/input/by-path/platform-i8042-serio-2-mouse"
    Option        "Protocol" "auto-dev"
    Option        "HorizEdgeScroll" "0"
    Option      "SHMConfig" "true"
EndSection

Notice the italicized lines. The real key here, from what I can tell (I tried copying and pasting many different configs from various forums), is the line

    Option        "Device" "/dev/input/by-path/platform-i8042-serio-2-mouse"

I first got the idea to add the above weird-looking line from a Gentoo wiki page on another HP laptop model. I bet the above format will work for any recent HP laptop, not just the G60. The only difference between the G60 and the Pavilion DV5000 is that the DV5000 uses “…serio-4-mouse”, whereas the G60 uses “…serio-2-mouse.” To figure out this critical “serio” number, use the following command:

less -eX /proc/bus/input/devices

The output for my G60 looked like this (other irrelevant devices omitted):

I: Bus=0011 Vendor=0002 Product=0007 Version=01b1
N: Name="SynPS/2 Synaptics TouchPad"
P: Phys=isa0060/serio2/input0
S: Sysfs=/devices/platform/i8042/serio2/input/input10
U: Uniq=
H: Handlers=mouse1 event10
B: EV=b
B: KEY=420 70000 0 0 0 0
B: ABS=11000003

Notice the italicized line. It says “serio2″. Anyway, the minimal xorg.conf settings work really well — senstivity and acceleration are handled quite nicely by default. Also, tapping is NOT enabled by default, so that’s why my settings do not say anything about it.’

NOTE: The touchpad by default works very well without synaptics (vertical scrolling and all) using the Linux Kernel 2.6.28 and the newer 2.6.29, with Xorg 1.5.3 and 1.6 — however, tapping is enabled by default. The above solution is the only way to disable (the rather annoying and extremely sensitive) touchpad tapping for the HP G60.

Sources:

http://en.gentoo-wiki.com/wiki/Hp_Pavilion_DV5000

http://www.linuxquestions.org/questions/slackware-14/synaptics-touchpad-in-slackware-12.2-on-hp-g60-702809/ (post#4)

April 11, 2009

Useful shell aliases for pacman/powerpill in Arch Linux

Filed under: Linux — Tags: , , , — Shinobu @ 11:49 pm

If you’re an Arch Linux user, you’ll find yourself typing “pacman -Sy <package>” or “powerpill -Syu” a lot. Here are some useful shell aliases (here, they are for Zsh) that I use to make things much, much easier:

alias p='sudo pacman'
alias pu='sudo pacman -Syu'
alias pa='sudo pacman -Sy'
alias pq='sudo pacman -Q'

alias pp='sudo powerpill'
alias ppu='sudo powerpill -Syu'
alias ppa='sudo powerpill -Sy'

And then all I have to type is ppu, and my system is upgraded. Also, I have in my /etc/sudoers file the following:

shinobu exelion=NOPASSWD: /usr/bin/pacman
shinobu exelion=NOPASSWD: /usr/bin/powerpill

This way, I don’t have to type my password every time I want to use pacman/powerpill.

March 31, 2009

Use your /etc/hosts file to save time

Filed under: FYI, Linux — Shinobu @ 7:40 am

I was installing Arch Linux on yet another machine (new laptop), and was pleasantly surprised by a little tidbit in the Arch Wiki pages — specifically the Beginner’s Guide (which is continually improved and updated fanatically by the Arch community). In short, you can alias host addresses in your /etc/hosts file. This can be for web addresses (www.google.com) or addresses on your LAN. Here is the example and explanation given by the wiki page:

“Tip: For convenience, you may also use /etc/hosts aliases for hosts on your network, and/or on the Web, e.g.:

64.233.169.103   www.google.com   g
192.168.1.90   media
192.168.1.88   data

The above example would allow you to access google simply by typing ‘g’ into your browser, and access to a media and data server on your network by name and without the need for typing out their respective IP addresses.”

Putting your hostname aliases in your /etc/hosts file is great, since all of your programs use it. Firefox, your terminal/shell, git, etc. They all reference this file. So if you’re using git, you can do git clone ssh://username@hostname-alias/… to clone a repo from your LAN, as I described in my post here, instead of ssh://username@192.168.0.2/…. So now all of your hosts on your LAN will be much easier to remember. Here’s my /etc/hosts file:

#
# /etc/hosts: static lookup table for host names
#
#<ip-address>    <hostname.domain.org>    <hostname>
127.0.0.1        aether.localdomain    aether
# aliases
192.168.0.20   exelion
192.168.0. 21 luxion.e
192.168.0.22   luxion.w
# End of file

I’ve aliased luxion.e to my laptop’s ethernet adapter, and luxion.w to its wireless adapter. After setting up your aliases in /etc/hosts, you should get in the habit of writing out your (sane) hostnames for all your LAN computers, instead of painfully writing out their numeric IP addresses. (And I’m sure you’d want to alias your hostnames even more with the advent of IPv6 — and its rather long IP address format.)

March 26, 2009

Clean Japanese and Korean Characters in URXVT

Filed under: Linux — Tags: , , , , , , , , , — Shinobu @ 7:45 am

I have files and directories using Japanese and Korean. Unfortunately, urxvt by default does not display these characters nicely (the hangul looks especially ugly) by default. The way to fix this is to have a good set of TrueType fonts for both Japanese and Korean, and to have URXVT use these as defaults.

If you’re on Arch Linux like me, you can install from the AUR the ttf-kochi-substitute and ttf-baekmuk fonts for Japanese and Korean fonts, respectively. (There is also ttf-sazanami and ttf-unfonts-core for more Japanese and Korean fonts, respectively — but here I’m going to use Kochi Gothic and Baekmuk Gulim in URXVT). Now, in your ~/.Xdefaults file, put this in:

urxvt*font: xft:Terminus:pixelsize=14,\
            xft:Kochi Gothic:antialias=false,\
            xft:Baekmuk Gulim

This makes it so that Terminus is used, then Kochi Gothic, then Baekmuk Gulim. It’s good to have Kochi Gothic on a higher priority than Baekmuk Gulim, since Kochi Gothic’s kanji glyphs look much better than Baekmuk’s (and since Japanese words often have kanji in them, whereas Korean files almost always have just hangul in them). Also, the pixelsize defined with Terminus is used for all succeeding fonts below. Now my URXVT looks really nice!

Adding Replay Gain in Linux Automatically

Filed under: Linux, Music, Python — Shinobu @ 7:25 am

Replay gain tags in mp3, flac, and ogg files lets the player adjust the volume accordingly to make the song sound not too loud and not too soft. The Music Player Daemon (aka MPD), my favorite music player, recognizes replay gain tags if present. However, for mp3 files, the popular APE format for replay gain tags are not supported; MPD can only read ID3 tags for replay gain for mp3’s. Luckily, an mp3 file can have both APE and ID3 tags. This means that we can use mp3gain (a cute, simple command-line app available in pretty much every Linux distro) to add APE tags into our mp3s with:

mp3gain [file(s)]

This will add APE replay gain tags into the mp3 file(s) chosen. Then, we can use a simple script to read these APE replay gain tags, convert them into ID3 tags, and put these ID3 tags into the same mp3s. Such a script, thankfully, already exists here. Now, the mp3 file will have both APE and ID3 tags for replay gain values! And MPD will happily use the ID3 values.

If you’re in a hurry, you’d automate this process for entire directories, recursively. That’s what this most excellent Linux page on replay gain page suggests. (You should REALLY bookmark that page, as it has the best information hands down about replay gain in Linux.) I’ve modified the scripts for metaflac there to make it work with mp3’s instead. Here are the two scripts:

Script A:

#!/bin/bash
# Define error codes
ARGUMENT_NOT_DIRECTORY=10
FILE_NOT_FOUND=11
# Check that the argument passed to this script is a directory.
# If it's not, then exit with an error code.
if [ ! -d "$1" ]
then
	echo -e "33[1;37;44m Arg "$1" is NOT a directory!33[0m"
	exit $ARGUMENT_NOT_DIRECTORY
fi
echo -e "33[1;37m********************************************************33[0m"
echo -e "33[1;37mCalling tag-mp3-with-rg.sh on each directory in:33[0m"
echo -e "33[1;36m"$1"33[0m"
echo ""
find "$1" -type d -exec ~/syscfg/shellscripts/replaygain/mp3/tag-mp3-with-rg.sh '{}' \;

Script B (the 'tag-mp3-with-rg.sh' script referenced above in Script A):

#!/bin/bash
# Error codes
ARGUMENT_NOT_DIRECTORY=10
FILE_NOT_FOUND=11
# Check that the argument passed to this script is a directory.
# If it's not, then exit with an error code.
if [ ! -d "$1" ]
then
	echo -e "33[1;37mArg "$1" is NOT a directory!33[0m"
	exit $ARGUMENT_NOT_DIRECTORY
fi
# Count the number of mp3 files in this directory.
mp3num=`ls "$1" | grep -c \\.mp3`
# If no mp3 files are found in this directory,
# then exit without error.
if [ $mp3num -lt 1 ]
then
	echo -e "33[1;33m"$1" 33[1;37m--> (No mp3 files found)33[0m"
	exit 0
else
	echo -e "33[1;36m"$1" 33[1;37m--> (33[1;32m"$mp3num"33[1;37m mp3 files)33[0m"
fi
# Run mp3gain on the mp3 files in this directory.
echo -e ""
echo -e "33[1;37mForcing (re)calculation of Replay Gain values for mp3 files and adding them as APE2 tags into the mp3 file...33[0m"
echo -e ""
# first delete any APE replay gain tags in the files
mp3gain -s d "$1"/*.mp3
# add fresh APE tags back into the files
mp3gain "$1"/*.mp3
echo -e ""
echo -e "33[1;37mDone.33[0m"
echo -e ""
echo -e "33[1;37mAdding ID3 tags with the same calculated info from above...33[0m"
echo -e ""
# the -d is for debug messages if there are any errors, and the -f is for overwriting any existing ID3 replay gain tags
~/syscfg/shellscripts/replaygain/mp3/ape2id3.py -df "$1"/*.mp3
echo -e ""
echo -e "33[1;37mDone.33[0m"
echo -e ""
echo -e "33[1;37mReplay gain tags (both APE and ID3) successfully added recursively.33[0m"
echo -e ""

And here is the APE to ID3 conversion script from the link above (the 'ape2id3.py' script called from Script B):

#! /usr/bin/env python

import sys
from optparse import OptionParser

import mutagen
from mutagen.apev2 import APEv2
from mutagen.id3 import ID3, TXXX

def convert_gain(gain):
   if gain[-3:] == " dB":
       gain = gain[:-3]
   try:
       gain = float(gain)
   except ValueError:
       raise ValueError, "invalid gain value"
   return "%.2f dB" % gain

def convert_peak(peak):
   try:
       peak = float(peak)
   except ValueError:
       raise ValueError, "invalid peak value"
   return "%.6f" % peak

REPLAYGAIN_TAGS = (
   ("mp3gain_album_minmax", None),
   ("mp3gain_minmax", None),
   ("replaygain_album_gain", convert_gain),
   ("replaygain_album_peak", convert_peak),
   ("replaygain_track_gain", convert_gain),
   ("replaygain_track_peak", convert_peak),
)

class Logger(object):
   def __init__(self, log_level, prog_name):
       self.log_level = log_level
       self.prog_name = prog_name
       self.filename = None

   def prefix(self, msg):
       if self.filename is None:
           return msg
       return "%s: %s" % (self.filename, msg)

   def debug(self, msg):
       if self.log_level >= 4:
           print self.prefix(msg)

   def info(self, msg):
       if self.log_level >= 3:
           print self.prefix(msg)

   def warning(self, msg):
       if self.log_level >= 2:
           print self.prefix("WARNING: %s" % msg)

   def error(self, msg):
       if self.log_level >= 1:
           sys.stderr.write("%s: %s\n" % (self.prog_name, msg))

   def critical(self, msg, retval=1):
       self.error(msg)
       sys.exit(retval)

class Ape2Id3(object):
   def __init__(self, logger, force=False):
       self.log = logger
       self.force = force

   def convert_tag(self, name, value):
       pass

   def copy_replaygain_tag(self, apev2, id3, name, converter=None):
       self.log.debug("processing '%s' tag" % name)

       if not apev2.has_key(name):
           self.log.info("no APEv2 '%s' tag found, skipping tag" % name)
           return False
       if not self.force and id3.has_key("TXXX:%s" % name):
           self.log.info("ID3 '%s' tag already exists, skpping tag" % name)
           return False

       value = str(apev2[name])
       if callable(converter):
           self.log.debug("converting APEv2 '%s' tag from '%s'" %
                          (name, value))
           try:
               value = converter(value)
           except ValueError:
               self.log.warning("invalid value for APEv2 '%s' tag" % name)
               return False
           self.log.debug("converted APEv2 '%s' tag to '%s'" % (name, value))

       id3.add(TXXX(encoding=1, desc=name, text=value))
       self.log.info("added ID3 '%s' tag with value '%s'" % (name, value))
       return True

   def copy_replaygain_tags(self, filename):
       self.log.filename = filename
       self.log.debug("begin processing file")

       try:
           apev2 = APEv2(filename)
       except mutagen.apev2.error:
           self.log.info("no APEv2 tag found, skipping file")
           return
       except IOError:
           e = sys.exc_info()
           self.log.error("%s" % e[1])
           return

       try:
           id3 = ID3(filename)
       except mutagen.id3.error:
           self.log.info("no ID3 tag found, creating one")
           id3 = ID3()

       modified = False
       for name, converter in REPLAYGAIN_TAGS:
           copied = self.copy_replaygain_tag(apev2, id3, name, converter)
           if copied:
               modified = True
       if modified:
           self.log.debug("saving modified ID3 tag")
           id3.save(filename)

       self.log.debug("done processing file")
       self.log.filename = None

def main(prog_name, options, args):
   logger = Logger(options.log_level, prog_name)
   ape2id3 = Ape2Id3(logger, force=options.force)
   for filename in args:
       ape2id3.copy_replaygain_tags(filename)

if __name__ == "__main__":
   parser = OptionParser(version="0.1", usage="%prog [OPTION]... FILE...",
                         description="Copy APEv2 ReplayGain tags on "
                                     "FILE(s) to ID3v2.")
   parser.add_option("-q", "--quiet", dest="log_level",
                     action="store_const", const=0, default=1,
                     help="do not output error messages")
   parser.add_option("-v", "--verbose", dest="log_level",
                     action="store_const", const=3,
                     help="output warnings and informational messages")
   parser.add_option("-d", "--debug", dest="log_level",
                     action="store_const", const=4,
                     help="output debug messages")
   parser.add_option("-f", "--force", dest="force",
                     action="store_true", default=False,
                     help="force overwriting of existing ID3v2 "
                          "ReplayGain tags")
   prog_name = parser.get_prog_name()
   options, args = parser.parse_args()

   if len(args) < 1:
       parser.error("no files specified")

   try:
       main(prog_name, options, args)
   except KeyboardInterrupt:
       pass

# vim: set expandtab shiftwidth=4 softtabstop=4 textwidth=79:

It’s pretty simple. Script A just calls Script B recursively on every directory found inside the designated directory. Script B finds mp3 files, and first tags them with APE replay gain tags with mp3gain. Then, it calls the APE to ID3 conversion script above to add equivalent ID3 tags into those same mp3s. I’ve modified Script B so that it first deletes any APE replay gain tags already present in the mp3 file before doing the replay gain calculations — but this is optional. I also added a bunch of ANSI color escape codes to Script A and B so that they look prettier. These three scripts work beautifully well together with mp3 files inside directories. However, the directories MUST be album directories, as all mp3 files found in a directory are treated as having come from the same album for album replay gain tags (track replay gain tags are always independent on a per-file basis).

I should probably rewrite Script A and B in Python to make it easier to maintain — but everything is pretty simple as it is. If you have 1 big folder full of mp3’s from different artists/albums, then you could change the

mp3gain "$1"/*.mp3

in Script B into something like

for file in $mp3files
do
    mp3gain "$file"
done

This way, mp3gain is called separately for each mp3 file (instead of being called once for all mp3 files in the folder). Now you don’t have to worry about the mp3’s in that folder being treated as having come from 1 album for those album gain tags. To top things off, you should edit your shell’s config (e.g., .zshrc), and alias Script A to something easy, like rgmp3, so that you can just do

rgmp3 [directory]

to get this whole thing to work. Now run this command on your master mp3 folder, take a nap, and come back. All your mp3’s will now have both APE and ID3 replay gain tags!

I hope people find this useful. I’ve googled and googled countless times about replay gain in the past, and until I discovered the excellent link mentioned above a couple days ago, I could never really get replay gain tags working for my mp3’s.

March 18, 2009

MPD – A Brief Guide

Filed under: FYI, Linux, Recreation — Tags: , , , , — Shinobu @ 5:24 am

The Music Player Daemon, aka MPD, is a great little application to handle music on your linux computer. Until recently (when Linux really started to take off and gain popularity), MPD has been notoriously difficult to get to work properly. I decided to write this small intro to MPD after having used it for about 6 months. This is a little guide to it to get you started on using MPD as your music manager.

The World of MPD

First, what MPD is:

  • MPD keeps track of your music library. It does this by defining a music directory, to which you can easily add symlinks to all the various places where you store your music on your computer.
  • MPD tracks changes to your music library (any time an mp3 is removed, or a new album added, you simply update the MPD database file — which takes three or four seconds at most).
  • MPD can store playlists.
  • MPD can play songs, and output this information to various devices, such as your physical speakers (via alsa usually, but it can be any audio driver for your system), or even a http port for web access (via icecast).

What MPD is NOT (what most people get confused about):

  • MPD cannot by itself determine: which songs are played and in what order. Nor does it come with any controls such as PAUSE, STOP, NEXT TRACK, etc. MPD, quite simply, can either: play a song, or do nothing while it is running. MPD’s behavior is controlled by clients, such as GMPC, Sonata, etc (my personal favorite is ncmpcpp).

The downside of this dichotomy between what MPD is and what it is not, is that you must install two applications to get the same functionality as any standard multimedia player like iTunes. However, there are many, many advantages of splitting up music-playing from music-controls. First, mpd will hardly crash and will be error-free due to its simple nature. Second, you gain the ability to access your large music collection from other computers in your home network without much fuss. This means that you could do things like: make your laptop act as a remote controller for your desktop’s speakers, or even create multiple MPD instances for specific purposes (e.g., alsa vs. icecast output).

Typical Setup

For most people, they only need one instance of MPD running — configured so that it plays songs directly to their computer speakers (with alsa output). So their MPD configuration file will look something like this (comment lines removed to save space):

music_directory "/home/shinobu/.mpd-untracked/music"
playlist_directory "/home/shinobu/.mpd-untracked/playlist"
db_file "/home/shinobu/.mpd-untracked/mpd.db"
log_file "/home/shinobu/.mpd-untracked/mpd.log"
error_file "/home/shinobu/.mpd-untracked/mpd.error"
pid_file "/home/shinobu/.mpd-untracked/mpd.pid"
state_file "/home/shinobu/.mpd-untracked/mpdstate"
user "shinobu"
bind_to_address        "localhost" # if no one else on the LAN wants to use this mpd; otherwise use local IP address
port                "6600"
audio_output {
    type            "alsa"
    name            "Exelion ALSA output"
}
mixer_type            "alsa"
mixer_device            "default"
mixer_control            "PCM"
replaygain            "track"
filesystem_charset        "UTF-8"
id3v1_encoding            "ISO-8859-1"

It’s very straightforward. All of the essential files and folders are customized and defined in the config file. (My mpd directory is .mpd-untracked, since these files/folders are not tracked by my git folder containing all of my dotfiles. See my previous two posts for more info on that setup.) The config file itself resides at /home/shinobu/sycfg/mpd/cfg-alsa — a customized path and filename, which is referenced each time MPD starts on bootup (in my .xinitrc file), with the command “mpd /home/shinobu/syscfg/mpd/cfg-alsa”. MPD will daemonize itself when it is called, so calling it from your .xinitrc file without the “&” argument at the end of the line is OK. Notice how the above configuration avoids running into permission problems, since everything resides under a normal user’s directory.

I then start up ncmpcpp, which by default looks to the mpd host as localhost and port as 6600 — matching the (default) values in my MPD config file above. And it all works rather beautifully.

Multi-MPD Setup for Music Sharing

Back when I still used iTunes (about 3 years ago), I remember it had this neat feature to share your music with anyone on your local network. You can do the same thing with MPD. The trick is to add a separate MPD instance for this very same feature. I myself have this multi-MPD setup on my desktop, with one MPD for alsa-only output (as seen above), and a second MPD for icecast-only output. Icecast, by the way, is a more general solution not wholly related to MPD for sharing music over the internet — we can use icecast since it plays rather well with MPD. Since my desktop has a second MPD instance so that it works with icecast, my laptop can now take advantage of the huge music collection that I have on it, remotely (well, as long as I’m on the home network in my case).

Here’s my MPD config file for icecast output:

music_directory "/home/shinobu/.mpd-untracked/music"
playlist_directory "/home/shinobu/.mpd-untracked/playlist"
db_file "/home/shinobu/.mpd-untracked/mpd.db"
log_file "/home/shinobu/.mpd-untracked/mpd-icecast.log"
error_file "/home/shinobu/.mpd-untracked/mpd-icecast.error"
pid_file "/home/shinobu/.mpd-untracked/mpd-icecast.pid"
state_file "/home/shinobu/.mpd-untracked/mpdstate-icecast"
user "shinobu"
bind_to_address		"192.168.0.7" # bind it to this computer's IP address if someone on the LAN wants to use this mpd
port				"6601"
audio_output {
    type			"shout"
    name			"exelion HD mpd stream"
    encoding		"ogg"			# optional
    host			"localhost"
    port			"8000"
    mount			"/mpd.ogg"
#--------------------------------------------------------------------------------------------------#
# See 'source password' in /etc/icecast.xml for password                                           #
#--------------------------------------------------------------------------------------------------#
    password		"hackme"
#--------------------------------------------------------------------------------------------------#
# Use 'bitrate' or 'quality'. Use 'bitrate' for static bitrate, or 'quality' for variable.         #
# Use one or the other -- not both!                                                                #
# bitrate                     "128"                                                                #
#--------------------------------------------------------------------------------------------------#
    quality			"10"
#--------------------------------------------------------------------------------------------------#
# 'format' is the audio format of the stream. The first number is the sample rate in Hertz (Hz);   #
# 44100 Hz is equal to cd quality. The second number is the number of bits per sample; again 16    #
# bits is the quality used on cd's. The third number is the number of channels; 1 channel is       #
# mono, 2 channels is stereo.                                                                      #
#--------------------------------------------------------------------------------------------------#
    format			"44100:16:2"
    protocol		"icecast2"		# optional
    user			"source"		# optional
    #description		"My Stream Description"	# optional
    #genre			"jazz"			# optional
    #public			"no"			# optional
    #timeout			"2"			# optional
}
mixer_type			"alsa"
mixer_device			"default"
mixer_control			"PCM"
replaygain			"track"
filesystem_charset		"UTF-8"
id3v1_encoding			"ISO-8859-1"

As you can see, much of the configuration is exactly the same as my first MPD config file. You’d want it like this, since you’d want it to use the same music collection and list of playlists. So the music directory, playlist directory, and database file are all the same. However, you’d want a different set of logfiles for this second MPD instance, so those values are changed. The same goes to the pid and state files.  The other noticeable differences are the bind_to_address value, set to my desktop’s IP address. The port number is also 6601, so that it doesn’t conflict with my first MPD. The final difference is the audio output, which is set to “shout” and not “alsa”. The “shout” output works with icecast, and the values in here, like port 8000 and “hackme” as the password, are taken straight from the default icecast intallation’s xml file (/etc/icecast.xml — of course, you can call icecast with a different config file location than the default one).

It’s important that you make this second MPD output solely to icecast. If you put in alsa output here, you’d have two different songs playing simultaneously on your desktop’s speakers! (Aside: MPD is perfectly capable of outputting to multiple audio outputs, so you could, if you wanted, make one MPD instance output to both alsa and icecast — although I don’t see why anyone would do this.) As for icecast itself, you can also call it from your .xinitrc file, after your mpd-calling lines, like so: icecast -b -c /home/shinobu/syscfg/icecast/cfg.xml. The -b flag daemonizes icecast (required since you’re calling it from your .xinitrc file), and the -c flag merely points to the correct config file location. My icecast config file looks like this:

<icecast>
    <limits>
        <clients>100</clients>
        <sources>2</sources>
        <threadpool>5</threadpool>
        <queue-size>524288</queue-size>
        <client-timeout>30</client-timeout>
        <header-timeout>15</header-timeout>
        <source-timeout>10</source-timeout>
        <burst-on-connect>1</burst-on-connect>
        <burst-size>65535</burst-size>
    </limits>
    <authentication>
        <source-password>hackme</source-password>
        <relay-password>hackme</relay-password>
        <admin-user>admin</admin-user>
        <admin-password>hackme</admin-password>
    </authentication>
    <hostname>localhost</hostname>
    <listen-socket>
        <port>8000</port>
    </listen-socket>
    <fileserve>1</fileserve>
    <paths>
        <basedir>/usr/share/icecast</basedir>
        <logdir>/home/shinobu/.icecast-untracked/log</logdir>
        <webroot>/home/shinobu/.icecast-untracked/icecast/web</webroot>
        <adminroot>/home/shinobu/.icecast-untracked/icecast/admin</adminroot>
        <alias source="/" dest="/status.xsl"/>
    </paths>
    <logging>
        <accesslog>access.log</accesslog>
        <errorlog>error.log</errorlog>
        <loglevel>1</loglevel> <!-- 4 Debug, 3 Info, 2 Warn, 1 Error -->
        <logsize>10000</logsize> <!-- Max size of a logfile -->
    </logging>
    <security>
        <chroot>0</chroot>
    </security>
</icecast

Here, the value for the <source-password> tag must match the password specified in your icecast-MPD’s config file. I’ve left it to the default “hackme” here for demnstration purposes — you should change it to something more secure (better safe than sorry!). My password is like @#$k23lsdf9. Make sure the directory in <logdir> exists. As for the <webroot> and <adminroot> directories, copy them straight out of /usr/share/icecast/web and /usr/share/icecast/admin, and place them into your custom location. This will take care of those annoying permission errors that users often face.

Now, on your remote computer, such as your laptop, you need (1) an MPD client, and (2) a music player, like mplayer, to intercept the raw .ogg music stream from your desktop with the icecast-MPD setup. In my ncmpcpp config file on my laptop, I have custom values for the mpd_host and mpd_port fields to be my desktop’s IP address 192.168.0.7 and 6601, respectively (matching the values in the second MPD config file above). So when I fire up ncmpcpp on my laptop, it connects over the LAN network into the music collection residing in my desktop, all with the magic of MPD. Once I activate the ogg stream by playing a file, I can then access this stream with mplayer — specifically, with mplayer -ao alsa -softvol -volume 10 -volstep 1 -prefer-ipv4 http://192.168.0.110:8000/mpd.ogg. I have this long command aliased, of course, to a hotkey in my XMonad configuration file. And within a couple seconds, I get a top-quality, ~500 kbps ogg stream right on my laptop (the quality setting in my MPD config file, set to “10″, is the maximum quality supported by the ogg format; 500 kbps, or a little over 60 KBps, should be nothing on your home ethernet LAN).

So here’s what’s happening with this second MPD instance on my desktop, configured with icecast output: MPD is keeps tabs on what my music collection consists of from port 6601, and “plays” to port 8000 — converting my current mp3/flac file into an ogg stream, which is then served by icecast. From my laptop, I then connect to this MPD by the desktop’s IP address and MPD port of 6601 with ncmpcpp. Then, once I tell MPD to start playing, I can then access the ogg stream with mplayer on my laptop. Mplayer simply looks to http://192.168.0.7:8000/mpd.ogg. When icecast on my desktop receives this request from my laptop’s mplayer, it gives the go-ahead and essentially acts as a bridge between my desktop’s icecast-output MPD and my laptop’s mplayer. Mplayer then takes this ogg stream, and outputs it onto my laptop’s speakers using the alsa drivers.

Here’s a picture to describe what’s going on. (The latest OpenOffice.org Drawing app is really a pleasure to use!)

mpd-configurationSo I hope this post proves useful to people who are new to Linux and are struggling to get MPD (and especially with Icecast) to work.

March 17, 2009

Unified Configuration File Setup Across Multiple Machines – Revisited

Filed under: FYI, Linux, Productivity/Perspectives — Shinobu @ 11:49 pm

My last post discussed how to get your config files, known as “dotfiles,” synchronized across multiple machines using a rudimentary makefile and git. I said that I hoped of achieving the “one folder for all config files” dream. I have achieved it, and it is pretty simple. Also, I will discuss how to handle files with passwords in them, and some other thoughts on this setup.

Keep track of system files, too

Essentially, my last post left out all but the system configuration files, such as /etc/fstab and the like. The /etc folder is owned by root, as well as the /boot folder. My first approach was to simply replace all such files with more symlinks, which would point to files owned by the normal user of the system. This approach had its drawbacks: (1) not all system files are symlinkable (e.g., /etc/sudoers is particularly security-conscious), and (2) the idea of deleting system files and replacing them with symlinks, on its face, sounded like I was setting myself up for a big grand screw-up.

So I thought: “Well, since system files are seldom ever edited anyway, why not just back them up periodically?” And that’s what I did instead. Now my makefile, as discussed in the previous post, has a section like this:

# copy contents of system files to keep track of them
syscopy:
ifeq ('$(HOSTNAME)','exelion')
	cat /boot/grub/menu.lst >       /home/shinobu/syscfg/sys/boot-grub-menu.lst-exelion
	cat /etc/X11/xorg.conf >        /home/shinobu/syscfg/sys/etc-X11-xorg.conf-exelion
	cat /etc/fstab >                /home/shinobu/syscfg/sys/etc-fstab-exelion
	cat /etc/hosts >                /home/shinobu/syscfg/sys/etc-hosts-exelion
	cat /etc/inittab >              /home/shinobu/syscfg/sys/etc-inittab-exelion
	cat /etc/makepkg.conf >         /home/shinobu/syscfg/sys/etc-makepkg.conf-exelion
	cat /etc/rc.conf >              /home/shinobu/syscfg/sys/etc-rc.conf-exelion
	cat /etc/rc.local >             /home/shinobu/syscfg/sys/etc-rc.local-exelion
	cat /etc/rc.local.shutdown >    /home/shinobu/syscfg/sys/etc-rc.local.shutdown-exelion
	cat /etc/yaourtrc >             /home/shinobu/syscfg/sys/etc-yaourtrc-exelion
	cat /etc/sudoers >              /home/shinobu/syscfg/sys/etc-sudoers-exelion # requires superuser privileges to read!
else
	cat /boot/grub/menu.lst >       /home/shinobu2/syscfg/sys/boot-grub-menu.lst-luxion
	cat /etc/X11/xorg.conf >        /home/shinobu2/syscfg/sys/etc-X11-xorg.conf-luxion
	cat /etc/fstab >                /home/shinobu2/syscfg/sys/etc-fstab-luxion
	cat /etc/hosts >                /home/shinobu2/syscfg/sys/etc-hosts-luxion
	cat /etc/inittab >              /home/shinobu2/syscfg/sys/etc-inittab-luxion
	cat /etc/makepkg.conf >         /home/shinobu2/syscfg/sys/etc-makepkg.conf-luxion
	cat /etc/network.d/luxion-wired > /home/shinobu2/syscfg/sys/etc-network.d-luxion-wired
	cat /etc/network.d/luxion-wireless-home-nopassword > /home/shinobu2/syscfg/sys/etc-network.d-luxion-wireless-home-nopassword
	cat /etc/rc.conf >              /home/shinobu2/syscfg/sys/etc-rc.conf-luxion
	cat /etc/rc.local >             /home/shinobu2/syscfg/sys/etc-rc.local-luxion
	cat /etc/rc.local.shutdown >    /home/shinobu2/syscfg/sys/etc-rc.local.shutdown-luxion
	cat /etc/yaourtrc >             /home/shinobu2/syscfg/sys/etc-yaourtrc-luxion
	cat /etc/sudoers >              /home/shinobu2/syscfg/sys/etc-sudoers-luxion
endif

I have in my /etc/rc.local the command “make -f /path/to/the/above/makefile -B syscopy“. So every time my system boots up, all of the config files are copied into their backup-equivalents in the syscfg/sys folder. Since git tracks changes in the syscfg folder, only changes in the config files are detected and tracked as changes (i.e., git doesn’t track changes in file modification times, which is a good thing here for our purposes — otherwise git would be saying that every time we boot up all of our system files have changed!). So now all of my system config files are tracked passively (by merely reading off them). Of course, if I manually edit a system file, I can still call make -B syscopy myself manually, and then run git diff in the syscfg folder to track those changes, and then git commit to solidify those changes into the git history.

For config files with passwords in them

DO NOT EVER PUT CONFIG FILES WITH PASSWORDS INTO YOUR TRACKED DOTFILES FOLDER! Not only does this mean that your password, in plain text, is tracked by git, but that should you ever change your password, git will notice the changes and track them as well! This will give anyone who gets access to your git repo a complete, timestamped history of your passwords for your applications (like icecast, irssi, etc.) So to get around this problem, I have it set up so that I have a copy of the password-containing config file, minus the passwords in them. Whenever I make changes to the original password-containing files, I update the changes into the copies, and then track these copies in git, not the originals.

Not all config files need symlinks

In my last post, I discussed how creating symlinks via commands in the makefile was the key to this whole setup. But for some (smarter) applications, symlinks are not needed, since they can intelligently be told which config file to use. Alpine, icecast, irssi, and mpd are like this, so I just have config files for them inside my syscfg folder, and just run these apps (which are all autostarted for me each time on boot) with commandline parameters pointing to the non-default config file locations.

Older Posts »

Blog at WordPress.com.