This post will be updated periodically (or even edited entirely) to reflect my currently knowledge of Linux. It’s mainly intended for people who’ve just gotten into Linux, and to help them know some essential things/config files about Linux. I am NOT a Linux/Unix expert — I merely present the following material to the best of my knowledge, an experience thus far, of using Linux.
The basic Linux layout for the modern user:
Whether you’re using Gnome, KDE, XFCE, etc, your system looks like this:
System -> X (aka X.org or xorg) server -> User Input/Output
As you can see, the X server acts as a bridge between you and the system — mainly in the form of a (pretty) graphical interface. I’ve worked with Ubuntu, Xubuntu, and Arch Linux, and they all use the 7th TTY (aka “virtual console”) for where the X server displays its out put. (That’s right, when you’re looking at X you are looking at the output on CTRL + ALT + F7; you can do CTRL + ALT + F1 through F6 to see 6 other places where you can log in and use Linux on a purely text-based, non-graphical level!! Try it out!) Some newbies get confused and do not know how to diagnose problems if their X server suddenly stops working (usually due to an upgrade to the latest, barely tested X version), and their computer doesn’t boot into the desktop environment (DE). It is important to realize that X is *not* a requirement to fix most if not all of your problems. As I’ve hinted above, you can switch to a different TTY/virtual console any time to access everything on a text-based level. This is a handy feature, since everything in Linux is text-based at the most fundamental levels.
Some vocabulary first:
Terminal/console (aka command-line interface, or “CLI”): a terminal or console is the text-based, “DOS prompt”-like interface that most Linux users love to use from their Gnome/KDE/other X environment (it emulates the TTY/virtual console described above, but from within the X environment — a useful feature indeed!). The “terminal” application on most desktop environments (DE’s) point to either gnome-terminal, xfce4-terminal, or some other terminal installed with the base installation. If you don’t have a DE, it’s still likely that you use X (I myself use Xmonad, which itself depends on X), in which case xterm will be installed. There are a plethora of terminal emulators you can choose from — I use the 256-color enabled version of rxvt-unicode, since it handles unicode text and window-resizing very well.
Shell: a shell is the program between your TTY/virtual console/terminal application and you.
System -> TTY/virtual console/terminal app -> Shell -> (text-only) user input/output
Most systems use the Bourne Again Shell or “bash” by default. Here again, there is a plethora of alternatives to choose from: zsh, fish, csh, tcsh, ksh, etc. I use zsh by default (it is the “login” shell for my username — i.e., whenever I go to a TTY/virtual console with CTRL + ALT + F1 through F6 and log in, the zsh shell takes over by default), since it has some really painless tab-completion (pressing TAB to cycle through various options) and prompt customization (a prompt is what you see after pressing “enter” — usually the location of your present working directory along with your username, and the name of the machine (aka “hostname”) you are on). Once you start up a shell session, either via a virtual console, or via a terminal emulator from the X environment, you are placed into a directory — usually your home directory (the shorthand is the “~” character). You can move around with “cd ..” to go up 1 parent directory, or “cd [directory name]” to go to that sub-directory.
Command: a command is a word that you type into your shell-of-choice, which invokes some other program or does something to the system to either (1) change the system or (2) produce some output. For example, the “ls” command outputs all the contents of your current directory.
Argument/parameter: an argument is a word that follows a command. E.g., for “echo hi”, echo is the command, and “hi” is the argument. An argument can be a flag (aka “option”), which is usually a hyphen with a letter, like “-r”, or two hyphens followed by a full word like “–option”.
Shell scripts: a shell script is a series of commands, written into a text file, to be used over and over again to save time. It can contain not just commands, but also loops, if/else statements, etc to become even more useful. The first line looks like “#!/bin/sh” in 99% of cases (and you’ll see that /bin/sh is symlinked to /bin/bash). The “#!” is called the shebang (I did not make this up!). The shebang is very important, as it tells Linux which program to use to execute those commands. That’s right, you can write a script in not just the language used for shells, but also Python (the programming language), Ruby, Perl, etc.
And now, some tips:
TIP #1: USE THE SHELL
Most people will say that the shell is “powerful” and that’s the reason why they like to use it instead of some GUI-based solution. While I agree that the shell is very powerful, the real reason why I like to use the shell is because it represents the closest distance between me and the computer. If run from within an X session, the shell essentially removes the graphical layer that separates me from the system. Even running a non-text-based program by invoking it from the shell has an advantage: you can see exactly what the program is thinking, because most programs output additional messages to the shell, but not to the GUI — a prime example is mplayer.
Using the shell, of course, requires some knowledge of basic commands. I already mentioned a couple: cd to change the current directory, and ls to list the contents therein. A really useful command you’ll use throughout your Linux experience is the man command. Type “man [some command/program name]” to access that command/program’s manual. Apparently, all manuals are grouped into 9 different sections — the most notable are section 1 for executable programs or shell commands (such as cd or ls) and 3 for C programming commands. Since cd is a shell command, you can do man 1 ls. However, man is smart enough to accept just man ls, since ls is unique. If a manual page has an ambiguous name in that the same name appears in multiple man sections, you’ll need to type man [section] [command/name] to get what you want. If you’re not sure whether the command you are typing has multiple entries in different sections, you can use the whatis command to tell us all the manpage sections with the given name. You can navigate through man’s output (called the “manpage”) with “j” for scrolling down, “k” for scrolling up, CTRL – D/U for scrolling multiple lines down and up respectively, and PAGEUP and PAGEDOWN to go up and down a whole page. Press “q” to exit. For some manpages, you’ll see a “SEE ALSO” section, e.g., “mandb(8)” or “less(1)”. The number in parentheses is simply the section number. So to go to mandb(8), you must type man 8 mandb. As both man and whatis are themselves commands, you can do “man man” or “whatis whatis” or “whatis man” or “man whatis” to get more information about them.
Note: the output of the man command is actually “piped” into the less program, and what you are actually interfacing with is the less program (i.e., all the hotkeys I described above for “man” are actually the hotkeys for less). All less does is it shows you all the given text, while still allowing you to “scroll back” to earlier lines and back down again, interactively. Because manual pages are often hundreds of lines long, outputting it directly to your shell would mean you’d have to scroll up and down from your shell — not a very pleasant experience, although it is doable. Piping long, multiline text it into less makes the experience more manageable. Another case where less is useful: say you have a directory with 500 subdirectories. Typing ls by itself would be a bit too much. Instead, type ls | less, and now it’s much more manageable. The pipe “|” operator is useful whenever you want to give one program’s output and feed it to another program.
Here are some other basic shell commands (use man to find out more about these commands):
- du: to find out how much disk space is being used
- df: to see how much free disk space is remaining
- rm: to delete a file (this is permanent — ou cannot ‘undo’ this command, so be sure to make a backup if you want to get it back later); to delete a directory (and thus all its contents), type rm -rf dirname
- mv: renames a file or directory; in the case of a directory, you can move it to a different location if you provide a nested path (e.g., “mv dir ../dir2″ — this says “go up one directory, and if the directory “dir2″ exists, move dir into dir2, otherwise rename dir into dir2); mv is useful because it is a move/rename operation combined into one command, depending on whether the renamed-to argument conflicts with an existing file/dir name
- cp: copies files from one location to another (e.g., “cp file file_backup” creates a copy of file, names it as “file_backup”, and places it in the current directory); to copy everything within a directory, do “cp -r” (the -r flag stands for “recursive”)
- ln -s [original file] [hot clone/ghost name]: creates a symbolic link — symbolic links are very useful: e.g., if you symlink “/bin/copy” to “/bin/cp”, then typing “copy” is exactly the same as typing “cp” for the copy command, and will take all the same flags as the regular cp. I’ve called the second argument a “hot” clone/ghost because modifying the symlink affects the original (except for the “rm” command). (If you’re a C programmer then, suffice it to say that ln -s creates essentially a new pointer to the given object). Another example: if you symlink a name to another directory, then cd-ing to that name will move you into that other directory, but Linux will treat that other directory as being present in the same parent directory as the name provided. I.e., it’s effectively a ghost/clone. Another example: say you are symlink-crazy and want to have a particular file to be present in every directory in the entire system. You can do this by creating a symlink to that particular file for every directory on the system — this would result in creating a ghost image of that file in every directory. You can do whatever with this ghost image, and it will affect the original file. The only exception is the deletion of that ghost image with the “rm” command — it will NOT delete the original file; it will only delete the symlink itself. If you want to see some good examples of heavy, nested ln -s usage, simply go to your /dev directory. Your /dev/cdrom file is most likely a symlink pointing to another file. (For all of my Arch Linux machines, I discovered that /dev/cdrom, /dev/cdrom0, /dev/cdrw, /dev/dvd and /dev/dvdrw ALL point to the single device file /dev/sr0, for the CD/DVD burner.)
- scp: acts just like cp, but over to a different machine over a network
- vi or vim [text file]: invokes vi or vim, depending on which one is installed, to edit a file; learning to use vim is a smart investment of time; note that a file in Linux is either text-based or binary. (Text files are the regular files that humans can read (it has words in it), while binary ones are things like picture files (.jpg), music/video files (.mp3, .mkv), and program executables (in Linux, they have no file extension). A text file need not have an extension.)
- touch [filename]: if the given filename is not present, it creates an empty text file with that name — otherwise, it updates the “last modified” metadata for that file to the current time; using touch is useful if you want to create a series of “placeholder/nameholder” files that you want to update/edit later, but not now.
- poweroff: turns the computer off
- reboot: turns the computer off, then back on again
- ssh: connects you to a different machine as a different user. ssh is arguably one of the most useful commands in the Linux world, as it allows you to remotely access a networked machine’s shell. ssh is helpful because it gives you a cryptographically secure shell connection to modify systems remotely. This means that you can reboot a computer on your home network without physically going to that computer — you can just ssh to it, and then type “reboot”
- eject: this is here for illustrative purposes — the eject command simply ejects the CD tray of the given optical device (and closes it if it is already open); it’s quite interesting in the same sense as the poweroff and reboot commands, in that it affects via software the hardware orientation of your computer. Isn’t cool to realize that, you can eject the CD tray of a computer on the other side of the world (as long as you can ssh to it)?
- sudo: this is another command you’ll be using for a long time — it executes whatever that follows it as if it were from the root user (the root user is a “user” that is installed with every Linux installation, and lives in the /root directory, who has the power to change core system files that are off-limits for normal “regular” users). If you forgot to prepend your argument with “sudo”, and Linux complains about permission issues, just do “sudo !!” and the “!!” will act as an alias for the last-typed command. Oftentimes, you’ll encounter statements like “do X as root” or “with root permission, do Y” — all this is saying is that you must run the command as if you were the root user — either by logging in as root, or by using some other method such as invoking the “sudo” command.
- su -m: this command changes the current user to the root user, while preserving the current shell (otherwise, the default shell of the root user, usually bash, will be used); this command is useful if you want to run a series of commands for modifying/accessing core system files in root-owned directories, like the /etc directory.
- UP ARROW: this is not really a command, but a hotkey present in all shells: pressing the up arrow key will go back to the previous command you typed; the size of the command history depends on the history size variable of the shell, which can be changed (for zsh, it’s defined in your ~/.zshrc file). The DOWN ARROW is the logical opposite.
- find: finds files/directories. Here’s a concrete example: “find -maxdepth 1 -name document” searches for a file named “document” in the current directory; if it is found, it outputs the path to the document. If not found, it outputs nothing.
Some things (quirks) about the shell:
- Everything is case sensitive. And I mean everything. “CD” is different than “cd”. The flag “-f” is different than “-F” (unless the command/program treats them the same way). A file called “file” is different than a file called “FILE” or “File” or “fiLe” or “filE”, etc. You get the idea.
- The “~” (called “tilde“) character: this character is an alias for your home directory; so if your username is “joe”, then the “~” character, as interpreted by your shell, is the SAME as “/home/joe”. Indeed, typing “~” is a lot easier, and faster, than /home/joe.
- The “.” (period) character: this character literally means “full path to current directory.” For example, if you type “cp file .”, it will fail because it really means “copy file into the present working directory”, and the name “file” is already present. The “.” character comes into play in two frequent situations, however: (1) when you want to run a program located in the present working directory (see the discussion about “./” below), and (2) when you want to create a symlink in the current directory (e.g., “ln -s /full/path/to/some/file/or/directory .” creates the symlink to some file or directory in the present directory, keeping the same name as the target file/directory.
- The “/” character: by itself, it means “the highest directory in the system”; e.g., “cd /” will take you to the highest directory that contains /etc, /root, /home, etc. If this character follows a word, it’s simply a (sub)directory delimiter. Combining “.” and “/” gives us “./”, which literally means “full path to current working directory”.
- The “*” character means “everything in this directory.” This is useful, for example, if you want to delete every file in the current directory, but keeping the current directory. You could just go up 1 directory, and invoke “rm -rf [directory name]“, but that would delete the directory itself as well. If the “*” character is mixed up with letters, it acts as a regular expression that matches any character. For example, “rm *.mp3″ will delete every mp3 file in the current directory. If you want to use the “*” character as an argument to pass to a program, use “\*” instead.
- The “\” character is the escape character. This is used to disambiguate identical-looking characters that mean different things. For example, say you have 3 files “hello,” “there”, and “hello there”. You want to edit the “hello there” file. Typing “vim hello there” would not work, as vim would interpret “hello” and “there” as two different files; the proper way is “vim hello\ there”. Here, the “\” character is followed by a space, serving to disambiguate the space character to mean “the space between the “hello” and “there” in the “hello there” file, and not the usual meaning of an argument list delimiter. If you have a long file name with a bunch of spaces in between, then typing all those “\” characters might be confusing; in this case, you could just enclose the wordy file with double quotes, so that vim “a filename with a lot of spaces in between” works as expected.
- Your shell has direct access to so-called “environment variables“. You can invoke the “printenv” program to list those environment variables set in your current shell session by typing “printenv” (“env” does the same thing if no arguments are supplied). Environment variables are simply configuration settings for different aspects of your system, that are used by programs (including shells!) to customize your user experience. Perhaps the most common variable is HOME, which is the path to your home directory. Another common one is PATH, which is simply a list of directories that have executable programs in them (like /bin or /usr/bin); any program that resides in a directory listed by PATH can be executed directly in the shell without specifying the full location of that program. E.g., you can type “/bin/cp” to invoke the cp program, but you can just as easily type “cp”, since the “/bin” directory is defined in PATH. (Note: if you want to invoke a program that is in your current directory, then prepend it with “./” — e.g., if you downloaded a program that is self-contained, then going into that program’s directory and typing “./program_name” will invoke that program; in fact, the use of “./” here is required as it ) Environment variables really start to shine when you use them in shell scripts.
TIP #2: BACKUP YOUR ESSENTIAL SYSTEM FILES
You should always keep your eye on your core system files. The most common ones are:
- /boot/grub/menu.lst: this holds the file that the GRUB bootloader reads on bootup to boot to the proper operating system; this file is especially important in dual-boot machines that have multiple, different operating systems installed
- /etc/X11/xorg.conf: this file is read by the X server each time it starts — among other things, here you can customize your keyboard’s locale (if you have an Apple keyboard, for example, you can tell Xorg that you are using such a keyboard, so that X can correctly interpret all the keys on the keyboard), as well as touchpad settings for synaptic touchpad users (how fast the touchpad scrolls, whether touchpad tapping is enabled, etc.)
- ~/.xinitrc: this file is executed by X once the /etc/X11/xorg.conf contents have successfully loaded, and all commands are executed as the logged-in normal user. Think of the .xinitrc file as a file that contains a list of shell commands that you want executed automatically for you every time you log into your computer. Usually, this file is used to either set environment variables for the entire X session, or to start daemons that require special arguments and settings (e.g., SCIM, urxvtd, xscreensaver, etc). For example, if you want to set your keyboard’s repeat key rate to a certain speed, then you can hav “xset r rate 400 50″ in your ~/.xinitrc file, and each time X is run it will set the key rate to the specified rate. The so-called “rc” files are configuration files, and here “xinitrc” stands for the configuration file for the xinit command (the command that is executed to start the X server — either manually or automatically if you have a DE; startx is a front end to the xinit command).
- /etc/fstab: this is the file that Linux uses to mount partitions/devices onto your system. The basic format of every line is [device or partition] [mount point directory] [options]. Your device can be, for example, /dev/sr0 for your CD/DVD burner, or a UUID (a hexadecimal-based string that is unique to a partition on a hard drive). The mount point directory simply represents what the mounted device looks like, in terms of directories; e.g., I have a second hard drive partition whose mount point is “/mnt/hd2backup1″ — here, if I go to /mnt, I will see a directory called “hd2backup1″, which contains all the contents of this hard drive partition. For ex-Windows users: a partition in Linux does NOT start with a capital letter followed by a colon, like “C:” or “D:”. Since everything in Linux is a file or directory (a directory being nothing more than a file that points to a collection of other files), even a hard drive partition does not deserve special distinction, and is merely, following my example above, a subdirectory under the /mnt directory. Do realize that the mount point directory is customizable — you can call it anything you want, as long as it exists on the system. For example, I have my /dev/sr0 and /dev/sr1 devices (my 2 CD/DVD burners) mounted to /mnt/media-sr0 and /mnt/media-sr1, respectively, as I find mounting them to the default /media directory a bit confusing (we mount devices to use them — we don’t “media” them).
- /etc/rc.local: this file is merely a shell script that is run before anything else is run on bootup — even before xorg.conf or ~/.xinitrc is read! The counterpart to this file is /etc/rc.local.shutdown, which is a shell script that executes, appropriately enough, on shutdown.
- /etc/sudoers: this file contains a list of users who can use the “sudo” prefix in their shell commands to run the command as the root user. It also can contain a list of executables that can be run by the specified normal user, so that the executable is run as root but without the (sometimes annoying) password prompt. This is useful if you run a root-level command often, but find the required password-typing a bit tedious (for example, both the poweroff and reboot commands require root user permission, since we want to avoid the situation where one user to turn off the computer if another user is also logged into the same machine (Linux is a truly multi-user-concurrency system!) — however, if you are the only user on the system, then you can avoid password prompts by including the poweroff and reboot commands in the /etc/sudoers file). The preferred way to edit the /etc/sudoers file is with the command “visudo” as root.
TIP #3: KNOW YOUR SHELL
Since your time spent at the shell will grow more and more as you become more familiar with Linux, you should try to learn how to make your shell more useful than simply treating it as a box to dump your commands. A good place to start is editing your shell’s configuration file. For zsh, it’s ~/.zshrc. Whether your shell is zsh or any other shell, the most obvious place to start is aliases. An alias is simply a word that acts as a much different (and usually more complicated) full-fledged command. E.g., I have “k” set up as “cd ..”, “kk” aliased to “cd ../..”, “kkk” aliased to “cd ../../..”, etc. It’s a lot easier and faster to type “k” than the more wordy “cd ..”. I also have “discon” aliased to “sudo mount /dev/sr0″ — and since I have the mount command in the sudoers file, I can just type “discon” and it will mount the CD or DVD to the given path (the command here only works without specifying the mount point directory because the mount point directory for /dev/sr0 is defined already in my /etc/fstab file). Similarly, I have “usbon” aliased to “sudo mount -w -t vfat -o uid=shinobu,gid=shinobu /dev/sdc1 /mnt/media-flash” to mount USB thumbdrives easily with just one word. My aliases “discof” and “usbof” unmount the above devices, respectively.
Once you get comfortable with your shell and that big black command-line prompt does not get so intimidating any more, you should consider learning how to write shell scripts. If you find the bash scripting language too arcane in appearance, consider using Python or Ruby instead — both of these languages can execute shell commands easily, with the added benefit of an extremely simple, low-learning-curve syntax. The biggest reason for learning how to write a shell script is this: you can automate anything. For example, the mp3 replay gain scripts I mentioned here owe their usefulness to bash/Python scripting. If you know how to use cron, then you can automate something to happen on every specified interval — be it hours, days, months, even years!
