SUMMARY: This post shows you how to sync multiple configuration files across multiple hosts with git and a makefile.
INTRODUCTION
If you have 20 different so-called ‘dotfiles’ like me, they can get difficult to keep track of. It can be even more difficult if you have multiple computers that you use often, and if you want them all to be updated to your latest settings.
For myself, I need to keep track of:
- .gitconfig
- .zshrc
- .zsh (folder)
- .vim (folder)
- .vimrc
- .gvimrc
- .vimperatorrc
- .Xdefaults
- .boxes
- .xmonad (folder)
- .xmonad/init.sh (custom init script that XMonad is told to call in the startup hook)
- .xmonad/xmonad.hs
- shellscripts (folder which has a growing number of custom shell scripts that I like to use every now and then, or at least keep as a reference on both my desktop and laptop)
- .xinitrc
Of course, this list will grow over time, as I start to learn more things and begin using more programs. What I want to do is (1) copy these files over to any other host that I use/own automatically and sync back to all the other machines any changes that I make to any one particular machine; and (2) have a unified config file structure, with a directory name for the application/setting, and a simple file called ‘cfg’ for the config file (which will be symlinked to what the application thinks is the true, appropriate location of the config file). The great thing about these two issues is that there is a simple, durable solution for these two precise concerns: git and make. So enough blabbering, let’s get to it!
Step 1: Put all config files into a new directory
It doesn’t matter where your new directory (let’s call it syscfg) is located. Move all of your config files that you want to keep track of into this directory. I suggest you rename all of them to fit some kind of unified naming scheme, and take note of what their former names/destinations used to be. For example, I use syscfg/vim to keep all of my vim things in there (instead of the default .vim, including a file called cfg to act as my .vimrc file).
Per-host (host-specific) settings
I highly suggest that you make all of your config files, such as your .xinitrc (or any other script or some sort) file, include per-host specific settings. Otherwise, you will have the same config settings on all of your systems! I.e., you’d want your .xinitrc to have something like:
do-something-universal-here
HOSTNAME=$(hostname)
case "${HOSTNAME}" in
hostname1)
do-something-here
;;
hostname2)
do-something-else-here
;;
*) # catches all other hostnames
do-something-else-that's-universal-here
;;
esac
do-something-else-that's-universal-here
The above syntax is for bash scripts (files that start with a “#!/bin/bash” at the very first line). If you are using zsh, you could also use this syntax for creating aliases that are specific to a certain host (e.g., my laptop doesn’t have 2 hard drives, so it doesn’t have aliases that point to my mount directory).
If your config file is painful to work with in implementing per-host settings (my xmonad.hs file is like this), you can still achieve per-host settings by making 2 config files, so for example cfg-host1 and cfg-host2, and symlink the correct one to .xmonad/xmonad.hs. You determine the correct config file to symlink to in the makefile. Read on.
Step 2: Create a makefile
Your sysconfig should now have a clean, uniform structure for all of your config files. Now, let’s create a makefile so that the program make can install and uninstall the symlinks as necessary. Here’s what my makefile looks like:
CFGROOT := $(shell pwd)
HOSTNAME := $(shell hostname)
all: boxes git shellscripts vim vimperatorrc xdefaults xinitrc xmonad zsh
boxes:
ln -fs $(CFGROOT)/boxes/cfg ${HOME}/.boxes
git:
ln -fs $(CFGROOT)/git/cfg ${HOME}/.gitconfig
shellscripts:
ln -fs $(CFGROOT)/shellscripts ${HOME}/shellscripts
vim:
ln -fs $(CFGROOT)/vim ${HOME}/.vim
ln -fs $(CFGROOT)/vim/cfg ${HOME}/.vimrc
ln -fs $(CFGROOT)/vim/cfg-gui ${HOME}/.gvimrc
vimperatorrc:
ln -fs $(CFGROOT)/vimperatorrc/cfg ${HOME}/.vimperatorrc
xdefaults:
ln -fs $(CFGROOT)/xdefaults/cfg ${HOME}/.Xdefaults
xinitrc:
ln -fs $(CFGROOT)/xinitrc/cfg ${HOME}/.xinitrc
xmonad:
ln -fs $(CFGROOT)/xmonad ${HOME}/.xmonad
ln -fs $(CFGROOT)/xmonad/init ${HOME}/.xmonad/init.sh
#--------------------------------------------------------------------------------------------------#
# Since it's really painful to do a unified config file across multiple hosts in XMonad v. 0.8.1, #
# I have to do it this way. #
#--------------------------------------------------------------------------------------------------#
ifeq ('$(HOSTNAME)','exelion')
ln -fs $(CFGROOT)/xmonad/cfg ${HOME}/.xmonad/xmonad.hs
else
ln -fs $(CFGROOT)/xmonad/cfg-luxion ${HOME}/.xmonad/xmonad.hs
endif
zsh:
ln -fs $(CFGROOT)/zsh ${HOME}/.zsh
ln -fs $(CFGROOT)/zsh/cfg ${HOME}/.zshrc
uninstall:
rm ${HOME}/.boxes
rm ${HOME}/.gitconfig
rm ${HOME}/.vim
rm ${HOME}/.vimrc
rm ${HOME}/.gvimrc
rm ${HOME}/shellscripts
rm ${HOME}/.vimperatorrc
rm ${HOME}/.Xdefaults
rm ${HOME}/.xinitrc
rm ${HOME}/.xmonad/init.sh
rm ${HOME}/.xmonad/xmonad.hs
rm ${HOME}/.xmonad
rm ${HOME}/.zsh
rm ${HOME}/.zshrc
This is where symlinks reveal their beauty. From what I know of Windows XP (and my knowledge is very limited because I hate M$ with a passion), you cannot do something like this. Anyway, the above is fairly obvious and straightforward, isn’t it? All this does is create symlinks, and remove them if desired. Since they are symlinks, you can still do something like “vim ~/.vimrc”, and vim will read (assuming again that our directory is syscfg) syscfg/vim/cfg, with all of the pretty syntax highlighting and so on. The -f flag for the ln command simply makes it create the symlink even if the symlink already exists at the destination. See man ln for more info.
If you run make -B all, it will create all the symlinks defined under the keyword all. (The -B flag, for forcing make to run, is required here given our situation with symlinks.) You could in the alternative select only those config files you wish to install; e.g., make -B xmonad for installing only those symlinks for vim’s config files, or make -B vim zsh for vim and zsh’s config files. Lastly, running make uninstall removes all of the symlinks from your system. Experiment to your delight. (Make sure to have the lines with ln and rm start with a TAB character, as make will otherwise throw an error.)
Also, note how the contents in syscfg do not matter at all in how they are named, since it’s really the with its symlinks that takes care of all the proper “dotfile” namings.
Lastly, since these are all symlinks, you can have in your, say, .gvimrc file, a line that says “source ~/.vim/cfg”, and it will still work since the .vim directory is symlinked to your syscfg/vim (i.e., you don’t have to refer to symlinks once you make the symlinks). This is just a long-winded way of saying that using this symlink approach preserves all of your old config file paths from within your config files.
Step 3: Fire up git
Now, fire up git, add your config files, and sync it across all your computers! Use my post here to do this. The only thing to note here for our purposes is that the makefile is stored under syscfg, and that syscfg is where git should be initialized (with git init). Also, only add the config files and any other files that the config files depend on. An example of a file you should NOT want to add to git is any sort of history file, such as zsh’s history file (specified with the HISTFILE option in zsh’s config file — in our case, syscfg/zsh/cfg), since you’d want different session history files for different machines. Another example would be vim’s session files for the session manager plugin. If you add such temporary history files (or any other file that the application automatically makes changes to), you will make git track these changes (very doable, but utterly worthless)! On the other hand, you’d probably want to add simply script-like files that your configs depend on, such as vim’s various plugins (mere .vim text files in the syscfg/vim directory), or even irssi’s perl plugins, if you use irssi (I don’t use IRC on my laptop, hence it’s exclusion from my sample syscfg and makefile above).
With git taking care of the syncing, you now have complete revision history, as well as guaranteed config file integrity across all of your systems. It’s only a matter of cloning, then simply pushing and pulling for all of your config file syncing needs. Personally, I have it set up so that I just do “sl” to ssh into my laptop (sl is aliased to the unbearably long “ssh username@192.168.0.102″; no password since I have ssh set up that way; again, see my post above to do this), then “d sy[TAB]” (“d” is aliased in my syscfg/zsh/cfg to mean “cd”, and I only have one directory starting with “sys” so I can use zsh’s TAB completion to do the rest), and then “gpl” (extremely shortened form for “git pull” — again, see my post on git to make git accept this instead of “git pull origin master”). Yes, all I do is sl, d sy[TAB], gpl, and my laptop is synced. No more checking/rechecking manually whether certain symlinks exist on my laptop, and whether certain vim plugins already exist on it. I wish I had thought of this sooner, as it would have saved me a lot of time.
Reminders and other tips
- Make sure that all of your crucial config files (like .xinitrc) work properly before implementing this setup! (It’s not fun fixing things in the virtual console a la CTRL+ALT+.)
- Make sure to have per-host settings in each of your config files (or, failing that, have your makefile link intelligently to different config files on a per-host basis)
- In my vim config files above, you’ll notice that I symlink my cfg-gui to .gvimrc. The actual cfg-gui file looks very simple, with a “source ~/.vim/cfg”, and all the gvim-specific commands following that. Make sure to have a check for any autocommands so that they are loaded only once so that gvim works properly. (I must say, I only use vim now, except when I feel like seeing 16+ million colors (GTK) as opposed to 256 (urxvt).)
- The makefile, and its contents, can be scripted in a different programming language if you don’t want to use make. I’ve noticed that some people use ruby to do this. But it could also be python, perl, bash, or any other script.
- Since we’re going to end up putting into syscfg most of our config files, it wouldn’t be a bad idea to add files that don’t actually need syncing (see my note on irssi above). You’d just put an if-statement in your makefile to exclude these files for certain hosts. The benefit to this approach is that, you would end up with ONE git repo for ALL of your config files. Even though I’m not at this stage yet, I feel myself inevitably being pulled toward this path. I want complete revision history for all my /etc/X11/xorg.conf, /etc/fstab, /etc/sudoers, and even /boot/grub/menu.lst files, if it’s possible to do so. It’s probably a security risk to symlink to these destinations (file permissions, which git isn’t good at, at least according to what I heard from Linus’s Google Tech Talk from 2007), but I’m the only human who has access (and cares about) the config files on my desktop/laptop anyway. I’ll update this post if I end up achieving this “one config directory to rule them all” dream.
This guide was prepared with the help of various internet websites (google is your friend), and also especially this site.
