Moved

Hello fans of Joe's Bit Bucket. This site has recently moved to NerdBoys.com. See you there!

Latest stories at NerdBoys.com:

• Case Mod How-To for the Alix Single Board Computer
  

Linux T-Shirts

When I'm not at the office, I live in T-shirts. As such, I go through a lot of t-shirts and I have to replenish my supply regularly. But what's a good geek supposed to do when he can't find one he likes at his local mall? That's right, he makes his own!

I recently used zazzle.com to make some cool Linux T-Shirts for my own use. When me friends saw me wearing them, they wanted to buy them too. So, I released my Linux T-shirts to the public. Check out my Linux T-shirts at my Zazzle store.

You'll notice that many of my Linux T-shirts feature Tux the penguin, the mascot of Linux. Tux was designed by Larry Ewing, using The GIMP.

If you decide to take a whirl at making your own shirts, please be respectful of others' intellectual property and either make your own graphics, use graphics that are in public domain or get the permission of the creator. For example, according to Larry Ewing's website, he grants you permission to use the Tux logo as long as you acknowledge both him and the GIMP.

Stupid sed Tricks

Here are some handy sed one-liners I've come across recently. Maybe someone else will find them handy.

Wrap each line with quotes (quotation marks) - version 1:

sed 's/^[^$]*$/"&"/'

Wrap each line with quotes (quotation marks) - version 2:

sed 's/.*/"&"/'

Replace space with escaped space:

sed 's/ /\\ /g'

How to Find Out Who is Logged On at a Windows Computer

Sometimes it's handy to find out who is logged on at a remote Windows computer. For example, before rebooting a critical server after hours, you might want to make sure your boss isn't logged in at his computer.

In Linux, this is easy. You simply login to the remote computer with ssh (or telnet...yikes) and run the who command. For example:

me@bosscomputer:~$ who
boss   tty7         2008-10-16 07:57 (:0)
me   pts/0        2008-11-14 15:25 (mycomputer)

In Windows, I found two ways of doing this. Both ways work with Windows XP and Windows 2003 (and perhaps other versions too).

The first way uses the psloggedon command, which is part of the pstools suite created by Mark Russinovich. For example:

C:\tools\pstools>psloggedon.exe \\bosscomputer

loggedon v1.33 - See who's logged on
Copyright ⌐ 2000-2006 Mark Russinovich
Sysinternals - www.sysinternals.com

Users logged on locally:
11/14/2008 9:35:31 AM    MYDOMAIN\BOSS

In the example above, we see that my boss is currently logged into BOSSCOMPUTER and he logged in at 11/14/2008 9:35:31 AM.

Often, you'll see error messages like these in the output, which you can ignore:

Error: could not retrieve logon time
NT AUTHORITY\LOCAL SERVICE
Error: could not retrieve logon time
NT AUTHORITY\NETWORK SERVICE

The neat thing about psloggedon is that it tells you not only about a logon from a user sitting in front of the computer but it can also tell you about remote logons to the computer via protocols like Remote Desktop Protocol (RDP). Here is an example of invoking psloggedon on a computer running Citrix Server (like Terminal Services):

C:\tools\pstools>psloggedon.exe \\citrixbox

loggedon v1.33 - See who's logged on
Copyright ⌐ 2000-2006 Mark Russinovich
Sysinternals - www.sysinternals.com

Users logged on locally:
11/14/2008 9:27:15 AM    MYDOMAIN\john
11/14/2008 6:44:56 AM    MYDOMAIN\mary
11/14/2008 6:29:42 AM    MYDOMAIN\steve
11/14/2008 6:34:21 AM    MYDOMAIN\martha

All of the users in the above example were logged in remotely (despite the misleading word "locally" in the output).

If you only care about logins from a user sitting in front of the computer, you can use another method: wmic.exe. For example

C:\tools\pstools>wmic /node:"bosscomputer" ComputerSystem GET UserName
UserName
MYDOMAIN\boss

In the above example, you can see that my boss is logged in at BOSSCOMPUTER. When using the wmic command, I recommend you get in the habit of wrapping the computer name in quotation marks. If you don't, and the name contains a hyphen, you'll see the error "Invalid Global Switch".

If this tip helped you, please leave me a comment or send me an email!

How to Find Uptime in Windows

Sometimes it's useful to find out the uptime for a Windows computer. For example, if you are rebooting a whole bunch of Windows computers and you aren't careful about keeping track of which ones you have just rebooted, you can check the uptime.

In Linux, it's easy to find out how long a computer has been up and running. You just runtime the good ol' uptime command. For example:

me@mycomputer:~$ uptime
09:17:32 up 23 days, 19:13,  3 users,  load average: 0.36, 0.54, 0.49

In Windows, it's almost as easy but not quite as intuitive. The following command should work on Windows XP, Windows 2003 and perhaps other versions: net statistics server. You could also use the shorthand version of that command: net stats srv. For example:

c:\temp>net stats srv
Server Statistics for \\MYCOMPUTER

Statistics since 11/13/2008 1:40 PM

Sessions accepted                  1
Sessions timed-out                 1
Sessions errored-out               1

Kilobytes sent                     3734
Kilobytes received                 491

Mean response time (msec)          0

System errors                      0
Permission violations              0
Password violations                0

Files accessed                     82
Communication devices accessed     0
Print jobs spooled                 0

Times buffers exhausted

Big buffers                      0
Request buffers                  0

The command completed successfully.

The line that starts with "Statistics since", shows when the computer came up. That's not quite the same as uptime but it's close enough. Microsoft just makes you do the math (like they always do).

---

FOLLOW UP:
I found an even easier way getting uptime in Windows. You can use psinfo, which is part of the pstools suite of tools created by Mark Russinovich. For example:

C:\tools\pstools>psinfo \\somecomputer Uptime

PsInfo v1.75 - Local and remote system information viewer
Copyright (C) 2001-2007 Mark Russinovich
Sysinternals - www.sysinternals.com

System information for \\somecomputer:
Uptime:                    0 days 4 hours 7 minutes 57 seconds

---

If this tip helped you, please leave me a comment or send me an email!

How to Create a Tar File That Excludes Hidden Files and Folders

Today I wanted to create a tar file that excludes hidden files and folders -- ones that start with a dot (i.e. '.').

Like a good boy, I read the fine man page (RTFM -- sometimes the F is replaced with a ruder word than "fine") but I couldn't get it to work after several different variations of the exclude pattern. It seems that the exclude pattern that tar expects doesn't follow any of the regular expression (a.k.a. regex) syntaxes that I'm familiar with. Someone enlighten me here if you know what it expects.

Anyway, after lots of unsuccessful googling, I finally found a relevant post. Here's an example of the magical incantation to do this trick (it seems to work in my brief testing using Ubuntu Linux):

tar -cf test.tar foo/ --exclude '.*'

If this tip helped you, please leave me a comment or send me an email!

How to Use sudo tar in a Script Without Password Prompt

If you are rolling your own backup shell script on your Ubuntu Linux box, chances are you might want to use tar or perhaps rsync somewhere in that script. For this example, let's say you've chosen to use tar.

At some point, you will probably want to use cron or some other mechanism to automate your backup. Furthermore, if you want to coordinate the backup of several computers from one central computer, you will probably end up running the backup by making an ssh connection from the central computer to each backup target computer. In that case, the user that is running the backup will probably not be root (unless you allow root logins on your ssh servers) and may therefore have limited privileges.

If that's the case, the user will not be able to backup some files with tar unless you use sudo tar. The problem with this is that sudo will prompt you for a password. If you want to prevent this prompt so that you can totally automate the backup over ssh, you'll need to do two things.

First, you'll need to use ssh private key authentication where the private key has no password. There are lots of tutorials for how to do this. If you're nervous about using a no-password private key, you could use a key ring mechanism instead. That's beyond the scope of this article so just google it.

Second (and this is the main point of this article) you'll need to add a NOPASSWD line for tar in your sudoers file. Using visudo, add the following line to your sudoers file (replace mybackupusername with the name of your backup user):

mybackupusername ALL = NOPASSWD: /bin/tar

That line tells sudo that the backup user can run sudo tar without prompting for a password. If you're paranoid, you can fine-tune that line so it only allows certain options (switches) for tar (you might want to prevent the -x switch). For example:

mybackupusername ALL = NOPASSWD: /bin/tar -czf * -C *

Here's another, slightly different example:

mybackupusername ALL = NOPASSWD: /bin/tar -c*

If this tip helped you, please leave me a comment or send me an email!

Windows DFS Brain Damage

So we just started using Windows' Distributed File System (or DFS for short). More specifically, we have been using the DFS Namespaces portion of DFS, which basically allows you to have one virtual share point to one or more underlying physical shares on any arbitrary physical server.

In theory, this level of indirection makes maintenance easier because on the client side you can map a drive letter to the virtual share, allowing you to change the physical share behind the scenes without any configuration changes on the client. For example, you could have a virtual share called \\myADDomain\corp\files that is configured to point to the physical share \\myFileServer\files. On the client, you can map a drive letter (e.g. Z:) to \\myADDomain\corp\files. Behind the scenes, that gets translated to \\myFileServer\files when the client uses the drive letter.

Today a user got the following error when logging into his Windows XP Pro machine: "An error occurred while reconnecting Z: to \\myADDomain\corp\files. Microsoft Windows Network: The system cannot find the file specified. The connection has not been restored."

My first instinct was to reboot the client, as suggested in many DFS troubleshooting tips. So, I tried rebooting but upon logging in, the user got the new error: "Z:\ refers to a location that is unavailable."

Sigh...

In DFS jargon, \\myADDomain\corp is called a DFS root and the files portion of \\myADDomain\corp\files is called a link. To make things more complicated, the DFS root points to these things called Root Targets, for example, \\myDC1\corp and \\myDC2\corp. Essentially, there are two levels of indirection here. \\myADDomain\corp points \\myDC1\corp (and \\myDC2\corp) and \\myDC1\corp\files (and \\myDC2\corp\files) points to \\myFileServer\files. I told you it was complicated.

The temporary workaround in this case was to map Z: drive directly to \\myFileServer\files. Of course, this renders DFS useless so I had to find a permanent fix. Below are the steps I followed:

1. Opened the DFS (Distributed File System) admininstrative tool on myDC1.
2. Noticed that \\corp.myADDomain.com\CORP (alias \\myADDomain\corp) DFS root was pointing to 2 root targets: \\myDC1\corp and \\myDC2\corp.
3. Right-clicked on each root target and clicked Check Status. Each one got a green check mark after the status check. This is good.
4. Noticed that the DFS root had the following DFS link: files.
5. Right-clicked on the link and clicked Check Status. It got a red X after the status check. This is BAD.
6. I then ran the DFS admin tool on myDC2. When I checked the status of root targets and link on this computer, ALL got green checkmarks. This is good.
7. On both myDC1 and myDC2, I ran Windows Explorer and opened the folder that maps to the CORP share. For example, the root target \\myDC1\CORP maps to C:\dfsroot on myDC1. Similarly, the root target \\myDC2\CORP maps to C:\dfsroot on myDC2.
8. In the C:\dfsroot folder on myDC1, the only child folder I saw was a hidden one called DO_NOT_REMOVE_NtFrs_PreInstall_Directory. However, on myDC2, the folder had the files child folder in addition to the funky hidden one.
9. AFAIK, you cannot manually recreate the missing child folder on myDC1 with Explorer because they it does not seem to be a real folder (right click to see the properties and you will see what I mean). In other words, it is a "magical" pseudo-folder.
10. Instead, in the DFS admin tool on myDC1, I right clicked on the \\myDC1\corp root target and clicked Remove Target.
11. Then I recreated the \\myDC1\corp root target by right clicking the DFS root (\\corp.myADDomain.com\CORP) and clicking New Root Target. For the server name I entered myDC1.
12. Then, magically, the files child folder in C:\dfsroot on myDC1 reappeared!
13. After that, the user logged out, logged in and Z: drive got mapped correctly. Woot!

How did myDC1 get into the bad state in the first place? I don't know. Go ask Microsoft...

---

FOLLOW UP (November 14, 2008): This has happened two more times since writing this article. It seems to happen whenever we reboot one of the domain controllers that is hosting a Root Target. I'm not sure how to prevent the problem but at least the workaround described above always fixes it. Sigh...

---

If this tip helped you, please leave me a comment or send me an email!

How To See the Members of a Group in Linux

As far as I know, there is no standard Linux command to find out the users that are members of a given group. Intuitively, you would think there would be a command like members that takes the group name as a parameter. Unfortunately, it seems that no such command exists. In lieu of this command, you can simply grep the file /etc/group like this:

grep "^thegroupname:" /etc/group

Obviously, you need to replace thegroupname with the group you are searching for. The character '^' means "beginning of line". The character ':' is the field separator between the group name and the group password in the group file.

If this tip helped you, please leave me a comment or send me an email!

How to Find Out Which Version of Linux You Are Running

This is a brief howto for finding out which version of Linux you are running. I say "brief" because this is certainly not comprehensive. There are a few ways to skin this cat, depending on which Linux distribution you are running. I will only cover Debian and Ubuntu. Similar techniques may work on other distros (especially Debian derivatives) but your mileage may vary.

But first, why should you even care which version you are running? There are many reasons why you should care, but perhaps the best reason is that it comes in handy when you are troubleshooting errors. For example, if you are a Linux newbie and you post an error message on a forum asking for help troubleshooting it, you can bet that the first response will include the question: "Which version of Linux are you running?". Knowing the version can sometimes also come in handy when you are installing new packages when the package is dependent on a certain version of Linux.

Note that the question "Which version of Linux are you running?" is actually sort of ambiguous. Sometimes you want to know the version of the Linux kernel and other times you want to know the version of the Linux distribution.

To get the version of the Linux kernel, you use the uname command, which I believe should be available on all distributions. For example:

me@mycomputer:~$ uname -r
2.6.18-5-686

In the above example, we see that the kernel version is 2.6.18-5-686. You can also use the -a option to get even more information:

me@mycomputer:~$ uname -a
Linux mycomputer 2.6.18-5-686 #1 SMP Fri Jun 1 00:47:00 UTC 2007 i686 GNU/Linux

To find out the version of the Linux distribution, there are a few ways, depending on the distro. In Ubuntu, version information is stored in the files /etc/issue and /etc/lsb-release. You can simply cat these files to see the information. Here is what I found on an Ubuntu 7.10 box:

me@mycomputer:~$ cat /etc/issue
Ubuntu 7.10 \n \l

me@mycomputer:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=7.10
DISTRIB_CODENAME=gutsy
DISTRIB_DESCRIPTION="Ubuntu 7.10"

On Debian and Debian derivatives (e.g. Ubuntu), version information is stored in the file /etc/debian_version. Here is what the file contained on a Debian 4.0 box:

me@mycomputer:~$  cat /etc/debian_version
4.0

And here is what the file contained on an Ubuntu 7.10 box:

me@mycomputer:~$  cat /etc/debian_version
lenny/sid

I did some googling and found that some other distros follow this /etc/*version convention. For example, apparently Red Hat stores version infomation in /etc/redhat-version. I don't have a Red Hat system lying around (I haven't had one since they split into Red Hat and Fedora) so I couldn't try it. Therefore, I'll just take it as a matter of faith that this is true. On other distros, chances are that you'll have a *version file in your etc directory.

Finally, another way you can find out the distro version is with the lsb_release command. Here is the output from an Ubuntu 7.10 box:

me@mycomputer:~$  lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 7.10
Release:        7.10
Codename:       gutsy

And here is the output from a Debian 4.0 box:

me@mycomputer:~$  lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 4.0r1 (etch)
Release:        4.0r1
Codename:       etch

I'm not sure if that command is available on all distros or just Debian and its derivatives. For other distros, I leave that as an exercise to the reader. If the command is not installed on your Debian or Debian derivative box, you can install it with: apt-get install lsb-release. Note that you might need to su or sudo before installing it. Note also that the package name has a hyphen ("-") whereas the command has an underscore ("_"). That difference got me the first time.

If this tip helped you, please leave me a comment or send me an email!

Bart's Boot CD with Broadcom NetXtreme 57XX Gigabit Controller

Are you having problems getting Bart's Boot CD to recognize the Broadcom NetXtreme 57xx Gigabit Controller network interface card?

I work with some Dell computers (models: Optiplex SX280, Optiplex GX620, Precision 690 and Precision 370) that have this NIC and the current version of Bart's Boot CD is unable to recognize it. The automatic NIC detection fails with a message to the effect of "could not find any device, you can try manually selecting a driver" (sorry, I can't remember the exact error).

Naturally, I looked on Bart's site to see if there was a new driver cab file that would fix this problem. There I found b57.cab, which had the description "Broadcom NetXtreme Gigabit Ethernet NDIS2 Driver v7.03 (031002)". That seemed like it would be the right one so I put it on my boot disk and tried it. Again, auto-detection failed.

I tried manually loading the driver in b57.cab but I got "Error 7306: The driver failed to initialize." I tried manually loading some of the other drivers on the disk but I got "Error 7321: Network-card drivers failed to load."

The good news is that I figured out how to fix this issue, after doing some research. Essentially, I made a custom version of b57.cab. Read on for more details. Note that you could apply these techniques for making custom versions of any of Bart's network driver cabs.

First, let me explain how (I think) the auto-detection works. Each of the network driver cabs includes a file called ndis.pci. Here are the contents of ndis.pci from the latest released version of b57.cab (ver 1.6):

ret="B57"
ven=14E4 "Broadcom"
dev=1645 "NetXtreme Gigabit Ethernet (BCM5701)"
1644 "NetXtreme Gigabit Etherent (BCM5700/5401)"
16A7 "NetXtreme Gigabit Ethernet (BCM5703X)"
165D "NetXtreme Gigabit Ethernet (BCM5705M 10/100/1000)"
165E "NetXtreme Gigabit Ethernet (BCM5705M 10/100/1000)"
166D "NetXtreme Gigabit Ethernet (BCM5705MFE 10/100)"
16A6 "NetXtreme Gigabit Ethernet (BCM5702 10/100/1000)"
164D "NetXtreme Gigabit Ethernet (BCM5702FE 10/100)"
1648 "NetXtreme Gigabit Ethernet (BCM5704C 10/100/1000)"
1696 "NetXtreme Gigabit Ethernet (BCM5782 10/100/1000)"

"Ven" stands for "vendor id", which identifies the manufacturer of the card. "dev" stands for "device ID", which identifies the model or chip set of the card. Operating systems use the vendor and device IDs to implement plug-and-play for PCI devices. When plug and play works, the operating system can automatically load the correct drivers for a device.

In this case, the manufacturer is Broadcom (vendor id 14E4) and all of the listed devices seem to be NetXtreme 57xx Gigabit devices of some sort. So why did autodetection fail? A clue is to look at the revision history of b57.cab. For example, look at this note: "v1.2 Updated by Oivind Pettersen from IBM, added dev=1644". Hmm, perhaps I need to add a new device ID to the list but how do I know which one to add?

To answer that question, I loaded Windows on a Dell Precision 690 and started poking around in Control Panel. I found the information I needed by drilling down to Control Panel | System | Hardware | Device Manager | Network Adapters | Broadcom NetXtreme 57XX Gigabit Controller | Details. There I found the string "PCI\VEN_14E4&DEV_1600&SUBSYS_01C01028&REV_02\4&F667E4F&0&00E0".

In that string, you can clearly see the vendor id, 14E4, and the device id, 1600. If you examine the device id list above, you'll note that 1600 is missing. Therefore, I added it. Similarly, for the Dell Optiplex SX280 and GX620, I had to add the device id 1677. I can't remember what the device id was for the Precision 370 but it was either one of the old ones of one of the two new ones.

After making the changes, my new ndis.pci file looked like this:

ret="B57"
ven=14E4 "Broadcom"
dev=1600 "NetXtreme Gigabit Ethernet (BCM57xx)"
1645 "NetXtreme Gigabit Ethernet (BCM5701)"
1644 "NetXtreme Gigabit Etherent (BCM5700/5401)"
16A7 "NetXtreme Gigabit Ethernet (BCM5703X)"
165D "NetXtreme Gigabit Ethernet (BCM5705M 10/100/1000)"
165E "NetXtreme Gigabit Ethernet (BCM5705M 10/100/1000)"
166D "NetXtreme Gigabit Ethernet (BCM5705MFE 10/100)"
1677 "NetXtreme Gigabit Ethernet (BCM57xx)"
16A6 "NetXtreme Gigabit Ethernet (BCM5702 10/100/1000)"
164D "NetXtreme Gigabit Ethernet (BCM5702FE 10/100)"
1648 "NetXtreme Gigabit Ethernet (BCM5704C 10/100/1000)"
1696 "NetXtreme Gigabit Ethernet (BCM5782 10/100/1000)"

After changing ndis.pci, I rebuilt b57.cab with CabPack (any cab building software would suffice).

To get the new cab into your boot disk, you need to do the following:

1. Put the cab in the folder bart\bcd\cabs\drivers\ndis
2. cd to the folder bart\bcd
3. build the boot sector: bcd -bab
4. create the ISO file: bcd -b corpmb
5. burn the ISO file to CD using Nero or other burning software

After building a new boot disk I tried it. This time, the card was automatically detected and the system tried to load the DOS network driver in b57.cab. Unfortunately, I got "Error 7306: The driver failed to initialize."

What else could be wrong? Perhaps I had an old version of the DOS driver? Sure enough, I found a newer version of the NetXtreme 57xx DOS NDIS2 drivers on Broadcom's website. The version number was 10.4.6 (I think the lastest version on Bart's site was 7.03).

To upgrade the driver, I simply replaced the B57.dos in b57.cab with the B57.dos included in the download from Broadcom. Then I followed the previously mentioned steps to rebuild the boot disk. This time, auto-detection worked AND the driver loaded correctly! Woot!

Using these steps, anyone should be able to quickly build a custom b57.cab. However, if you're technically challenged (or just plain lazy), you can try sending me a polite email request for the cab. If I'm in a good mood, I might just respond to your request.

Note that you could also use these steps to upgrade the cab files for other network drivers. Just keep these tips in mind:

1. If auto-detection fails, you first must find the cab file that contains an ndis.pci with your card's vendor id. If you find the vendor id, you will need to add a device id to ndis.pci. If you don't find the vendor id in any of the cabs, you'll need to make a brand new cab file. I don't have any specific instructions on that but you should be able to figure it out from my notes above.
2. If you get "Error 7321: Network-card drivers failed to load", you probably have the wrong DOS driver for your card. For example, you tried to use a Broadcom driver for an Intel card. Try a different driver.
3. If you get "Error 7306: The driver failed to initialize", you probably have the right driver but it's too old. Try a newer version of that driver.

If this tip helped you, please leave me a comment!

Book Review: Head First HTML with CSS & XHTML

First and foremost, I am a persistence layer software developer. I don't do much UI work and what little UI work I do usually involves rich client UIs. I rarely do any web UI work.

Having said that, I think it's important to know at least a little bit about web UI technologies, even if you are a back-end developer. Why? Well, at the very least, you will be able to understand the vocabulary of the UI guys you work with. Furthermore, you never know when you might have to look for a new job and there's a good chance they'll want you to know about some of that web UI stuff.

With that in mind, I recently picked up a copy of Head First HTML with CSS & XHTML to see if I could learn a thing or two. I just finished the book and I wanted to share my thoughts on it. Please read on.

There are a lot of HTML books out there so you might be wondering why I got this particular HTML book . One reason is that it got a rating of 4.7 out of 5 stars from 193 reviewers on Amazon.com. That's a pretty good score! Another reason is because I've skimmed some of the other books in the Head First series and I think they are pretty darn fun to read.

So what do I think of this book? It's fantastic! It's fun to read, engaging, informative and full of humor. It has numerous interesting activities and exercises to reinforce learning. It has lots of diagrams, photos and colors. I has plenty of jokes. And, on top of all that, the authors seem to really know their stuff.

If you read this book, you will learn the fundamentals of HTML, XHTML and CSS. You will learn it quickly and you won't feel like you're being tortured. Most importantly, you will have fun.

No book is perfect however. My biggest quibble is the horrible index. I had a hard time finding anything and I had to keep on adding my own items to the index with a pencil. You CANNOT expect to use this book as a reference. It should be viewed strictly as a tutorial.

Also, by no means is this book a comprehensive resource on CSS. For example, it didn't even talk about the min-width property for setting the minimum width of your page. Lots of sites use this feature. I can't believe the authors didn't mention it in their chapter on layouts.

The authors also spend quite a bit of time modularizing headers, footers and sidebars but they neglect to mention that it won't help you that much if you are have a purely static HTML website, where you would have to go through much pain to make any changes to them (how much fun would it be to make the same change in 100 pages?). I feel the authors should have at mentioned this problem and recommended that you investigate server-side-includes, PHP, JSP or ASP as a solution to this problem.

Contrary to what the book's cover says, I don't think you'll "launch your web career in one chapter". However, this book is definitely an excellent starting point for anyone who is new to these technologies.

Pros: Extremely fun to read. Teaches most of the fundamentals. Teaches standards compliance.
Cons: Horrible index.
Rating: 4.5/5

How to Copy a File and Get a Progress Bar, in Linux

Today I had to copy a really big file (in this case, a Ghost image) from a Linux box to a Windows box, using Samba. My first instinct was to use the cp command. The problem with cp is that it does not show you any sort of progress (I couldn't find any relevant option in the cp man page). What I wanted was a progress bar display like the one wget displays.

I did some googling and found a few decent candidates:

• Lamby's "can you get cp to give a progress bar like wget?" bash script (cp_p)
• Midnight Commander (mc)
• Last File Manager (lfm)
• Pabix's "cp with progress bar" bash script (gcp)
• Pipe Viewer (pv)

While all of them might work, the only one I tried was pv and I loved it. It did exactly what I wanted. The result was sort of like "cp with a progress bar".

Under Debian Linux (sarge version) pv was easy enough to install: apt-get install pv.

After reading the pv man page, I was still a little unclear on exactly how to use it to copy files (yes, I guess I'm dumb). However, after a little bit of experimentation, I figured it out:

me@mycomputer:/home/ghost$ pv bigfile > /mnt/ghost_on_windows_box/bigfile

The source file is bigfile, in the current directory. The target file is bigfile, on a share called /mnt/ghost_on_windows_box. Obviously, the share is a SMB share on a Windows box. On the Linux box, I mounted that share with the CIFS file system (via an entry in /etc/fstab).

Here is what the output looked like, at three different stages:

205MB 0:00:17 [8.03MB/s] [=======>                           ] 25% ETA 0:00:49
436MB 0:00:31 [40.7MB/s] [=================>                 ] 54% ETA 0:00:25
801MB 0:00:44 [  18MB/s] [=================================>] 100% ETA 0:00:00

Just confirm that the copy worked, I ran md5sum on both the source file and the target file. It indeed worked.

One minor problem I noticed, however, is that it took several seconds to get my command prompt back after pv showed the copy was 100% complete. After some investigation, it seems that it wasn't pv's fault. Rather, it had something to do with Samba. I'm not even sure if it was the Linux Samba client's fault or the Windows SMB server's fault.

I confirmed that it was not a pv problem by copying the file to a local folder on the Linux box. In that case, pv returned control to the shell immediately after the progress bar went to 100%.

As a workaround for this Samba copying problem, I added an extra command to the original command:

me@mycomputer:/home/ghost$ pv bigfile > /mnt/ghost_on_windows_box/bigfile; echo ***DONE***

Now, several seconds (or even a minute or so) after the progress bar reaches 100%, I get the message ***DONE*** in the console. Then I know it is really done.

For my needs, pv works great.

If this tip helped you, please leave me a comment!

State Pattern Persistence with Hibernate

In my software development job, I do a lot of persistence layer implementation, especially using the Hibernate O/R mapper. I work closely with a clever domain layer programmer who tends to use a lot of design patterns, perhaps because he likes the woman of the cover of Head First Design Patterns.

Sometimes, these design patterns create a bit of a challenge for me in the persistence layer, especially since I always tell my domain layer programmer that I can persist things transparently, without having to muck up his code with persistence layer artifacts.

A case in point is the State pattern. I have now persisted 5 different sets of states so it's no longer a challenge. However, it took me a little while to figure how to do it. Naturally, I started by checking out Hibernate's forums but much to my chagrin, I found several people asking the same question but no one offering an answer. Therefore, I will provide an answer in this blog post. Read on!

Imagine you have a Customer class, which is playing the "context" role in the design pattern. The customer has a simple state machine with two states: active and inactive. I've implemented State pattern persistence in two ways: without an enumeration and with an enumeration. First, here is a sample implementation of the pattern in the domain layer, without using an enumeration:

public interface CustomerState
{
  /**
   * Checks whether the CustomerState is 'Active'.
   * @return true if the CustomerState is 'Active'.
   */
  boolean isActive();

  /**
   * Checks whether the CustomerState is 'Inactive'.
   * @return true if the CustomerState is 'Inactive'.
   */
  boolean isInactive();

  /**
   * Activates the specified Customer.
   * @param customer the Customer being activated.
   */
  void activate(Customer customer);

  /**
   * Deactivates the specified Customer.
   * @param customer the Customer being deactivated.
   */
  void deactivate(Customer customer);
}

/**
* All concrete subclasses of this class are stateless (they have no instance
* variables). Therefore, we have overridden equals and hashCode (see below)
* because any instance of a particular subclass should be considered equal to
* any other instance of that sublcass.
*/
public abstract class AbstractCustomerState implements CustomerState
{
 public static final CustomerState ACTIVE = new CustomerStateActive();
 public static final CustomerState INACTIVE = new CustomerStateInactive();

 public boolean isActive()
 {
   return false;
 }

 public boolean isInactive()
 {
   return false;
 }

 public void activate(Customer customer)
 {
   throw new UnsupportedOperationException();
 }

 public void deactivate(Customer customer)
 {
   throw new UnsupportedOperationException();
 }

 /**
  * Returns true if the Object's class name matches this Object's class name.
  */
 public final boolean equals(Object obj)
 {
   if (this == obj)
   {
     return true;
   }
   if (obj == null)
   {
     return false;
   }
   return getClass().getName().equals(obj.getClass().getName());
 }

 /**
  * Returns the hashCode of the Object's Class name because all instances of
  * this Class are equal.
  */
 public final int hashCode()
 {
   return getClass().getName().hashCode();
 } 
}

public class CustomerStateActive extends AbstractCustomerState
{
 public boolean isActive()
 {
   return true;
 }

 public void deactivate(Customer customer)
 {
   // do some checks to make sure it's okay to deactivate this customer
   // ...

   // set the state
   ((AbstractCustomer) customer).setState(AbstractCustomerState.INACTIVE);
 }
}

public class CustomerStateInactive extends AbstractCustomerState
{
 public boolean isInactive()
 {
   return true;
 }

 public void activate(Customer customer)
 {
   // do some checks to make sure it's okay to activate this customer
   // ...

   // set the state
   ((AbstractCustomer) customer).setState(AbstractCustomerState.ACTIVE);
 }
}

Notice in my example that I have made the states stateless. Yes, I realized that a stateless state seems like an oxymoron but it's true; the above state classes do not have any instance variables. Since they are stateless, any instance of a state is equal to any other instance of that same state. That's why I overrode the equals() and hashCode() methods. Since the states are stateless, I also created constants for each state (ACTIVE and INACTIVE) in the abstract base class so that we don't have to keep on instantiating them all over the place.

Anyway, the code above is what the domain layer developer would hand over to me. Then it would be my job to persist it. The first task is to make a Hibernate UserType implementation:

public class CustomerStateUserType implements UserType
{
 public static final int ACTIVE_STATE_DB_VALUE = 1;
 public static final int INACTIVE_STATE_DB_VALUE = 2;

 public int[] sqlTypes()
 {
   return new int[] {Types.INTEGER};
 }

 public Class returnedClass()
 {
   return CustomerState.class;
 }

 public boolean isMutable()
 {
   // CustomerStates are stateless, therefore they are clearly immutable
   return false;
 }

 public Object deepCopy(Object value) throws HibernateException
 {
   // CustomerStates are immutable
   return value;
 }

 public Serializable disassemble(Object value) throws HibernateException
 {
   // CustomerStates are immutable
   return (Serializable) value;
 }

 public Object assemble(Serializable cached, Object owner)
   throws HibernateException
 {
   // CustomerStates are immutable
   return cached;
 }

 public Object replace(Object original, Object target, Object owner)
   throws HibernateException
 {
   // CustomerStates are immutable
   return original;
 }

 public boolean equals(Object x, Object y)
   throws HibernateException
 {
   if (x == y)
   {
     return true;
   }
   if (x == null || y == null)
   {
     return false;
   }
   return x.equals(y);
 }

 public int hashCode(Object x) throws HibernateException
 {
   return x.hashCode();
 }

 public Object nullSafeGet(ResultSet rs, String[] names, Object owner)
   throws HibernateException, SQLException
 {
   final int stateValue = rs.getInt(names[0]);
   if (rs.wasNull())
   {
     return null;
   }

   CustomerState state = null;
   switch (stateValue)
   {
     case ACTIVE_STATE_DB_VALUE:
       state = AbstractCustomerState.ACTIVE;
       break;
     case INACTIVE_STATE_DB_VALUE:
       state = AbstractCustomerState.INACTIVE;
       break;
     default:
       throw new RuntimeException("Unknown CustomerState value ["
         + stateValue + "]");
   }
   return state;
 }

 public void nullSafeSet(PreparedStatement st, Object value, int index)
   throws HibernateException, SQLException
 {
   if (value == null)
   {
     st.setNull(index, Types.INTEGER);
   }
   else
   {
     if (value.equals(AbstractCustomerState.ACTIVE))
     {
       st.setInt(index, ACTIVE_STATE_DB_VALUE);
     }
     else if (value.equals(AbstractCustomerState.INACTIVE))
     {
       st.setInt(index, INACTIVE_STATE_DB_VALUE);
     }
     else
     {
       throw new RuntimeException("Unknown CustomerState [" + value + "]");
     }
   }
 }
}

As you can see, most of the interesting code is in nullSafeGet() and nullSafeSet(), which map the states to an integer column in the database. Next, you need to map it:

<hibernate-mapping>
<class name="Customer" table="customer">
 <property name="state" column="state" type="CustomerStateUserType"
   access="field"/>
 <!-- ...other properties -->
</class>
</hibernate-mapping>

Pretty easy, huh? The only problem with this implementation is that if the domain layer developer subsequently adds another state, I have to modify my UserType to account for the new state. If the domain layer developer forgets to tell me about the new state, it could be awhile before the error is discovered. Of course, one way around this is to make nullSafeGet and nullSafeSet map each state to the state's class name. For example:

public class CustomerStateUserType implements UserType
{

 public int[] sqlTypes()
 {
   return new int[] {Types.VARCHAR};
 }

 //...other methods

 public Object nullSafeGet(ResultSet rs, String[] names, Object owner)
   throws HibernateException, SQLException
 {
   final String stateValue = rs.getString(names[0]);
   if (rs.wasNull())
   {
     return null;
   }

   CustomerState state = null;

   try
   {
     state = (CustomerState) Class.forName(stateValue).newInstance();
   }
   catch (Exception e)
   {
     throw new RuntimeException("Can't instantiate instance of ["
       + stateValue + "]", e);
   }

   if (!(state instanceof CustomerState))
   {
     throw new RuntimeException("Unknown CustomerState value ["
       + stateValue + "]");
   }

   return state;
 }

 public void nullSafeSet(PreparedStatement st, Object value, int index)
   throws HibernateException, SQLException
 {
   if (value == null)
   {
     st.setNull(index, Types.VARCHAR);
   }
   else
   {
     if (value instanceof AbstractCustomerState)
     {
       st.setString(index, value.getClass().getName());
     }
     else
     {
       throw new RuntimeException("Unknown CustomerState ["
         + value + "]");
     }
   }
 }
}

Of course, the problem with storing the class name in the database column is that it takes up more space than an integer; consider the relative cost of storing a million records with an integer state value versus a value like "com.mycompany.someapp.domain.customer.CustomerStateActive".

The second style of implementing this pattern involves using an enumeration, specifically Apache's Jakarta-commons Enum class (of course, you could probably use Java 1.5's new enum feature too). Here are the relevant domain layer classes:

import org.apache.commons.lang.enums.Enum;

public abstract class AbstractCustomerState extends Enum implements CustomerState
{
 public static final CustomerState ACTIVE = new CustomerStateActive();
 public static final CustomerState INACTIVE = new CustomerStateInactive();

 protected AbstractCustomerState(String name)
 {
   super(name);
 }

 /**
  * Overrides superclass method to allow enums to be subclasses, yet still be
  * part of the same enumeration. This is what allows an enum to play the role
  * of a State in the State design pattern. The Javadoc comments for Enum call
  * this type of Enum a "Functional Enum".
  * @see org.apache.commons.lang.enums.Enum#getEnumClass()
  * @see org.apache.commons.lang.enums.Enum
  */
 public final Class getEnumClass()
 {
   return AbstractCustomerState.class;
 }

 public static AbstractCustomerState getEnum(String name)
 {
   return (AbstractCustomerState) getEnum(
     AbstractCustomerState.class, name);
 }

 public static Map getEnumMap()
 {
   return getEnumMap(AbstractCustomerState.class);
 }

 public static List getEnumList()
 {
   return getEnumList(AbstractCustomerState.class);
 }

 public static Iterator iterator()
 {
   return iterator(AbstractCustomerState.class);
 }

 //----------------start of CustomerState methods---------------------
 public boolean isActive()
 {
   return false;
 }

 public boolean isInactive()
 {
   return false;
 }

 public void activate(Customer customer)
 {
   throw new UnsupportedOperationException();
 }

 public void deactivate(Customer customer)
 {
   throw new UnsupportedOperationException();
 }
}

public class CustomerStateActive extends AbstractCustomerState
{
 CustomerStateActive()
 {
   super("ACTIVE");
 }

 public boolean isActive()
 {
   return true;
 }

 public void deactivate(Customer customer)
 {
   // do some checks to make sure it's okay to deactivate this customer
   // ...

   // set the state
   ((AbstractCustomer) customer).setState(AbstractCustomerState.INACTIVE);
 }
}

public class CustomerStateInactive extends AbstractCustomerState
{
 CustomerStateInactive()
 {
   super("INACTIVE");
 }

 public boolean isInactive()
 {
   return true;
 }

 public void activate(Customer customer)
 {
   // do some checks to make sure it's okay to activate this customer
   // ...

   // set the state
   ((AbstractCustomer) customer).setState(AbstractCustomerState.ACTIVE);
 }
}

There is a tradeoff with this implementation. Notice the extra code at the top of AbstractCustomerState? I must override several methods of the Enum class, as per the "Functional Enums" section of Enum's JavaDocs. For the UserType, I started by making an abstract base class for all enumerations:

/**
* Maps a org.apache.commons.lang.enums.Enum to a Hibernate type. This
* allows you to persist Enum instances. This class was adapted from an
* example posted in this forum
* thread  and an example on page 210 of "Hibernate in Action" (first
* edition).
*/
public abstract class EnumUserType implements UserType
{
 private Class enumClass;

 public EnumUserType(Class enumClass)
 {
   this.enumClass = enumClass;
 }

 public int[] sqlTypes()
 {
   return new int[] {
     Types.VARCHAR
   };
 }

 public Class returnedClass()
 {
   return enumClass;
 }

 public boolean equals(Object x, Object y) throws HibernateException
 {
   if (x == y)
   {
     return true;
   }

   if (x == null || y == null)
   {
     return false;
   }

   return Hibernate.STRING.isEqual(x, y);
 }

 public Object nullSafeGet(ResultSet rs, String[] names, Object owner)
   throws HibernateException, SQLException
 {
   String value = rs.getString(names[0]);
   return rs.wasNull() ? null : EnumUtils.getEnum(enumClass, value);
 }

 public void nullSafeSet(PreparedStatement st, Object value, int index)
   throws HibernateException, SQLException
 {
   // make sure the received value is of the right type
   if ((value != null) && !returnedClass().isAssignableFrom(value.getClass()))
   {
     throw new IllegalArgumentException("Received value is not a["
       + returnedClass().getName() + "] but [" + value.getClass() + "]");
   }

   if (value != null)
   {
     Enum enum = (Enum) value;
      
     // convert the enum into its persistence format
     String enumName = enum.getName();

     // set the value into the resultset
     st.setString(index, enumName);
   }
   else
   {
     st.setNull(index, Types.VARCHAR);
   }
 }

 public Object deepCopy(Object value) throws HibernateException
 {
   // Enums are immutable - nothing to be done to deeply clone it
   return value;
 }

 public boolean isMutable()
 {
   // Enums are immutable
   return false;
 }

 public int hashCode(Object object) throws HibernateException
 {
   return object.hashCode();
 }

 public Serializable disassemble(Object value)
   throws HibernateException
 {
   return (Serializable) value;
 }

 public Object assemble(Serializable cached, Object owner)
   throws HibernateException
 {
   return cached;
 }

 public Object replace(Object original, Object target, Object owner)
   throws HibernateException
 {
   return original;
 }
}

Then, I extended that base class to create a CustomerStateUserType.

public class CustomerStateUserType extends EnumUserType
{
 public CustomerStateUserType()
 {
   super(AbstractCustomerState.class);
 }
}

Notice that the derived UserType is a piece of cake to make.

If this tip helped you, please leave me a comment!

Stupid Cron Tricks

Have you ever wanted to make cron run a scheduled task on some weird interval like every 29 days? I was faced with that problem just the other day.

I was starting to get annoyed with DynDNS's nag emails, asking me to either log in and "touch" my WRT54GL router's host record every 30 days or to upgrade to a paid service that does not require a periodic touch.

Of course, you can create a cron job that calls programs like ez-ipupdate to automate this periodic touching. The problem is, even though cron is quite flexible, it does not support intervals of 29 days.

Why 29 days? If you touch the record any earlier than 28 days since the previous touch, DynDNS may flag you for abuse and disable your host record (in practice, I have been able to touch the record a handful of times within 28 days without being flagged, but why take a chance?). On the other hand, if you don't touch the host record within 30 days, you get a nag email (and if you don't touch it within 35 days, the record is deleted by DynDNS altogether). So, I figured that 29 days would be a safe value to use.

I thought I had solved the problem with the technique I described in this article. It did automatically touch the record during within the 28 to 35 day window but since it was based on a monthly cron job, I still got nag emails during 31-day months because 31 is greater than the 30 day nag email threshold.

So how did I solve the problem the second time around? I based my solution on Stephan Sokolow's Crontab Timing Tricks article, but with a few minor changes. Here is the script that does most of the work:

#!/bin/sh
#
# Runs ez-ipupdate periodically (every $INTERVAL days).
# To automate this, call this script from a daily cron job.

let INTERVAL=29
let upSeconds=`cat /proc/uptime | cut -f1 -d.`
let days=$((${upSeconds}/86400))

echo `date`
echo interval = ${INTERVAL}
echo uptime in days = ${days}

if [ $days -lt $INTERVAL ]; then
  echo uptime less than interval, so no touch needed
elif [ $(( $days % $INTERVAL )) -ne 0 ]; then
  echo uptime is not a multiple of interval, so no touch needed
else
  echo uptime is a multiple of interval, a touch is needed
  echo doing touch
  rm -f /tmp/ez-ipupdate.cache
  /usr/sbin/ez-ipupdate -c /etc/ez-ipupdate.conf
fi

As you can see, ez-ipupdate is only called every 29 days, based on the system uptime.

Next, I created a cron job to automatically call the script daily:

1 0 * * * /usr/sbin/periodic-ez-ipupdate > /tmp/periodic-ez-ipupdate-cron.log 2>&1

Ya, It's pretty cheesy but it works.

One caveat with my solution is that if the router is rebooted sooner than 29 days since the last reboot, my periodic-ez-ipupdate script will not be called. That's okay though because the stock configuration of ez-ipupdate in OpenWrt White Russian RC6 automatically calls ez-ipupdate upon reboot. Note, however, if you frequently reboot your router, you will be flagged for abuse for sending too many unnecessary updates to DynDNS. Fortunately, I go several months between reboots of my router.

An obvious solution to the reboot problem would be to store the date of the last touch in a file and compare that to the current system date to see if 29 days had passed. The is a great solution if your router has a hard drive. However, the WRT54GL only has flash memory, which has a finite number of write cycles. Many manufacturers will tell you that you have at least 10,000 write cycles. But I have read that some chips can give you as little as 1,000 write cycles, so why take a chance? That's why I used the uptime-based solution.

If this tip helped you, please leave me a comment!

A Virtual Visit from Paul Wouters

Soon after posting yesterday's blog entry, How To: VPN Between RV082 (or RV042) and WRT54GL (or WRT54G), I received a comment from Paul Wouters:

Use dpdaction=restart

btw yout ike/ipsec lifetimes are insanely short. you should not do that. leave them default, and the shortest one of the other device will be used.

Paul, if you're reading this, thanks for the tips!

I wasn't sure how I could have missed that useful dpdaction=restart setting so I went back and checked the ipsec.conf man page this morning. Sure enough, the dpdaction=restart setting was missing from the man page. That's why I missed it! See! I did RTFM!

Anyway, I did some googling to find out more about dpdaction=restart and I came across this Openswan mailing list message, authored by none other than Paul Wouters:

On Thu, 10 Nov 2005, Duncan Reed wrote:

> I was interested in seeing how the new(ish) dpdaction=restart option
> differed from clear and hold. Looking through the ipsec.conf man and
> other docs they don't appear to have been updated yet.
>
> How does restart differ from hold? And in what situations would I use
> restart instead of the other options.

clear: tear down the broken tunnel. Allow cleartext pckets to the destination
hold: tear down the broken tunnel. Disallow cleartext packets, wait for the remote to re-estbalished a new IPsec SA (responder)
restart: tear down the broken tunnel. Disallow cleartext packets, attempt to restart the IPsec tunnel (initiator)

clear is mostly used for roadwarriors. You likely won't be talking to that remote IP again, as the roadwarrior might show up elsewhere.

hold is for a connection, usually with a remote subnet defined, that vanished but is on dynamic IP. you cannot restart it, but you also want to prevent
sending cleartext packets for the defined remote subnet.

restart is for a connection to a remote static ip, which you can initiate yourself.

Compare it to rekey=no. If you have right=%any, you need to have rekey=no, and you cannot use dpdaction=restart. If this was a single roadwarrior, you clear, if it was for a subnet, you want to hold it.

Paul

I love finding out about undocumented settings! Hehe!

Now, as for my "insanely short" values for keylife and ikelifetime, where did I come up with those? Well, when I was trying to connect the RV082 to the WRT54G, I figured that if I had any problems with bad settings, the RV082 side would be the pickiest. So, I tried to stick mostly to the RV082's default values. RV082's default value for Phase1 SA Life Time is 28800 seconds (8 hours) and the default value for Phase2 SA Life Time is 3600 seconds (1 hour).

My mistake was in my mapping of the RV082's settings names to the OpenSwan settings names. I incorrectly mapped Phase1 SA Life Time to keylife and Phase2 SA Life Time to ikelifetime. The correct mapping is Phase1 SA Life Time to ikelifetime and Phase2 SA Life Time to keylife. How did I found that out? From another Paul Wouters mailing list message, of course:

On Tue, 27 Sep 2005, Agent Smith wrote:

> ike_life is set by the keyword ikelifetime which is
> phase 1 timeout (maximum supported is 8 hr.)
>
> ipsec_life is set by keylife keyword and that is phase
> 2 timeout.
>
> phase 2 timeout is typically less then phase 1
> timeout.
>
> correct?

Yes

Paul

So what values should you use for ikelifetime (phase1) and keylife (phase2)? I emailed this question to Paul and here is his answer:

The RFC leaves it open. Some people prefer phase 2 to last longer then phase1, some vise versa. I believe using phase2 that is bigger then phase1 is good, since a phase 1 rekey failure won't immediately blow your tunnel out of the water, since phase2 is still valid for a few more hours.

Based on Paul's advice, I assume the Openswan default values are appropriate. That is, ikelifetime=1 and keylife=8. Note that the default values on the RV082/RV042 appear to be the exact opposite of this, so you should probably reverse the values on the RV082/RV042.

BTW, I found out that Paul has authored a book about Openswan called Openswan: Building and Integrating Virtual Private Networks. I recall seeing this book at the bookstore when I was attempting to create my VPN. At the time I thought "Great book -- I should get it". However, I didn't get it because I didn't feel like begging my boss to buy me yet another book. I probably should have got it...

How To: VPN Between RV082 (or RV042) and WRT54GL (or WRT54G)

Introduction

Using the OpenWrt Linux distribution, you can configure a LinkSys WRT54G (also WRT54GS and WRT54GL) router as an IPSec VPN endpoint. This IPSec VPN functionality is provided by an Openswan package that was built specifically for OpenWrt.

In this How To, I will show you how to create a VPN between a LinkSys WRT54GL and a LinkSys RV082 (RV042 could be used also).

As usual, if the post helps you, please have the common courtesy to leave me a thank you comment!

Openswan vs Freeswan - What's the difference?

Openswan is a fork of the Freeswan project. Apparently, Openswan has more features that Freeswan. Anyway, it's moot because only Openswan has been ported to OpenWrt. Note that much of the Freeswan documentation, tips and how-tos also apply to Openswan.

Software and Hardware Versions

I used the following hardware and software versions for the OpenWrt end of the VPN tunnel:

• LinkSys WRT54GL v1.1 (serial number starts with CL7B).
• OpenWrt firmware version: White Russian RC6
• Openswan 2.4.4-1

On the other end of the VPN tunnel, I used:

• LinkSys RV082
• Stock firmware version: 1.3.3.5
• From skimming the GPL source code for that firmware version, it appears that it uses Freeswan (perhaps version 1.99) for its VPN server.

Network Diagram

This how-to was created in a "lab" environment. To simulate the internet we used private IP addresses. Here is the network diagram:

From the point of view of the WRT54G, the WRT54G side of the tunnel is called the local side or the left side and the RV082 side of the tunnel is called the remote side or the right side.

Here are the relevant IP addresses, as shown in the above network diagram:

• Left WAN interface ("left") = 192.168.3.2
  
• Left LAN subnet ("leftsubnet") = 192.168.2.0/24
  
• Left LAN interface ("leftsourceip") = 192.168.2.1
  
• Right WAN interface ("right") = 192.168.3.1
  
• Right LAN subnet ("rightsubnet") = 192.168.1.0/24
  
• Right LAN interface ("rightsourceip") = 192.168.1.1
  

Step 1: Configure the RV082 Side of the VPN

• Login to the RV082 Web Admin Site
• Browse the VPN page.
• Click the Add New Tunnel button. The following "Mode Choose" screen will appear. Click the Add Now near the top (i.e. in the Gateway to Gateway section).

• Enter the settings as shown on the following two screenshots, then press Save Settings:

Step 2: Configure the WRT54G Side of the VPN

• Install OpenWrt White Russian RC6, if you haven't done so already.
• If necessary, update OpenWrt's package list: ipkg update
• Install the openswan package: ipkg install openswan. ipkg should automatically detect and install openswan dependencies like kmod-openswan.
• Create the file /etc/ipsec.secrets, which will hold your preshared key. Here are the contents of my file (note that you can configure keys for specific tunnels or all tunnels):
  

# PSK for tunnel from specific ip to specific ip
#192.168.3.2 192.168.3.1: PSK "abc123"

# PSK for tunnel from any ip to any ip
: PSK "abc123"

• Edit the file /etc/ipsec.conf. Here are the contents of my file:

version 2.0     # conforms to second version of ipsec.conf specification

# basic configuration
config setup
    plutodebug="none"
    klipsdebug="none"
    nat_traversal=no
    interfaces=%defaultroute

# Add connections here
conn TestVPN
    authby=secret
    keyexchange=ike
    ikelifetime=480m
    keylife=60m
    pfs=yes
    left=192.168.3.2
    leftsubnet=192.168.2.0/24
    leftnexthop=%defaultroute
    right=192.168.3.1
    rightsubnet=192.168.1.0/24
    auto=start
    leftsourceip=192.168.2.1

#Disable Opportunistic Encryption
include /etc/ipsec.d/examples/no_oe.conf

• Edit /etc/firewall.user. Here is what I appended to the end of the file:

### IPSec VPN
# allow IPSEC
iptables -A input_rule -p esp -s 192.168.3.1              -j ACCEPT
# allow ISAKMP
iptables -A input_rule -p udp -s 192.168.3.1 --dport 500  -j ACCEPT
# allow NAT-T
iptables -A input_rule -p udp -s 192.168.3.1 --dport 4500 -j ACCEPT
# disable NAT for communications with remote LAN
iptables -t nat -A postrouting_rule -d 192.168.1.0/24     -j ACCEPT
# Allow any traffic between tunnel LANs
iptables -A forwarding_rule -i $LAN -o ipsec0 -j ACCEPT
iptables -A forwarding_rule -i ipsec0 -o $LAN -j ACCEPT

Step 3: Start up the Tunnel

• If you are starting the tunnel for the first time, you can start it by either rebooting the WRT54G, or by executing: /etc/init.d/S60ipsec --start

Step 4: Verify the Tunnel is Up

There are numerous ways to verify that the tunnel is up and working:

• RV082 System Log - You should see a message like this: Tunnel Negotiation Info - Quick Mode Phase 2 SA Established, IPSec Tunnel Connected
• RV082 VPN connection status (in the web admin tool) - The button in the "tunnel test" column should have the caption "Disconnect" and the "status" column should say "connected". For example:

• ssh to the WRT54G and run the command ipsec auto --status. The output should look something like this:

000 #2: "TestVPN":500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE in 2514s; newest IPSEC; eroute owner
000 #2: "TestVPN" esp.82fdd248@192.168.3.1 esp.96ea10e4@192.168.3.2 tun.1002@192.168.3.1 tun.1001@192.168.3.2
000 #1: "TestVPN":500 STATE_MAIN_I4 (ISAKMP SA established); EVENT_SA_REPLACE in 27519s; newest ISAKMP; lastdpd=0s(seq in:0 out:0)

What you want to see are the phrases are "ISAKMP SA established" and "IPSec SA established", with the relevant connection name (in this case, "TestVPN"). Note that ipsec auto --status will tell you only what states have been achieved, rather than the current state. Since determining the current state is rather more difficult to do, current state information is not available from Linux FreeS/WAN. If you are actively bringing a connection up, the status report's last states for that connection likely reflect its current state. Beware, though, of the case where a connection was correctly brought up but is now downed: Linux FreeS/WAN will not notice this until it attempts to rekey. Meanwhile, the last known state indicates that the connection has been established.

• ssh to the WRT54G and run the command logread. The output should look something like this:

Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: initiating Main Mode
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: received Vendor ID
payload [Dead Peer Detection]
Jan  1 00:05:13 (none) daemon.err ipsec__plutorun: 104 "TestVPN" #1:
STATE_MAIN_I1: initiate
Jan  1 00:05:13 (none) daemon.err ipsec__plutorun: ...could not start conn
"TestVPN"
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: transition from
state STATE_MAIN_I1 to state STATE_MAIN_I2
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: STATE_MAIN_I2: sent
MI2, expecting MR2
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: I did not send a
certificate because I do not have one.
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: transition from
state STATE_MAIN_I2 to state STATE_MAIN_I3
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: STATE_MAIN_I3: sent
MI3, expecting MR3
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: Main mode peer ID is
ID_IPV4_ADDR: '192.168.3.1'
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: transition from
state STATE_MAIN_I3 to state STATE_MAIN_I4
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: STATE_MAIN_I4:
ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=oakley_3des_cbc_192
prf=oakley_md5 group=modp1024}
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #1: Dead Peer Detection
(RFC 3706): enabled
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #2: initiating Quick
Mode PSK+ENCRYPT+TUNNEL+PFS+UP {using isakmp#1}
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #2: Dead Peer Detection
(RFC 3706): enabled
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #2: transition from
state STATE_QUICK_I1 to state STATE_QUICK_I2
Jan  1 00:05:13 (none) kern.warn pluto[1646]: "TestVPN" #2: STATE_QUICK_I2: sent
QI2, IPsec SA established {ESP=>0x82fdd26a <0x454556ce xfrm="3DES_0-HMAC_MD5" natd="none" dpd="">

Again, what you want to see are the phrases are "ISAKMP SA established" and "IPSec SA established".

• Ping the RV082's LAN address from the WRT54G.
• Ping the WRT54G's LAN address from the RV082.

Troubleshooting

If the connection does not come up:

• Check the VPN Log on the RV082. There might be some useful error messages.
• Check the dmesg log on the WRT54G (ssh to the WRT54G and run the command dmesg). For more verbose messages, try changing the plutodebug and klipsdebug settings in /etc/ipsec.conf from "none" to "all" or another supported log level. HOWEVER, please make sure to change both settings back to "none" when you have finished debugging otherwise the VPN performance will suffer a lot! You have been warned!

How to Get the VPN to Reconnect Automatically

If the WAN link between the two VPN endpoints goes down, the VPN link will go down too. To get the VPN to reconnect when the WAN link comes up again, you have two options:

• Enable the "Keep-Alive" setting on the "advanced" section of the VPN tunnel configuration on the RV082:

Note that this keep-alive setting does not seem to work if the WAN link is down for more than about 4 minutes. After that, you will probably have to manually reconnect the tunnel.

• Enable the "dead peer detection" settings on the WRT54G in the file /etc/ipsec.conf. The relevant settings are dpddelay, dpdtimeout and dpdaction. Please see the Openswan ipsec.conf man page for a description of each setting. According to the man page, those settings should have default values, however, they didn't seem to work so I added them manually to ipsec.conf. Here are the settings I used:

dpddelay=10
dpdtimeout=30
dpdaction=hold

Note that the RV082 also has dead peer detection (DPD) but it does not seem to work like DPD on the WRT54G. Here is a description of the RV082's DPD feature:

Dead Peer Detection (DPD): When DPD is enabled, the RV082 will send the periodic HELLO/ACK messages to prove the tunnel liveliness when both peers of VPN tunnel provide DPD mechanism. Once a dead peer detected, the RV082 will disconnect the tunnel so the connection can be re-established. The Interval is the number of seconds between DPD messages. The default is DPD enabled, and default Interval is 10 seconds.

As you can see, RV082 DPD only seems to remove the dead connection. It does not seem to try to reconnect. That's what the RV082's keep-alive feature is for, as described here:

Keep-Alive: This mechanism helps to keep up the connection of IPSec tunnels. Whenever a connection is dropped and detected, it will be re-established immediately.

The WRT54G's DPD feature seems to do both jobs -- disconnect the dead connect and reconnect.

I have found that a combination of the above strategies works well for keeping connections up. That is:

• Enable DPD on RV082
• Enable keep-alive on RV082
• Enable DPD on WRT54G

If those strategies don't work, you can also try pinging each VPN endpoint periodically. This is a "roll your own" keep-alive solution.

How to Manually Reconnect the Tunnel

If the tunnel does not automatically reconnect after a WAN link goes down and comes back up, you can manually reconnect it from either side of the tunnel:

• To reconnect from the WRT54G side, ssh to the WRT54G and run /etc/init.d/S60ipsec restart. (Alternatively, you can reboot the WRT54G but this will bring down all WAN connectivity for a few seconds.)
• To reconnect from the RV082 side, press the tunnel's Connect button on the VPN Summary page in the web admin tool. (Alternatively, you can reboot the RV082 but this will bring down all WAN connectivity for a few seconds.)

Split DNS on WRT Side of the Tunnel

Let's say you have an internal DNS server on the RV side of the tunnel and you want all DNS lookups for the internal domain to be delegated to that DNS server but still have the WRT handle other (i.e. internet host) DNS lookups. The solution for this problem is to use a feature called split dns. Split DNS allows a DNS server to delegate lookups to another DNS server for certain domains.

To enable this feature, you need to add a "server" setting to the file /etc/dnsmasq.conf on the WRT (and, of course, you would need to restart dnsmasq). For example:

server=/internal.mydomain.com/192.168.1.145

internal.mydomain.com is the domain that you want to split requests for to the internal DNS server across the VPN. 192.168.1.145 is the address of that DNS server.

Joining a Domain on WRT Side of the Tunnel

Let's say you have an active directory domain controller on the RV side of the tunnel and you have a computer on the WRT side that you want to join to the domain.

To join a domain across the VPN, you need to add a "srv-host" setting to the file /etc/dnsmasq.conf (and, of course, you would need to restart dnsmasq). For example:

srv-host=_ldap._tcp.dc._msdcs.internal.mydomain.com
,mydomaincontroller.internal.mydomain.com,389,0,100

In this example, mydomaincontroller.internal.mydomain.com is the name of the domain controller and internal.mydomain.com is the domain. I'm not sure if this only works with active directory or if it also works with NT4 domains. I leave that as an exercise to the reader.

Links

Following are some useful how-to and informational links about installing and running Openswan on a WRT54G. Note that none of these links provide a comprehensive how-to. I had to piece together information from all of these links to create this page.

• Openswan


• The VPN Follies (Part 1) -- Terminating VPN Connections with Linux and other Stuff -- Introduction


• VPN Follies Part 2 -- Linux 2.4, OpenSWAN, and WRT54G


• IPSec on OpenWrt - from the OpenWrt Wiki


• Openswan ipsec.conf man page


• Openswan ipsec.secrets man page


• resolv.conf man page


• dnsmasq setup


• Openswan wiki


• Can I use Network Neighborhood (Samba, NetBIOS) over IPsec?


• Configuring Linux VPNs

Addendum

For some corrections and further information about this how-to, please see the following:

• A Virtual Visit from Paul Wouters

How to Hide the Outlook Envelope Icon When You Get Spam

In Outlook 2003, I have enabled the setting "show an envelope icon in the notification area". With this setting enabled, whenever I get a new email, an envelope is displayed in the Windows system tray. I like this feature because it is less obtrusive and distracting than the methods of new email notification (e.g. Desktop Alerts, playing a sound, etc).

The only problem with this setting is that, by default, the envelope icon gets displayed even when the new email is spam (i.e junk email). This is a big problem for me because probably 90% of my email is spam and I actually lose a few minutes every day by switching to Outlook whenever the envelope icon appears, only to discover that the new mail is spam. It is so annoying!

With prior versions of Outlook, this was a tricky problem to solve. My solution back then was to hack out a VBA script and hook it into Outlook with a rule. My code was based off this example. It was a pain in the butt to create and install and sometimes it failed with non-descriptive error messages that said something like "rule failed".

When I installed Outlook 2003 the other day, I noticed an email folder called "Junk E-Mail". When I saw that, it gave me hope for a better solution. So, I started reading about this new folder in Outlook 2003's help file. I read and I read and I read. To my chagrin, I found no relationship between the Junk E-Mail folder and the envelope icon in the help file.

Undaunted, I thought I would try an experiment anyway because I thought there might be something magical about this new folder. After all, it can't be deleted like user-created folders.

Before I tell you about my experiment, I'll give you some background information. At our office, we run Exchange 2003. However, we front this with a Spam Assassin server running on a Linux box. All incoming mail first goes through the Spam Assassin server, which detects spam and marks it by appending a special string (e.g. **THIS IS FRIGGING SPAM**) to the subject line before forwarding it on to Exchange.

For my experiment, I created the following Outlook rule, using the rules wizard thingy:

Apply this rule after the message arrives
with **THIS IS FRIGGING SPAM** in the subject
move it to the SPAM folder
except if from {anyone in my office}
or except if from {my wife}
or except if from {etc, etc, etc}
stop processing more rules

The SPAM folder shown in the above rule is a folder I had created. When spam arrived, it was correctly moved to the SPAM folder but the envelope icon still appeared, which is NOT what I wanted.

Next, I edited the rule, changing the SPAM folder to the magical "Junk E-mail" folder. Voila! It worked! The next time I got a new spam message, the envelope icon did NOT appear.

If this tip helped you, please leave me a comment!

How to Force ez-ipupdate to Touch Dyndns Record Every Month

Like most home internet users, I have a dynamic IP address and I use Dyndns to map it to a static host name so that I can remotely access my router (and computers behind it) without having to know its current IP address.

My router is a LinkSys WRT54G, which is running the OpenWrt custom firmware. OpenWrt comes with a package called ez-ipupdate, which can be configured to automatically update your Dyndns host record whenever the router's address changes. For detailed instructions, see the DDNSHowTo page in the OpenWrt wiki.

The instructions in OpenWrt's wiki are pretty good but they don't deal with the situation where the time between address changes is greater than 30 days. This is a problem if you are using Dyndns's free Dynamic DNS service. The Dynamic DNS service FAQ states the you must periodically "touch" the host record, even if the address hasn't changed, or your host record will be deleted. Here is the policy as at the time of writing this blog post:

A Dynamic DNS hostname only needs to be updated when your IP address has changed. Any updates more frequently than this - from the same IP address - will be considered abusive by the update system and may result in your hostname becoming blocked. Any script which runs periodically should check to make sure that the IP has actually changed before making an update, or the host will become blocked. An exception to this is for users with mostly static IP addresses; you may update 24-35 days after your previous update with the same IP address to "touch" the record and prevent it from expiring. Users will receive an e-mail notification if a host has been unchanged for 30 days, hosts are deleted after 35 days without being updated unless you've purchased upgrade credit(s).

Here is an example of the e-mail notification (i.e. "nag" email) that you get when you haven't touched the host record in 30 days:

A hostname you have registered with Dynamic Network Services at DynDNS, SomeDynamicHostName.dyndns.org, with current IP address 192.168.1.1, will expire in the next 5 days. This expiration is due to an automatic timeout; your host has not been updated for 30 days, and hosts are removed after not being updated for 35 days. This is our policy to prevent a stagnant DNS system.
Users with static IP addresses can use the Static DNS system, which does not have this timeout.

You can "touch" your hostname and prevent it from being deleted simply by performing a manual, or forced, update. The system will not consider this
single update abusive. If you wish to allow your hostname to expire,
simply do nothing. You will receive a notification when this occurs.

You can also purchase an account upgrade credit at https://www.dyndns.com/+upgrades/add.html - users with any credits on their account are exempt from this expiration policy.

https://www.dyndns.com/+dyndns/SomeDynamicHostName.dyndns.org is a direct URL to a page where you can manually update your hostname. Simply submit the form (by clicking the "Modify Host" button), and your hostname will not expire unless it is left without updates for another 35 days.

Please DO NOT contact the support department and ask to have your host updated. Following the instructions above is the ONLY way to keep your host from being removed by our automated system.

If you do not use your account any longer, please delete it using the tools at . We regret seeing you go.

Replies to this e-mail address will be discarded. Please contact the support department at support@dyndns.com if you have questions, queries, or comments.

Sincerely,
The DynDNS Team

To prevent the deletion of my host record, I created a cron job that touches the host record on the first day of each month. To create a cron job in OpenWrt, you must put a job definition in the file /etc/crontabs/root. Here is my job definition (must be all on one line):

1 0 1 * * rm -f /tmp/ez-ipupdate.cache
; /usr/sbin/ez-ipupdate -c /etc/ez-ipupdate.conf
> /tmp/ez-ipupdate-cron.log 2>&1

Note that without the first command, rm -f /tmp/ez-ipupdate.cache, ez-ipupdate will not touch the record at dyndns.org and will instead show the message "no update needed at this time". So, the first command it critical to force a "touch" of the record. Without the first command, the log file /tmp/ez-ipupdate-cron.log would show the following, which is NOT what we want:

ez-ipupdate Version 3.0.11b8
Copyright (C) 1998-2001 Angus Mackay
no update needed at this time

With the first command, you should see this in the log file, which IS what we want:

ez-ipupdate Version 3.0.11b8
Copyright (C) 1998-2001 Angus Mackay
connected to members.dyndns.org (63.208.196.95) on port 80
members.dyndns.org says that your IP address has not changed since the last update

Even though it says "your IP address has not changed" it has still "touched" the record, which is what we want.

If this tip helped you, please leave me a comment!

Update: Here is a better solution.

Just the File Size Please

Yesterday I wrote a custom log file rotator script in bash for one of my embedded Linux boxes (yes, I know about logrotate but this was a special situation that required a custom script).

One of the building blocks I needed for this script was a function that could return the size of a file. The wc command seemed appropriate for this:

---

me@mycomputer:~$ wc --help
Usage: wc [OPTION]... [FILE]...
Print newline, word, and byte counts for each FILE, and a total line if
more than one FILE is specified. With no FILE, or when FILE is -,
read standard input.
-c, --bytes print the byte counts
-m, --chars print the character counts
-l, --lines print the newline counts
-L, --max-line-length print the length of the longest line
-w, --words print the word counts
--help display this help and exit
--version output version information and exit

---

Yep, the -c option was the one that I wanted. Here is an example of retrieving the size of the file called foo:

---

me@mycomputer:~$ wc -c foo
1071 foo

---

The size of the file is 1071 bytes. But what's with the extra "foo" string? I didn't want that. I justed wanted the 1071. What's a boy to do? awk to the rescue. Awk is a general purpose programming language that is designed for processing text files and streams. For example, you can use it to parse data fields from text that is piped in from another command. Here is how I used awk to parse just the file size from the output of wc:

---

me@mycomputer:~$ wc -c foo | awk '{print $1}'
1071

---

I hope this helps someone else. If you have a better way of getting just the file size, please leave a comment.

UPDATE

Here is an easier (and better performing?) way of getting just the file size:

stat -c%s filename

Accessing Windows Shares Across a VPN

Just for laughs, I recently created a point-to-point VPN (a.k.a. gateway-to-gateway VPN) between my home and the office. At the office, the VPN endpoint is a LinkSys RV042, which, as far as I can tell, runs Freeswan VPN software. At home, the VPN endpoint is a LinkSys WRT54GL, which I hacked to run the OpenWrt linux distribution and OpenSwan VPN software. A future post will detail exactly how I did this but the purpose of this post is to discuss how I got my Windows file shares to work across the VPN.

When I first setup the VPN, I initially joined my home Windows XP Pro computer to the Active Directory domain at the office. That made it easy to share files back and forth. However, I didn't like that other domain admins could have full access to my home computer. Therefore, I unjoined my home computer and reverted it back to workgroup mode.

After reverting back to workgroup mode, I could still access shares on office computers. Upon accessing a share on my office computer, I would be prompted to enter a username and password. I this case, I entered my domain username and password. I can't remember if I put in the short username (e.g. myusername) or the fully qualified username (e.g. mydomainname\myusername) but the point is that it worked.

On the other hand, I couldn't access any home shares from the office. Upon attempting access, I would either get an "access denied" error or a "credentials supplied conflict with an existing set of credentials" error.

The problem is, when you access a workgroup share from a domain computer, Windows assumes you want to login with your domain username and password and it doesn't prompt you to enter your workgroup username and password. Since my domain username and password are different from my workgroup username and password, I couldn't be authenticated and access was denied.

The solution was to set up a username and password on my home computer that was identical to my domain username and password. This solution works fine but the only problem is that the new user appears on the Windows XP Welcome Screen. Thankfully, there is a way to hide users from the Welcome Screen:

1. Open the Registry Editor (i.e. run regedit).
2. Navigate to the following registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\SpecialAccounts\UserList
3. Create a DWORD value where the name is the username of the user you want to hide from the Welcome Screen. For example, "myusername".
4. Set the value of the new registry entry to 0 (it should default to this value automatically).
5. Close the Registry Editor.
6. You might have to reboot but I didn't have to.

You might be wondering how I accessed computers at the remote end of the VPN using Windows Explorer. Well, there are a few options. One way is to enable NetBios over TCP. If you do that, you should be able to automically see the computer names in "My Network Places". I didn't use that method but if you want to try it, see Can I use Network Neighborhood (Samba, NetBIOS) over IPsec? in the Openswan FAQ.

Another method is to register computers at both ends of the VPN with a DNS server at the office. If you do that, you should be able to enter "\\somecomputer" into Windows Explorer's location bar to see a list of shares on a remote computer. I tried that and it worked great. In my case, I found dnsmasq's "split DNS" feature to be particularly useful. For now, I'll leave it as an exercise to the reader to find out more about this feature but for a hint, read about the "server" setting for the file "dnsmasq.conf" in the dnsmasq manpage.

Finally, you can also access the remote computers by IP address. In this case, enter "\\someaddress" (e.g. \\192.168.1.10) into Windows Explorer's location bar.

Arp Tip

I finally used Windows' arp command for something useful.

Last week I was configuring numerous brand new LinkSys routers for various branch offices. After successfully configuring the first one, I plugged in the second and tried to connect to it's web admin page. Unfortunately, my connection attempt kept on timing out. After a couple of minutes of pulling my hair out, I remembered something I had learned about in school a few years ago -- Address Resolution Protocol or ARP for short.

ARP is a protocol for determining a host's hardware address given its network address. In the common case of TCP/IP-over-ethernet networks, the hardware address is known as the MAC address and the network address is known as the IP address. This protocol is important because a host's hardware address must be known before you can communicate with it over a network.

If you're familiar with LinkSys routers, you'll know that they all have the default IP address of 192.168.1.1. On the other hand, every networked device (including LinkSys routers) has (or should have) a unique MAC address. For example, I had two routers with MAC addresses of 00-00-24-c5-94-f4 and 00-00-24-c6-66-84 respectively.

When I plugged the first one in and tried to connect to http://192.168.1.1, Windows silently used ARP to translate the IP address 192.168.1.1 into the MAC address 00-00-24-c5-94-f4. This worked without a hitch and I was able to configure the router.

The problem arose when I plugged in the second router. Apparently, Windows (and perhaps other operating systems) caches the IP-address-to-MAC-address for a period of time. When I tried to connect to http://192.168.1.1, Windows looked that IP address in its arp cache and found the MAC address of 00-00-24-c5-94-f4. However, the MAC address it really needed to use was 00-00-24-c6-66-84.

How long does Windows cache these arp mappings and how do you delete the cache? The answers can be found on Microsoft's TechNet. According to TechNet:

Dynamic ARP cache entries - These entries are added and deleted automatically during normal use of TCP/IP sessions with remote computers. Dynamic entries age and expire from the cache if not reused within 2 minutes. If a dynamic entry is reused within 2 minutes, it may remain in the cache and age up to a maximum cache life of 10 minutes before being removed or requiring cache renewal by using the ARP broadcast process.

To view the current contents of the ARP cache, go to a command prompt and run arp -a. To delete the entire cache, run arp -d. To delete only a particular mapping, run arp -d (for example, arp -d 192.168.1.1).

Here is an example session to show you what I'm talking about. Note that ping fails (i.e. times out) if you have an incorrect IP-address-to-MAC-address mapping:

---

C:\> arp -a

Interface: 192.168.1.25 on Interface 0x9000006
Internet Address Physical Address Type
192.168.1.1 00-00-24-c5-94-f4 dynamic

C:\> ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:

Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64
Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64
Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64
Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64

Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

(AT THIS POINT, I PLUGGED IN THE OTHER ROUTER)

C:\> ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\> arp -a

Interface: 192.168.1.25 on Interface 0x9000006
Internet Address Physical Address Type
192.168.1.1 00-00-24-c5-94-f4 dynamic

C:\> arp -d 192.168.1.1

C:\> ping 192.168.1.1

Pinging 192.168.1.1 with 32 bytes of data:

Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64
Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64
Reply from 192.168.1.1: bytes=32 time< 10ms TTL=64

Ping statistics for 192.168.1.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 1ms, Average = 0ms

C:\> arp -a

Interface: 192.168.1.25 on Interface 0x9000006
Internet Address Physical Address Type
192.168.1.1 00-00-24-c6-66-84 dynamic

C:\>

How to Clear syslog Log Files in Linux

Many Linux distributions are preconfigured to automatically and periodically rotate syslog log files by means of cron jobs that call logrotate or a similar log rotation script. However, if you want to clear a log file manually, you can simply use the greater than sign followed by the log file name. For example, to manually clear the file kern.log, you can run this command:

me@mycomputer:> /var/log/kern.log

The beauty of this is that the cleared out file still retains its original permissions and ownership.

VPN Between RV082 (or RV042) and WRT54GL (or WRT54G)

Great news! I have managed to get a site-to-site (a.k.a. gateway-to-gateway) VPN working between a LinkSys RV082 and a LinkSys WRT54GL running OpenWrt! It should also be possible to use similar hardware like the RV042, RV016, WRT54G, WRT54GS, etc.

The full how-to is here.

Displaying Masqueraded Connections

I often use Linux's netstat command to do things like figuring out which ports are listening on a computer and what are the active connections on a computer. It's very handy for troubleshooting networking problems.

Today I wanted to figure out what sites a certain user on my network was connecting to and how much traffic the user was using. I connected to the router with SSH and ran netstat with no options. I was surprised to see only my SSH connection. Then I remembered that the user in question gets onto the internet with a masqueraded address. Well, duh!

I figured netstat was still the command that I needed to use but that I just needed to pass some option. So, I looked at the man page for netstat and sure enough I found the -M option, whose description is "Display a list of masqueraded connections."

I tried netstat -M and got the following error message: "netstat: no support for `ip_masquerade' on this system."

That sucks! After some googling, I found out that netstat -M only works on pre-2.4 kernels. On 2.4 and later kernels, you need to use this:

cat /proc/net/ip_conntrack

The format of ip_conntrack is kind of cryptic but if you look at it for a few moments, you can figure it out. You can also use grep to filter out the stuff you don't need. For example, to see only the connections involving a particular address, say 192.168.1.10, you can do this:

cat /proc/net/ip_conntrack | grep 192.168.1.10

Apparently, there is a command out there that can nicely format the contents of ip_conntrack. It's called conntrack viewer. Unfortunately, I couldn't connect to that site today. Perhaps the link is dead.

How to Kill Outlook Before Backing Up Your PST file

I use Cobian Backup to automatically backup my Outlook mail file (i.e. PST file) as well as other types of files. There is a catch with backing up PST files with Cobian Backup -- Outlook cannot be running. If Outlook is running, Cobian will not backup the PST file because it will be locked by Outlook.

Luckily, Cobian has a feature called "events", which can be used to run arbitrary tasks before and/or after a backup job. I use this feature to kill Outlook before running my backup job, which removes the file lock and allows Cobian to backup the PST file.

If you want to use this feature to kill Outlook, the first step is to create a batch file that can kill Outlook from the command line. I called my batch file "kill_outlook.bat". Here are the contents:

taskkill /f /im outlook.exe

The taskkill command is available on Windows XP Pro and, I believe, also Windows XP Home and Windows Server 2003. On Windows 2000, instead of taskkill, you need to use kill, which is not included on the Windows 2000 install disk. To get kill, you must install the Windows 2000 Support Tools.

After you make the batch file and test it, you need to configure a "before backup" event in Cobian. Open your backup job configuration and go to the Events tab. Under the "Before Backup" box, press the "Add" button and click "Execute and wait" on the popup menu. In the file browser dialog that appears, browse to and select the batch file you just created. Then click OK. Finally, click OK on the job properties dialog and you are done!

Good luck!

Linux traceroute vs Windows tracert

Anyone who has used Linux traceroute and Windows tracert knows that the two programs have very different command line options. But did you know that there is a fundamental difference in the way that these two programs work?

According to RFC1393, traceroute implementations are supposed to use the ICMP protocol. Indeed, the windows implementation does use ICMP. However, by default, the Linux implementation uses UDP, unless you apply the "-I" option, in which case it will use ICMP.

I googled to find out why Linux traceroute uses UDP by default but I couldn't find any definitive reasons. In practice, I've found that there is no real performance benefit; both the UDP flavour and the ICMP flavour seem to take about the same amount of time to do their work.

I guess the real advantage of being able to choose between the UDP and ICMP flavours is that if a firewall along the route is configured to block ICMP, you can try UDP. Similarly, if a firewall blocks UDP you can try ICMP.

If want to allow someone to traceroute your internet host, you need to configure your firewall to allow it. For ICMP, you must accept (i.e. iptables -j ACCEPT) ICMP packets of type 0 (Echo Reply), 8 (Echo) and 30 (Traceroute).

The UDP firewall configuration is slightly different. For UDP, you must not drop (i.e. iptables -j DROP) UDP packets in the destination port range of 33434 to 33600. Note that you do not have open these ports (i.e. iptables -j ACCEPT). It is sufficient to simply reject (i.e. iptables -j REJECT) on these ports. Just don't use "iptables -j DROP", which will cause the UDP flavour of traceroute to fail. Note that there is no traceroute service for these ports (i.e. you don't have to run any sort of traceroute server/daemon); indeed, for traceroute's "-p" option, the man page says:

Set the base UDP port number used in probes (default is 33434). Traceroute hopes that nothing is listening on UDP ports base to base + nhops - 1 at the destination host (so an ICMP PORT_UNREACHABLE message will be returned to terminate the route tracing). If something is listening on a port in the default range, this option can be used to pick an unused port range.

Good luck and happy tracerouting!

How to Sort Files By Modification Time in ls

To sort files by modification time when using ls, use the -t option. For example, to sort in descending order:
[me@mycomputer usr]$ ls -lt
total 224
drwxr-xr-x 88 root root 65536 Sep 1 17:39 lib
drwxr-xr-x 14 root root 4096 Sep 1 16:00 local
drwxr-xr-x 2 root root 12288 Jul 26 04:04 sbin
drwxr-xr-x 174 root root 4096 Jul 25 19:01 share
drwxr-xr-x 2 root root 40960 Jan 25 2006 bin
drwxr-xr-x 7 root root 4096 Jan 16 2006 libexec
drwxr-xr-x 3 root root 4096 Oct 16 2005 java
drwxr-xr-x 61 root root 12288 Mar 13 2005 include
drwxr-xr-x 4 root root 4096 Mar 13 2005 src
lrwxrwxrwx 1 root root 10 Mar 13 2005 tmp -> ../var/tmp
drwxr-xr-x 6 root root 4096 Feb 27 2005 kerberos
drwxr-xr-x 2 root root 4096 Feb 22 2005 etc
drwxr-xr-x 2 root root 4096 Feb 22 2005 games
drwxr-xr-x 7 root root 4096 Feb 21 2005 X11R6

To sort in descending order, add the -r option:
[me@mycomputer usr]$ ls -ltr
total 224
drwxr-xr-x 7 root root 4096 Feb 21 2005 X11R6
drwxr-xr-x 2 root root 4096 Feb 22 2005 games
drwxr-xr-x 2 root root 4096 Feb 22 2005 etc
drwxr-xr-x 6 root root 4096 Feb 27 2005 kerberos
lrwxrwxrwx 1 root root 10 Mar 13 2005 tmp -> ../var/tmp
drwxr-xr-x 4 root root 4096 Mar 13 2005 src
drwxr-xr-x 61 root root 12288 Mar 13 2005 include
drwxr-xr-x 3 root root 4096 Oct 16 2005 java
drwxr-xr-x 7 root root 4096 Jan 16 2006 libexec
drwxr-xr-x 2 root root 40960 Jan 25 2006 bin
drwxr-xr-x 174 root root 4096 Jul 25 19:01 share
drwxr-xr-x 2 root root 12288 Jul 26 04:04 sbin
drwxr-xr-x 14 root root 4096 Sep 1 16:00 local
drwxr-xr-x 88 root root 65536 Sep 1 17:39 lib

How to Add a User to a Group in Linux

Today I had to add an existing user to an existing group in Linux. It's been so long since the last time I did it that I had actually forgotten how to do it. Well, good ol' man pages to the rescue. Here's how to do it:

adduser user group

If you want to verify that it worked, use grep:

grep group /etc/group

In grep's output, you should see the user's name. For example:

group:!:50:tom,dick,harry,user

Enjoy!

Disabling Password Authentication on Dropbear SSH Server in OpenWrt Running on WRT54G

I have a LinkSys WRT54G router which runs OpenWrt (version White Russian RC5). For console access to the router, I have enabled the Dropbear SSH server, which works well.

Recently, I opened up the SSH port to allow remote access to the console. To secure this as best I could, I disabled password authentication and enabled public key authentication, by following the instructions in the Dropbear Public Key Authentication Howto.

After doing disabling password authentication, I thought to myself "What if I lose my private key?" The answer is "I'm probably screwed", because there is no serial port on the WRT54G for local access. (Actually, there are some hardware mods you can do to add a serial port but I'm not that handy or brave).

So I thought to myself "Would't it be great if you could configure Dropbear to disallow password authentication for remote connections but allow it for local connections?"

I thought, perhaps, that Dropbear might have an option for this. Here are the command-line options for Dropbear 0.48:

• -b bannerfile = Display the contents of bannerfile before user login (default: none)
• -d dsskeyfile = Use dsskeyfile for the dss host key (default: /etc/dropbear/dropbear_dss_host_key)
• -r rsakeyfile = Use rsakeyfile for the rsa host key (default: /etc/dropbear/dropbear_rsa_host_key)
• -F = Don't fork into background
• -E = Log to stderr rather than syslog
• -m = Don't display the motd on login
• -w = Disallow root logins
• -s = Disable password logins
• -g = Disable password logins for root
• -j = Disable local port forwarding
• -k = Disable remote port forwarding
• -a = Allow connections to forwarded ports from any host
• -p port = Listen on specified tcp port, up to 10 can be specified (default 22 if none specified)
• -i = Start for inetd

As shown above, the -s option can be used to disallow password authentication (thereby forcing public key authentication). However, I can't see an option that would do exactly what I wanted (i.e. disable password authentication for remote connections but still allow password authentication for local connections).

After thinking about the problem for a few minutes, I realized that I could solve the problem by starting two instances of the Dropbear daemon. The only thing you have to do is make a small change to /etc/init.d/S50dropbear.

Here is the last line from the stock version of that file in White Russian RC5:

/usr/sbin/dropbear

What we want to do is change that file so that we start a second instance of dropbear that disallows password authentication. Here is the last line, plus a new line that starts the second instance:

# failsafe for local access - port 22, pw auth allowed
/usr/sbin/dropbear

# secure for remote access - port 50022, pw auth not allowed
/usr/sbin/dropbear -s -p 50022

The line new line /usr/sbin/dropbear -s -p 50022 starts a second instance of dropbear that disallows password authentication. Note that it starts the second instance on port 50022 instead of the default port 22; you can use another unused port number instead, if you so desire.

After making those changes you will have to reboot the router.

Next, you have to allow remote access to port 50022 and disallow remote access to port 22. If you are running the WRT54G on a LAN, behind an internet-facing router, just port forward from some port on the internet-facing router to port 50022 on the WRT54G. Do NOT port forward to port 22.

If you are running WRT54G as your internet-facing router, open port 50022 on the firewall on the WAN side. Do NOT open port 22 on the WAN side (by default, it should already be open on the LAN side).

The only downside I can see with my "second instance strategy" is slightly higher memory usage. Hopefully, in later releases of OpenWrt, you'll be able to use the web admin interface (webif) to disable/enable password authentication so that you can run just one instance of Dropbear.

QuickBooks Help - Page Cannot Be Displayed

I installed QuickBooks Pro 2006 yesterday. When I tried to access the help files, I got the error: "Page Cannot Be Displayed".

I found lots of suggested solutions in Google. I tried many of them one-by-one and they did not work. Then I started trying them in combination. Finally, it worked. However, I'm not sure which combination did the trick. If you are having the same problem, please try the various tips on the page CHM help files error: The page cannot be displayed.

I think this tip is important:

regsvr32 hhctrl.ocx

But you might have to create the MaxAllowedZone registry key first (see above link).

Again, I'm not sure exactly which combination did the trick. Just keep trying all of the suggestions on that page (perhaps in various orders) until it is fixed. Then post a comment back here, describing what worked for you.

BTW, Intuit's knowledge base sucks shit! It should have the solution for this problem, which seems to be a common one, judging from the number of posts about it in the QuickBooks forums.

QuickBooks Pro 2006 - Shitty Out of Box Experience

I just installed QuickBooks Pro 2006. The installation went okay but when I tried to start QuickBooks for the first time, I ran into problems. The splash screen came on for about a half a second and then the program closed. I tried it again and then same thing happened.

I figured if Intuit was a typical software company, they would have put some error message into a log file. Sure enough, I found a log file at "C:\Program Files\Intuit\QuickBooks Pro\QBWIN.log". Inside the log I found several "missing module" messages:

=================== COMRegAll ===================
Sat Sep 30 16:42:37 2006


Missing module: 'C:\WINDOWS\system32\mfco42d.dll'
Missing module: 'C:\WINDOWS\system32\msvcirtd.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\addinmgr2.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qfill.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbwps.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\COMObjectFactory.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\GraphSeriesCol.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\cominifile.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\ViewSrcColumns.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrCustHighlights.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrUnpaidBills.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrVendPayHist.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbcscbvw.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\CtrViewGraph.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\ctrviewtable.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctripmds.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\ctrviewhtml.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\customercontact.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\openbalbycustjob.dll'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\excelpayrolldatasource.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctraddin.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctrcustinvsummary.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctrcustpaysummary.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctropenpobycust.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctropenpovendor.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctrprofitloss.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctrqview.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrTenLstProfCust.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrTenMostProfCust.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrUnbillTimeCost.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\storageclasses.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\TenLeastProfitJob.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\TenMostProfitJob.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\vendorcontact.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbctrcashpos.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\ComEvtBroadcaster.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\Qbctrspacer.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\Qbctrtitle.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBInstanceFinder.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\viewsource.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrItemProfit.DLL'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrTenMostProfSvcs.DLL'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrTenLstProfSvcs.DLL'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\OverdueBalance.dll'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\CookieMonster.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\AXLBridge.exe'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\QBXLAdin.DLL'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\CtrViewToolTip.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\tcmaddin.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBCtrTRDS.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\CoLocator2.DLL'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\ECredit.DLL'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\bbfdepcalc.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBDTView.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBDTRatios.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBXMLRP2.DLL'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\sdksubscription.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\qbsdkcomutil.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\sdkparse.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBUpdate\QBUServiceMgr.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBUpdate\QBUpdate.exe'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBUpdate\QBMsgMgr.exe'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBUpdate\QBMsgRequestMgr.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBUpdate\QBOneStepUpdate.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\QBUpdate\QBUpdateCtrl.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\AnswerWorks 4.0\AWAPI4.DLL'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\components\HR\bin\HROrganizer.dll'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\components\HR\bin\wrapper.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\QuickBooks\ctGrid.ocx'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\CashFlow\1.0\CashFlowProjector.dll'
Missing module: 'C:\Program Files\Common Files\Intuit\QuickBooks\\Intuit\LoanManager\1.0\LoanManager.dll'
Missing module: 'C:\Program Files\Intuit\QuickBooks Pro\dbctrs8.dll'
=================================================

Looking at the messages, I noticed an obvious error with a part of the path that was duplicated: "Intuit\QuickBooks\\Intuit\QuickBooks". I googled for a solution and didn't find one. Then I checked Intuit's so-called "knowledge base" and still did not find a solution.

That's when I noticed the batch file "C:\Program Files\Intuit\QuickBooks Pro\reboot.bat". I opened the file and noticed the comment "Post-reboot registration commands, for QuickBooks 2003" at the top of the file. Then I noticed a whole bunch of calls to "regsvr32" -- for example:

C:\WINDOWS\system32\regsvr32 /s "C:\Program Files\Common Files\Intuit\QuickBooks\addinmgr2.dll"

Obviously, that batch file was supposed to run automatically the first time I rebooted after installing QuickBooks. Now I know what you're thinking but let me assure you that I DID reboot after installation. QuickBooks just effed up and didn't run the batch file.

Anyway, I manually ran the batch file and it fixed the problem. I was able to run QuickBooks.

If you have the same problem and this blog post helps you, please leave a comment.

Cheers!

Norton GoBack...To the Stone Age

My boss came to me yesterday with her notebook and said "My notebook has been really slow for the last week or so. Can you please fix it?"

Of course I can! I started by turning on her notebook and noticed that it started up very slowly. Then, after logging in, I noticed that there was constant hard drive activity, even when the notebook was left idle for 30 minutes.

I ran msconfig and noticed that there were several stupid and unnecessary "services" configured to startup automatically at boot time. For example:

• iPodService and iTunesHelper - Apparently, these just make iTunes easier to launch and/or make iTunes launch faster. Whoop-de-doo...
• vzfw - A Sony Vaio program which takes up about 35-40 MB of memory and which apparently searches your hard drive for multimedia files to serve up to other users on the LAN -- as useless a "service" as I have ever seen.
• the winzip "quick launch" thingy
• the quicktime "quick launch" thingy
• ati2evxx - The ATI External Event Utility, which apparently just provides hot keys for changing video setting. Who needs that shit? Is it really that hard to right-click on the desktop, click properties and then click settings?

I set all of the above "services" (and a few other stupid ones) to start up manually, or not at all. As a result, the computer booted much faster and used much less memory. However, I still noticed constant HD drive activity. That's when I noticed that Norton GoBack 4.0 was recently installed on the computer.

Here is what GoBack is supposed to do:

• Reverses system crashes, failed software installations, user errors, and more
• Rolls PCs back minutes, hours, or even days before onset of a problem
• Lets you try software safely, with a fast uninstall if you don’t like it
• Recovers accidentally deleted or modified files
• Prevents unauthorized users from rolling back a hard drive
• Automatically schedules hard drive restorations to a set configuration

In other words, GoBack is for idiots who are too lazy to perform regular backups. It's not that hard to do, People, really.

When I googled "norton GoBack" "constant hard drive", I found someone with a similar story. After he removed GoBack, the constant hard drive activity went away.

So, I removed GoBack too. And guess what? That fixed the problem.