Tried normal GUI upgrade using upgrade-manager -d command. I was told I need to download around 1.5G to complete upgrade. I wasn't surprised since I have 10s of window managers and other apps installed which I play around with.
First thing I did was uninstall everything GUI based and bring down the system to bare bones. I did achieve this by uninstalling kdebase, libgtk, openoffice and games etc. And did apt-get autoremove to remove unnecessary packages.
Since I don't have any window manager installed I had to do everything from virtual terminal. I used following command to do upgrade from CLI. us.archive.ubuntu.com was incredibly slow for me and I had to use tw.archive.ubuntu.com. Took me around 3hrs on 512kbps connection.
sudo do-release-upgrade -d
After completion of system upgrade installed default GNOME environment.
sudo apt-get -y install ubuntu-desktop
Finally ath5k started supporting my atheros wireless chipset. nVidia driver was automatically installed and configured.
I didn't see any change in my system boot time. It was neither fast nor slow. Needs further investigation.
My system is now super responsive its either due to changes in jaunty or removing crap out of my system. :-P