Download a website in ubuntu

By -

We will use wget to save entire webiste. Syntax is

wget <options> http://yoursitename.com

Following are the useful options you can use with wget.

-c / --continue Resume interrupted download.
-nd / --no-directories Don't create directory hierarchy.
-P / --directory-prefix=Save downloaded files to specified directory.
-U / --user-agent=Spoof user agent string.
-r / --recursiveGo crazy :)
-l / --level=use 0 for infinite depth level or use number greater than 0 for limited depth.
-k / --convert-linksModify links inside downloaded files to point to local files.
-p / --page-requisitesGet all images, css, js files which make up the web page.
-np / --no-parentDon't download parent directory contents.

I normally use following command to download a website.

wget -r --level=0 -convert-links --page-requisites --no-parent  <url>

Comments

Anonymous said…
Thanks! I used this to save a copy of our website that we created to chronicle our trip to China to adopt our son. I want to use the space for other things now, but didn't want to lose the memories recorded on the website. One detail though: you list "-convert-links" as a parameter, but it needs to be "--convert-links" in order for the links to be changed to match the local file system. Thanks for the help!

Brett
February 13, 2010 at 8:13 PM
Unknown said…
Is there some way to prevent some particular file format from being downloaded..? Like, the images etc.
March 23, 2010 at 11:14 PM
Anonymous said…
I normally use following command to download a website.

wget -r -v --level=0 --convert-links --page-requisites --no-parent

cause is verbose and you can know what is going on
September 6, 2010 at 9:25 PM
shrek said…
Good job friend. really good to save the html tutorials.

Thank you very much.
February 4, 2011 at 12:30 AM
Anonymous said…
There is a typo in the example of the command you usually use. You put '-convert-links'. It should be '--convert-links' (two dashes instead of one. It doesn't convert the links unless this is right.

Thanks for this post.
April 12, 2011 at 7:22 PM
Francois said…
That is fantastic! I am working on a project to create a presentation based on a website and hopefully this will help with the files I need to do that.

Thank you!
September 20, 2012 at 10:23 AM
Post a Comment

Popular posts from this blog

Multi touch for any,all synaptics touchpad

By -
Multi-touch became trendy after iphone came up with it. We are seeing many new laptops with multi-touch. At hardware level, there is nothing special that you need, to make multi-touch work. You can also do two-finger scroll and two-finger tap to right/middle click. Also, three-finger tap, but, for me it is very hard to press three fingers at the same time(1 out of 10). Create new file gksudo gedit /etc/hal/fdi/policy/11-x11-synaptics.fdi Paste the following code into the file and save it. <?xml version="1.0" encoding="ISO-8859-1"?> <deviceinfo version="0.2"> <device> <match key="info.capabilities" contains="input.touchpad"> <merge key="input.x11_driver" type="string">synaptics</merge> <merge key="input.x11_options.SHMConfig" type="string">On</merge> <merge key="input.x11_options.EmulateTwoFingerMinZ" ...
Read more

Easy network traffic shaping on your ubuntu system

By -
Traffic shaping on Linux system used to require knowledge of lot of things like iptables, qdiscs & networking protocols. Things have changed and now there are couple of tools which handle all of this complexity without requiring much knowledge. And one of them is wondershaper . First install wondershaper by issuing following command. sudo apt-get -y install wondershaper Use following command template to make it work right away. sudo wondershaper <interface name> <down speed> <up speed> Where interface name is your network interface name like eth0 or eth1 or wifi0 and down speed , up speed are self explanatory and specified in kilo bits per second. So sample command will look like sudo wondershaper eth0 512 512 To disable wondershaper from controlling particular interface use following command. sudo wondershaper clear <interface name> To make these changes permanent add following lines to /etc/network/interfaces under relevant interface section. ...
Read more

winetricks - install wine application perquisites with ease

By -
As most of us know wine is used to run windows applications on linux. But setting it up always has been pain. For example if your windows program requires some prerequisites that aren't shipped or not implemented in wine then you have to go for hunting on net. Examples include mscore fonts, Internet Explorer, DCOM, etc. And this guide explains how to install winetricks on ubuntu. Crossover office has a little wizard that will download install all the prerequisites of a particular software but It costs money to use. But there is one little known tool called winetricks which you can use to automate that stuff for free. Finally you can concentrate on something productive rather than hunting for some obscure stuff. Download winetricks Use following commands to get winetricks and set it up. wget http://www.kegel.com/wine/winetricks chmod +x winetricks Using winetricks For example if you want to install Microsoft Core fonts use following command. ./winetricks corefonts ...
Read more