GNU Wget is a command line utility for downloading files from the Web. With Wget, you can download files using the HTTP, HTTPS, and FTP protocols. Wget offers a number of options that allow you to download multiple files, resume downloads, speed up bandwidth, recursive downloads, background downloads, mirror a website, and much more.
In this tutorial, we will show you how to use the Wget command through practical examples and detailed explanations of the most common Wget options.
- Install Wget on Linux
- Wget command syntax
- How to download a file with Wget
- How to download a single file using wget
- Use Wget to save the downloaded file under a different name
- Use Wget to download a file to a specific directory
- How to limit download speed with Wget
- How to resume a download with Wget
- How to download in the background with Wget
- How to download multiple files with Wget
- Use Wget to download FTP
- Use Wget to create a mirror website
- How to bypass certificate verification with Wget
- How to download standard output with Wget
- conclusion
Install Wget on Linux
Currently, the wget package is pre-installed on most Linux distributions.
To check if the Wget package is installed on your system, open your console, type wget
and hit Enter. If you have wget installed, the system will print wget: missing URL
; otherwise it will print the wget command not found
.
If wget is not installed, you can easily install it using your distribution’s package manager.
Installing wget on Ubuntu and Debian
sudo apt update
sudo apt install wget
Installing wget on CentOS and Fedora
sudo yum install wget
Wget command syntax
Before we get into how to use the wget
, let’s start by looking at the basic syntax.
The expressions of the wget
have the following form:
wget [options] [url]
- options: the
options
Wget . url
: URL of the file or directory that you want to download or synchronize.
How to download a file with Wget
In its simplest form, when used without any options, wget will download the resource specified in the URL to the current directory.
In the following example, we are downloading the Linux kernel tar file:
wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.2.tar.xz
Wget starts by resolving the domain’s IP address, then connects to the remote server and starts the transfer.
During the download, Wget displays the progress bar along with the file name, file size, download speed, and estimated time to complete the download. Once the download is complete, you can find the downloaded file in your current working directory .
To disable Wget output, use the option -q
If the file already exists, Wget will append .N
(number) to the end of the file name.
How to download a single file using wget
To download a single command line file using wget, use the following syntax:
wget https://noviello.it/percorso/file.tar.gz
wget ftp://ftp.noviello.it/percorso/file.tar.gz
wget -O output.file https://noviello.it/percorso/file.name.tar.gz
Use Wget to save the downloaded file under a different name
To save the downloaded file with a different name, pass the -O
followed by the chosen name:
wget -O latest-hugo.zip https://github.com/gohugoio/hugo/archive/master.zip
The above command will save the latest hugo zip file from GitHub as the latest-hugo.zip
place of its original name.
Use Wget to download a file to a specific directory
By default, Wget will save the downloaded file to the current working directory. To save the file to a specific location, use the option-P
wget -P /mnt/iso http://mirrors.mit.edu/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1804.iso
With the above command we are telling Wget to save the CentOS 7 iso file to it /mnt/iso
.
How to limit download speed with Wget
To limit the download speed, use the option --limit-rate
By default, speed is measured in bytes / second. Add k
for kilobyte, m
for megabyte g
for gigabyte.
The following command will download the Go binary and limit the download speed to 1MB:
wget --limit-rate=1m https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz
This option is useful when you don’t want wget to consume all the available bandwidth.
How to resume a download with Wget
You can resume a download using the option -c
This is useful if the connection is interrupted while a large file is downloading and instead of starting the download from scratch, you can continue with the previous one.
In the following example we resume the download of the Ubuntu 18.04 iso file:
wget -c http://releases.ubuntu.com/18.04/ubuntu-18.04-live-server-amd64.iso
If the remote server does not support resuming downloads, Wget will start the download from the beginning and overwrite the existing file.
How to download in the background with Wget
Sometimes when downloading a file, the remote server can be configured to block Wget User-Agent. In situations like this, use the -U
to emulate a different browser.
wget --user-agent="Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" http://wget-forbidden.com/
The above command will emulate Firefox 60 requesting the page from wget-forbidden.com
How to download multiple files with Wget
If you want to download multiple files at the same time, use the -i
followed by the path to a local or external file that contains a list of URLs to download. Each URL must be on a separate line.
In the following example, we are downloading the Arch Linux, Debian and Fedora ISO files with the URLs specified in the file linux-distros.txt
wget -i linux-distros.txt
http://mirrors.edge.kernel.org/archlinux/iso/2018.06.01/archlinux-2018.06.01-x86_64.iso
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
https://download.fedoraproject.org/pub/fedora/linux/releases/28/Server/x86_64/iso/Fedora-Server-dvd-x86_64-28-1.1.iso
If you specify -
as the filename, the URLs will be read from standard input.
Use Wget to download FTP
To download a file from a password-protected FTP server, specify your username and password as shown below:
wget --ftp-user=FTP_USERNAME --ftp-password=FTP_PASSWORD ftp://ftp.example.com/filename.tar.gz
Use Wget to create a mirror website
To mirror a website with Wget, use the option -m
This will create a complete local copy of the website by following and downloading all internal links and website resources (JavaScript, CSS, images).
wget -m https://example.com
If you want to use the downloaded website for local navigation, you will need to pass some additional arguments to the above command.
wget -m -k -p https://example.com
-k
-k option will allow Wget to convert links in downloaded documents to be suitable for local viewing. The -p
will tell wget to download all the files it needs to display the HTML page.
How to bypass certificate verification with Wget
If you want to download a file over HTTPS from a host with an invalid SSL certificate, use the --no-check-certificate
:
wget --no-check-certificate https://domain-with-invalid-ss.com
How to download standard output with Wget
In the example below, Wget -q
option -q) and it will send the latest version of WordPress to stdout ( -O -
) and forward it to the one tar
it will extract the file to /var/www
.
wget -q -O - "http://wordpress.org/latest.tar.gz" | tar -xzf - -C /var/www
conclusion
With Wget, you can download multiple files, resume partial downloads, mirror websites, and combine Wget options according to your needs.
For more information on Wget, visit the GNU wget man page.