Fun With Curl

cURL is a Linux command that is used to transfer multiple data types to and from a server. It operates utilizing the libcurl library, which allows it to use the following protocols to move information:

• Telnet
• HTTP
• FTP
• IMAP
• POP3
• SCP
• SFTP
• SMTP
• TFTP
• LDAP
• Files

Check The Status of A Site

To check if a website is available, we often use the following commands to get the apache response code which indicates if the site is up, down, or in a condition that needs to be addressed:

curl -Is http://domain.com | head -n 1
HTTP/1.1 200 OK

curl -I http://domain.com
HTTP/1.1 200 OK
Date: Sun, 16 Oct 2016 23:37:15 GMT
Server: Apache/2.4.23 (Unix)
X-Powered-By: PHP/5.6.24
Connection: close
Content-Type: text/html; charset=UTF-8

or, to get the status code itself, pipe the curl command to head -n 1|cut -d$’ ‘ -f2

curl -I http://domain.com | head -n 1|cut -d$' ' -f2

To follow a redirect use:

curl -L http://domain.com

Downloading With cURL

If you need to download a file to the current folder you are in and want to keep the same filename, run:

root@server:~/test$ curl -O http://ipv4.download.thinkbroadband.com/5MB.zip

root@server:~/$ curl -O http://ipv4.download.thinkbroadband.com/5MB.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k  100 5120k    0     0  2450k      0  0:00:02  0:00:02 --:--:-- 2450k
root@server:~/$ 

or, if you need to choose a different location or filename, use

curl -o website.html https://domain.com/

root@server:~/$ curl -o newfile.zip http://ipv4.download.thinkbroadband.com/5MB.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5120k  100 5120k    0     0  2311k      0  0:00:02  0:00:02 --:--:-- 2311k
root@server:~/test

In the above command, we downloaded a small file originally named 5MB.zip and saved it as newfile.zip. cURL can also be used to download multiple files simultaneously, as shown in the examples below:

curl -O [URL1] -O [URL2] -O [URL3] 
or
curl -O https://domain.com/file.pdf -O https://domain.com/file2.pdf

Download A File Over SSH

Another popular method for using cURL is to download files securely via SSH using the following command:

curl -u root sftp://host.domain.com/path/to/file

FTP Using cURL

To access an FTP server with cURL, use the following command:

curl ftp://ftp.domain.com --user username:password

You can also download files using cURL over FTP:

curl ftp://ftp.domain.com/file.txt --user username:password

Additionally, we can upload a file onto the FTP server via cURL:

curl -T file.txt ftp://ftp.domain.com/ --user username:password

Resume A Broken Download Using cURL

If you are downloading a file and lose your internet connection for some reason, cURL can resume the download where it left off:

curl -O -C - https://domain.com.file.pdf

Use cURL To Get Your IP Address

curl -s ip.liquidweb.com
curl checkip.amazonaws.com
curl ifconfig.me

Verify If A Website Supports HTTP/2 With Curl

curl -I --http2 -s https://liquidweb.com/ | grep HTTP
HTTP/2 200

Use cURL To Expand Shortlinks

If an untrusted short link is sent to you, verify its destination using the following command:

curl -sIL https://lqwb.us/2kvCtvm | grep ^Location;

Transfer Large Files Using cURL And transfer.sh

The website transfer.sh is a simple file sharing service for the command line that allows you to transfer files up to 10G, and stores the file for up to 10 days. They also allow for the encryption of a file if needed. We’re going to use it to upload and download an encrypted file:

# Encrypt files with password using gpg
$ cat /test/test.txt|gpg -ac -o-|curl -X PUT --upload-file "-" https://transfer.sh/test.txt

#enter a passphrase twice and transfer.sh will then provide a link to download the encrypted file.

# To download and decrypt
$ curl https://transfer.sh/1h4tfs/test.txt|gpg -o- > /test/test.txt

Check EMail Using cURL

You can connect to Gmail via the command line to check unread mail using cURL

curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | grep -oPm1 "(?<=)[^<]+" | sed '1d'

curl -u username --silent "https://mail.google.com/mail/feed/atom" | perl -ne 'print "\t" if //; print "$2\n" if /<(title|name)>(.*)<\/\1>/;'

cURL The Weather!!!

curl http://wttr.in/lansing
curl http://v2.wttr.in/lansing

or the moon phase!

curl wttr.in/Moon

cURL For A Random Number

curl "https://www.random.org/integers/?num=1&min=1&max=100&col=1&base=10&format=plain&rnd=new"

Use cURL To Grab A Cheatsheet

curl https://cht.sh

Need A QR Code? cURL

curl qrenco.de/liquidweb.com

cURLing And Social Media

Grab the latest YouTube video title and URL by username, the newest Tweet from Liquidweb, or check your favorite Twitch channel.

curl -s https://decapi.me/youtube/latest_video?user=liquidwebinc
curl -s https://decapi.me/twitter/latest?name=liquidweb
curl -s https://decapi.me/twitch/uptime?channel=CHANNELNAME

Also, you can tweet using cURL:

curl -u user:pass -d status="Tweeting from the shell" http://twitter.com/statuses/update.xml

Convert A Document Using cURL

curl \
  http://c.docverter.com/convert \
  -F from=html \
  -F to=pdf \
  -F input_files[]=@<(echo hello)

Define A Word Using cURL

cURL can connect to dict.org and define a word for you. Just add the word you need to identify after the “d:” like so;
(we also pipe the command through ‘less’ so we have a better view of the output)

curl dict.org/d:website |less
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   550    0   550    0     0   3179      0 --:--:-- --:--:-- --:--:--  3160
220 pan.alephnull.com dictd 1.12.1/rf on Linux 4.4.0-1-amd64 <auth.mime> <81911058.2765.1568247019@pan.alephnull.com>
250 ok
150 1 definitions retrieved
151 "website" wn "WordNet (r) 3.0 (2006)"
website
    n 1: a computer connected to the internet that maintains a
         series of web pages on the World Wide Web; "the Israeli web
         site was damaged by hostile hackers" [syn: {website},
         {website}, {internet site}, {site}]
.
250 ok [d/m/c = 1/0/28; 0.000r 0.000u 0.000s]
221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

check this one out yourself:

curl dict.org/d:curl |less

Final Thoughts

Just like many other Linux commands, cURL has multiple benefits and uses. It is a versatile and functional part of a more extensive toolkit you have available to meet various needs. Can you think of a few situations where cURL can be useful to you?

Learn More Today!

Have thoughts, questions, or concerns about this or any other topic?
We have Support and Solution Teams standing by around the clock who have an in-depth knowledge of these and other web hosting technologies, especially those discussed in this article. Give us a call today at 1.800.580.4985, or open a chat or ticket with us to explore how you can take advantage of these or other opportunities we have available.
We look forward to hearing from you!