Mastodon

December 21, 2019

259 words 2 mins read

Curl as a daily driver

Curl as a daily driver

cURL is an amazing tool, chain it along with jq and you’ve got a JSON parsing API reader right there in your shell.

But it’s so much more, so here are a few ways I’ve used it in the past.

Firstly say you want to verify a file of URLs to check they all return 200, you can do that in one line: 

cat file.txt |while read line; do echo -n $line returns: && curl -s --max-time 2 -o /dev/null -w "%{http_code}" $line;echo;done
Outputs: 
www.bbc.co.uk returns:301
blah.com returns:000
www.google.co.uk returns:200
https://www.google.co.uk returns:200

Load Balancers are there to help right… well sometimes they don’t; sometimes they fail in odd ways or maybe someone didn’t push to all nodes of a clustered load balancer and you’re now getting 20% of connections fail. Sure you could edit you /etc/hosts file to point your url to each node of the load balancer… five times.

Or you could use this script replace the IPs in the 3rd line with the IPs of your load balancer ./curlmany.sh mytestdomain.com and you’ll quickly see where the problem is.

#!/usr/bin/bash
  
for i in 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14;

  do echo -en "$i:\t"; curl --resolve "$1:443:$i" -so /dev/null -w "%{remote_ip}\t %{http_code} -> %{redirect_url}"  https://$1:443;
    echo
  done

Curl has many options, go see the man himself or read the book

curl