HEAD requests

Please keep in mind that this post is about 3 years old.
Technology may have changed in the meantime.

A quicky this time.

I needed a simple script that would make a HEAD request for a certain URL, and then make a new HEAD request if the result was a redirect, etc.

Well, this is it:

#!/usr/bin/env bash

# This script makes a HEAD request for the given URL.
# If the result has a status 301 or 302,
# a new HEAD request is made for the new URL.
# Etc.

# For more info, see
# https://www.ohreally.nl/2020/12/10/head-requests/

################################################################################

# Copyright (c) 2020 Rob La Lau 

# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:

# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

################################################################################

# Dependencies.
# `ncat' is part of Nmap.
AWK=`which awk`   || exit 1
NCAT=`which ncat` || exit 1
SED=`which sed`   || exit 1

# Usage message.
usage() {
	echo "Usage: $0 "
	echo
	echo "Only http and https URLs are supported."
	exit 1
}

# First (and only) parameter is the URL.
url=$1
[ -z "${url}" ] && usage

# URL parts.
proto=${url%%://*}
uri=${url##*://}
host=${uri%%/*}
path=${uri##*/}
if [ "${host}" = "${path}" ]; then
	path="/"
else
	path="/${path}"
fi

# SSL/TLS, or not?
case ${proto} in
	http)
		param=""
		port=80
		;;
	https)
		param="--ssl"
		port=443
		;;
	*)
		usage
		;;
esac

# Make request and process result.
while read line; do
	echo ${line}
	case ${line} in
		HTTP/*)
			status=$(echo ${line} | "${AWK}" '{print $2}')
			;;
		Location:*)
			# Line ends in CRLF.
			location=$(echo ${line} | "${SED}" -E 's/^[a-z:]+ (.*)/\1/i;s/\r$//')
			;;
		*)
			# Ignore.
			;;
	esac
done < <(printf "HEAD ${path} HTTP/1.1\r\nHost: ${host}\r\n\r\n" | "${NCAT}" ${param} ${host} ${port})

# Rinse repeat.
[ ${status} -eq 301 -o ${status} -eq 302 ] && $0 ${location}

Looks good, doesn’t it…?

Note:
With a badly configured web server, this script may loop forever. If that happens, hit ^C (Ctrl-C).

This script can also be downloaded from my Github account.

REPUBLISHING TERMS

You may republish this article online or in print under our Creative Commons license. You may not edit or shorten the text, you must attribute the article to OhReally.nl and you must include the author’s name in your republication.

If you have any questions, please email rob@ohreally.nl

License

Creative Commons License AttributionCreative Commons Attribution
HEAD requests