So, I was testing with curl today and I know that it’s possible to direct to /dev/null to suppress the page. But that’s not very handy if you are checking whether html page loads, so I came up with some better body checks to use.
A Basic body check using wc -l to count the lines of the site
time curl https://www.google.com/ > 1; echo "non zero indicates server up and served content of n lines"; cat 1 | wc -l
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 167k 0 167k 0 0 79771 0 --:--:-- 0:00:02 --:--:-- 79756
real 0m2.162s
user 0m0.042s
sys 0m0.126s
non zero indicates server up and served content of n lines
2134
A body check for Google analytics
$ time curl https://www.groundworkjobs.com/ > 1; echo "Checking for google analytics html elements string"; cat 1 | grep "www.google-analytics.com/analytics.js"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 167k 0 167k 0 0 76143 0 --:--:-- 0:00:02 --:--:-- 76152
real 0m2.265s
user 0m0.042s
sys 0m0.133s
Checking for google analytics html elements string
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
Such commands might be useful when troubleshooting a cluster for instance, where one server shows more up to date versions, (different number of lines). There’s probably better way to do this with ls and awk and use the html filesize, since number of lines wouldn’t be so accurate.
Check Filesize from request
$ time curl https://www.groundworkjobs.com/ > 1; var=$(ls -al 1 | awk '{print $5}') ; echo "Page size is: $var kB"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 167k 0 167k 0 0 79467 0 --:--:-- 0:00:02 --:--:-- 79461
real 0m2.170s
user 0m0.048s
sys 0m0.111s
Page size is: 171876 kB
Pretty simple.. but you could take the oneliner even further… populate a variable called $var with the filesize using ls and awk , and then use an if statement to check that var is not 0, indicating the page is answering positively, or alternatively not answering at all.
Check Filesize and populate a variable with the filesize, then validate variable
$ time curl https://www.groundworkjobs.com/ > 1; var=$(ls -al 1 | awk '{print $5}') ; echo "Page size is: $var kB"; if [ "$var" -gt 0 ] ; then echo "The filesize was greater than 0, which indicates box is up but may be giving an error page"; fi
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 167k 0 167k 0 0 78915 0 --:--:-- 0:00:02 --:--:-- 78950
real 0m2.185s
user 0m0.041s
sys 0m0.132s
Page size is: 171876 kB
The filesize was greater than 0, which indicates box is up but may be giving an error page
The second exercise is not particularly useful or practical as a means of testing, since if the site was timing out the script would take ages to reply and make the whole test pointless, but as a learning exercise being able to assemble one liners on the fly like this is an enjoyable, rewarding and useful investment of time and effort. Understanding such things are the fundamentals of automating tasks. In this case with output filtering, variable creation, and subsequent validation logic. It’s a simple test, but the concept is exactly the same for any advanced automation procedure too.