Yesterday I created a proof of concept script, which basically goes off and identifies the hitcounts of a website, and can give a technician within a short duration of time (minutes instead of hours) exactly where hitcounts are coming from and where.<\/p>\n
This is kind of a tradeoff, between a script that is automated, and one that is flexible.<\/p>\n
The end goal is to provide a hitcount vs memory commit metric value. A NEW TYPE OF METRIC! HURRAH! (This is needed by the industry IMO).<\/p>\n
And also would be nice to generate graphing and mean, average, and ranges, etc. So can provide output like ‘stat’ tool. Here is how I have progress<\/p>\n
\r\n#!\/bin\/bash\r\n#\r\n# Author: \tAdam Bull, Cirrus Infrastructure, Rackspace LTD\r\n# Date: \tMarch 20 2017\r\n# Use:\t\tThis script automates the analysis of webserver logs hitcounts and\r\n# \t\tprovides a breakdown to indicate whether outages are caused by website visits\r\n#\t\tIn correlation to memory and load avg figures\r\n\r\n\r\n# Settings\r\n\r\n# What logfile to get stats for\r\nlogfile="\/var\/log\/httpd\/google.com-access.log"\r\n\r\n# What year month and day are we scanning for minute\/hour hits\r\nyear=2017\r\nmonth=Mar\r\nday=9\r\n\r\necho "Total HITS: MARCH"\r\ngrep "\/$month\/$year" "$logfile" | wc -l;\r\n\r\n# Hours\r\nfor i in 0{1..9} {10..24};\r\n\r\ndo echo " > 9th March 2017, hits this $i hour";\r\ngrep "$day\/$month\/$year:$i" "$logfile" | wc -l;\r\n\r\n # break down the minutes in a nested visual way thats AWsome\r\n\r\n# Minutes\r\nfor j in 0{1..9} {10..60};\r\ndo echo " >>hits at $i:$j";\r\ngrep "$day\/$month\/$year:$i:$j" "$logfile" | wc -l;\r\ndone\r\n\r\ndone\r\n<\/pre>\nThing is, after I wrote this, I wasn’t really happy, so I refactored it a bit more;<\/p>\n
\r\n#!\/bin\/bash\r\n#\r\n# Author: \tAdam Bull, Cirrus Infrastructure, Rackspace LTD\r\n# Date: \tMarch 20 2017\r\n# Use:\t\tThis script automates the analysis of webserver logs hitcounts and\r\n# \t\tprovides a breakdown to indicate whether outages are caused by website visits\r\n#\t\tIn correlation to memory and load avg figures\r\n\r\n\r\n# Settings\r\n\r\n# What logfile to get stats for\r\nlogfile="\/var\/log\/httpd\/someweb.biz-access.log"\r\n\r\n# What year month and day are we scanning for minute\/hour hits\r\nyear=2017\r\nmonth=Mar\r\nday=9\r\n\r\necho "Total HITS: $month"\r\ngrep "\/$month\/$year" "$logfile" | wc -l;\r\n\r\n# Hours\r\nfor i in 0{1..9} {10..24};\r\n\r\ndo\r\nhitsperhour=$(grep "$day\/$month\/$year:$i" "$logfile" | wc -l;);\r\necho " > $day $month $year, hits this $ith hour: $hitsperhour"\r\n\r\n # break down the minutes in a nested visual way thats AWsome\r\n\r\n# Minutes\r\nfor j in 0{1..9} {10..59};\r\ndo\r\nhitsperminute=$(grep "$day\/$month\/$year:$i:$j" "$logfile" | wc -l);\r\necho " >>hits at $i:$j $hitsperminute";\r\ndone\r\n\r\ndone\r\n<\/pre>\nNow it’s pretty leet.. well, simple. but functional. Here is what the output of the more nicely refined script; I’m really satisfied with the tabulation.<\/p>\n
\r\n[root@822616-db1 automation]# .\/list-visits.sh\r\nTotal HITS: Mar\r\n6019301\r\n > 9 Mar 2017, hits this hour: 28793\r\n >>hits at 01:01 416\r\n >>hits at 01:02 380\r\n >>hits at 01:03 417\r\n >>hits at 01:04 408\r\n >>hits at 01:05 385\r\n^C\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"Yesterday I created a proof of concept script, which basically goes off and identifies the hitcounts of a website, and can give a technician within a short duration of time (minutes instead of hours) exactly where hitcounts are coming from … Continue reading