First we need to isolate our access-logs. We can do that with a quick find.
find / -iname 'access-logs'
This should print out a list of access-logs (usually per domain).
Let’s search within out logs now.
grep "<Date/Here>" /file-location | grep "bot"|less -N
The above command looks for a date (22/Sep) within the file for all instances of “bot” we then “less” that and print the line numbers out (-N). We “less” as these files can become quite large and we only want to see a handful of results at a time to give us a manageable chunk of data
Bonus: group IP’s by count
For those of you that are interested in counting the amount of times a particular ip (or phrase) occurs in the access log.
grep 'IP.HERE' /location/of/access-log | cut -d' ' -f1 | sort | uniq -c | sort -r
Sort: sort lines of file
Cut: takes specific columns/chars out of the file given flags
Cut -d: take a whole word as opposed to characters
Cut -d -f1: Combined with -f1 to extract the first column.
Uniq: groups the keyword together, appending “-c” prints out the amount of results… this is the important part!
Finally with a trailing “sort” we list top of the bottom the amount of times the IP appears (-r placing the “most” at the top).