I called upon the services of a good friend and a self proclaimed geek.
He looked into something supposedly called the system log, and soon enough he found the problem and fixed it. Now my favourite OS is working as it should.
But this set me thinking. What was this enigma my dear friend had called 'The System Log'.
What was this awesome thing that helped Linux maintain its favourite position in my eyes.
That was when I decided I had to learn everything I could about system logs
and that was when i decided to share this knowledge with the rest of the world.
As it comes out, a system log is a bunch of text files maintained by (almost) all operating systems where they record everything that is taking place inside the OS. You boot your PC, it will be recorded in the log. You insert a pen drive, it will be recorded in the log. You start utorrent, it will be recorded in the log. In short, there is no escaping this monster.
When your systems are running smoothly, you should take some time out to learn and understand the contents of various log files, which will help you when there is a crisis and you have to look though the log files to identify the issue.
Please note that Ubuntu(Debian based Linux) will be used in the following examples.
For a first look at a log file, fire up terminal and type-
# tail -f /var/log/kern.log
By default, the tail command displays the last 10 lines in a file (kern.log here). You can also specify the number of lines(from the end) that you desire by using tail -
the -f option is used to print every new line updated in the file, after the commend has been given. If -f is used, press ctr+c to exit the 'follow' mode
Other options for looking at log files are
# less /var/log/messages
# vi /var/log/messages
# more -f /var/log/messages
Now that we are comfortable with accessing a log file, lets have a look at some important log files in Linux.
/var/log/messages – Contains global system messages, including the messages that are logged during system start up. There are several things that are logged in /var/log/messages including mail, cron, daemon, kern, auth, etc.
/var/log/dmesg – Contains kernel ring buffer information. When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process. These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten.
/var/log/auth.log – Contains system authorization information, including user logins.
/var/log/boot.log – Contains information that are logged when the system boots
/var/log/daemon.log – Contains information logged by the various background daemons that runs on the system
/var/log/dpkg.log – Contains information that are logged when a package is installed or removed
/var/log/kern.log – Contains information logged by the kernel. Helpful for you to troubleshoot a custom-built kernel.
/var/log/lastlog – Displays the recent login information for all the users. This is not an ASCII file. You should use lastlog command to view the content of this file.
/var/log/maillog or /var/log/mail.log – Contains the log information from the mail server that is running on the system. For example, sendmail logs information about all the sent items to this file
/var/log/user.log – Contains information about all user level logs
/var/log/Xorg.x.log – Log messages from the X
/var/log/alternatives.log – Information by the update-alternatives are logged into this log file. On Ubuntu, update-alternatives maintains symbolic links determining default commands.
/var/log/btmp – This file contains information about failed login attempts. Use the last command to view the btmp file. For example, “last -f /var/log/btmp | more”
/var/log/cups – All printer and printing related log messages
/var/log/anaconda.log – When you install Linux, all installation related messages are stored in this log file
/var/log/yum.log – Contains information that are logged when a package is installed using yum
/var/log/cron – Whenever cron daemon (or anacron) starts a cron job, it logs the information about the cron job in this file
/var/log/secure – Contains information related to authentication and authorization privileges. For example, sshd logs all the messages here, including unsuccessful login.
/var/log/wtmp or /var/log/utmp – Contains login records. Using wtmp you can find out who is logged into the system. who command uses this file to display the information.
/var/log/faillog – Contains user failed login attempts. Use faillog command to display the content of this file.
Apart from the above log files, /var/log directory may also contain the following sub-directories depending on the application that is running on your system.
/var/log/httpd/ (or) /var/log/apache2 – Contains the apache web server access_log and error_log
/var/log/lighttpd/ – Contains light HTTPD access_log and error_log
/var/log/conman/ – Log files for ConMan client. conman connects remote consoles that are managed by conmand daemon.
/var/log/mail/ – This subdirectory contains additional logs from your mail server. For example, sendmail stores the collected mail statistics in /var/log/mail/statistics file
/var/log/prelink/ – prelink program modifies shared libraries and linked binaries to speed up the startup process. /var/log/prelink/prelink.log contains the information about the .so file that was modified by the prelink.
/var/log/audit/ – Contains logs information stored by the Linux audit daemon (auditd).
/var/log/setroubleshoot/ – SELinux uses setroubleshootd (SE Trouble Shoot Daemon) to notify about issues in the security context of files, and logs those information in this log file.
/var/log/samba/ – Contains log information stored by samba, which is used to connect Windows to Linux.
/var/log/sa/ – Contains the daily sar files that are collected by the sysstat package.
/var/log/sssd/ – Use by system security services daemon that manage access to remote directories and authentication mechanisms.
But how does log work
whenever the OS does any task, it stores a description of the task in the log file
initially the log file kern.log looks like
Linux also allows us to access multiple dynamically updated log files in the same terminal.
the latest version of the tail command allows multiple -f arguments, as shown below
$ tail -f /var/log/syslog -f /var/log/kern.log
Download multitail as
$ apt-get install multitail
after which you can use the following command
$ multitail /var/log/syslog /var/log/auth.log
Multitail utility has a lot of additional features as explained in the mutitail home page.
For example, you can scroll back in a log file, display log files in colors and search inside log files while using multitail.
$ gunzip a.gz to unzip a.gz
also note that you can use
$ gzip a.txt so as to zip a file a.txt
substituting the name of the log file in logfilename
The output of
A USEFUL SHELL SCRIPT TO PARSE THROUGH LOG FILES
Our very ambition in life is to make your life simpler, while not making our life too complicated. Thus we have created this useful(atleast we think its useful) shell script(bash) which can parse through a few important log files.
The six subdirectories are Firewall, HTTPERR, Scm, Srt, WMI, WUDF. The text log files SrtTrail and httperr1 in Srt and HTTPERR are two of the important files. Particular information can be found out by searching for the relevant keyword in the text files.
Under Windows Logs, there are 5 options for various type of logs arranged under Application, Security, Setup, System and Forwarded Events. The Application logs store information about the starting and stopping of any service or application whereas the Setup and System logs journal the setting up and installation of services and every system message respectively.
When a new device is introduced
When we connected a pen drive, new logs were viewable after refreshing the window which read similar to:
Driver Management concluded the process to install driver FileRepository\wpdfs.inf_amd64_neutral_fc4ebadff3a40ae4\wpdfs.inf for Device Instance ID WPDBUSENUMROOT\UMB\2&37C186B&0&STORAGE#VOLUME#_??_USBSTOR#DISK&VEN_KINGSTON&PROD_DT_100_G2&REV_1.00#001CC0EC321AFB6187162150&0# with the following status: 0x0.
Rudresh Pande 2011093