<h1 id="nagios-basics">Nagios Basics</h1>
<h2 id="introduction">Introduction</h2>
<p>I noticed nagios on the requested topics page. I am far from being an expert with nagios and there is a lot I do not know. I have a working knowledge of most of the basic nagios principles. So, hopefully, I can give a useful introduction and review some one the principles of nagios along the way</p>
<p>Nagios is a network monitoring tool. You define some things for nagios to check, and nagios will alert you if those checks fail.</p>
<p>Nagios has a web UI that is normally used to see the status of the checks. There are some basic administration tasks you can do from the web UI</p>
<ul>
<li>enabling/disabling notifications</li>
<li>Scheduling Downtime</li>
<li>Forcing immediate checks</li>
</ul>
<p>Nagios is primarily configured with text files. You have to edit the nagios config files for things like</p>
<ul>
<li>adding servers</li>
<li>customizing commands</li>
</ul>
<h2 id="nagios-core-vs-nagiosxi">Nagios core vs NagiosXI</h2>
<p>NagiosXI is the commercial version of nagios. NagiosXI requires a paid license and includes support. NagiosXI has some extra features including wizards for adding hosts and easy cloning of hosts.</p>
<p>I have used NagiosXI, and personally don't find the extra features very useful. Probably the biggest reason to use NagiosXI is Enterprise that requires commercial support</p>
<p>The <code>community</code> version of nagios is normally referred to as <code>nagios core</code> This episode will focus on the nagios core</p>
<h2 id="nagios-documentation">Nagios Documentation</h2>
<p>I don't like the official nagios core documentation. A lot like man pages, It is a good reference, but can be hard to follow.</p>
<p>Maybe is it possible for someone to read the documentation and be able to install and configure nagios for the first time. But it took me a lot of trial and error to get a functional nagios server following the nagios documentation</p>
<p>Outside of the official documentation, Most of the nagios installation guides I found online recommend downloading and building nagios from the nagios site. My general policy is to use OS provided packages whenever possible. Normally, sticking to packages eases long the term maintenance.</p>
<p>You may not always get the latest feature release, but installation and updates are usually easier. I know not everyone will agree with me here, and will want to build the latest version. Regardless of the install method, most of the nagios principles I go over will still apply</p>
<p>I am making the assumption that most listeners will be most familiar with Debian/Ubuntu, so I will go over installing nagios on Ubuntu using the nagios packages from the Ubuntu repository</p>
<h2 id="hosts-and-services">Hosts and Services</h2>
<p>Before I go over the installation, I'll talk a bit about some of the pieces that make up nagios Nagios checks are for either hosts or services.</p>
<p>From the Nagios documentation</p>
<blockquote>
<p>A host definition is used to define a physical server, workstation, device, etc. that resides on your network.</p>
</blockquote>
<p>Also from the nagios documentation</p>
<blockquote>
<p>A service definition is used to identify a "service" that runs on a host. The term "service" is used very loosely. It can mean an actual service that runs on the host (POP, SMTP, HTTP, etc.) or some other type of metric associated with the host</p>
</blockquote>
<p>Normally, hosts are checked using ping. If the host responds to the ping with in the specified time frame, the host is considered up. Once a host is defined and determined to be UP, you can optionally check services on that host</p>
<h2 id="installation-and-setup">Installation and setup</h2>
<p>Install the packages</p>
<pre><code>apt install nagios4</code></pre>
<p>One of the dependencies is the monitoring-plugins I'll talk more about the monitoring-plugins package when we dig in to the checks</p>
<p>The primary UI for nagios is a cgi driven web app usually served via apache. Following the nagios4 installation, the web UI isn't functional. So we need to make a few configuration changes</p>
<p>The nagios config file for apache contains a directive that is not enabled by default</p>
<p>Enable 2 Apache modules</p>
<pre><code>a2enmod authz_groupfile
a2enmod auth_digest
systemctl restart apache2</code></pre>
<h2 id="nagios-authentication">Nagios authentication</h2>
<h3 id="enable-users-in-the-nagios-ui">Enable users in the nagios UI</h3>
<p>In <code>/etc/nagios4/cgi.cfg</code> change the line</p>
<pre><code>&#39;use_authentication=0&#39;</code></pre>
<p>to</p>
<pre><code>&#39;use_authentication=1&#39;</code></pre>
<h3 id="modify-apache">Modify Apache</h3>
<p>In <code>/etc/apache2/conf-enabled/nagios4-cgi.conf</code> change</p>
<pre><code>Require all granted</code></pre>
<p>to</p>
<pre><code>Require valid-user</code></pre>
<p>And if needed, remove the IP restriction by removing the line that starts with</p>
<pre><code>Require ip</code></pre>
<p>And finally we need to add a nagios basic auth user. I normally use <code>nagiosadmin</code>, but it can be any username</p>
<pre><code>htdigest  -c /etc/nagios4/htdigest.users Nagios4 nagiosadmin</code></pre>
<h4 id="restarts">Restarts</h4>
<p>Restart apache and nagios and the nagios UI will be fully functional</p>
<h2 id="check-commands">Check commands</h2>
<p>Nagios uses a collection of small standalone executables to perform the checks. Checks are either OK, Warning, or Critical, depending on the exit code of the check.</p>
<table>
<thead>
<tr class="header">
<th style="text-align: center;">Exit Code</th>
<th>Status</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">0</td>
<td>OK/UP</td>
</tr>
<tr class="even">
<td style="text-align: center;">1</td>
<td>WARNING</td>
</tr>
<tr class="odd">
<td style="text-align: center;">2</td>
<td>CRITICAL</td>
</tr>
</tbody>
</table>
<p>The check commands are standalone applications that can be run independent from nagios. Running the checks from the shell is helpful to better understand how the nagios checks work. The location of the check commands can vary depending on how nagios was packaged. In this case, they are in <code>/usr/lib/nagios/plugins</code></p>
<p>Looking at the names on the files can give you an idea of their purpose. For example, it should be obvious what <code>check_http</code> and <code>check_icmp</code> are for.</p>
<pre><code>cd /usr/lib/nagios/plugins
$ ./check_icmp localhost
OK - localhost: rta 0.096ms, lost 0%|rta=0.096ms;200.000;500.000;0; pl=0%;40;80;; rtmax=0.218ms;;;; rtmin=0.064ms;;;;
$ ./check_http localhost
HTTP OK: HTTP/1.1 200 OK - 10977 bytes in 0.005 second response time |time=0.004558s;;;0.000000;10.000000 size=10977B;;;0</code></pre>
<p>Most checks can be run with <code>-h</code> to print usage help</p>
<p>The checks can be in any language as long as is it is executable by the nagios server. Many are compiled C but Perl and shell scripts are also common</p>
<pre><code>file check_icmp
check_icmp: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=46badf6e4322515a70d5553c8018a20e1e9b8206, for GNU/Linux 3.2.0, stripped</code></pre>
<h2 id="nagios-config-files">Nagios config files</h2>
<p>The primary nagios config file is <code>/etc/nagios4/nagios.cfg</code></p>
<p>nagios.cfg has a directive that will load additional user generated files</p>
<pre><code>cfg_dir=/etc/nagios4/conf.d</code></pre>
<p>I like to put all my additions to nagios in this directory and use git for both version control and backup.</p>
<h2 id="nagios-commands">Nagios commands</h2>
<p>Nagios doesn't run the check executable directly The checks have to be explicitly defined in as a command Some predefined commands are in <code>/etc/nagios4/objects/commands.cfg</code></p>
<p>Debian package <code>monitoring-plugins-basic</code> contains several command definitions that are loaded by nagios.cfg <code>cfg_dir=/etc/nagios-plugins/config</code></p>
<p>Lets look in the <code>/etc/nagios-plugins/config</code> at <code>ping.cfg</code> for an example of how commands are defined</p>
<pre><code># &#39;check-host-alive&#39; command definition
define command{
    command_name    check-host-alive
    command_line    /usr/lib/nagios/plugins/check_ping -H &#39;$HOSTADDRESS$&#39; -w 5000,100% -c 5000,100% -p 1
    }</code></pre>
<p>Commands require <code>command_name</code> and <code>command_line</code> The command line is that path to the executable that will perform the check and optional arguments. Most checks require <code>-H</code> for the host address to check The check-host-alive command also contains arguments to set the critical and warning thresholds with <code>-c</code> and <code>-w</code></p>
<p>The check_ping command is similar the check-host-alive command except it requires 2 arguments to set the critical and warning thresholds.</p>
<pre><code>define command{
        command_name    check_ping
        command_line    /usr/lib/nagios/plugins/check_ping -H &#39;$HOSTADDRESS$&#39; -w &#39;$ARG1$&#39; -c &#39;$ARG2$&#39;
        }</code></pre>
<h2 id="templates">Templates</h2>
<p>Hosts and services require a lot of reused variables. Object definitions normally use <strong>templates</strong> to avoid having to repetitively set the same variables on each host. Nagios normally ships with predefined templates for hosts and services that will work for most cases.</p>
<p>In Ubuntu, the templates are defined in <code>/etc/nagios4/objects/templates.cfg</code>. Template definitions are the same as other object definitions, except they contain <code>register 0</code> which designates the object as a template. I'll show how the templates are used when I go over the host and service definitions.</p>
<h2 id="notifications">Notifications</h2>
<p>By default, notifications are sent via email to nagios@localhost. The easiest way to get notifications is to configure the nagios server to forward emails to a monitored email address. Since many networks block sending email directly via SMTP, email forwarding may be challenging.</p>
<p>In a follow up episode I will cover setting up postfix to relay mail through a mail sending service and maybe some other methods for sending alerts</p>
<h2 id="localhost">Localhost</h2>
<p>By default, nagios is set to monitor localhost. Having the nagios server can be useful but you probably want to add some additional servers.</p>
<p>Have a look at <code>/etc/nagios4/objects/localhost.cfg</code> if you want to see how the checks for localhost are defined</p>
<h2 id="adding-a-new-host-to-monitor">Adding a new host to monitor</h2>
<p>We will use google.com as an example and create a file named <code>google.cfg</code> and place it in in the cfg_dir <code>/etc/nagios4/conf.d</code>.</p>
<p>The files can be named anything that ends in <code>.cfg</code>. My preference is one file per host that contains all the checks for that host. The content of <code>google.cfg</code> is included new the end of the show notes.</p>
<p>First, we need to define the host. <code>host_name</code> is the only field required to be set. The remaining requirements are met by using the <code>generic-host</code> template.</p>
<p>We can add a service check to google.com using the same file. The easiest to add is a http check <code>host_name</code>, <code>service_description</code>, and <code>check_command</code> have to be set the remaining requirements are met by using the <code>generic-service</code> template.</p>
<h2 id="restarting-nagios">Restarting Nagios</h2>
<p>Nagios has to be reloaded to pick up the configuration changes. Prior to restarting nagios, you can verify the nagios configuration is valid by running:</p>
<pre><code>nagios4 -v /etc/nagios4/nagios.cfg</code></pre>
<p>This will print a summary of the configuration. Any warnings or errors will be printed at the end.</p>
<p>Warnings are not fatal, but should probably be looked at. Errors will keep nagios from restarting; if there are no errors, it is safe to restart nagios</p>
<p>Check the nagios UI at <code>https://SERVER_IP/nagios4</code> and you should see 2 hosts, localhost and google.com as well as the service checks for the hosts</p>
<h2 id="next-episode">Next Episode</h2>
<p>Since I have already made the mistake of mentioning a follow up episode, I know I am now committed to making additional episode, Next time I will try to cover some enhancements to nagios, including</p>
<ul>
<li>some notification options</li>
<li>monitoring-plugins packages</li>
<li>writing custom checks</li>
<li>using SNMP to monitor load average and disk usage</li>
</ul>
<p>Leave a comment if there are other aspects of nagios you would like me to try to cover. No promises, but I will do my best.</p>
<p><strong>Thanks for listening and I will see you next time.</strong></p>
<h2 id="files">Files</h2>
<h3 id="playbook">Playbook</h3>
<pre><code>---
- hosts: nagios
  tasks:
  - name: install nagios
    apt:
      name:
        - nagios4
      update_cache: yes

  - name: Enable the Apache2 modules
    command: a2enmod &quot;{{item}}&quot;
    with_items:
    - authz_groupfile
    - auth_digest
  - name: modify nagios cgi config to require user
    replace:
      path: /etc/nagios4/cgi.cfg
      regexp: &#39;use_authentication=0&#39;
      replace: &#39;use_authentication=1&#39;
  - name: nagios require valid user
    replace:
      path: /etc/apache2/conf-enabled/nagios4-cgi.conf
      regexp: &quot;Require all  granted&quot;
      replace: &quot;Require valid-user&quot;
  - name: remove IP restriction
    lineinfile:
      regexp: &quot;Require ip&quot;
      path: /etc/apache2/conf-enabled/nagios4-cgi.conf
      state: absent
  - name: move auth requirements out of File restrictions
    lineinfile:
      path: /etc/apache2/conf-enabled/nagios4-cgi.conf
      regexp: &#39;^s*&lt;/?Files&#39;
      state: absent
  - name: nagios user
    copy:
      dest: /etc/nagios4/htdigest.users
      src: htdigest.users
  - name: restart apache
    service:
      name: apache2
      state: restarted
  - name: copy nagios configs
    copy:
      src: &quot;{{item}}&quot;
      dest: /etc/nagios4/conf.d
    with_items:
      - google.cfg
  - name: restart nagios
    service:
      name: nagios4
      state: restarted</code></pre>
<h3 id="google.cfg">google.cfg</h3>
<pre><code>define host {
  host_name google.com
  use generic-host
}

define service {
  use generic-service
  host_name google.com
  service_description HTTP
  check_command check_http
}</code></pre>
<h3 id="htdigest.users">htdigest.users</h3>
<pre><code>nagiosadmin:Nagios4:85043cf96c7f3eb0884f378a8df04e4c</code></pre>
