Cool Solution - ElasticSearch

From Univention Wiki

Revision as of 14:23, 1 December 2017 by Hpeter (talk | contribs) (→‎FileBeat)
Jump to: navigation, search
Produktlogo UCS Version 4.2

Note: Cool Solutions are articles documenting additional functionality based on Univention products. Not all of the shown steps in the article are covered by Univention Support. For questions about your support coverage contact your contact person at Univention before you want to implement one of the shown steps.

Also regard the legal notes at Terms of Service.
Note: This article is not yet reviewed.


Elastic Search together with Logstash allows the collection and processing of log files and metric data. ElasticSearch is the default backend for Kibana and can also be used with Grafana, both of which display the state of your environment.

Elastic Search in a VM

Please note, ElasticSearch is a database that communicates with other aspects of your environment. Therefore, it does not like to be suspended. While we did not experience any long-lasting issues, we had short-term inconsistencies during the test, which can create a lot of false positive errors in the frontends.

Installation

Install Java

Download Java JRE from Oracle and copy it onto the server using SCP.


Create a directory under the optional directory

mkdir /opt/jdk

extract Java to the new directory

tar zxf <file name> -C /opt/jdk

Add Oracle Java to the update-alternatives

update-alternatives --install /usr/bin/java java /opt/jdk/<java version>/bin/java 100
update-alternatives --install /usr/bin/javac javac /opt/jdk/<java version>/bin/javac 100

Install sudo

Install sudo from the Univention Repository

univention-install sudo

Set the Repository

Add the Elastic GPG Key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

Add the repository

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list

Install Elasticsearch

Install Elasticsearch

univention-install elasticsearch

Add elasticsearch to autostart

systemctl enable elasticsearch
systemctl start elasticsearch

Install Logstash

Logstash can be installed through the elastic repository

univention-install logstash

Add elasticsearch to autostart

systemctl enable logstash
systemctl start logstash

Restart

Restart Elasticsearch

systemctl restart elasticsearch

Configure Logstash

First Logstash needs multiple input and output plugins to process Data. Within this website, we will look at Beats and Syslog for input and output the Data to Elasticsearch.

Beats

Install the plugins for beats and syslog:

/usr/share/logstash/bin/logstash-plugin install logstash-input-beats
/usr/share/logstash/bin/logstash-plugin install logstash-output-elasticsearch

Edit the config file at /etc/logstash/conf.d/beats.conf

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

Logstash needs to have port 5044 open to accept Beats input. To open the port set the following UCR variable

ucr set security/packetfilter/tcp/5044/all=ACCEPT \
security/packetfilter/udp/5044/all=ACCEPT
service univention-firewall restart

Syslog

Install the plugins for beats and syslog:

/usr/share/logstash/bin/logstash-plugin install logstash-input-syslog
/usr/share/logstash/bin/logstash-plugin install logstash-output-elasticsearch

Edit the config file at /etc/logstash/conf.d/syslog.conf

input {
  syslog {
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][syslog]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

By default logstash will be using the default syslog port 514. You need to open it in the UCS firewall with:

ucr set security/packetfilter/tcp/514/all=ACCEPT \
security/packetfilter/udp/514/all=ACCEPT
service univention-firewall restart

Collect Data

Elasticsearch uses Beats to collect data. This article will look into Filebeat for collecting Logfiles and Metricbeats for collecting data on the server. We will also show you how to enable the Syslog input.

FileBeat

The FileBeat is used on every server to collect Log files. While traditionally Syslog has been employed for this task and forwarding data, there are some drawbacks in Syslog. The most important one is that some services do not use Syslog but merely write into a file. Also, the FileBeat is better at shaping traffic, if you direct a lot of routine traffic but need to react fast to errors.

You can install the FileBeat from the repository

sudo univention-install filebeat

Autostart the FileBeat

update-rc.d filebeat defaults 95 10

To configure the FileBeat, open /etc/filebeat/filebeat.yml and add the input and output section

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/*.log
    - /var/log/apache2/*.log
    - /var/log/apt/*.log
    - /var/log/samba/*.log
    - /var/log/univention/*.log
    - /var/log/syslog
output.logstash:
  hosts: ["<Hostname Logstash Search Server>:5044"]

Upload the index file to elastic search

curl -XPUT 'http://<Hostname Elastic Search Server>:9200/_template/filebeat' -d@/etc/filebeat/filebeat.template.json

Start File Beat

service filebeat start

Metricbeats

Metricbeats collects statistics of the system and sends them to elastic search.

You can install the Metricbeats from the repository

sudo univention-install metricbeat

The MetricBeat can be automatically started on boot with

update-rc.d metricbeat defaults 95 10

To configure the Beat, open

/etc/metricbeat/metricbeat.yml

and add the input and output section

metricbeat.modules:
- module: system
  metricsets:
    - cpu
    - load
    - core
    - diskio
    - filesystem
    - fsstat
    - memory
    - network
    - process
  enabled: true
  period: 10s
  processes: ['.*']
- module: apache
  metricsets: ["status"]
  enabled: true
  period: 1s
  hosts: ["http://127.0.0.1"]

name: "<server name>"
tags: ["UCS Server"]

output.logstash:
  hosts: ["<Hostname Logstash Search Server>:5044"]

logging.level: error

Start the beat with

service metricbeat start
Personal tools