Difference between revisions of "Cool Solution - Grafana"

From Univention Wiki

Jump to: navigation, search
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Version|UCS=4.2}}  
+
{{Version|UCS=4.3}}  
 
{{Cool Solutions Disclaimer}}
 
{{Cool Solutions Disclaimer}}
 
{{#seo:
 
{{#seo:
 
|title={{#replace:{{#replace:{{#replace:{{#replace:{{FULLPAGENAME}}|'|'}}|&|&}}|"|"}}|Cool Solution - |}} - {{SITENAME}}
 
|title={{#replace:{{#replace:{{#replace:{{#replace:{{FULLPAGENAME}}|'|'}}|&|&}}|"|"}}|Cool Solution - |}} - {{SITENAME}}
Grafana offers an open source dashboards for visualizing your data. It can be used, for example, for visualizing data from [[ElasticSearch]]
+
|description=Grafana offers an open source dashboards for visualizing your data. It can be used, for example, for visualizing data from Elastic Search
 
}}
 
}}
{{Review-Status}}
 
  
 
Grafana offers an open source dashboards for visualizing your data. It can be used, for example, for visualizing data from [[ElasticSearch]].
 
Grafana offers an open source dashboards for visualizing your data. It can be used, for example, for visualizing data from [[ElasticSearch]].
  
 
== Installation ==
 
== Installation ==
 
 
=== Set the Repository===
 
=== Set the Repository===
 
 
Add the Grafana GPG Key
 
Add the Grafana GPG Key
 
 
<pre>
 
<pre>
wget -qO - https://packagecloud.io/gpg.key | sudo apt-key add -
+
wget -qO - https://packagecloud.io/gpg.key | sudo apt-key add -
 
</pre>
 
</pre>
  
 
Add the repository
 
Add the repository
 
 
<pre>
 
<pre>
echo "deb https://packagecloud.io/grafana/stable/debian/ jessie main" | tee -a /etc/apt/sources.list.d/elastic-5.x.list
+
echo "deb https://packagecloud.io/grafana/stable/debian/ stretch main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list
 
</pre>
 
</pre>
  
 
=== Install Grafana ===
 
=== Install Grafana ===
 
+
Install Grafana
Install Grafana using apt
 
 
 
 
<pre>
 
<pre>
apt-get update && apt-get install grafana
+
univention-install grafana
 
</pre>
 
</pre>
  
Start Grafana and add it to the autostart list,
+
Start Grafana and enable it as an autostarting service
 
 
 
<pre>
 
<pre>
systemctl daemon-reload
 
systemctl start grafana-server
 
 
systemctl enable grafana-server.service
 
systemctl enable grafana-server.service
 +
systemctl start grafana-server.service
 
</pre>
 
</pre>
  
 
=== Open the Firewall ===
 
=== Open the Firewall ===
 
 
The user can reach Grafana by default using the port 3000. This port has to be opened in the firewall using the following ucr command
 
The user can reach Grafana by default using the port 3000. This port has to be opened in the firewall using the following ucr command
 +
<pre>
 +
ucr set security/packetfilter/tcp/3000/all=ACCEPT \
 +
security/packetfilter/tcp/3000/all/en="Grafana"
 +
</pre>
  
 +
Restart the univention Firewall
 
<pre>
 
<pre>
ucr set security/packetfilter/tcp/3000/all=ACCEPT
+
systemctl restart univention-firewall
ucr set security/packetfilter/tcp/3000/all/en="Grafana"
 
 
</pre>
 
</pre>
  
Restart the univention Firewall
+
== Grafana LDAP Integration ==
 +
The LDAP integration makes use of the memberOf overlay, please see the [http://docs.software-univention.de/manual-4.2.html#groups::memberof documentation] about activating it.
 +
 
 +
To enable the LDAP integration open the configuration file '''/etc/grafana/grafana.ini''', search for the LDAP authentication and activate the integration by setting the variable '''allow_sign_up''' to true.
 +
 
 +
The following script configures the actual LDAP integration to match your system. Please note that it will change your Domain Admins to be Grafana Admins and your Domain Users to be Editors. You might want to adapt it to your specific environment.
 +
<pre>
 +
cat > /etc/grafana/ldap.toml <<_EOL_
 +
 
 +
[[servers]]
 +
host = "$(ucr get ldap/server/name)"
 +
port = $(ucr get ldap/server/port)
 +
use_ssl = false
 +
start_tls = true
 +
ssl_skip_verify = false
 +
root_ca_cert = "/etc/univention/ssl/ucsCA/CAcert.pem"
 +
bind_dn = "$(ucr get ldap/hostdn)"
 +
bind_password = "$(cat /etc/machine.secret)"
 +
search_filter = "(&(uid=%s)(univentionObjectType=users/user))"
 +
search_base_dns = ["$(ucr get ldap/base)"]
 +
 
 +
# Specify names of the ldap attributes your ldap uses
 +
[servers.attributes]
 +
name = "givenName"
 +
surname = "sn"
 +
username = "uid"
 +
member_of = "memberOf"
 +
email =  "mailPrimaryAddress"
 +
 
 +
# Map ldap groups to grafana org roles
 +
[[servers.group_mappings]]
 +
group_dn = "cn=Domain Admins,cn=groups,$(ucr get ldap/base)"
 +
org_role = "Admin"
 +
 
 +
[[servers.group_mappings]]
 +
group_dn = "cn=Domain Users,cn=groups,$(ucr get ldap/base)"
 +
org_role = "Editor"
 +
 
 +
[[servers.group_mappings]]
 +
group_dn = "*"
 +
org_role = "Viewer"
 +
 
 +
_EOL_
 +
</pre>
  
 +
Now restart Grafana again.
 
<pre>
 
<pre>
systemctl restart univention-firewall
+
systemctl restart grafana-server.service
 
</pre>
 
</pre>
  
 
== Configure Grafana ==
 
== Configure Grafana ==
 
 
Grafana utilizes multiple plugins to import and visualize data. Navigate your web browser to
 
Grafana utilizes multiple plugins to import and visualize data. Navigate your web browser to
 
 
<pre>
 
<pre>
http://<ip of your server>:3000
+
https://<ip of your server>:3000
 
</pre>
 
</pre>
  
Line 67: Line 103:
  
 
=== ElasticSearch ===
 
=== ElasticSearch ===
 
 
Please ensure, that you have configured not only ElasticSearch but also, that you have already collected Data in the Database.
 
Please ensure, that you have configured not only ElasticSearch but also, that you have already collected Data in the Database.
  
During the configuration, you will be asked to configure a Datasearch. Here you can select [[ElasticSearch]]. If you followed our guide, than select the URL '''http://localhost:9200''' and set Access to '''proxy'''
+
During the configuration, you will be asked to configure a Datasearch. Here you can select [[ElasticSearch]]. If you followed our guide, select the URL '''http://localhost:9200''' and set Access to '''proxy'''
  
The index in our example is composed out of the '''[logstash-info]-YYYY.MM.DD'''. For the Pattern select '''Daily'''. Leave the Time field name and select version '''5.x'''. Most beats for ElasticSearch write every minute. Thus '''1m''' in Min interval should be sufficient. If you are monitoring some servers more frequently, you might want to change it to something different, e.g., '''10s'''.
+
The index in our example is composed out of the '''[filebeat-6.2.3-]YYYY.MM.DD'''. For the Pattern select '''Daily'''. Leave the Time field name and select version '''5.6+'''.  
 +
Most beats for ElasticSearch write every minute. Thus '''1m''' in Min interval should be sufficient. If you are monitoring some servers more frequently, you might want to change it to something different, e.g., '''10s'''.
 +
 
 +
The index for metricbeat data has the same format '''[metricbeat-6.2.3-]YYYY.MM.DD'''. You can find more indices with following command
 +
<pre>
 +
curl -XGET 'localhost:9200/_cat/indices?v'
 +
</pre>
  
 
=== Dashboard ===
 
=== Dashboard ===
 
 
Depending on how much data you have in your database, you might need to wait a couple of minutes before you can start creating a new Dashboard. To create a dashboard merely select the desired components and drag them into the Dashboard.
 
Depending on how much data you have in your database, you might need to wait a couple of minutes before you can start creating a new Dashboard. To create a dashboard merely select the desired components and drag them into the Dashboard.
  
[[Category:EN]]
+
== Archive ==
 +
* There is a version of this article for [https://wiki.univention.de/index.php?title=Cool_Solution_-_Grafana&oldid=13410 UCS 4.2].

Latest revision as of 14:06, 6 April 2018

Produktlogo UCS Version 4.3

Note: Cool Solutions are articles documenting additional functionality based on Univention products. Not all of the shown steps in the article are covered by Univention Support. For questions about your support coverage contact your contact person at Univention before you want to implement one of the shown steps.

Also regard the legal notes at Terms of Service.

Grafana offers an open source dashboards for visualizing your data. It can be used, for example, for visualizing data from ElasticSearch.

Installation

Set the Repository

Add the Grafana GPG Key

wget -qO - https://packagecloud.io/gpg.key | sudo apt-key add -

Add the repository

echo "deb https://packagecloud.io/grafana/stable/debian/ stretch main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list

Install Grafana

Install Grafana

univention-install grafana

Start Grafana and enable it as an autostarting service

systemctl enable grafana-server.service
systemctl start grafana-server.service

Open the Firewall

The user can reach Grafana by default using the port 3000. This port has to be opened in the firewall using the following ucr command

ucr set security/packetfilter/tcp/3000/all=ACCEPT \
security/packetfilter/tcp/3000/all/en="Grafana"

Restart the univention Firewall

systemctl restart univention-firewall

Grafana LDAP Integration

The LDAP integration makes use of the memberOf overlay, please see the documentation about activating it.

To enable the LDAP integration open the configuration file /etc/grafana/grafana.ini, search for the LDAP authentication and activate the integration by setting the variable allow_sign_up to true.

The following script configures the actual LDAP integration to match your system. Please note that it will change your Domain Admins to be Grafana Admins and your Domain Users to be Editors. You might want to adapt it to your specific environment.

cat > /etc/grafana/ldap.toml <<_EOL_

[[servers]]
host = "$(ucr get ldap/server/name)"
port = $(ucr get ldap/server/port)
use_ssl = false
start_tls = true
ssl_skip_verify = false
root_ca_cert = "/etc/univention/ssl/ucsCA/CAcert.pem"
bind_dn = "$(ucr get ldap/hostdn)"
bind_password = "$(cat /etc/machine.secret)"
search_filter = "(&(uid=%s)(univentionObjectType=users/user))"
search_base_dns = ["$(ucr get ldap/base)"]

# Specify names of the ldap attributes your ldap uses
[servers.attributes]
name = "givenName"
surname = "sn"
username = "uid"
member_of = "memberOf"
email =  "mailPrimaryAddress"

# Map ldap groups to grafana org roles
[[servers.group_mappings]]
group_dn = "cn=Domain Admins,cn=groups,$(ucr get ldap/base)"
org_role = "Admin"

[[servers.group_mappings]]
group_dn = "cn=Domain Users,cn=groups,$(ucr get ldap/base)"
org_role = "Editor"

[[servers.group_mappings]]
group_dn = "*"
org_role = "Viewer"

_EOL_

Now restart Grafana again.

systemctl restart grafana-server.service

Configure Grafana

Grafana utilizes multiple plugins to import and visualize data. Navigate your web browser to

https://<ip of your server>:3000

The default username is admin and the password likewise is admin

ElasticSearch

Please ensure, that you have configured not only ElasticSearch but also, that you have already collected Data in the Database.

During the configuration, you will be asked to configure a Datasearch. Here you can select ElasticSearch. If you followed our guide, select the URL http://localhost:9200 and set Access to proxy

The index in our example is composed out of the [filebeat-6.2.3-]YYYY.MM.DD. For the Pattern select Daily. Leave the Time field name and select version 5.6+. Most beats for ElasticSearch write every minute. Thus 1m in Min interval should be sufficient. If you are monitoring some servers more frequently, you might want to change it to something different, e.g., 10s.

The index for metricbeat data has the same format [metricbeat-6.2.3-]YYYY.MM.DD. You can find more indices with following command

curl -XGET 'localhost:9200/_cat/indices?v'

Dashboard

Depending on how much data you have in your database, you might need to wait a couple of minutes before you can start creating a new Dashboard. To create a dashboard merely select the desired components and drag them into the Dashboard.

Archive

  • There is a version of this article for UCS 4.2.
Personal tools