APM with Elastic Agent 9.x and Fleet Server

Introduction

This is a follow up to using APM Binary with Elasticsearch and Kibana. In this example, we will use Docker to start up a NodeJS application , Elastic Agent with Fleet Server, and Elasticsearch plus Kibana as the APM Log Server and Visualizer.

Much of the docker explanation is explained in this Elasticsearch + Docker Compose: Full Explanation video.

Download Code: Coming soon

Requirements

ELK - ubuntu 24.04, 8gb memory
App - ubuntu 24.04, 1gb memory

Steps

Step 1 - Install Elasticsearch Kibana

On the Elasticsearch Server, Place this install.sh script on to the server you wish to use as your Elasticsearch Kibana APM logging server.

Update the variable baseip at the top of the file with the IP address of your Elasticsearch Kibana APM server.

Run chmod +x install.sh && ./install.sh

When things are complete, reset the elastic super user password by running /usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u elastic.

Create an enrollment token with this command /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana. Copy the enrollment token, you will need it later.

Type systemctl status kibana to get the url you are supposed to visit. It should look something like https://0.0.0.0/?code=129302. Visit this url in your web browser (You may need to replace the 0.0.0.0 with the actual IP address of your elasticsearch server. Paste in your enrollment token and submit it.

Now you can login to the Elasticsearch and Kibana with the password you created for the elastic user.

Step 2 - Make TLS For Fleet Server

From the elasticsearch server, run this:

fleetip='ip.of.app.server' mkdir -p /etc/certs /usr/share/elasticsearch/bin/elasticsearch-certutil cert \ --out /etc/certs/fleet.zip \ --name fleet \ --ca-cert /etc/elasticsearch/certs/http_ca.crt \ --ca-key /etc/elasticsearch/certs/http_ca.key \ --ip $fleetip \ --pem;

If you choose to use domain names, replace --ip $fleetip with --dns your.domain.or.hostname.

Then run these commands from your elasticsearch server:

fleetip='ip.of.app.server' ssh root@$fleetip 'mkdir -p /etc/certs/elasticsearch/' scp /etc/elasticsearch/certs/http_ca.crt root@$fleetip:/etc/certs/elasticsearch/http_ca.crt scp /etc/certs/fleet.zip root@$fleetip:/etc/certs/ ssh root@$fleetip 'cd /etc/certs/ && unzip fleet.zip'

Step 3 - Setup Fleet Server

Go to Kibana.

Go to Fleet » Settings » Add a Fleet Server » Advanced.

Use these settings:

Create Fleet Server Policy (you can collect systems metric)
Choose a Deployment Mode: Production
Add your Fleet Server Host: Name Anything you want, URL: https://ip.address.of.app.server:8220, then press Add Host
Click Generate Service Token

curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-9.2.4-linux-x86_64.tar.gz
tar xzvf elastic-agent-9.2.4-linux-x86_64.tar.gz
cd elastic-agent-9.2.4-linux-x86_64
sudo ./elastic-agent install --url=https://{ip.of.app.server}:8220 \
  --fleet-server-es=https://{ip.of.elasticsearch.kibana}:9200 \
  --fleet-server-service-token={autogenerated} \
  --fleet-server-policy=fleet-server-policy \
  --fleet-server-es-ca-trusted-fingerprint={autogenerated} \
  --certificate-authorities=/etc/certs/elasticsearch/http_ca.crt \
  --fleet-server-cert=/etc/certs/fleet/fleet.crt \
  --fleet-server-cert-key=/etc/certs/fleet/fleet.key \
  --fleet-server-port=8220 \
  --install-servers

The --certificate-authorities is for the ca that signed --fleet-server-cert.

Step 4 - Setup APM

Go to Kibana » Integrations » APM » Add Elastic APM.

Use these fields:

Integration Name: Give a name
Host: localhost: 8200
Url: http://localhost:8200
Agent Authorization » Secret Token: anything you want
Existing hosts » Agent policies: Fleet Server policy name from Step 3

Step 5 - Install Application

See the Testing Applications of the Beginner's Guide to install and test APM.

Step 6 - Observe Additional Services

The benefit of Elastic Agent and Fleet Server is the ability to observe more services. Let's demonstrate by installing Apache2 webserver.

apt-get install -y apache2

Now visit http://ip.of.app.server to verify web page is running.

Go to Kibana » Integrations » Apache HTTP Server » Add Apache HTTP Server.