Getting MetricBeat to talk to the Elastic Stack in Docker

Getting a 3 node Elasticsearch Cluster with a separate Kibana node is surprisingly easy with Elastic’s well documented Docker support.

I’m wanting to explore using MetricBeat to retrieve statistics from Azure using the Azure Monitor metricset. So I figured that running MetricBeat in Docker would also make sense.

My experience with Docker is limited as I don’t use it day to day so when the Metricbeat ‘setup’ step was failing with ‘connection refused’ I started to research Docker networking.

The instructions for running the Metricbeat setup prompts you to substitute your Kibana and Elasticsearch hosts when running:

docker run \
docker.elastic.co/beats/metricbeat:7.8.0 \
setup -E setup.kibana.host=kibana:5601 \
-E output.elasticsearch.hosts=["elasticsearch:9200"]

I first attempted to switch the Elasticsearch host to -E output.elasticsearch.hosts=["localhost:9200"] as Elasticsearch responds to queries to localhost:9200 after the cluster is spun up. This resulted in an error saying

Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://localhost:9200: Get http://localhost:9200: dial tcp 127.0.0.1:9200: connect: connection refused]

A short lesson in Docker bridge networks later it was clear my amendment wasn’t fit for purpose. The 3 Elasticsearch nodes and Kibana were all running in the 172.18.0.X ‘elastic’ network defined in docker-compose.yml.

networks:
  elastic:
    driver: bridge

While reading a StackOverflow answer which tipped me off about the --network= parameter for docker run, I attempted to switch my execution to the following as th docker-compose.yml seemed to refer to the bridge network as ‘elastic

docker run --network=elastic docker.elastic.co/beats/metricbeat:7.8.0 setup -E setup.kibana.host=kibana:5601 -E output.elasticsearch.hosts=["localhost:9200"]

The error message changed to Error response from daemon: network elastic not found. This felt odd as that’s what it was referred to in the YAML and looked similar to the StackOverflow answer.

The missing piece of my understanding was when I asked Docker to list the networks to find that the bridge was named elastic-stack_elastic by executing a docker network ls command.

Changing the Setup command got me one step further by running the following and switching the Elasticsearch Host IP to the internal network address:

docker run --network=elastic-stack_elastic docker.elastic.co/beats/metricbeat:7.8.0 setup -E setup.kibana.host=kibana:5601 -E output.elasticsearch.hosts=["172.18.0.3:9200"]

Then I need to replace the kibana hostname. After a bit of trial and error I discovered it was running on 172.18.0.1:5601.

So the final version of the command which worked is:

docker run --network=elastic-stack_elastic docker.elastic.co/beats/metricbeat:7.8.0 setup -E setup.kibana.host=172.18.0.1:5601 -E output.elasticsearch.hosts=["172.18.0.3:9200"]

This ran the index setup and loaded the Metricbeat sample dashboards into Kibana.

Next step is to create the configuration file and start collecting data. That’s for another blog post. Happy gathering.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.