Getting MetricBeat to talk to the Elastic Stack in Docker

Getting a 3 node Elasticsearch Cluster with a separate Kibana node is surprisingly easy with Elastic’s well documented Docker support.

I’m wanting to explore using MetricBeat to retrieve statistics from Azure using the Azure Monitor metricset. So I figured that running MetricBeat in Docker would also make sense.

My experience with Docker is limited as I don’t use it day to day so when the Metricbeat ‘setup’ step was failing with ‘connection refused’ I started to research Docker networking.

The instructions for running the Metricbeat setup prompts you to substitute your Kibana and Elasticsearch hosts when running:

docker run \ \
setup -E \
-E output.elasticsearch.hosts=["elasticsearch:9200"]

I first attempted to switch the Elasticsearch host to -E output.elasticsearch.hosts=["localhost:9200"] as Elasticsearch responds to queries to localhost:9200 after the cluster is spun up. This resulted in an error saying

Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://localhost:9200: Get http://localhost:9200: dial tcp connect: connection refused]

A short lesson in Docker bridge networks later it was clear my amendment wasn’t fit for purpose. The 3 Elasticsearch nodes and Kibana were all running in the 172.18.0.X ‘elastic’ network defined in docker-compose.yml.

    driver: bridge

While reading a StackOverflow answer which tipped me off about the --network= parameter for docker run, I attempted to switch my execution to the following as th docker-compose.yml seemed to refer to the bridge network as ‘elastic

docker run --network=elastic setup -E -E output.elasticsearch.hosts=["localhost:9200"]

The error message changed to Error response from daemon: network elastic not found. This felt odd as that’s what it was referred to in the YAML and looked similar to the StackOverflow answer.

The missing piece of my understanding was when I asked Docker to list the networks to find that the bridge was named elastic-stack_elastic by executing a docker network ls command.

Changing the Setup command got me one step further by running the following and switching the Elasticsearch Host IP to the internal network address:

docker run --network=elastic-stack_elastic setup -E -E output.elasticsearch.hosts=[""]

Then I need to replace the kibana hostname. After a bit of trial and error I discovered it was running on

So the final version of the command which worked is:

docker run --network=elastic-stack_elastic setup -E -E output.elasticsearch.hosts=[""]

This ran the index setup and loaded the Metricbeat sample dashboards into Kibana.

Next step is to create the configuration file and start collecting data. That’s for another blog post. Happy gathering.

Community builder

I chatted with Jess White last Friday lunchtime about the Women In Tech meetup again for my Twine Radio podcast.

After getting back to the Tech Nottingham Slack #general channel, Tom asked how I chose the people for the podcast. It’s made me take a step back and remind myself where I was, what I was doing and who, in no small part, was responsible for this.

I’ve mentioned before that the idea for Twine Radio came from being part of the Nottingham Tech community and being inspired by the Christmas Party that Tech Nottingham organised two years ago.

At each Tech Nottingham event Andrew Seward reminds us of the driving force behind Tech Nottingham is principally to “make Nottingham a better place to live and work in technology”. And the call to action is to talk to one another, to share ideas, meet new people, make new connections.

So as I was reflecting on recording another Twine Radio podcast, I reminded myself I was doing just that. Going out and meeting new people, recording their stories and helping them spread the word about more opportunities to do all of the above.

It’s not that Nottingham wasn’t already the host of some of the UK’s best tech talent. It’s not that Nottingham wasn’t full of people doing creative and interesting things with technology. It’s not that we weren’t blogging & sharing things individually.

However, it’s in no small part through his enthusiasm, energy and passion that Andrew Seward has lit a fire in the hearts of those who have heard his “call to arms”.

And to hammer home the point, on the same day, Andrew was out and about spreading the word at Experian and then London.

This was hot on the back of last Monday’s City AM piece where Andrew reminded us that “we’ve created this by working together”.

Let’s just pause and remind ourselves that Friday was Andrew’s birthday. He was like “nope, no celebrating for me, I need to go out and network & spread the word for Nottingham!”


Thank you Andrew for all the inspiration, hard work & dedication that has made us come together in ways we’d never have thought of. Long may it continue. You are a true asset to Nottingham and we’re lucky to have you fighting our corner.


Hack24 2016

The wait is over. Winter may be coming but so is Hack24 2016 and I can’t wait. Clear your calendar for the weekend of 19th & 20th March 2016 because you’ll want to be free to take part in this excellent event.

The first Hack24 was such a roaring success I’m sure 2016’s event will build on the great work all the team put in across the board.

Of course, they’re going to keep us on the edge of our seats for a while longer to find out exactly where it’ll take place, when the tickets will be available & all of the sponsorship news as it unfolds. This truly feels like an early Christmas present 🙂

If you’d like to sponsor Hack24 2016 get in touch with them at


Fixing upgraded Windows 10 HP Desktop with broken System Restore partition

I know that this post is very niche but given I spent around 11 hours of last Saturday fixing an Hewlett Packard All-In-One Computer that couldn’t boot I thought I’d share my findings in the hope it saves someone else the pain.

The computer had stopped booting into Windows and was showing a flashing cursor on a black background after the HP splash screen. Also, the F11 System Restore option was missing meaning the ‘quick fixes’ weren’t instantly available.

The HP Compaq 6000 Pro All-In-One I was attempting to fix was a refurbished model which had come with a Windows 7 Pro for Refub PC product key. It had been recently upgraded to Windows 10 using an in-place upgrade which at the moment is still free of charge.

When I got the machine to look at, I started off booting in to Linux Mint off a USB stick to try and get to the bottom of it. After booting, Mint showed me the contents of the NTFS drive so it wasn’t a total drive failure and allowed me to back up the user’s Documents folder. I did fix some drive errors in case it was bad sectors that were preventing it booting.

The ‘achilles heel’ to all of this is HP’s decision to create at least two partitions on the main hard drive: a small System recovery partition and a larger Windows O.S. partition. The recovery partition was created at the time the original Windows 7 O.S. had been installed and contains a WinRE branded O.S. to guide you through HP’s restore options.

A lot of advice was guiding me to try a System Repair so initially I started with a Windows 10 bootable USB stick that I’d created off my Windows 10 laptop. Upon trying to run System Repair it failed and told me that it’d logged the error to a file (not massively helpful). Once I’d swapped in to a Command Prompt in the Advanced options, I found it told me that the System partition wasn’t fixable by this version of Windows.

With a copy of a Windows 7 CD I attempted to System Repair the recovery partition using the initial OS. However, this too failed as it tries to find a valid Windows Installation to add to the Master Boot Record (MBR) and of course it couldn’t see one as Windows 7 had been replaced by Windows 10.

So it felt I was in an impossible ‘chicken and egg’ scenario as a result of HP’s decision to make the recovery partition the bootable one. I got sidetracked a little assuming the EFI Boot directory was essential until I discovered this PC didn’t support EFI.

The breakthrough came when I realised that the System Repair partition was currently marked as the Bootable partition and not the O.S. drive. Switching this flag over allowed the Windows 10 System Repair to make the O.S. drive bootable and found a valid Windows 10 installation on the bootable partition. A nervous reboot later I found I was greeted by the Windows 10 login screen once more.

Given the eleven hours I spent fixing it in total, the relief and joy was palpable and needless to say the family member for whom I was fixing the PC was overjoyed. I had avoided needed to wipe the drive and start again which was increasingly becoming a possibility as time marched on. Arguably with a proper backup of the user data this would have always been the saner option.

I did at one point engage HP’s online chat help who were very friendly but when I blurted out my solution, overjoyed, I’m pretty sure the operator dealing with my inquiry wouldn’t have necessarily added my fix to their list of suggestions.

So the moral of this story is, if you’re taking advantage of Microsoft’s free Windows 10 installation on a Windows 7 HP PC maybe remove the System Recovery partition before you start? I found an option to do that in the HP Support tool somewhere.

Alternatively if you’re in the ‘broken O.S. state’ I was in you can hopefully see the light at the end of the tunnel. Either way, good luck 🙂

Tech Nottingham springs forward

There’s lots of exciting things going on in Nottingham at the moment. In fact as a city there’s always been a buzz around the place; an ‘anything is possible’ feeling all the way from the City Council down to those who live and work in it. We are also lucky to have a strong technology sector based in the city in a wide range of sectors.

The buzz is not limited to energy applied within companies as we’re lucky to have a community of like-minded individuals willing and keen to share ideas, passions and best practices in community events out of business hours.

Tech Nottingham is a monthly event that is launching from May 2014 and will be based out of Antenna which is already the focus for a lot of great events like Nott Tuesday and Second Wednesday.

It’s based on the great foundation of GeekUp Nottingham which Gemma Cameron setup in 2010. I’ve been an attendee and a speaker on a small handful of occasions and thoroughly enjoy the chance to meet up and share ideas with people I don’t get the chance to chat to in my day to day job at Esendex.

I’m pleased and supportive of the rebranding to Tech Nottingham for two main reasons.

First is that I personally don’t feel the event to date has gained much from the association with the GeekUp brand. It’s a good ‘ice breaker’ to give people a clue to what it might be about. However it also has the potential with the word ‘Geek’ to maybe turn some folk away.

I’m sure each GeekUp in the land is different and there is no right or wrong way to run one. I’ve just not witnessed any central ‘force’ pushing the success of GeekUps; no cross-retweeting, no central campaigning, no push to local media etc.
The success of GeekUp in Nottingham has been very much down to Gemma, Andrew Seward (its current Captain) and its members. So I don’t think breaking association harms the event at all.

Secondly, Tech Nottingham is aiming to continue to differentiate from other tech social events in the area by throwing its net even wider. The successful format already sees talks by experts intersperced with katas, dojos and retrospectives to name but a few of the different session types tried over the years.

In Andrew’s latest post on the new Tech Nottingham website he paints the picture of the future involving

hack events, campaigns, spin-off meetups on specialist subjects, workshops, training, help for startups, charities and community groups”. 

This exciting mission statement is the perfect time for a rebrand to something that can reflect all that it is hoped this community can become. It’s also clear that Andrew is looking to the members themselves to help him deliver this which I look forward to seeing how I can assist.

Personally being involved with GeekUp and other community projects like Code Club is that it’s not just personal development that can come out of voluntarily meeting up with other folk. There’s a chance to continue to share the knowledge to a wider, even non technical, audience through activities that can benefit others than the participants. The simplest goal of making others know that there’s meet ups like this

The first meeting (if you will) of Tech Nottingham will be at Antenna on Monday, 12th May at 6.30pm. Hope to see you there!

N.B. I’m pleased that we didn’t end up with ‘Tech Monday’ 🙂

6 steps to update Jenkins Slave Agents on Windows

After having updated Jenkins to the latest Long Term Service release (1.509.4) I wanted to ensure the Windows Slave Agents that we’ve got running were up to date. The Windows Slave agents had previously been installed using the Java Web Start process and then turned into Windows Service slave agents.

To reinstall the Slave Agents we had to do the following:

  1. Stop the ‘Jenkins Slave’ Windows service
  2. Launch a Command Prompt (ensuring its run as an Administrator)
  3. type the following (assuming your Slave Agent was installed at c:jenkins)
    • sc delete jenkinsslave-c__jenkins
  4. Browse to your Node definition on your Master Jenkins instance in a web browser (Jenkins -> Manage Jenkins -> Manage Nodes -> Node)
  5. Click the Java Web Start process & run it on your Slave machine
  6. Select the option to install as a Windows Service from the File menu.

The ‘magic sauce’ was finding the ‘sc delete’ command which allowed us to quickly upgrade all of our Windows Slaves.

Hope it helps you maintain your Jenkins instances.

Getting Windows Git Bash to Hitch

On a Windows 7 machine that already had Windows Git bash version 1.7.9-preview20120201 on it, I wanted to add ‘hitch’. Hitch is a great little Ruby program that manipulates the Git author settings to properly attribute pair programming work. Rather than a commit having a single author, Hitch creates a unique email address from a known base email address and adding the pair names after a + symbol.

This is useful as when we push code up to GitHub it becomes clear which pair have been working on a task. You can also create a Gravatar for the unique concatenated email address so the pair combination can have their own logo too.

The installation is simple: gem install ruby. The Windows machine didn’t have Ruby on it so I went to Ruby Installers for Windows and downloaded v1.9.3-p125. make sure you tick the box in the install wizard to add the Ruby executables to your PATH. When Git Bash starts it’s loaded with the Windows PATH settings anyway so it should be able to find Ruby to install hitch.

Hitch itself is a breeze to setup using your GitHub usernames.
hitch username1 username2

Hitch will ask if you want to add the usernames to its local hitch_pairs files to save you entering their full name again. Enter their full name and you’re good to go. Now each subsequent git commit will have the correct author information. Pair programming is great and Hitch allows both members of the pair to have their work attributed.

Foreign Raspberries

The exciting Raspberry Pi project is nearing fruition but it seems their attempts to boost the UK economy by getting the boards made over here has been thwarted by lead times and an insane tax law.

It seems UK manufacturers end up paying tax on components, but if they get the boards made overseas and then shipped over then that’s ok because they’re completed products rather than raw materials. Bottom line is that this insanity means that foreign outsourcing of manufacturing gets even more attractive.

So in these times of austerity it would seem the sensible thing to do to support UK manufacturing by lobbying for a change in those tax laws. We have a lot of manufacturing talent in the UK and we should be supporting it.

If you want to support a lobby on this issue there’s an e-Petition been set up to register your support at

12 steps to using GitHub with XCode 4

The Git integration in XCode 4 is very welcome however it seems to be a bit flaky when it comes to dealing with remote repositories. For clarity I’m using Mac OS X version 10.6.7 and XCode 4.0.2 (4A2002a).

If I create a new XCode 4 Project and accept the option to create a local Git repository, all is well. I can commit locally and that’s great.

When I want to push the new project to GitHub I want to add a remote repository and push my local changes up to GitHub. I can add a GitHub repository to XCode 4 as a new Repository in the Organizer window but there’s no method of setting it as a remote repository for my existing project.

So instead here’s the 12 steps that I followed to get GitHub to work with a new XCode 4 project

Continue reading 12 steps to using GitHub with XCode 4