Recent content in Home of Robert Fletcher on My Ramblings
A friend of mine came to me with a problem. He had a bunch of games servers that was being hosted internally on a old pc turned into a proxmox server. Currently the way to access these servers was opening more and more ports on the ISP provided router turning the firewall into something resembling swiss cheese. Furthermore being on a home internet connection there was no grantee that the External…
GitOps is getting a lot of buzz as the next evolution of DevOps, to put it in practice I am going to use it to manage my new Kubernetes cluster. I wanted to manage everything from application to DNS entries through an automated process, if I could I did not want to use any UI or type any command. History time… If we turn back the clock 15 years before the terms like DevOPS were not in wide…
As a Software developer when starting a new project, the first thing you must do is setup your workspace. Sometimes that is just git clone the repo to your laptop and you have all the tools installed. But I find most of the time you are presented with a blank server where you will do all your development in. Depending on how mature the project is and the makeup of the team you might just get a…
God Dam it Microsoft, fine just take my money! The story starts last month when one of my parent’s fell for one of the gift card scammers emails. To be fair to my parents the hacker did not compromise their account, I have tried to teach them good password quality. What the scammers did was compromised their friend’s account and send the email. Being unsure about this out of the blue…
Over the past 6 months, I have slowly been moving all my sites to be built using static site generators. I am currently enjoying the ease of editing some markdown and it producing a site. My last post I moved this blog from ghost to being built by a static site generator. The static site generator I chosed was Hugo. I like the simplicity, it is fast and has a good amount of support especially…
The title is not clickbait this entire site and webserver is being hosted in a container that is smaller than one of the photos I have taken. Ok slight cavitate my photos are taken in RAW and can average around 50mb per image. So maybe hosting an entire blog in 50mb. But before you guess no it not using volume mounts or anything like that all the images and post content is bundled in the…
I current maintain my own public gallery over the weekend I wanted to try and automatic image classification using Go and Tensorflow and to understand how PhotoPrism classification system works
Backing up my data with Rclone, Nextcloud to Backblaze
Its 2021 and a lot of changes have happened since I last did a blog post on my home-lab. Something big happened in 2020 which resulted in doing most of my work from home. Luckily for me things like desks monitors and an office chair I always had since leaving university. Other changes since my last blog post is that I have swapped a monthly rent to this mortgage thing which means I now have the…
Its the end of 2020 due to all the fun events that caused us all to stay at home, or in my case just business as usual, I been tasked to add more and more services to my ever expanding homelab (blog post coming soon ). This has caused me to go out and buy a bigger VPS and required me to move the existing services from one server to another.
This document wuill go through how to setup rbac with ldap and hpow to extrac varibles from the ldap user to be used to perform specific quiries on a index to restic data based on a number of key fields and a set of custom logic where by a user must have all values of certain fields and have at lest 1 of another field Setup Elastic cluster This requires a elasticsearch cluster to have xpack…
Its summer 2019 time to upgrade the homelab services. As you can see below there are several to upgrade: from the Wiki (Bookstack) a taiga instance, monitoring provided by grafana and prometheus the password manager and authentication Services For the most part the process was simple. As discussed in eariler blog enteries all these services are in docker containers and each service has its own…
Some detail documentation on how I have setup almost single sign-on to Gitea, Taiga.io, Grafana, Portainer and Bookstack using Openldap and Keycloak This post has spun from this reddit post over at r/selfhosted about how to get openldap working with keycloak. While I am no expert it did remind me about the pain it was and so it be a good idea to fully document it somewhere for later reference.
Over the past week I added a small pc to the network to help massively upgrade the services and simplify the authentication process. The problem I wanted to solve was that I do not want to run the HP DL380 server 24/7 it’s loud and more important consumes 200-300 watts idle rather a lot of power for a server doing nothing. But I do want some key services Git and Scrum management to run 24/7…
Summer 2018 and I am back on a train from conference on Elasticsearch and the solutions it provides.
I am currently on a train heading down to London for Elasticon and been reading the whole Microsoft is buying GitHub. The reaction from places like Twitter and hacker news is that the sky is falling and we should all move to gitlab. Gitlab.io rather then setting up a self hosted version While I think that the whole sky is falling drama is way over-blone, Microsoft has taken some good steps in the…
its spring time to clean the server, setup openstack and sort out the mess of nginx configs
First of all I should just mention what is openshift for those who are not familiar with it. Openshift is a platform as a service (PaaS). While you can use docker to manage a few containers and tools like docker-compose for services (multiple containers that make up an application, website+ database for example). Things get way way more complex once you have multiple physical hosts and want…
SSH on windows has always sucked now I found a better way Terminus. OK although I am a bit of a linux advacate my main workstation at home is windows 10 pro while at work is Ubuntu which is probably a little unsual and to complete the circle my laptop is a macbook air I am truly multiplatform. But the one of things that has fustrated my about my workstation is that there is no good ssh client.
Over the past month I have experimented with many different tools for documentation and monitoring my home-lab and cloud.
Over christmas I been visiting family and friends so I was away from my house and my small homelab but I still wanted a way to do work / check up on my cluster. Previously I mentioned the vpn tunnel I had setup between the raspberry pi and the webserver hosted by OVH. This allowed me to ssh into the server without exposing my home IP or opening ports since I have a the standard BT home router and…
The nights are drawing in and its getting colder outside the leaves on the trees are changing to that wonderful autumn colour so time for some cluster upgrades. Most blogs document the install procedure of thing X but are less likly to document the running of thing X, Yes this is probably not that interesting for anyone but myself to read but as this blog is in part to document work I have done on…
So its bank holiday Monday I just come back from a lovely walk on the hills, just made a fresh cup of tea and I get this phone called claiming there are from BT. The say that they have seen some unusual traffic on my internet connection and they say its not secure. This raised my spidey-sense that this is another of these scams where they want to install something on your computer.
Its been a while since I last did a update on the current infrastructure. Firstly there has been no major hardware changes. The HP server still runs libvirt managed QEMU Virtual Machines, and all is working well. The main change is that I finally have some new services deployed in to these virtual machines. Before I start here is the overview diagram of the current architecture with the focus on…
First of I need to state before I start that I am not a data scientist nor am I any expert in R, so the code in this post should not really be used as a guide. A bit of back story I been wanting to have a go at doing something that is more data science then some other software engineering project. A good hello world seemed to me was do some Sentiment analysis, it felt a step up from just doing…
Why did I choose to learn Go, In part it was to add another language to my repertoire, I seen it listed as the main technologies behind some of the latest services for cluster orchestration and minoring such as Prometheus.io . As I am a kind of person that when learning something new likes to have a project associated with it and I don’t want to just build the age old todo app I though I…
The main work here was looking at how hard it was to setup a ELK stack for metric collection and was where I was storing the IRC data before closing the doors on that project for the moment. For me that are lots of great things to say about using a ELK stack for metric collection and processing log files form all small different data sources. I can see why it has become the almost de facto…
Its been over 3 months since the last blog post. Where I mentioned some of project ideas I was planning on working on. Out of the 3 projects( Golang/React Project, Sentiment Analysis, Cluster Architecture) I have done about 1 and 1/2 of them so not that bad. I ended up dropping the IRC Sentiment Analysis partly because my data source (Gamealtion IRC Chat) dried up as moved over to Discord (which…
Over the next few months there are several projects I want get started to explore several new technologies from new languages to Cluster Architecture technologies with some machine learning in top. I have broken the new technologies into 3 separate projects each one covers something different. These projects are listed below. Microstack a Golang/React Project The first is a project called…
Its been over a week since the last update to the home lab so below is the following changes that I have made and some experiments with VPN tunnels Below is the network diagram of the lab so far, to make the diagram more simple I have removed the power component (see part 1 for the full diagram) We will ignore the OVH VM for the moment which I will explain in a bit.
In My adventures of creating my home lab that’s a tiny version of AWS so I can spin up several VMS into a small cluster to experiment with some of the big data tools out there and give them a metaphorical kicking to see what they can do and if the be useful for me. In the previous post a created a small script that does the boiler plate work of setting up a vm.
Time to get some virtual machines running. The main plan is to use Ubuntu cloud images, they are small 300MB approx compared to Centos7 minimal 700MB. The problem with using cloud images is that they are designed to be run in large data center with something like AWS or OpenStack. Since I am running one machine I cant’t use OpenStack (its really designed for 3 host or greater) it can be run…
I think Github is up there as one of the greatest contributions in producing software there has been since the creation of the web but I thinks it has some problems. The main crux of my problem is the centralisation of all the code even from big companies such as google and important projects like python. I can see why the large centralisation of the community in one place allows for easy…
My new project for the past months and for the next few months is the construction of my home lab. The purpose of this lab is to learn some new technologies that I could not with using a hosted solution as I did for the past 3 years using a server from OVH. The Diagram below shows the current state of my little lab that been made up from some scavenged components form old companies hardware that…
Its the start of a new year (2017) and its time to be a little bit more proactive and try something new. Well not new this is not 2004 but for me this blog will be a place that I can dump bits of my brain for future reference and may be it be useful to you the reader if you come across it. It will a be the place to record my adventures in playing around with new technology many focused on…