Recently when deploying a new Kubernetes service I ran into this error... Nothing I could find online described what the issue was and it took a few days to realize the answer was simple. I had put the imagePullSecrets inside the......
Day 41 included updating my Unifi Flex Mini switch to the 2.0 firmware and adding yet again more scenes to Home Assistant....
Day 40 included setting up some more automations and scenes in Home Assistant....
Day 39 focused on instaling the smart lock from Day 38, and configuring some home assistant scenes. The lock install went as smoothly as it possibly could have. Given that its a zigbee device, all I had to do was fire up ZHA in Home Assistant and pair......
Day 38 was the first day back after taking some time off for the birth of my daugter. The first thing I did was create a new automation in home assistant so we can easily turn on some lights to low levels at night to feed and change her without waking......
Day 37 of #100DaysOfHomelab was spent adding and customizing some Home Assistant automations....
Day 36 of #100DaysOfHomelab was spent fixing some issues with ESPHome devices which had fallen out of Home Assistant with the dreaded unavailable erorr. They were all still online and could be talked to through the ESPHome editor......
Day 35 of #100DaysOfHomelab was spent updating dependencies on my Kubernetes cluster via the GitOps pull requests that automatically come through....
Day 34 of #100DaysOfHomelab was spent upgrading ESPHome and the related 20+ devices running it as firmware, as well as configuring more firewall rules that I'm porting from my previous PfSense installtion over to the Unifi Dream Machine Pro....
Day 33 of #100DaysOfHomelab was spent figuring out some auth issues I was having between Authelia, Traefik and my docker containers. All of my containers are running behind Authelia as a sigle sign on auth system. Since migrating my network and moving......
Day 32 of #100DaysOfHomelab was focused on getting my local DNS resolution working the way it should be from Day 31's cutover to the Unifi Dream Machine Pro. What I discovered early on, is that I had not set the DNS server on each subnet's DHCP settings......
Day 27 of #100DaysOfHomelab was the cutover day from PfSense into the Unifi Dream Machine Pro. A lot went well, and a lot went not so well. The migration itself went well, I took backups of everything on the PfSense side, including taking screenshots of......
Day 30 of #100DaysOfHomelab was spent migrating services from the HAProxy instance I have currently on my PfSense router, to an instance of Traefik running on a Raspberry PI 4. This is in preperation to cut my network over to using the Unifi Dream......
Day 29 of #100DaysOfHomelab was spent planning my network migration to use the new Unifi Dream Machine Pro I just bought from my current PfSense router. Unifi has come a long way in the past year or so with the featurues I originally move to PfSense for,......
Day 27 of #100DaysOfHomelab was spent adding Home Assistant entities for my Son's new room. He's transitioning out of the Nursery in preparation for our 2nd child and I had to add the door sensor, his sound machine, the camera we have mounted and a......
Day 27 of #100DaysOfHomelab was spent updating Home Assistant to 2023.6.2 and updating various outdated plugins via HACS....
Day 26 of #100DaysOfHomelab was spent updating TrueNAS from 22.12.2 to 22.12.3 . While that's mostly straight forward, it does require taking down everything else on the network which depends on the NAS's storage. That includes......
Day 25 of #100DaysOfHomelab was spent updating the various Raspberry Pis I have on my network. The first was a POE powered Raspberry Pi 4 which runs in the server rack and is used for monitoring the network. The second was a Raspberry Pi 3B which runs......
About a year ago, Laravel added strict mode which lets you turn off some convenience features that can impact performance or introduce bugs if you're not careful. It's a great thing to enable by default in your non production environments so that your......
I've been using Jigsaw to build this blog for the past few weeks. Static site generators have the advantage of being fast and secure, but they also have the disadvantage of being a bit more difficult to work with. One thing that is a bit of a pain is......
Day 24 of #100DaysOfHomelab was focused on the SSD pool errors I encountered yesterday. The errors did come back overnight, so I swapped the drive with another 500GB SSD I had lying around. Thankfully TrueNAS + ZFS makes this a pretty easy process. I......
Day 23 of #100DaysOfHomelab had me focusing on fixing some disk errors thrown recently by TrueNAS and a new backup strategy. The disk errors were on my SSD cluster of 5 500GB drives that I use to host VMs, Kubernetes Volumes, and shared configuration......
Day 22 of #100DaysOfHomelab was about fixing some more deprecated configurations with the latest version of Home Assistant. In the latest version they changed how command line switches are configured, so it required a few tweaks to get working again. I......
Jigsaw is an awesome static site generator that is built on PHP and Laravel components. In fact as of last week, it now powers this blog. One thing that I have noticed while creating posts is that if you add a new tag (morphed from the default......
Day 21 of #100DaysOfHomelab was focused around fixing some issues with my CCTV system. I use both Frigate and BlueIris for different functions. Frigate handles object detection using AI and triggers notifications through home assistant. BlueIris is what......
Day 20 of #100DaysOfHomelab was focused around upgrading HomeAssistant to the latest major version and fixing some mis-behaving Zigbee devices that have been dropping off the network randomly. The zigbee changes ended up being easy. Simply factory......
Day 19 of #100DaysOfHomelab was around reconfiguring my Frigate instance to only record on motion events. I already use BlueIris to record 24/7 and was only experimenting with Frigate continuous recording. This will allow me to reduce the storage on my......
Day 18 of #100DaysOfHomelab was a slow day of mostly applying update PRs to the K3s cluster that Renovate automatically created. I also backed up my old ghost blogs that I previously hosted in my lab, that I have now moved over to Cloudflare Pages and......
Day 17 of #100DaysOfHomelab involved me switching this very blog to not be in my homelab anymore! Much like the Return Early website, I moved this blog to Cloudflare Pages to take advantage of the insane speed that a statically generated site can......
Day 16 started with another fun power outage. This time with a transformer blowing and knocking out the power for 4 hours. Thankfully in Day 15 I had changed out the batteries with the new ones and they performed great. It gave me time to double check......
Today was about replacing the worn out batteries in my CyberPower UPS which have caused issues over the past two weeks during an abnormally frequent amount of power outages. Thankfully the replacement kit I ordered seems to have worked flawlessly. It......
The first thing on the to-do list today was moving the website for ReturnEarly, LLC out of my homelab and into a static site genrator hosted on Cloudflare Pages. Due to recent power outages and extended downtime at home, it was no longer viable to host......
As a continuation of Day 12, I focused on fixing the NVIDIA GPU issue I was facing on my docker host. After some back and forth, I discovered that the nvidia-headless-525 package doesn't include both libnvidia-encode-525 or......
Another day, another random power outage that caused my UPS to go haywire and not do it's job... In better news I did finally order new batteries for it to prevent the issue from happening a third time. In the fallout of the power outage, my TrueNAS box......
Today's work is all about HomeAssistant. The first thing I had to take care of was a bunch of updates to the ~30 or so ESPHome based devices we have. This is mostly Martin Jerry smart switches flashed with ESPHome along with some smart plugs of various......
Today was another continuation of debugging the storage issues I've been having between my Kubernetes cluster and my Truenas SCALE box. I tried swapping out the storage driver to use the hpe-storage but I'm still getting the same result: So that......
As part of my ongoing struggles getting NFS working with the Democratic-CSI driver in Kubernetes, I have started to look into alternatives. The one that has stood out so far is the one by hpe-storage: GitHub - hpe-storage/truenas-csp: TrueNAS CORE......
Cert Manager Issue As of yesterday I was getting this error from cert-manager when it was trying to start up in my K8 cluster. As it turns out, the repo this was referencing had changed and the file no longer existed. The fix was easy enough,......
Day 7 started by investigating some issues with Prowlarr which proxies searching for... Linux ISOs... automatically. It was having trouble fetching from a tracker and ended up being a new field that needed filled out with a valid UserAgent. I had to......
Day 6's progress was all about software updates for the core services on my homelab. First was pfSense, which has an update for pfSense+ to 23.05 Next were my two Proxmox servers which had various debian package updates that needed to be installed.......
Day 5 included cleaning up some duplicated CCTV footage from my TrueNAS server, 22 terabytes of it. I run both BlueIris (for 24/7 recording) and Frigate for AI categorized motion alerts but didn't realize I had turned on full recording on Frigate and......
I was recently reminded of a tweet I saw a few weeks ago talking about tech debt and the issues Slack went through having to pause real work for 18 months that allowed Microsoft Teams to catch up. Tech debt is the #1 reason product/market fit erodes for......
Day 4 has been all about getting some services moved over to the K8s cluster. I've been starting first with my uptime-kuma instance, but have been getting an odd error around the SOPS secret management / encryption. I'm pretty sure this is an issue......
Day 3 involved some network planning. I currently run pfSense+ as my router of choice on a dedicated 1U SuperMicro server. I'm planning some shifts to my VLAN / subnetting setup and at the same time considering moving over to a Unifi Dream Machine SE.......
Day 2 was eventful as it started with a 4 hour power outage that unfortunately showed me that something is wrong with my CyberPower UPS which immediately shut off even though it was reporting 34 mins of run time available. That's usually more than enough......
It's time again to jump back into #100DaysOfHomelab after starting it late last year but stopping after some family / job stuff got in the way. This time, I'm starting off by getting a Kubernetes cluster setup. I used the flux-cluster-template from......
Recently when upgrading to MySQL 8 at work, we started to run into this error when we would run older migrations. It turns out that as of MySQL 8, column names are limited to 53 characters. That should always be enough for everyone, but apparently it......
If you're like me, the 5 minute inactivity timeout with TrueNAS's Web UI is too short and leads to frustration. You'll be in the middle of setting up datasets / shares and switch tabs for a few minutes only to come back to a login screen and losing your......
Recently while renaming some Proxmox cluster nodes (which I now know to be not recommended) I ran into an issue where my VM configuration got cleared. All of a sudden I had no VMs over two host machines. I knew I had everything backed up via daily......
If you've recently changed your Proxmox cluster, you may find that one or more of the nodes is unable to view VNCProxy for it's VMs. In the error message you'll see something like this: The solution here is pretty simple. We must update our keys for......