Massive Breakthrough in MQTT provision!

Even though we will only run it over IP4 local network, this protocol has now been tested daily on more than 2 billion devices. VAULT-SYS will support plug-n-play any MQTT device and Modbus-TCPIP first version out of the box.

This was such a dramatic breakthrough this weekend I decided to make it into an official stack instead of just a note.

1. I had a gigantic bloated implementation to simply talk with the Mosquito MQTT server as a separate process via HTTP calls into it’s API. Seemed like a good idea at the time until I got it running a month ago and realized it sucked and was too distributed.

2. As usual, in the process I learned quite a bit about how MQTT worked and realized that once you got through all the complexity you are initially presented with, it’s actually a baby simple format and structure. It occurred to me you could write the server in a couple dozen lines and it would be simple enough anybody could glance at it and see how it worked.

3. People have talked about how the solid future of the data management of the Internet-Of-Things is realized in MQTT to structure your whole enterprise (no matter what it is) as the equivalent of a file tree. I got to thinking while I was writing and testing the code Saturday and Sunday for my own MQTT implementation … what if it is the future of VAULT-SYS?

4. I have already made strides to converting incoming Modbus-TCPIP into MQTT structures on the fly, except I was trying to store them in the Mosquito server. This was stupid if I could get my own MQTT running and could chuck Mosquito. I did it! That changes everything. Now the whole enterprise is a colossal data tree, including all servers online, which will have their own MQTT clients to register. If it exists, it adds itself to the tree with all the data required to speak to it. It will probably be as simple as single flag in the .INI configuration file where “MQTT-Server=yes” is all it takes to make the server boot MQTT hub, otherwise activate MQTT Client connection.

5. You know you have the right paradigm when you eliminate a bunch of redundant code. Now I can toss my UDP discovery code (I must have spent 6 months on that over the years.) and my own custom server information manager, which had been in there at least for 20 years!!! Powered by HTTP at one point, then UDP. I can scrap that. The less code, the better your paradigm is getting.

I was shocked when my MQTT barebones tests for my new code were working fine. I had trouble believing I had replaced all that with just a couple lines of C99.

CONNECT-SUBSCRIBE-PUBLISH-CLOSE-REQUEST verbs for MQTT all supported in about 30 lines of code now in VAULT-SYS!!! So tiny compared to my former implementation just communicating with Mosquito server it seems impossible!

I love when it shrinks and know it is not right whenever the code grows! You just know there is a better way to do it!!

This is all that is needed for a shelter and it is reflective with other nodes already through VAULT-SYS connection sharing! My village idiot system of calling the same REST API that any given server uses makes sharing sensor and controls over an infinite area as simple as talking to the local server.

Compared with Mosquito server I was running as an interactive process for MQTT support, my implementation is a million times simpler, clearer and less obfuscated than Mosquito ever was … plus it is much faster response time with no weird delays. All run on a polling system with no threads of any kind … much stabler long term.

I have run this overnight for testing but I have a feeling after a year there would be problems with fragmentation if the tree is always assembled at startup from live IOT devices (best case) so that’s a problem I need to solve in version 1.0 or so.

At present I am allocating and releasing nodes once they have connected, right now I am trying to build a static heap allocator that only runs once at startup, like everything else. Remember, a system intended to run 50 years at a time should not dynamically allocate any memory or run any threads. My design specifications since 1997 when I wrote the first version in MS-DOS Visual Basic.

I think this is a pretty solid heap that has been tested on tiny and colossal systems and performs well at any scale …

OHeap For Embedded Devices or Supercomputers

Will probably be replacing my simple tree with a proper “trie” structure on a static heap so risk for fragmentation drops to zero long term.

This looked pretty solid for trees that can be persisted (serialized in SQLITE.)

This could be serialized out to SQLite on the fly at almost no cost to polling times.

The great thing about SQLite is that if you run the journal, the database will catch up when it can and does not need to jam everything up when serializing data to-from disk. Another example of how incredible Richard Hipp’s public domain code is.

Been using SQLite since 2005 long before the rest of the world discovered how amazing it is.

I know what you are thinking. I’ve been thinking about it too, since 1996 when I found out empty Pringle’s cans can extend an ordinary WIFI transmitter into a network that covers several hundred miles from your retreat/shelter.

A Pringles can can be turned into a homemade directional waveguide antenna, known as a “cantenna,” to focus Wi-Fi signals. While it extends point-to-range connections over hundreds of feet, it does not build a true wide-area network (WAN) on its own. Instead, it acts as a high-gain directional tool to bridge two specific points.

• Shape and Lining: The metal-lined interior of a Pringles cylinder naturally reflects and bounces 2.4 GHz radio frequency waves used by standard Wi-Fi.

• Directional Focus: Unlike normal router antennas that broadcast in every direction, a cantenna concentrates power into a narrow, focused beam to send or receive signals from a single targeted destination.

• Limitations: Users on Reddit have a consensus that while a cantenna boosts distance and signal direction, it cannot bypass encryption or create a broad network layout without proper routers and credentials.

• Construction: Makers typically drill a precise hole in the side of the tube to insert a copper wire element soldered to an N-type connector attached to a wireless card.

• Alternative Cans: Enthusiasts note that a standard Pringles tube is slightly narrow for optimal 2.4 GHz math, meaning other metal cylinders can sometimes perform better.

• Legal Constraints: Pointing high-gain homemade directional antennas at distant or unauthorized networks can exceed local regulatory power limits and violate agency rules like the FCC.

• Explore instructions on building a wireless link from Building Wireless Community Networks. (Bought this book secondhand about 15 years ago!)

• Read a guide on construction steps at MakeUseOf.

• Learn about early testing trends from the BBC News.

I have been looking for the Holy Grail for a long time. After 25 years, I think I just found it this weekend.

• Regards, Tex