A blog dedicated to exploring The Information Age through Enterprise Architecture and Data Management
Before staring I should mention that my main contract ends in December this year (2022) and I am looking for contract work for the 2023 year. I am a lifelong learner and data & enterprise architect with 28 years experience. I also love tinkering with new technologies. I can be contacted at: neilhepburnjob@gmail.com or on LinkedIn at: www.linkedin.com/in/costie On with the article… If you have been…
Cletus and Koriolis (by Neil Hepburn) Preface This is a story about truth, beauty, and reality. Cletus and Koriolis A long long time ago, around after 1000 BCE, there was a Greek youth who went by the name of "Cletus". Cletus like his father was hard working and ambitious and a fisherman. But Cletus was also a fiercely independent thinker who was always searching for his own light - a curious…
The headline of this post is a quote borrowed from the Canadian author Witold Rybczynski who when asked by the New York Times what the greatest invention over the past millennium was, responded that it was the screwdriver (I would have chosen computer - by I'm not very good with home renovations), and went on to say that the Robertson screwdriver in particular was the "biggest little invention"…
A recent study published out of The University of East Anglia has shown – what many have long suspected - that the human brain treats well designed tools (like a spoon or a hammer) as an extension of the human body. More importantly, this study shows us that well-designed tools are interpreted by the brain differently than poorly designed tools which the brain simply treats as an external object.…
Recap In the last article we looked at the differences between Power BI and Qlik’s indexing engines with respect to their Link-based (as opposed to Merged Cube-based) indexing technologies. The reason why we performed this experiment is that after reviewing a Qlik patent (granted in April 2020) it became clear to me that Power BI and Qlik’s technologies, while appearing similar from the outside -…
Update 2021-01-12 An astute reader (thank you reader!) has pointed out a flaw in part of the experiment. I have since updated and re-run this part of the experiment to address the flaw. In summary, the flaw was that I had five (5) Measures in both Power BI and Qlik. But in Power BI these Measures were all being executed as separate queries whereas in Qlik they were being executed as a single…
TL/DR: Snowflake’s $63 billion valuation shortly after their IPO is primarily based on user network effects rather than its database innovations. Tech journalists and other analysts don’t want to sound cynical by reminding people of “lock in” and would rather discuss innovative features. But technical innovation is a distraction here. Whether you are an investor or a customer, it behooves us to…
These days when we hear the word 'diversity', different things might come to mind for different people. Diversity - with modern connotations of immigration or affirmative action has infused the word with with notions of multiculturalism, gender equality, and other forms of social levelling. I am supportive of that, but what I want to discuss here is diversity from a more abstract perspective. Lest…
Over the past five years or so I have taken a deeper interest in philosophy. More to the point, I have become especially curious as to WHY certain philosophies are adopted over others. If you have had the chance to listen to my podcast “ Cradle of Analytics ”, you may know that I spent ten episodes (and nearly nine hours) explaining the “Origins of Analytics”, and why I believe that so-called…
This post is a follow-up to my original posting (and paper) titled “A Modular Approach to Solving the Data Variety Problem”. In response to that posting a LinkedIn commenter (Mark B.) asked the following [paraphrased] question to understand how he might use modular approach to build a modular data analysis system to handle the following scenario: “As a digital marketer, I would like to see how the…
I have been working on a paper in my spare time on the weekends for a number of months now. My goal with this paper is to change the thinking around data and ultimately bridge the chasm between how IT and the Business think about data management and in particular Data Warehousing and Business Analytics. I am publishing the full paper here as a PDF and will publish portions of this paper in…
A few weeks ago I stumbled on this article " A co-Relational Model of Data for Large Shared Data Banks " ("coRel" hereon) in the on-line version of ACM Queue (this is the Association of Computing Machinery's magazine). The article was authored by two employees (Erik Meijer and Gavin Bierman) of Microsoft. The authors summarize their thesis as thus: "Contrary to popular belief, SQL and noSQL are…
A common problem faced by database developers involves keeping schemas and data in sync between environments. The problem isn't even restricted to development. Often databases must be kept synchronized in operational scenarios. Data is often replicated for reasons of high availability and performance. In this posting, I want to illustrate a real-world scenario that frequently comes up, and a…
Earlier this year I blogged about an alleged case of silver manipulation as raised by Andrew Maguire to the CFTC. If you haven't already, I urge you to read that post before continuing on with this post. I ended my post with a challenge to any serious journalist to properly investigate Andrew Maguire's case. So far I have yet to see anything resembling a real investigation undertaken by the media…
Over the past couple months I've been jumping between three different ETL tools (well PowerShell is not exactly an ETL tool but has some overlapping functionality). The experience has given me new perspective on the strengths and weaknesses of each tool. I hope to share with you my experiences, opinions, and recommendations. This report is not going to be as structured, or cover as many tools as…
"What do you think of the long form census" I ask inquisitively, adding "You know, the census form, which one out of every five persons needs to fill out. In order to provide information about stuff like ethnicity, religion, and income". "I don't want to give out information to the government if I don't have to." I'm sternly told. "Why not?" I ask politely. "It will never get released." I…
In August 2008 economic world history was made, as a major world record was smashed. You probably never heard this news - even if you read the business section of a major newspaper, or work in the finance industry. Instead, what you may have heard was the consequence of this news. Namely: gold and silver prices hitting a major slump. But behind those price slumps was the biggest and most…
It's been a long while since I've updated this blog. I've been very busy with a number of other projects. But with the wife and kids away this weekend I've found some time to get some ideas out that I've been sitting on for the past few weeks. This blog post is on Metadata. If you're not sure what Metadata is, you can read an older post I published a couple years ago here . My goal is to go over…
I first heard about column-based data warehouses in the Feb. 25th print edition of Information Week . The article interviewed Michael Stonebraker (Stonebraker was one of the original architects of the relational database, and released the first low cost RDBMSs under the company Ingres ) , and has fully embraced this new architecture which takes a fundamentally different approach to storing and…
I just noticed this ad on the back of a recent print edition of Information Week. I'd seen these ads before (mainly on CNET's web site), but not in print form in a magazine geared towards data management professionals. The ad campaign looks like it's targeting 12 year old boys, but I suppose it must be targeting the 12 year old boy within us all... or geeky software developers. But before I start…
The culture of data and analytics is starting to take foot among the general population. Books like "Competing on Analytics" and "Supercrunchers" herald a new era of business whereby every little decision is vetted through intense fact-based scrutiny. Well, it would seem to be that way. The truth is, most processes within most businesses (even the businesses discussed in the aforementioned books)…
I've been working a lot these days with data visualizations and presentation reports. I must admit that I've learned a thing or two about how people approach data, both from the IT side and the business side. However, after looking at dozens and dozens of data visualizations and executive reports, I have realized that there are effectively two kinds of reports that you can present to an executive,…
I've decided to hop on the dumbing down bandwagon for once. But instead of just offering your typically mind-numbing advice, and providing a bunch of pointless examples to prove my point. I'm going to try to address the skeptics head-on. So what is this "secret", and these "three simple steps" you might ask? I will get right to the point, and then substantiate my arguments for the skeptics. That…
I've been meaning to put this entry down for weeks now, but in each passing week it seems I come to new realizations about social networks and the direction they are going. It's been about two weeks since I've had any new thoughts on the matter, and for now my thinking as settled down. But before getting into my thoughts on social networks and how I feel they might be applied to the enterprise,…
Before getting into my post, I wanted to mention an interview I read in this month's Wired with Eric Schmidt, CEO of Google . I want to share with you a small excerpt: Wired: Google’s revenue and employee head count have tripled in the last two years. How do you keep from becoming too bureaucratic or too chaotic? Schmidt: It’s a constant problem. We analyze this every day, and our conclusion is…