I’m a geek. Always have been, always will be. One of my geekier traits is my fascination with computer chess. It was an obsession that started when I was about 14 years old, running tournaments between ChessMaster 4000 personalities on a 386 16MHz PC that cost my parents way too much money. While the popular kids were doing whatever popular kids did (sports, I think) I was busy meticulously…
The Three-Ring Data Platform Circus: Fabric, Databricks & Snowflake Short on time? Jump to the Summary Table and “Patterns I’m Seeing in the Wild ” section for a one-page cheat sheet. The Cloud Data Platform market has steadily evolved over the years, and what has traditionally been a two-act show featuring Snowflake and Databricks has shifted to a three-ring circus with the recent and compelling…
TLDR: Bottom Line: Two new protocols—Anthropic’s Model Context Protocol (MCP) and Google’s Agent-to-Agent (A2A)—are solving enterprise AI’s biggest challenge: getting AI agents to work with your existing systems and collaborate with each other, rather than building expensive, disconnected point solutions. What This Means for Your Organization: MCP gives AI agents secure access to your databases,…
AI has created an interesting and dynamic inflection point for our society. It’s ushered in an era of rapid technological advancements that are revolutionizing industries and reshaping the landscape of businesses on a global scale, and at the same time it’s also created an array of substantial risks that can not be underestimated. Recognizing the critical importance of safeguarding against these…
Table of Contents Intro Type 1 - 101 Modern Type 1 Diabetes Management The $1,000,000 Question Disclaimer What this blog post is not: What this blog post is: The Data A Note on Non-Stationarity The Approach The Code Data Loading and Creating our Time Series Objects Scaling our Data Model Definition Fitting the Model and Predicting RMSE and Quantiles Out-of-Sample Results Wrap-up Conclusion and…
TLDR : I provide a “chat with your docs” framework for using a cloud environment to have a local embedding model vectorize an obscure fiction book and then have a Llama-based text generation / LLM model answer questions about it. This approach is easily extended to read PDF’s or other proprietary and domain-specific information like financial reports, legal decisions, repair manuals, performance…
Local LLMs - Getting Started with LLaMa on AWS EC2 As the world of AI continues to evolve, large language models (LLMs) have become increasingly popular. These models offer powerful capabilities for tasks such as text generation, summarization, translation, and more. In this blog post, we’re going to walk through running your own copy of a local, open LLM on the cloud (in this case AWS). Before…
This is a series of articles in 5 parts. Each part investigates a specific challenge organizations usually run into, and the final part wrap-ups up with some of my additional thoughts and suggestions. You can navigate the series using the links in the list below. Navigation Challenge #5 – Lift-and-Shift vs Refactoring Challenge #4 – Why no one wants to use your new tools / platform! Challenge #3 -…
This is a series of articles in 5 parts. Each part investigates a specific challenge organizations usually run into, and the final part wrap-ups up with some of my additional thoughts and suggestions. You can navigate the series using the links in the list below. Navigation Challenge #5 – Lift-and-Shift vs Refactoring Challenge #4 – Why no one wants to use your new tools / platform! Challenge #3 -…
This is a series of articles in 5 parts. Each part investigates a specific challenge organizations usually run into, and the final part wrap-ups up with some of my additional thoughts and suggestions. You can navigate the series using the links in the list below. Introduction I’ve been at the intersection of technology and data for over 20 years. I’ve spent half of that, a solid decade, of my…
Overview Finding true time series data at billion row level scale is hard. It’s usually either generated from standard RNG’s which leave out the very important temporal dependencies through their use of Gaussian or Uniform distributions, or it’s too large to easily move around for testing purposes. In this post I show you how to synthesize billions of rows of true time series data with an…
Deep Dribble 1.2 , has crunched the numbers and the answer is Gonzaga. Same answer as last year. Deep Dribble now has a favorite team! I like this pick quite a bit, and I’m also glad to see the predictions followed my rankings to a ‘T’. The bracket was made by predicting every team against every other team both as home AND away, and taking the difference between the two. Some games were very…
The latest iteration of the NCAA model, Deep Dribble 1.1 , saw a number of changes for the 2016-2017 season. The most interesting change was the daily rankings function which was a unique challenge for me. I’d never built out a rankings system before and I think it worked pretty well for the first pass. Ken Massey picked up the rankings and included them in his weekly NCAA composite (Deep Dribble…
TLDR - Daily (early season model) predictions here . Well hello there! Been a while, right!? Hope you’ve been well. I looked back at some of the posts and clearly it’s been some time since the site has been active. There is a reason for that, and her name is Claudia . Claudia was born in June and has been fun, adorable, and at the same time a drain on my free time like I never would have imagined.…
Deep Dribble has had some ups and downs this year, but overall I’m happy with the first full 100% Python implementation of the model. I’ve definitely got some ideas for next season and I’m looking forward to testing them out when I have some free time. We’ll see how the bracket predictions stack up for this year, but it’s always a crap-shoot! I’ve got the entry added to the 2016 Machine Madness…
Note - If you’ve already read this, and just want to jump to the forecast vs actual table, click here . I realized long ago that when I’m actually interested in a topic, I learn it much more quickly and easily than topics I have difficulty relating to, or have no interest in. I recently had some exposure to time series analysis and while I think it’s a fascinating field, it can very quickly become…
Over the last few months the UChicago team and I have been focusing on detecting credit card fraud utilizing Markov-chain based models and streaming transaction data. One of the main issues that we ran into developing our modeling strategy was that our initial results were based on small-ish sized data sets (~10GB), and that those results didn’t necessarily reflect performance at “Big Data” scale.…
Last year I spent quite a bit of time building a number of models to predict regular season NCAA basketball games. I spent about 2-3 weeks before I ended up with a basic classification model (Away team wins = 1/0) that I was happy with. Then I ran into Scott Turner who pointed me in the direction of predicting score differentials, and I spent another month or so tearing everything down and…
Normally on this blog I try and focus on more ‘pure’ data science and statistics as I spend the majority of my day focused on the IT-related side of data science. There’s also quite a bit of information already out there around the technical aspects of data science such as configuring python, installing R or MySQL, general Linux how-to’s, etc… so I don’t really feel the need to duplicate it.…
If you’re not familiar with TrueSkill, it’s a rating system developed by Microsoft in order to assist in player matchmaking with their XBOX Live service. It’s a pretty cool and flexible system, and allows for unbalanced teams, ties, partial play, etc… I originally learned about it, as I do with most sports-related math theory, from Scott Turner and his website . The general idea is that a player’s…