Adventures in the land of AI & scalable data systems
An earlier post covered building a local LLM stack with Ollama, Open WebUI, and Continue. That stack was about getting something running — a chat box, a model in the IDE, a feel for what local inference looks like. This follow-up takes the next step: turning that stack into an actual coding agent that can read files, run tools, and edit code on your behalf, on a laptop GPU with only 4 GB of VRAM.
LangChain has quietly built the most capable stack for production AI agents. Between the graph-based orchestration of LangGraph and the new DeepAgents harness, the developer experience is elite.
Retrieval Augmented Generation (RAG) has revolutionized how Large Language Models (LLMs) access and utilize external knowledge, moving beyond static training data to deliver more accurate, relevant, and up-to-date responses. While basic RAG setups are powerful, achieving peak performance and addressing complex real-world scenarios often requires a more sophisticated approach. This post explores…
In the rapidly evolving world of Large Language Models (LLMs), the ability to run these powerful tools locally on your hardware is likely appealing. Running AI locally grants you unparalleled control, endless customization, and complete data privacy. This guide will walk you through building your personal AI playground, assembling an open-source stack from the model server to a feature-rich user…
The concept of a data lakehouse is gaining serious traction, and for good reason! It combines the flexibility and cost-effectiveness of a data lake with the data management features and ACID transactions of a data warehouse. Let’s dive into building a practical lakehouse setup using some fantastic open-source and cloud-native tools: DuckDB with its DuckLake extension, Google Cloud Storage (GCS)…
Popular data processing platforms offer users the ability to inject an external process into the data processing pipeline. The data flowing through the data pipeline is fed as input to the external process, while the output produced by the process is fed back into the pipeline. The external process runs an executable or a script. This pattern resembles the popular Unix pipelines (or pipes). This…
In an earlier post, we looked at how to create a Docker image for SciDB. The image built in that post followed the SciDB Community Edition Installation Guide very closely. The image is functional and a good learning resource, but not very efficient. The image uses around 6GB of space and cannot be build automatically on Docker Hub due to long build time. In this post, we revisit this topic and try…
One of the strengths of SciDB over other database management systems is its extensibility.1 SciDB allows the user to add new data types, functions, and operators. In this multi-part post, we discuss various aspects of extending SciDB. In this post we look at the available documentation and how to setup the development tools.
In part 1 and part 2 of this multi-part post, we looked a how to load data from multiple files while capturing information present in the file name. In this post, we look at how to load data files organized as tables with a possibly large number of columns and header rows.
SciDB has extensive documentation but there is no official tutorial or getting started guide. In this post, we go over some of the tutorials and getting started materials available online.