About Low Hanging Data
Low Hanging Data is a project built on a simple conviction: the best data analysis is the simplest analysis that answers the question. Before reaching for a model, a complex pipeline, or an elaborate visualization, pick the fruit that's already within reach.
About the Author
Michael Petrillo is a data engineer, data scientist, and data analyst with a focus on data operations. He writes about practical data workflows, pipeline architecture, and applied analysis techniques that prioritize clarity over complexity.
Connect with Michael on LinkedIn
The Three Principles
- Concise — Good analysis fits in a headline with supporting evidence. If a finding requires three paragraphs to explain, it either isn't clear yet or wasn't worth finding. See: Visualizing Data with Python, Organizing Data with SQL.
- Transparent — Sources, assumptions, and limitations are stated plainly. Reproducibility is not optional. If someone else can't verify the work, the work isn't finished. See: Data Cleaning and Validation, Building Your First Data Pipeline.
- Low Hanging Fruit First — Counts, sums, averages, and distributions come before clustering, forecasting, or causal inference. Complexity is only added when it changes a conclusion. See: Getting Started with Data Collection, DuckDB for Financial Analysis.
Who This Is For
Anyone who works with data and wants to produce analysis that decision-makers actually trust and use:
- Analysts who want their work to have more impact with less overhead.
- Developers building data features who need to communicate findings to non-technical stakeholders.
- Researchers who want their methodology to be reproducible and their conclusions defensible.
- Anyone who has ever been asked "so what does this actually mean?" after presenting data.
Articles
Practical guides that apply these principles to real tasks, each leading with the simplest viable approach:
- Getting Started with Data Collection — data sources, formats, and the basic workflow
- Pulling Data from REST APIs — authentication, pagination, rate limiting
- WebSockets & Streaming Data — real-time feeds for market data, news, and social platforms
- Working with CSV and JSON — the two most common formats and their edge cases
- Data Cleaning and Validation — missing values, duplicates, type mismatches, outliers
- Python & Pandas for Data Wrangling — clean, reshape, and prepare messy data
- Organizing Data with SQL — the queries you'll actually use day to day
- Parquet and DuckDB — query millions of rows faster than pandas, no server required
- DuckDB for Financial Analysis — rolling indicators and a local analytics layer over stock price data
- Visualizing Data with Python — matplotlib, seaborn, and plotly for static and interactive charts
- Building Your First Data Pipeline — combine collection, cleaning, and storage into a repeatable workflow
- Scheduling and Automating Pipelines — cron, Python schedulers, and lightweight orchestration
- Low-Hanging Data Sources for Stock Prediction — freely accessible datasets and supporting research
- Building a Stock Prediction Classifier with scikit-learn — SVM on Reddit sentiment, Google Trends, and price data
Posts & Project Writeups
The Posts section covers shorter posts and project retrospectives — what was built, what worked, and what didn't:
- Building a Stock Prediction Pipeline — combining Reddit sentiment, Google Trends, and NYSE price data with an SVM classifier
- How BD Used AWS to Stop Guessing When Medical Devices Would Fail — a real-world case study using AppFlow, Lambda, Athena, and QuickSight
Contact
Have a question, found an error, or want to discuss something from an article? Reach out by email:
Legal
- Privacy Policy — how this site handles cookies and advertising data
- Terms of Use — content license, disclaimers, and site policies