A few months ago, NVIDIA CEO Jensen Huang announced their version of NanoClaw, which promised to offer better privacy, security and performance compared to the popular open-source tool OpenClaw. OpenClaw, an agent framework with easy tooling, was mired with numerous security and privacy problems involving loss of sensitive data like …
There's been a lot of chatter about guardrails that's more marketing than science, so here's a real take on guardrails so that you can make real decisions about y:our machine learning architecture. Note that this is a mega-post that I'm going to keep updated with the latest references. Should …
If you've read Practical Data Privacy , you'll know I'm not a huge fan of synthetic data. As a data scientist and machine learning person, usually I want to use high quality data and only introduce slight deviations by using protections for privacy (like differential privacy). In my book, I clarify …
I've been exploring privacy and security aspects of AI-assisted coding and also experimenting with those workflows for my own work. In doing so, I've got a pretty robust setup for using Claude Code with both the Anthropic backend and a locally hosted GPU machine in my at home AI lab …
AI usage (both intended and not) is increasing in our products, software and lives. For some, this is a welcome way to automate tedious tasks; for others, an intrusion that doesn't seem to end. For everyone, AI changes how you might think about and evaluate data privacy. At workplaces, there's …
You've learned in the last few articles about how differential privacy works and some of the common pitfalls of actually using it in deep learning scenarios. In this article, you'll learn about tracking differential privacy: through parameter choice, accounting and auditing. If done well, these choices and methods reduce memorization …
If you're just getting started with local AI and local-first development, one of the initial hurdles will be getting your data local. More of an audio-visual person? Check out the accompanying YouTube video on the Probably Private channel if you'd rather watch and listen. Why should you store data locally …
So, you wanna do at-home AI? Yes, you do! There's a bunch of great reasons to run your own AI including having more control over your data and models, learning more about how deep learning works, testing out new ideas without having to pay extra cloud or subscription costs and …
In the last article in the series on addressing the problems of memorization in deep learning and AI , you learned about differential privacy and how to apply it to deep learning/AI systems. In this article, you'll explore what can go wrong when using differential privacy training in deep learning …
Differential privacy influenced both privacy attacks and defenses you've investigated in this series on AI/ML memorization . You might be wondering: what exactly is differential privacy when it's applied to deep learning? And can it address the problem of memorization? Are you a visual learner? There's a YouTube video on …
In the past articles, you've been exploring the field of machine unlearning , investigating if you can surgically remove memorized or learned data from models without retraining them from scratch or from an earlier checkpoint. Unlearning is one proposed solution to the AI/ML memorization problem explored in this multi-article series …
Building on our understanding of machine unlearning and its varied definitions , in this article you'll learn common approaches to implementing unlearning. To effectively use these approaches, you'll first want to define what unlearning definition and measurement fits your needs. In current unlearning research, there are three main categories of unlearning …
Machine unlearning sounds pretty cool. It is the idea that you can remove information from a trained model at will. If this was possible, you'd be able to edit out things you don't want the model to know, from criminal behavior, racialized slurs to private information. It would solve many …
It seems like every week my LinkedIn feed is filled with new just released AI risk taxonomies, threat models or AI governance handbooks. Usually these taxonomies come from governance consultants or standards authorities and are a great reference for understanding the wide variety of risks AI systems 1 bring with …
You've probably at some point heard the term "guardrails" when talking about security or safety in AI systems like LLMs or multi-modal models (i.e. models that include and produce multiple modalities, like speech and image, videos, image and text). Are you a visual learner? There's a YouTube video for …
One common way to control memorization in today's deep learning systems is to fix the problem by building software around it. This software can also be used to deal with other undesired behavior, like producing hate speech or mentioning criminal activities. Are you a visual learner? There's a YouTube video …
In this article series , you've been able to investigate memorization in AI/deep learning systems -- often via interesting attack vectors. In security modeling, it's useful to explicitly define the threats you are defending against, so you can both discuss and address them and compare potential interventions. Prefer to learn by …
I'm excited to open-source a project that I've been using for the past 2 and a half years: a private/personal reader and recommender. It works with: RSS feeds Reddit HackerNews Lobste.rs and comes with an example Jupyter Notebook for training your own text-based recommendation model once you have …
In this article, the last in the problem exploration section of the series , you'll explore adversarial machine learning - or how to trick a deep learning system. Adversarial examples demonstrate a different way to look at deep learning memorization and generalization. They can show us how important the learned decision space …
At this point in reading the article series on AI/ML memorization you might be wondering, how did the field get so far without addressing the memorization problem? How did seminal papers like Zhang et al's Understanding Deep Learning Requires Rethinking Generalization not fundamentally change machine learning research? And maybe …
You've heard claims that we will "run out of data" to train AI systems. Why is that? In this article in the series on machine learning memorization you'll explore model size as a factor in memorization and the trend for bigger models as a general problem in machine learning. Prefer …
So far in this series on memorization in deep learning , you've learned how massively repeated text and images incentivize training data memorization , but that's not the only training data that machine learning models memorize. Let's take a look at another proven memorization: novel examples. Prefer to learn by video? This …
In this article in the deep learning memorization series , you'll learn how one part of memorization happens -- highly repeated data from the "head" of the long-tailed distribution. Prefer to learn by video? This post is summarized on Probably Private's YouTube . Recall from the data collection article that some examples are …
In this article in the series on machine learning memorization , you'll dive deeper into how typical machine learning training and evaluation happens, a crucial step in ensuring the machine learning model actually "learns" something. Let's review the steps that lead up to training a deep learning model. High-level steps to …
Hello! We may not know each other, but here you are on my website -- perhaps because you saw a post or someone shared a link. I'm resourceful, determined, intelligent and looking for new challenges. Welcome! Wenn Deutsch einfacher ist, schreiben Sie mir bitte per Email (katharine at kjamistan punkt com …
I recently had the wonderful experience of keynoting PyData Paris , thanks again for the invite! When deciding on a topic, I was considering my recent research about how AI/ML systems memorize data . As I've mentioned in a few talks, if we indeed embraced the fact that machine learning systems …
In this series , you've learned a bit about how data is collected for machine learning, but what happens next? You need to turn the collected data -- images, text, video, audio or even just a spreadsheet -- into numbers that can be learned by a model. How does this happen? TLDR (too …
You probably are already aware that many machine learning datasets come from scraped internet data. Maybe you received the infamous GPT response: "Please note that my knowledge is limited to information available up until September 2021." You might have also read fear-mongering opinions and articles that companies will "run out …
When's the last time that ChatGPT parroted someone else's words to you? Or the last time a diffusion model you used recreated someone's art, someone's photo, someone's face? Has Copilot given you someone else's code without permission or attribution ? If this happened, how would you know for sure? In this …
Want to learn more about how, when and why machine learning, particularly deep learning systems memorize data? By studying memorization, you'll learn more about how machine learning systems really function, along with how privacy works from a technical point-of-view. You'll also be better able to decide how, when and where …
Building a newsletter is a fairly common activity these days, with many creators, writers and thinkers making part of their living via subscribers willing to give small amounts of money out per year or month to get exclusive access. Beyond the paid subscriptions, there's an increasing demand for free, or …
After months of searching, lots of fun (and some less fun) interviews and hours of self-reflection, I am excited to announce I am the new Head of Product at Dropout Labs ! 🎉 The interview and decision process was quite iterative and disruptive! I am somewhat to blame for this as I …
Hello! We may not know each other, but here you are on my website -- perhaps because you saw a post or someone shared a link. I'm resourceful, determined, intelligent and looking for new challenges. Welcome! Here's more about me, in case it is news to you: [About Me] Co-founder of …
When I first was introduced to the idea of adversarial learning for security purposes by Clarence Chio's 2016 DEF CON talk and his related open-source library deep-pwning , I immediately started wondering about applications of the field to both make robust and well-tested models, but also as a preventative measure against …
I presented a keynote at PyData Warsaw on moving toward interpretable reliable models. The talk was inspired by some of the work I admire in the field as well as a fear that if we do not address interpretable models as a community, we will be factors in our own …
Unless you have been avoiding all news, you have likely heard of the coming changes in European privacy regulations which go into effect in May 2018. The changes are covered under the General Data Privacy Regulation Directive, whose final text was made available in May 2016. I presented a talk …
I was invited to give a talk at 404 Dublin , a really cool conference joining community groups w/ tech folks and art installations. When thinking of what topics might be of interest to the audience, I selfishly went to one of my (side) passions.. following artists who are doing amazing …
Finding ways to determine fake news from real news is a challenge most Natural Language Processing folks I meet and chat with want to solve. There is significant difficulty in doing this properly and without penalizing real news sources. I was discussing this problem with Miguel Martinez-Alvarez on my last …
I gave a long and opinionated tutorial at EuroPython 2017 about how we should do unit testing and validation within a data science scope . The GitHub repository for the course (which is part of my O'Reilly Live Online training ) is https://github.com/kjam/data-cleaning-101 . I will continue editing and …
This past Wednesday, I had the pleasure of giving a keynote at EuroPython 2017 . I covered a historical view of ethics in computing. The slides are shared here, but it was also recorded so I will post a video when it is available. (Updated: video added!) In addition, a series …
Practical Data Cleaning Resources (O'Reilly Live Online Training) This week I will be giving my first O'Reilly Live Online Training via the Safari platform. I'm pretty excited to share some of my favorite data cleaning libraries and tips for validating and testing your data workflows. This post hopes to be …
I was kindly asked by the PyData Amsterdam organizers to keynote the conference. As a passionate fan of ethical machine learning and the great research being done by data scientists and academics around the world -- I am very enthused to present the topic to the conference. My slides are currently …
The saddest moment for me at conferences is when I'm in the middle of an interesting conversation with a bright person and I ask her when her talk is and she says, "Who me?" The number of folks I speak with every year at conferences who have amazing stories to …
Over the new year holiday time I had a chance to get away from it all, and snuck up to Finland to sit in a lodge on the Gulf of Finland, sip coffee, take saunas and read. I brought along a few books, the only programming one being Brian W …
I'm really excited to announce a new Python video course with O'Reilly on data pipelines . If you are interested in learning some of the popular options available for workflow automation and management in Python, take a look! In the course, I cover: Using Celery for simple automation Setting up Hadoop …
I gave a talk about Directed Acyclic Graphs (DAGs) and Dask at PyConCZ 2016 . It was super fun and I had a great time at the conference. If you want to read my slides below, here they are! There will be videos available later, so I'll post the link / video …
PyConCZ 2016 was such a fun conference! First off, it was the first time I got to see Jackie Kazil since we started writing our O'Reilly book Data Wrangling with Python together, HOORAYYYY! OMG PYTHONISTAS! @JackieKazil & I are together for the first time since we started the @OReillyMedia Data Wrangling …
I participated in a two-day PyDataBerlin Hackathon event in early-October and decided to build a scraper for European Parliament. This was after I found the Europarl parallel corpus a bit underwhelming as it is messy and not tagged for party, speakers or topic (this is understandable, as it is primarily …
I dunno about you, but I often find myself bored with NLP (natural language processing) datasets. Too often they are older, based around something that is not particularly interesting to me or something I've analyzed or used before. For me, IRC has often been a source of community, fun, sometimes …
I gave a talk at PyCon UK 2016 on automating your data cleanup with Python. I want to again thank the organizers for having me and thank the folks who attended. If you have any questions or are interested in talking about data cleaning problems, feel free to reach out …