Recent content on Zepworks
I’m happy to announce that DeepDiff 9.0.0 is released. Major releases are always a little tricky. On one hand, they are exciting because you get to clean up behavior, fix long-standing issues, and move the project forward. On the other hand, even small backward incompatible changes can surprise people. This release has a few of those changes, especially around cases where ignore_order=True is…
TLDR TLDR; I had a brilliant idea to write logs to a file on Minikube and have those logs persist between the restarts. I had to jump through some hoops. The main one was realizing that I had to change the folder ownership of where the files will persist within Minikube itself and use hostPath volume type on the pod so that I could persist the logs in Minikube. Why you don’t need this on a…
Bye bye class Config The Pydantic V1 behavior to create a class called Config in the namespace of the parent BaseModel. Now, you need to add a class attribute call model_config . class Schema ( BaseModel ): class Config : orm_mode = True becomes class Schema ( BaseModel ): model_config = ConfigDict ( from_attributes = True ) Even the names of the attributes have changed. orm_mode is renamed to…
💥 Qluster Qluster recently graduated from the USC Marshall School Of Business Incubator and the Columbia Business School Accelerator (CAVA). Where it came from? The insight came when I was running the Fair Search Team (RIP). We had just raised over $400 million in equity, burning cash like no other -WeWork excluded. The global search bar was the main interface to Fair’s car buying…
DeepDiff 5.2.1 comes with the new commandline feature! Get sample data Let’s start by looking at the Google Analytics Sample dataset on Kaggle . Here is a sample query taken from paultimothymooney’s notebook Let’s run this for a few different months in 2017: 2017 04 2017 05 2017 06 2017 07 Let’s export the output of these queries for each month into individual csv files…
Note: After writing this article I talked to the maintainer of Minikube and turns out the recommended way to remotely access Minikube is to do SSH tunneling. In fact kubectl port forwarding and everything works fine over the ssh tunnel. For more details visit https://github.com/kubernetes/minikube/issues/4733 If you are like me, you have some older hardware sitting around catching dust. The other…
It is June 2020 and I have some exciting news to share. DeepDiff has surpassed 1.7 million downloads a month! Notable DeepDiff Users The percentage of downloads per Python version Python version percent downloads 3.8 38.99% 667,561 3.6 29.98% 513,385 3.7 11.49% 196,773 2.7 9.78% 167,391 3.5 9.26% 158,587 Total 1,712,177 Date range: 2020-05-01 - 2020-05-31 Python 3 downloads of DeepDiff have taken…
DeepDiff 5 is finally here! Delta Object The Delta object is introduced. DeepDiff Delta is a directed delta that when applied to t1 can yield t2 where delta is the difference between t1 and t2. The Delta objects are like git commits but for structured data. You can convert the diff results into Delta objects, store the deltas, and later apply to other objects. Example: >>> t1 = [ 1 , 2 , [ 3 , 5 ,…
Intro Disclaimer: Fair is Sep Dehpour’s employer at the time of writing this article. If you don’t know about Fair , we are a vehicle subscription app. So when it came to naming the monolith that was dealing with vehicles, it was a no-brainer to call it vehicle-service . As the team grew, soon the on-boarding, testing, and development became slow and painful. We needed to release…
This tutorial is written based on DeepDiff 4.0.6. One of the features of DeepDiff that comes very handy is comparing nested data structures that include numbers. There are times that we do care about the exact numbers and want it to be reported if anything slightly changed. from pprint import pprint from deepdiff import DeepDiff t1 = { "key" : [ 1.2 , 1.5 ]} t2 = { "key" : [ 1.20 , 1.50 ]} >>>…
Part 1: Into Auto Complete: Summary Back in 2018, I was given the task to improve our autocomplete experience. We already had an Autocomplete backend solution using Elasticsearch, but what we needed was something that was faster, more customizable which could handle complex queries such as red bmw x5 in napa 2019 . This is the story of how I approached it, what I learnt, and the final product that…
Update May 22: Here is the video of the talk: The Proposal This was my talk proposal for Pycon 2017 which got accepted. The proposal is slightly modified to match the final talk better. For example originally I was going to talk about writing a Redis Client too but I ended up removing that from the final talk. I will be giving this talk tomorrow on Saturday May 20th! The code samples used in this…
This is based on a talk I gave at SoCal Python meetup. The PDF of this talk is available here. All the code is run in Python 3.5 but it works with Python 2 too. Overview When it comes to performance, lazy loading can help a lot. I remember when I first started using Django and Django’s lazy queries were magic to me. In this article I will try to dimistify lazy objects in Python, mainly…
As a person who loves dynamic typing, I have always been frustrated with Py-Redis. It is a great library, but it is too low level. It maps the actual Redis commands directly. I always wanted to just say the.path = "something" and have it magically saved into Redis. Then somewhere else say print(the.path) and get the value from Redis. As simple as that! And here it is: RedisWorks Install pip…
From time to time I need to do some simple file/folder transfer/query from a Python app to a S3 bucket. There is a great Python library called Boto that offers a comprehensive low level interface to AWS. I consider it too low level for simple operations on S3. On the other hand there is a fantastic command line library called s3cmd written in Python but it is designed to be used from command line…
The Talk Here is the actual talk! I gave a talk at Pycon 2016 about how DeepDiff does what it does. The proposal This is my Python talk proposal for 2016. It is about how my DeepDiff does what it does. I will be presenting this talk on June 1st at Pycon. https://us.pycon.org/2016/schedule/presentation/1862/ Edit (June 2016): read more about how the presentation went at Pycon presentation after…
Why Distributed Task Queues? Offload long jobs to background processes example: video conversion Offload too many [small] jobs to background processes example: commenting system Keep track of jobs, monitor, auto-restart Schedule jobs Replace cron jobs. Cron jobs can run maximum once a minute. That is a limitation. And they don’t have the best interface. Message Queue vs. Task Queue Message…