about me
pandas 3.0 has just been released. This article uses a real‑world example to explain the most important differences between pandas 2 and the new pandas 3 release, focusing on performance, syntax, and user experience. A note on pandas versioning Before diving into the technical details of pandas 3 …
Introduction Comma-separated values (CSV) are an extremely popular format to store tabular data because of their simplicity and how easy is to write them. The file can be directly read by a human, as opposed to more efficient binary formats like parquet, for example: name,age Maryam,23 Mèng yáo …
Introduction At the time of writing this post, we are in the process of releasing pandas 2.0. The project has a large number of users, and it's used in production quite widely by personal and corporate users. This large use based forces us to be conservative and make us …
The problem We want to find out which are the top #5 American airports with the largest average (mean) delay on domestic flights. Data We will be using the Data Expo 2009: Airline on time data dataset from the Harvard Dataverse . The data consists of flight arrival and departure details …
This week we organized a round table with people involved in the management of data projects. Mostly data science. The idea came after the Executives at PyData session organized earlier this year. And discussions with few people on the challenges when trying to deliver data projects. Ian Ozvald , founder of …
Some context This post is mainly a technical post on what's the status of the pandas documentation. But let me provide a bit of context on where this comes from. It's a personal opinion, but I think pandas is one of the clearest examples of how open source is transforming …
Some exciting news. After some years of organizing sprints , and maintaining open source, I've been thinking on a more efficient workflow for projects with high volume of activity, like pandas . An exaggerated example would be that I want to create 1,600 issues in pandas. One for each docstring of …
Last week took place in Bilbao, Spain, EuroSciPy 2019 . This year we introduced the maintainers track a room dedicated to discussions among maintainers. The idea is similar to the birds of a feather or unconference sessions of other conferences. But focussed on open source maintainers and contributors. And we scheduled …
Leo Breiman was a distinguished statistician at UC Berkeley, known among other things for his major contributions to CART (decision trees), and ensemble techniques, mainly bootstrap aggregation. Combining both, he was able to define one of the most popular machine learning models even today (18 years after the publication of …
Today I've got my new Dell XPS (with Ubuntu preinstalled), and this is the procedure to set it up, and get my perfect working environment. This is expected to be useful mainly for my future self , but sharing it here in case someone else can find ideas or tips that …
While git is surely one of my favorite tools, and increases my productivity in a sometimes unbelivable way (like when working on 3 or 5 features at the same time), some times there are operations that can be a bit tricky. There are plenty of git tutorials and guides to …
It's been a while since I wanted to move my blog out of blogger. Today I finally did it. :) print ( 'hello world (from Pelican)' ) This new blog uses Pelican, and is hosted on GitHub pages. Which will let me create blog posts by simply using restructuredText, or Jupyter notebooks. You …
The past 10th of March took place #pandasSprint . To the best of my knowledge, an unprecedented kind of event, where around 500 people worked together in improving the documentation of the popular pandas library. As one of the people involved in the organization of the event, I wanted to write …
Just as a quick disclaimer, this post is about my personal experience and opinions at NIPS 2017, and I'm not an AI researcher, I work as a data scientist in the industry. For a more technical summary of the talks and papers presented, you may want to check this document …
Contributing to open source is one of the most rewarding experiences one can find. Just finding a bug or a new cool feature of a widely used library, working on it, and sharing it with the rest of the users. This is how open source has become so great and …
This is a post about my experience at PyData London 2017 . About what I liked, what I learnt... Note that having 4 tracks, and so many people, my opinions are very biased. If you want to know how your experience would be, it'll be amazing, but different than mine. :) On …
This last weekend I went to my third PyData, the one in London, and it's been such a great experience. Before, I went to PyData Amsterdam, and PyData Madrid, also this year. After the three conferences, which were very similar, but quite different at the same time, I just wanted …
What do I do after installing Fedora 23 MATE-Compiz? Install Google Chrome Merge both panels to the bottom, and auto-hide it Change mouse setup to allow touchpad click and double finger scroll Change look and feel setup to select window when the mouse moves over it Disable screensaver Change terminal …
This is a short note about how I set up my "data scientist" environment. Different people have different tastes, but what I use, and what I set up is: conda for environment and package management (equivalent to virtualenv and pip to say) Latest Python (yes, Python 3) Jupyter (aka IPython …
Installing Google Earth in Fedora is trickier than it should. Here is a short HOWTO: Download 64bits Fedora version from [Google Earth site](https://www.google.com/earth/download/ge/agree.html) sudo yum install google-earth-stable_current_x86_64.rpm OOOPS!!! You got **file /usr/bin from install of google-earth-stable-7.1.2.2041-0 …
Here there is a blog post on how to install Skype on Fedora 21 by quickly creating an RPM package. [http://mariuszs.github.io/blog/2014/skype_for_fedora_21.html](http://mariuszs.github.io/blog/2014/skype_for_fedora_21.html) It's also a great simplified tutorial on how to build any RPM.
I think LATEX is simply amazing by itself. More when writing an academic document, but for any kind of doc, using LATEX is really time and pain saving. The concept of creating a document class (defining all the styles of the document), and then simply forgetting on formats, and focusing …
For some reason, brother printers (at least mine) do not take into account the settings specified for the printer in the regular way (Gnome settings in my case). But mysteriously, there is a command which can be used to change them properly ( brprintconf_mfc235c) . In my case, I was having problems …
While there are some cool things in Gnome 3, I think mostly everyone will agree that there are many design mistakes. To me, it looks like a bad copy of the Mac desktop, and it's specially annoying that they also brought the "let Steve Jobs decide it for you" philosophy …
While I love Postgres, I get some problems every time I want to do the simple operation of creating a database with an associated user if it's been a while since the last time I did it. There are several posts on the Internet about Postgres authentication, but I couldn't …
After all these days at EuroPython, there is a thought that keep me thinking. It is about how Python have different ways to represent what it could be considered the same thing. On today's talk, Alex Martelli pointed out that "def" and "lambda" are actually the same concept. This was …
While it's been a long time since the first stable version Python 3 was released, it's not yet available on several operating systems. Looking for a repository with Python 3 rpms, I found IUS Community , but I had some problems with it, and I thought on building my own rpms …
Today I discovered Joel test, a test to evaluate software companies. While the article is pretty out-of-date, and there are some points that are exclusively for companies working on compiled programming languages, the article is still very interesting. I think every software company should took the test. Also I think …
This is a simple guide on how to do simple branching operations in Mercurial. First of all, let's comment the two different options for branching on Mercurial, and in most distributes source control systems. First option is to create a clone of the original repository to create a branch. This …
First step is to let Google host your JavaScript library of choice for you. Google Libraries API hosts JQuery, Mootools, Prototype... and you can directly link to them from your website. More info at: http://code.google.com/apis/libraries/devguide.html Second step is to compress you CSS file …
Python debugger (pdb) doesn't work on App Engine SDK as usual. After adding to my project: import pdb; pdb.set_trace() I got: Blocking access to skipped file " /.pdbrc" File "/usr/lib/python2.6/bdb.py", line 46, in trace_dispatch return self.dispatch_line(frame) File "/usr/lib/python2.6/bdb.py …
Today I've been researching on Linux and Debian booting. There is an excellent article from IBM, which explains the procedure, and the involved parts: http://www.ibm.com/developerworks/linux/library/l-linuxboot/ Basically: - BIOS checks CMOS and choose the booting device Control is given to device's MBR (physically first 512 …
Just few hours ago, Django's new localization system has been commited to trunk. As some of you know, I did most of the work as my Google Summer of Code project, this year. Of course, together with Jannis Leidel , who also did the final steps, including the commit. Summarizing, with …
Here you have my proposal for Google Summer of Code 2009. It was approved previous week, and I'll be working on it during this summer. The problem While Django provides an amazing system to translate texts, and displays localized dates in some parts of the admin; it has many data …
Until now, if you had to develop an online store in django, you had two options, use satchmo , or write your own code. Satchmo is a huge application that tries to provide everything for all the cases, so for a simple shop you've to deal with hundreds of features that …
Some times it can be useful to serve our site content with little differences depending on the visitor operating system. I really think it's a bad idea changing the content or doing some big changes, depending on it, but this post can be useful for it as well. So, while …
Those days there is some activity in the django model translation area, specially for the two new projects that joined django-multilingual and transdb to achieve this: django-transmeta and django-modeltranslation . While there are some intentional differences among some projects (for example django-modeltranslation is the only one that can translate models without …
Here you've a simple function to avoid a python script to be executed more than once at the same time: def use_lock(func, lockfile): if not os.path.exists(lockfile): with open(lockfile, 'w') as f: f.write(str(os.getpid())) func() os.remove(lockfile) return True else: return None …
Some time ago I needed to add an IP field to my model with more records (some hundred thousands). I was going to just add Django's IPAddressField, but I realized that it stores the data as text on the database, and I didn't like the idea. Basically, and IP address …
django-transmeta is a new project that provides django field translations in a simpler way than existing ones like django-multilingual and transdb . The basis of that simplicity is creating a field in the database table for every translation, so internally we'll have something like: CREATE TABLE app_model ( [...] myfield_en varchar, myfield_ca varchar …
Modern browsers escape urls automatically before sending them to the server, but what happens if your application serves http requests to clients that doesn't escape urls? The answer is that can get unexpected results if you server works in Django (and probably in any python framework/application). That's because python's …
Today I find a solution for one of the most common problems I had with CSS, vertical centering. I really don't like CSS, and one of my examples to explain why, it was the lack of a elegant way to vertical align an element inside a div. First page I …
I had to compare two XML strings for some unit tests, and if you want to do it without considering the indentation, or the newlines, it is a little bit tricky. I thought that parsing the original xml and returning it again (using minidom), I'd got a raw string without …
Some time ago I purchased a Brother multifunctional priner/scanner/fax, the MFC-235C . Today, after reinstalling my laptop, I had to install it again, and it has been as easy as the first. Specially because Brother has GNU (aka Linux) support, including GPL drivers for all printer features. As well …
Everything is changing on Django those days, and many people contacted me because StdImage stopped working with trunk. Basically all major changes (except GeoDjango) affected it, such as change from newforms to forms, signal refactoring, and file storage refactoring. Now it's up to date. Remind that django-stdimage is a Django …
Few days ago a new search engine was published, and many media covered the it. I supposed for that, that it should be something important, but I think that in few days it'll only be important for its creators. Why I think that? Well, here you have my arguments for …
Some time ago, I wrote a post in this blog about the essential Firefox extensions for developers. After some time, that post is out of date, so I changed some of those extensions, and started using some new. Here is the updated list: Firebug : Javascript debugging, graphical check of CSS …
In this article you'll find some tips, that could be useful for avoiding problems or extra work when translating your Django application. 1. Setting up the environment Doing some trivial changes to your project structure, can avoid you of translating many string (the ones that are already translated in Django …
On July 13th, DjangoCon 2008 was announced, and the illusion and nervousness started for many djangonauts. The main problem here, is that due to the lack of time and experience, the number of attendees is limited to 200. That makes sense, but there are other things that doesn't make sense …
Here you have what, from my point of view, is the status of django i18n. Comments will be very welcome, specially from people from countries with other i18n needs than mine (based on the idea that Django i18n is perfect for people in the US, here is the troubleshooting for …