Latest Posts on Fully-Faltoo Blog
We originally created Screener AI using WebSockets. While Django’s WebSockets worked pretty well, they had a few issues: The sessions were linked to the browser connection. If the user’s browser disconnected, the answer generation also stopped in the backend. This was especially problematic on mobile, where browsers are stopped after 5 seconds if the user switches apps or turns off the screen. The…
Last night, a simple delete query brought screener.in down. This is how the things went. One of the tables in the database had around 80 million rows. I ran a MySQL query to delete around 60 million rows. I went off to sleep and my remote shell got disconnected. 7 hours later, I woke to see the website performing very slowly. The disk utilisation showed writes of 80MB/s for the last 7+ hours.…
Radhika and I had our second baby. He shares the birthday date with Radhika - 3rd September.
Enso , a writing tool, has a beautiful little feature called CoffeeMode . It enables writing privately in public places. Anything we write appears as a •. I was sitting in a library recently, scribbling my random thoughts in my jrnl . This CoffeeMode sounds perfect for such moments (such as on airplanes). Interestingly, it was easy to implement this mode in Neovim using the conceal feature.…
Yesterday I wrote a script to create backups for my server. Just when I ran the script, I realised it would take an hour to complete. It was 7pm. I needed to leave for home. I cursed myself for not doing bgrun bash backup.sh instead of bash backup.sh . The thing is bgrun doesn t exist. The closest thing I found was: nohup your_command > output.log 2>&1 & disown . Let’s break it down: nohup = no…
A bad carpenter blames his tools I implemented metaWeblog APIs on this blog over the weekend. This allows me to use MarsEdit (or any other blogging tool) for writing posts. I remember using Microsoft s Live Writer a lot in the early days. The desktop interface reduces the friction of opening a webpage, logging in, and navigating various menus. I created my own blogging system to avoid the…
Handling millions of requests from thousands of IPs is hard! A flood of millions of requests clogs one drain after another. This is what happened when we had a DDoS attack this week. First, our hard-disk got filled due to access logs Then our hard-disk got filled due to too many open files error logs Various services started hitting file descriptor limits The system started dropping packets and…
This post by Andrej Karpathy stuck with me. There are a lot of videos on YouTube/TikTok etc. that give the appearance of education, but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a…
Flyer for my Neovim Workshop Last week, I did a workshop on Neovim. I haven't seen any technical workshops in Lucknow before. I thought of starting one. I created a standee and placed it right outside the TCS gate. Standee for Workshop I didn't expect too many registrations because of the niche topic. I wanted to attract only the nerds-and-geeks for the first event. I used HasGeek for…
Last week, I transferred my domains from Squarespace to PorkBun. All 12 of them! Porkbun's Domain Dashboard Squarespace Billing Nuisance I was frustrated with the billing page on Squarespace. Their auto-payments didn't work. This was a major risk as I could lose my domains. Part of it was because of RBI regulation in India. But more frustrating part was jumping between the pages finding the…
I changed my laptop recently. I was unable access GitHub from command line after that. I was getting this error. remote: Support for password authentication was removed on August 13, 2021. remote: Please see https://docs.github.com/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication. fatal:…
We recently started using OKRs for our Screener team. We are a team of 7. - 4 Developers: 3 local + 1 remote (part-time) - 3 Support: 1 local + 2 remote Four of the team members work remotely. Most of the tasks are picked, planned and prioritised by me. I find it hard to assign tasks, do continuous follow-ups and provide feedbacks. This is specially hard on our remote team. This centralised…
Speed Reading by Kam Knight I saw this Tweet by Arpit Ranka. I didn't believe it. Speed Reading sounds like a gimmick to me. I have read a speed reading book before. I have also tried various speed reading tools . They were all placebos and didn't last long. I was so much in disbelieve that I had to ask Arpit Bhaiya in person when I met him. "Bhaiya, I was surprised to see your Tweet on Speed…
I was a VS Code user for last many years. Though it "just worked", it started getting slower. It became too magical and too bundled. It became harder to configure and customise. I had too many plugins and I didn't know what was overriding what. This is when I read about Zed . I liked their simplicity and speed. Zed provides very limited configuration options . This makes it easy to configure and…
Screener was quite a bit slow yesterday. But more importantly, once in a while we were getting a "504 bad gateway" error. One thing I remember the most from Max Kanat-Alexander's "Understanding Software" is to never fix a problem until you can reproduce it. Sometimes people have a very hard time debugging. Mostly these are people who believe that in order to debug a system, you have to think about…
Phalut top - with The Sleeping Buddha in the backdrop This was my first high altitude trek. After returning, I am surprised, it didn't feel that hard. I can to do it again! Probably the walk on day 4, from Phalut to Aal changed everything. It was a walk I will remember for a long time. That day I saw the valley of clouds. The ocean of clouds floating beneath us. The little hail (kind of…
There have been lots of interesting developments over last few weeks: Added a WYSIWYG editor (Squire) Created a new app: DumbFill (RAG) Had a small family get-together Caught "The 4-Hour Work Week" bug While the first 3 were more creative and fun; the last one has been occupying me the most. It is an experiment I would like to talk about. Warning: I have still read only 50% of the book. Trying to…
I attended PyDelhi conference this week. It was refreshing to talk about tech, non-stop, for two days. These are some notes from the talks at the conference. Talk on Zen Of Python by Shantanu. Pic by PhysicsWorm. NetworkX for network analysis Navya gave a simple and quick overview of the NetworkX library with practical examples. I want to try this library. Though I don't have an immediate…
I travelled to Chennai last week with my wife and daughter. We visited temples in Kanchipuram, did some saree shopping, and travelled around the city. I stayed for two more days and visited some listed companies after my wife returned. I was surprised to see zero encroachment in the city. In Lucknow and most parts of India, the foothpaths are taken over by street vendors. Just outside our house,…
I get stuck sometimes. And I can do nothing until I solve that problem. When I am stuck; My emails start piling-up; Pending payments go unnoticed; People get offended as I don't reply to their DMs; Meetings as hastened or cancelled; The error logs start increasing; All the KMPs take a sudden nose-dive. I get stuck fixing the music player, while all the guests are waiting for the dinner. I get…
My Brother and I were at Flame University last week. We attended Prof. Sanjay Bakshi's course of Case Studies and Business Models . Ayush Bhai and me at Flame Campus Prof Sanjay Bakshi and Flame Sanjay Bakshi Sir's lectures have changed the way we invest. We always looked for undervalued and unpopular companies going through a transition. We were number guys and focused mostly on financials of the…
I recently shared about Mastodon. The tech conversations are pretty interesting there. Found some good libraries and podcasts on it. DHH Podcast on Minimalist Management Sometimes I have too many things to do. - Pending code reviews - Unanswered support queries - Overflowing inbox - Increasing error logs - Operational stuff such as hiring, planning and other things And while this never ending todo…
I tried Mastodon multiple times. It was hard to use. - Following new people was hard. - The timeline didn't refresh and was broken. - Following conversations was hard. - Seeing what others liked and going down the rabbit hole was not possible. Until I discovered Mona . Screenshot of multi-column view in Mona I read about Mona on Daring Fireball . It is such a pleasant software to use. It provides…
I spend a lot of time in terminal. Most of it goes in reviewing code, managing servers, running tests etc. These are the tools I use. I hope to keep this list updated. It should be helpful during system re-installs. Libraries Shell: zsh This is what runs the commands. ZSH is installed by default. Prompt: Starship Prompt is the information we see about the current env. Starship automatically shows…
I stopped posting the weekly notes for sometime. I thought, I would instead post small updates as and when they happened. But I was unable to do that. Restarting with weekly updates. Joined MMA Matrix I have been wanting to learn boxing or street combat. It is good to have proper skills. To have confidence to help others as well as ourselves when needed. My boxer friend, Tabish, recommended me MMA…
I feel proud of my new record as I never thought I could run 5 kilometers. I haven't felt healthier ever before. The credit goes to my Garmin Coach. Details of my 5k run on Strava I started running in November 2022. I needed a watch to track my runs. My requirements were: The watch should have an inbuilt GPS as I don't carry a smartphone. It should have a good battery life. It is good to have…
MailBrew is one of my favourite services. It allows us to create custom email digests. We can use it to track blogs, Tweets of favorite people and other sources. It is like creating a custom newspaper for yourself. I have 3 daily digests on MailBrew. 1. Daily Morning Digest - 8am This one contains: - Tweets of people who Tweet rarely and I don't want to miss them. - RSS feed for DD News . To track…
These are few of the things I want to do this year. Run thrice a week - do 150 runs I did a Chattisgarh Jungle Trek in 2022. It was one of the most magical experience. Being alone in the jungle for 4 days. No horns - no noise; only the chirping of birds, blowing of breeze and crackling of leaves. Walking on the river, swimming in it, stargazing, sitting by the lotus lake and having conversations…
We recently added a "search everywhere" feature on Screener. It allows us to do a full-text search on all the concall transcripts, announcements and key-insights of all the listed companies. This was our second attempt on it. There are various ways to implement full-text search: - Using full-text index in relational database such as MySQL / PostgreSQL - Using services such as ElasticSearch,…
Yesterday, Screener.in stopped working in the evening. I logged in via remote shell, ran ` top` and saw mysql using all the memory. Next I did ` tail -f /var/log/mysql/error.log ` and saw this frightful error: 10:20:39 UTC - mysqld got signal 11 ; Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. Thread pointer: 0xffed4075d9f0 Attempting backtrace. You…
Django 4.1 release notes - update in Queryset.bulk_create() I upgraded Screener to Django 4.1 last week. I was surprised to see the support for conflicting updates in bulk_create method. This uses "insert...on duplicate key update" under the hood. [Django Docs] . INSERT...DUPLICATE UPDATE ( MySQL docs ) is one of my favourite SQL hacks. It allows us to create and update rows in bulk. We pass the…
Monkey doing a code-review - Dall-E "Every line of code, every comment, every line-break should have a purpose." We usually use Github's review feature for code reviews. It is hard, however, to jump between files when the changes are large. An ideal way would be to manually mark each and every line as checked during the review. I use a combination of git reset and git add -p to do just that.…
We have been of multitasking between tax returns, feature prototypes and code reviews for the last two weeks. We discovered Section 54F this time. This section exempts capital gains (with no upper limit) if the proceeds are invested in buying or constructing a house. Very interesting. This is something that my Dad does. He creates a physical asset in every bull run. Last year we discovered section…
I love reading how software companies work. We are a very small tech team at Screener. Just 4-5 of us. Yet it becomes chaotic sometimes. At the same time, there are companies which have 10s, 100s and 1000s of developers. They have fabulous products and keep rolling out new features. I admire them and try to learn a few tricks from them. About how they distribute work, how they keep the codebase…
We had some trivial engineering challenges over last 2 weeks. Reducing diff speed from 30+ seconds to 300ms We have a wiki feature at Screener. It allows anyone to modify and add key-insights for companies. In the backend, we use Google's diff-match-patch library. It generates diffs between multiple versions and merges them. The Python version of the library had a long processing time for some…
This was a busy week loaded with code-reviews, designs, meetings and coffees. Investor Presentations on Screener We added a section for investor presentations. It was built on the similar lines as concall transcripts. The backend was developed by Mukesh and me. The frontend was trickier due to the paucity of space. Design versions for investor presentation feature Fun with Aeropress While I have…
Paul Graham suggests avoiding matters of money and disputes from taking up our mental bandwidth. This week had a bit of both. B2B customers at Screener We haven't served B2B customers on Screener because of two primary issues. 1. Data rights 2. Complexity The raw data for personal use is costly. Data for commercial or website use is 20x costlier. And it still comes with certain restrictions. We…
We had a good family get together this week as one of my cousins entered courtship. Progress on Concall Tools I have been wanting to extract structured information from concall transcripts . We started by trying to extract the names of various call participants and their associated firms. We made some progress last week. The algorithm is working pretty well with most concall transcripts now. We…
This was a rather busy week. I was mostly catching up with the pending emails and logs. Extracting data from Concall Transcripts Concall Transcripts contain a wealth of information. I started working on a library to extract structured information from concall transcripts. Initially, we are trying to understand how to proceed with PDFs. We started with a single function for extracting the names of…
Hey, I missed last week's update. I was travelling to Guwahati. Spirit of Entrepreneurship in Young India - A panel discussion I was part of a panel discussion at IIT Guwahati. The purpose was to encourage today's youth toward entrepreneurship. I enjoyed the conversations with other panellists and moderator Jay . Wifey, daughter, me, Jay and Jay's Father at Guwahati Biotech Park Fun at Pobitora,…
It was largely a bug-fixing week. Not the best of the weeks. What's brewing Got a new coffee-making machine: Mukka Express My brother and I are coffee addicts. Not the "black-coffee" kind, but the "instant coffee" ones who can't work without it. We have been wanting to get a coffee machine for our office. I researched and came across Bialetti . Their Moka Pot is a traditional kitchen piece of…
This was a little hobby-works week. I got to play on some new toy projects. Major Developments Created a custom Django starter template . Deploying and setting up a new Django project takes 3-4 days. Most of the time I am copying my setup and deployment scripts from other projects. This starter template reduces the setup time to 10 minutes. It mainly splits up the setting and requirements files…
I love to read weekly notes of some of my favourite techies: - Simon Willison - Chir.ag - Monica - Dejal It is a wonderful insight into their lives. We get a sneak preview of the things they are working on and how they are thinking. I too hope to publish updates that often. Kind of an open life journal. Not because I have too many interesting things to share. But because I want to get back to…
We recently added a full-text search feature in Screener. It searches for all the exchange announcements. We initially faced a few issues while implementing it. However, we were easily able to find a way around them by creating a simple query parser. The problems MySQL offers 2 main types of searches. BOOLEAN and NATURAL MODE. The natural mode is a user-friendly mode. It takes the input and…
I loved this essay by PG. He explains the concept of "schlep blindness". It explains the reason why no one solved payments before Stripe. Every coder knew that problem. Every coder wanted a solution. Yet no one solved it! Because everyone thought that it was a hard problem. It will involve making deals with banks. And then take a lot of risks because of the flow of money involved. These are…
This is such a cool CSS trick. Cross-browser. No JS. The tabs are implemented as different sections with different ids. Then the tab contents are hidden using: section { display: none; } The CSS which toggles them is: section:target { display: block; } Noice!
We saw a sudden increase in disk usage last month. We store all persistent data on /data-volume . I checked what is using the space using: sudo du -h --max-depth 1 /data-volume | sort -h It showed MySQL folder was using around 150 GB. It was huge. Last week it was ~120 GB. I was very concerned about this increase and dug more into it. I thought it was because of the full-text search index. But I…
I loved this chart. How much time can we spend in automating it? I have often wasted hours on automating tiny things. And sometimes never even used those scripts again. This chart should come handy next time.
I loved this interview of AR Rahman. The piece that I loved the most was where he explains how he thinks about creativity [ 10.06 ]. I think it is a constant seeking. When we try 500 things out of which 5 become amazing; I think it's a blessing. We can try 1000 years and still not crack something. But if you get 1 line that lingers on in your heart, in people's hearts, I think that's a blessing.…
Loved this short essay by Taleb. He highlights the difference in the way nature and we humans behave: Nature builds with extra spare parts (two kidneys), and extra capacity in many, many things (say lungs, neural system, arterial apparatus, etc.), while design by humans tend to be spare, overoptimized, and have the opposite attribute of redundancy, that is, leverage—we have a historical track…