Recent content on Learnings in IT
Introduction Nowadays, much of the online commentary related to Generative AI is focussed on Automated code Generation and rightly so. But Lets look at alternate use case of it. Traditionally, Business Intelligence (BI) relies on custom-built dashboards and reports that query a data warehouse. This often requires heavy intervention from engineering teams to analyze requirements, develop artifacts,…
Introduction This article explores various ways in which Dapr helps with development and operations of distributed applications. If you build software today, you are likely building a distributed system. Whether it’s two services talking or a monolith calling a lambda, you have crossed the process boundary. Distributed applications are software systems that consist of multiple components or…
Introduction This article compiles my notes on an excellent architectural deep-dive by Jeffrey Richter. He outlines how to architect robust AI Agents using Google’s Agent-to-Agent (A2A) protocol for coordination and Anthropic’s Model Context Protocol (MCP) for tool integration. It is a must-read for anyone moving beyond simple chatbots to complex agentic workflows. You can read the original…
Introduction Good software architecture is crucial for a business to reach its goals. It provides a structured way to build software that is easy to maintain, reliable, and performs well. Key drivers of software architecture Several factors influence a software’s architecture: Business requirements: What the software needs to do. Quality attributes: How well the software performs, such as…
Introduction A Software Bill of Materials (SBOM) is a list of all the components, libraries, and modules that make up a software, providing transparency into its composition. It describes various packages and dependencies that go into creating a software artifact. Software products are composed of many different components, some of which might come from third party sources. These third-party…
Introduction In the ever-evolving landscape of AI and machine learning, Google’s MCP Toolbox for Databases stands out. This open-source server enables developers to connect generative AI applications to enterprise databases, facilitating prompt-based querying and natural language processing (NLP). Whether you’re setting up your LLM on-premises using OLLAMA or leveraging providers like…
Background There was a requirement to perform series of tasks, involving generation of output files, such that the required throughput is achieved. These tasks involve database read operation, external API invocation and file i/o. Generally, benchmarking showed that executing them in sequential way was not helpful. What if asynchronous programming be used to perform this task. So Lets Start.…
Location Mumbai, Maharashtra, India Professional Title Senior Technical Architect at Worldline India What do you do ? I write code. In my current role, I am responsible for Architecture for all the systems in Payments space covering Acquiring and Issuance of Cards, UPI (QR based payment system in India). I am involved in creating and maintaining the Architecture of various systems, integration…
Introduction Artificial Intelligence, especially Large language models (LLMs) are all in high demand. Since OpenAI released ChatGPT, interest has gone up multi-fold. Since 2023, Powerful LLMs can be run on local machines. Local Large Language Models offer advantages in terms of data privacy and security and can be enriched using enterprise-specific data using Retrieval augmentation generation…
Java Language Articles, E-books Getting Started with Java in 2023 Finding Java Thread Leaks With JDK Flight Recorder and a Bit Of SQL
Background Ever encountered a scenario where REST API consumption works from tools like curl , Web Browser but not from Application. Lets dive in. The requirement is as simple as consuming REST API from a Application over TLS. Problem Statement The REST API, to be consumed, is standard API interface which requires access over TLS. The client in this case is Windows 2016 server. During Development,…
Background A Client has E-commerce Application consisting of services aimed at specific domains of business functionality it serves. One of these services is responsible for accepting the order, authenticating it and forwarding it for further processing in terms of inventory checks, payment and so on. For Authentication, this service sends SMS to Customer’s Mobile number (and e-mail id) and…
PostgreSQL General Articles Mistakes to avoid while using PostgreSQL The Part of PostgreSQL We Hate the Most We are sticking with postgresql Free Postgres database (or SQLite) from fly.io Using generate_series feature for reporting When Postgres blocks: 7 tips for dealing with locks PostgreSQL - Don’t do this Is PostgreSQL good enough? Online event processing by Martin Klepmann PostgreSQL rocks,…
Introduction While embarking on building any new server application, one of the key requirement is whether it needs durable, persistent storage of data (and in most cases, it does). This is followed by evaluating suitable data store. Likely evaluation criteria is Application’s Requirement (Tolerance for eventual consistency, High Availability etc.), Team’s familiarity, Costs, Tech.…
Introduction We develop a piece of software with aim to fulfil specific business requirements in terms of resource usage, throughput, availability among others. Profiling and benchmarking are approaches that developer has in his/her arsenal to gain continuous feedback on whether a piece of code is behaving optimally and adhering to it’s objectives. Lets look at what they mean, Profiling is…
Introduction I have been reading excellent Database Reliability Engineering book and below are my notes from it. Key Incentive(s) for Automation Elimination of Toil - Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows. Important System Characteristics Latency…
Introduction Suppose you have a distributed application running in production and it is based on Micro services/Service Oriented Architecture and have SLA of being “always on” (be available 24*7, barring deployments of course !!). In such cases, having proper monitoring of Application health in place is absolutely essential. What if Monitoring is an afterthought (i.e. application is…
Introduction One of the design considerations stressed upon by Jeffrey richter about APIs (Read more here ) is that “API is expected to be stable over long period of time”. Recently,for a .NET based project, we decided to upgrade some of the ASMX (legacy SOAP based approach) based APIs and were immediately reminded by Customer(s) to avoid any kind of impact on existing users. This…
Introduction In a company/enterprise, typically there are multiple sources of data. This could be result of M&A (where each of those add in a new data store) or result of multi year process of using data stores that are in vogue at that time. Result is combination of various types of relational databases, flat file systems, queues and so on. This results in Data Silos. This scenario is typically…
Introduction While gathering data for Analytics, one often has to source data from multiple sources. Traditionally, the approach has been to do ETL (Extract-Transform-load) where, Extract - typically involves retrieving data from source. This could also be via streaming Transform - Apply transformation to the extracted data. Load - Loading the data in Operation Data store (ODS) or data warehouse…
Background Recently, i went through excellent video series on Designing & Versioning HTTP_REST APIs presented by Jeffrey Richter . It is available here . In the past, i had read Jeff’s books on CLR and found his writing to be very clear and understandable. So is my experience with this Video Series. Below is summary of learnings from this Video Series. I do not claim that every aspect is…
Background In a typical workflow of software development, Developer implements a Unit/component, tests it and pushes the changes to source control repository. It then goes through Continuous integration, automated testing, provisioning and deployment. Given High availability requirements expected (or should i say assumed) nowadays, As much as functional correctness of the Unit, it is also…
Background In a typical business Application, there are often requirements for, Batch processing - Often long running Tasks like data import/export, End of day processing etc. These tasks are often scheduled to be executed at pre-defined interval or on occurance of an Event. Asychronous processing - Tasks, often part of business process / workflow, that can be performed asychronously or offloaded.…
Background How many times have we landed up in a meeting staring at random slowness or such production issues in a distributed Application ? only to experience helplessness with limited (or often times no) visibility available about the runtime behavior of the Application. It often ends up in manually correlating whatever diagnostic data available from Application and combining it with trace/logs…
Background I primarily work on Windows for development purposes. Whenever its about writing code in Golang, invariably one comes across usage of Make. A quick check on popular Go projects on Github will show Makefile being used to automate tasks like linting, build, testing and deployment. Being on Windows, i have been looking for alternative build tool that is easy to setup (i.e. doesn’t…
Background I started this blog, https://sachinsu.github.io few months back . In this relatively short period of time, Blog has sizeable number of useful links across various categories in addition to the detailed blog post like this one. As an ongoing activity, I think that it is necessary to verify links mentioned on this blog. So how can it be done ? obviously one way is to do it manually by…
Background I recently had opportunity to support team who has been battling with Intermittent (scary i know :)) issues with TCP connectivity in Production. Simplified deployment Architecture is as below, High Level Architecture Technology Stack used is Microsoft .NET Framework 4.8 using ODP.NET for Oracle Connectivity (Oracle Server is 8 CPU box). Each of Web Servers in cluster have IIS hosted on…
Background I recently came across bounty by Balaji Srinivasan to send Direct Message to all twitter followers. Currently, i do not intend to participate in bounty and this is mere exercise. This is an attempt to write CLI tool in Golang in response to it. For detailed requirements, refer here Approach In Brief, CLI should, accept arguments like Twitter API Key,Auth token, DM Message Download all…
Section covering resources for Online learning etc. Exploring basics of Computer Science, bit by bit Exploring basics of Distributed Systems Awesome List of Free Learning Resources Collection of online learning resources Complete intro to Linux and CLI Linux System Administration - Skill challenge
Python has become pervasive all throught Data Science be it Machine learning, Deep learning, Data Processing and general purpose tasks like Web Development. Courses, Trainings Python Tutorials Python for Beginners from MSDN Nice Collection of trainings per level of complexity Python Programming And Numerical Methods: A Guide For Engineers And Scientists Practical Python Projects Articles Getting…
At my current workplace, All Applications are expected to adhere to PCI DSS standards meant for Data protection, Access Regulation and so on. Dedicated SOC Team,consisting of Security analyst who are continously on the prawl to identify breach, conduct periodic auditing of Applications, hardening of Servers. While all our .NET applications adhere to below guidelines, ASP.NET Security Overview…
Over the last many years, de-facto language of the Web (specifically front-end) has been Javascript (and variants like Typescript, ECMAScript versions and so on). The Web development has been revolving around HTML+CSS+Javascript trio. It all started with support for Javascript in browsers, followed by addition of XMLHTTP API, Rich DOM Manipulation Support in Javascript. To induce order and apply…
Background The source code of tracfee.com is hosted on Github Private. At a High level, Tracfee’s Architecture involves, Single Page Application using VueJS, deployed on Netlify API in Go , deployed on Oracle Cloud So far, API testing has been automated and we were looking at ways to automate deployment of both UI and API. Steps required to deploy API are less since we are using Docker to…
Useful links for deep diving in various Cloud Ecosystems Articles Cost of Cloud - Paradox Cloud vs. On-premise Computing Zero dollar Infrastructure stack Cloud Server Performance, Prices, Specs and Features The Cost of Cloud, a Trillion Dollar Paradox Why we are leaving cloud General Guidelines when working as Cloud Engineer Articles (GCP) 13 sample architectures to kickstart your Google Cloud…
Useful links related from Generative AI, ML space Collections Gen AI- Collection of Articles on AI Code Generation and its pros and cons AI Guide by Mozilla Collection of resources related to Applied ML List of for MLOps Prompt Engineering Playbook for Programmers Free courses Fast AI by Jeremy Howard AI Canon - List of resources around GPT Free Deep learning course Articles AI native software…
User Interface / User Experience Design Approach Modern Web - Guides, tools and libraries for modern web development. How Stripe Designs Beautiful Websites Tools for non artistic developers Principles of Design Micro frontends - Techniques, strategies and recipes for building a modern web app with multiple teams that can ship features independently. Thoughts on SPAs 33 JavaScript concepts every…
Background Recently, we had requirement wherein a process should, Periodically (Poll) or Asynchronously (Pub-sub) listen on incoming requests/messages. The whole process is expected to be long running. Should also implement clean disposal of in-flight requests and subsequent cleanup using something similar to Cancelble Context in Go The first of the objective is somewhat dependent on mechanism…
Background Oftentimes, we come across situation where code does not perform as per expectation. What is typically approch to address it, Performance Testing - Visual Studio Load Tests or Third party tools like Locust , Vegeta , Gatling etc. Visual Studio Diagnostics Tools Or Use tools like Perfview/dotTrace/dotMemory to diagnose bottlenecks What if it is possible to Benchmark code for, Set of…
Background A Web Application, developed in ASP.NET Core (Runtime Version 3.1.100) using Razor Pages and Web API, is expected to be launched from within third-party Web Application in iframe, with complete HTML being rendered. During the Development, a mock HTML Page was developed to simulate launching of ASP.NET core based Web Application in iframe. Note that this page as well as Application was…
Perspectives Section covering Business, project/programming perspectives Laws of Frugal Architecture Cognitive load is all that matters Stick to boring Architecture Your tech stack is not the product Architecture anti-patterns Don’t call yourself a programmer Grasp Responsibility Patterns Things every programmer should know Guiding principles after 20 years of programming Programmers: Before you…
General Purpose tools Section covering useful tools for every day activities, Online learning etc. Plane - Open source alternative to JIRA ShareX - Screen capture, file sharing and productivity tools (Windows only) Dark Lang - Declarative platform to build serverless backend OBS Studio - Free and open source software for video recording and live streaming. Open source Wiki platform Open source 3D…
Background Recently, i got involved in assignment where in an application was facing issues with throughput. Expectation is to support more than 500 transactions per second while load testing results were indicating system was experiencing high latency beyond 100+ transactions per second. This application is developed in .NET Framework + .NET Core and primarily uses Relational Database for…
Oracle Database Performance, Best Practices Connection Strategies for Database Applications Using High-Speed Data Loading and Rolling Window Operations with Partitioning Designing Applications for Oracle Real-World Performance Best Practices for Extreme Performance with Oracle Data Warehousing Blog on Oracle Performance troubleshooting Using PL/SQL Bulk processing features Auditing tables using…
NOSQL Databases MongoDB FerretDB - MongoDB Interface with underlying PostgreSQL as database engine Amazon DynamoDB Data Modelling in DynamoDB Must follow Twitter handle of Rick Houlihan Best Practices for Secondary Indexes with DynamoDB Apache Cassandra 7 mistakes when using Apache Cassandra Apache Geode How Mastercard fights fraud with Apache Geode Apache Pinot Pinot- Enabling Real-time Analytics…
MySQL Links Query analytics for the day-to-day developer with MySQL 8.0 Schema Change Management for MySQL Temporal Data tables in MariaDB MySQL 8.0 Indexes, Histograms, and Other Ways to Speed Up Your Queries Maxwell - MySQL to Kafka change data capture LetsEncrypt setup for MariaDB How LetsEncrypt has built Next Gen Database Servers 18 things you can do to remove mysql Bottlenecks due to High…
Database Knowledge base around general database related topics. General Links Which Data Architecture to choose Prisma’s Data Guide - A growing library of articles focused on making databases more approachable. Query optimization guide Database performance for Developers Heimdall data -Database scale-out without Application changes Database of databases Modern SQL in databases Eventual consistency…
Below are some of my project(s), Tracfee , One stop for Tutors to manage students, track fees. Developed as SPA in VueJS + Quasar using API in Golang , Oracle Database and hosted on Netlify . RSS APP RSS Reader app, to be used in lieu of Google Reader. Developed in Python with MongoDB as database.
I am a software architect with more than 2 decades of experience, currently working at @worldlineglobal . This is my personal site where i share helpful content (gathered or authored) on Technology (and other topics). I appreciate any ideas/suggestions you have on how I can improve this site.
Microsoft .NET Platform where i have spent most time till now. General Links What is .NET Aspire Compared to Dapr What is .NET? by Scott Hanselman Async in Depth Using Async/Await in WCF or ASMX with AsyncEx Comparing Async/Await with GoRoutines .NET Presentations - Events in a Box Building Microservices in .NET Materialized View Pattern for Cross Service Queries Oracle DB and .NET - Optimizing…
Go Language My current Favorite Language Articles, E-books About Go - Compiler, packaging etc. When (and when not to) to use Generics in Go Ver. 1.8 High performance GO Workshop Learnings from Production usage of Go Thoughts on Go performance optimization Effective Go Handling 1M websockets connections in Go Notes on Go language Standard Go Project Layout 10 things you (probably) don’t know about…