Rants on the daily grind of building software. This has been moved to https://slott56.github.io. Fix your bookmarks.
All of the content has been moved And reformatted. And lightly edited to clean up a few of the long-standing problems. Moved to https://slott56.github.io/ . Nothing new will be posted here. All the new stuff will go to https://slott56.github.io/ . Fix Your Bookmarks Why are you still reading? Click the link and go to https://slott56.github.io/ and fix your bookmarks.
Check out DZone's Coding zone: https://dzone.com/coding . Hover over the "Coding" drop-down menu. Notice anything lacking? I'll give you a hint: Python. They have "Frameworks", "Java", "Javascript", "Languages" and "Tools". The "Languages" seems to be general programming, and the posts include Java, JavaScript and numerous other languages. It leaves me with questions about the basis for the…
Apple Books Pivot to Python A Guide for professionals and skilled beginners https://books.apple.com/us/book/pivot-to-python/id1586977675 I've recently updated this to fix some cosmetic problems with title pages, the table of contents and stuff like that. The content hasn't changed. Yet. It's still an introduction to Python for folks who already know how to program, they want to pivot to…
Some background: FAERIE DUST™ , Obstinate Idiocy , Obstinate Idiocy, Expanded , and even Permutations, Combinations and Frustrations . I want to set up algorithm design as the diametric opposite of Obstinate Stupidity. To do that, let's look at Obstinate Stupidity. The theme? We did something wrong, and we don't want to fix it. I emphasize this because it takes many forms. Another common variant…
Real Question (somewhat abbreviated): "My algorithm performs badly, do I need asyncio?" Short answer: No. Long answer: Sigh. No. Do you need a slap upside the head? Here's how it plays out: Q: "We figured that if we 'parallelize' it, then we can apply multiple cores, and it will run 4x as fast." Me: "What kind of I/O are you doing?" Q: "None, really. It's compute-intensive." Me: "Async is for I/O.…
When I talk about functional programming in Python, folks like to look for place where functional programming isn't appropriate. They latch onto finite-state automata (FSA) because "state" of an automata doesn't seem to fit with stateless objects used in functional programming. This is a false dichotomy. It's emphatically false in Python, where we don't have a purely functional language. (In a…
This isn't about details of pySpark. This is about the philosophy of testing when working with a large, complex framework, like pySpark, pandas, numpy, or whatever. BLUF Use data subsets. Write unit tests for the functions that process the data. Don't test pyspark itself. Test the code you write. Some History I've worked with folks -- data scientists specifically -- without a deep background in…
See https://towardsdatascience.com/building-generator-pipelines-in-python-8931535792ff I'm delighted by this article. I was shown only the first, horrible, example. I think the idea was to push back on the idea of complex generators. I fumed. Then I read the entire article. Now I'm fuming at someone who posted the first example -- apparently having failed to read the rest of the post. This idea of…
I've been trying to help some folks who have a "search" algorithm that's slow. They know it's slow -- that's pretty obvious. They're -- unfortunately -- sure that asyncio will help. That's not an obvious conclusion. It involves no useful research. Indeed, that's a kind of magical thinking. Which leads me to consider the process of over-engineering. The Problem Over-engineering is essentially a…
This is bonus content for the forthcoming Functional Python Programming 3rd edition book. It didn't make it into the book because -- well -- it was just too much of the wrong kind of detail. See this " Tough TCO " document for some thoughts on Tail-Call Optimization that can be particularly difficult. This isn't terribly original, but I think it's helpful for folks working through more complex…
I started with the Apple Books platform because it's an easy default for me. Pivot to Python A Guide for professionals and skilled beginners https://books.apple.com/us/book/pivot-to-python/id1586977675 I've recently updated this to fix some cosmetic problems with title pages, the table of contents and stuff like that. The content hasn't changed. Yet. It's still an introduction to Python for folks…
In the long run, I think there's a small book here. See 8 reasons Python will rule the enterprise — and 8 reasons it won’t | InfoWorld . The conclusion, "Teams need to migrate slowly into the future, and adopting more Python is a way to do that," seems to be sensible. Some of the cautionary tales along the way, however, don't make as much sense. TL;DR. There are no reasons to avoid Python. Indeed,…
I almost made a terrible blunder. See https://github.com/slott56/py-web-tool for some background. This is a "Literate Programming" tool. I started fooling around with this kind of thing back in '05 (maybe even earlier.) This is not the blunder. The whole idea of literate programming is not very popular. I'm a fan of Jupyter{Book} as the state of the art in sophisticated literate programming, if…
First, there's Pivot to Python A Guide for professionals and skilled beginners https://books.apple.com/us/book/pivot-to-python/id1586977675 I've recently updated this to fix some cosmetic problems with title pages, the table of contents and stuff like that. The content hasn't changed. Yet. It's still an introduction to Python for folks who already know how to program, they want to pivot to…
A presentation I did recently. https://github.com/slott56/bashing-the-bash Folks were polite and didn't have too many questions. I guess they fundamentally agreed: the shell is awful, we can use it for a few things. Safe Shell Scripts Stay Simple: Set the environment, Start the application. The Seven S's of shell scripting. Many many thanks to Code & Supply for hosting me.
This is the best part about Python -- you can build something quickly. And it really works. But. What are the next steps? While there are a *lot* of possibilities, I'm focused on an "enterprise work group" application that involves a clever web service/RESTful API built in Flask. Maybe with NLP. Let me catalog a bunch of things you might want to think about to "productionize" your great idea.…
Enterprise COBOL is both a liability and an asset. There's tangible value hidden in the code. See https://github.com/slott56/looking-at-cobol I've tweaked the presentation a little. The essential ingredients in coping with COBOL are these: Use something like Stingray Reader to parse COBOL DDE's and process the data in the native format. Analyze the Job Control Language (JCL) to work out the…
My literate programming tool, pyWeb, has moved to version 3.1 -- supporting modern Python. Next up, version 3.2. This is a massive reworking of the data structures involved. The rework lets me use Jinja2 for templates. There's a lot of fiddliness to getting the end-of-line spacing right. Jinja has the following: {% for construct in container -%} {{construct}} {%- endfor %} The easy-to-overlook…
As noted in My Shifting Understanding and A Terrible Design Mistake , I had a design that focused on serialization instead of proper modeling of the objects in question. Specifically, I didn't start with a suitable abstract syntax tree (AST) structure. I started with an algorithmic view of "weaving" and "tangling" to transform a WEB of definitions into documentation and code. The weaving and…
I've been fascinated by Literate Programming forever. I have two utterly divergent takes on this. See https://github.com/slott56/PyLit-3 for one. See https://github.com/slott56/py-web-tool for another. And yet, I've still done a really bad design job. Before we get to the design, a little bit of back story. Back Story Why two separate literate programming projects? Because it's not clear what's…
When writing about code in LaTeX, the essential strategy is to use an environment to format the code so it stands out from surrounding text. There are a few of these environments available as LaTeX add-on packages. The three popular ones are: verbatim . I think this is built-in to LaTeX. It's not very clever, but it is simple to use. listings . See https://www.overleaf.com/learn/latex/Code_listing…
In Static Site Blues I was wringing my hands over ways to convert a ton of content from a two different proprietary tools (the very old iWeb, and the merely old Sandvox) into something I could work with. After a bit of fiddling around, I'm delighted with Pelican . First, of course, I had to extract all the iWeb and Sandvox content. This was emphatically not fun. While both used XML, they used it…
I have a very large, static site with 10+ years of stuff about my boat. Most of it is pretty boring. http://www.itmaybeahack.com/TeamRedCruising/ I started with iWeb. It was very -- well -- 2000-ish look and feel. Too many pastels and lines and borders. In 2012, I switched to Sandvox. I lived on a boat back then. I don't have reliable internet. Using blogger.com, for example, required a sincere…
I've been an on-and-off user of LaTeX since the very, very beginning. Back in the dark days when the one laser printer that could render the images was in a closely-guarded secret location to prevent everyone from using it and exhausting the (expensive) toner cartridges. A consequence of this is I think the various algorithm environments are a ton of fun. Pseudo-code with math embedded in it. It's…
I was asked to review code that -- well -- was evil. Not like "shabby" or "non-pythonic". Nothing so simple as that. We'll get to the evil in a moment. First, we have to suffer two horrible indignities. 1. Busy Waiting 2. Undefined Post-Conditions. We'll beat all three issues to death separately, starting with busy waiting. Busy Waiting The Busy Waiting is a sleep-loop. If you're not familiar,…