a personal blog by ethan miller about topics related to software engineering but sometimes extending into the intersection of technology and society.
We all want code (and systems) that are simple. Rich Hickey, the creator of Clojure, distinguished simple from easy . The easiest thing is not necessarily the simplest. But the software engineer Dan Palmer takes this further, pointing out that there are multiple conceptions of simplicity, and they exist in tension with each other: But what is simplicity? Despite this near universal agreement that…
Daily electricity demand patterns from Victoria, Australia (vic_elec dataset) — each line is one day, colored by year I recently relaunched tablecloth.time with version 1.00-alpha-6 . The goal of this project, which remains experimental, is to build a composable extension for temporal analysis on top of tablecloth , the primary dataframe library in the Clojure data science ecosystem. Tablecloth…
How curious that these very different applications have all become similar tasks in the world of AI, approached through similar pieces of engineering. Comprehension questions are treated as translation questions, and translation models are further simplified into language models — essentially, predict the next word, given previous words. How could this interchangeability possibly work? How could…
The New Column API Today we at SciCloj deployed a new Column API ( tablecloth.column.api ) into the data processing library Tablecloth (available as of version 7.029.1 ). This new API adds a new primitive to the Tablecloth system: the column . Here’s how we use it: (require '[tablecloth.column.api :as tcc]) (tcc/column [1 2 3 4 5]) ;; => #tech.v3.dataset.column<int64>[5] [1, 2, 3, 4, 5] The new…
This post is the second regular update for my “Column for Tablecloth” project, which has been generously funded by Clojurists Together . The goal of this project – explained in more detail in my first update – is to give the column a presence within the data processing library Tablecloth . In other words, by the end of the project, users of Tablecloth should be able to interact exclusively with a…
For a few years now, I have been working with a unique, global group of people associated with the SciCloj community, many of whom are interested in promoting the use of Clojure for data-centric computing. As part of this effort, I recently applied for and then received funding from Clojurists Together – an organization in the Clojure community that provides funding for open source work on Clojure…
Automatic keyword extraction is a frequently encountered problem in natural language processing that poses some interesting challenges. The task of an automatic keyword extractor is to extract set of words and/or phrases that effectively summarize a given text. Such a tool can be useful in a variety of situations. Generating keywords for human consumption is one use. Additionally, because keyword…
Managing Python development environments can be confusing. Tools like pyenv and virtualenv , while powerful, have a steep learning curve. I encountered this recently while trying to find a sensible way to set up Anaconda the popular Python and R distribution for data science frequently used with Jupyter Notebooks. The problem I experienced stemmed from the fact that Anaconda has its own version of…
In this post, I explain my reasons for building whatishistory.net , and, in doing so, I also present some ideas about the meaning of the notion of “openness” in the domain of culture as it is being pursued by so-called digital humanists. What is the website? Whatisistory.net is a simple tool for collecting definitions of the field of history. It provides a form for entering a definition, which…
Recently, I attended PolyConf ( #PolyConf ) in Poznan, Poland. The conference was organized around the idea of polyglot programming, a philosophical approach to programming that stresses the value of coding in multiple languages. Set against a tendency that often finds developers lamenting the chaos of programming languages as if they’d entered the Tower of Babel, the polyglot programmer instead…
In the last few years, the field of digital humanities has emerged to some prominence. “Digi Labs” are suddenly appearing all across the academic landscape, and one now meets graduate students who are not only writing PhDs in one of the classic disciplines, but who also belong to digital lab or institutes of some sort. What I want to do in this post is define in an expansive way what I think this…