I have run hundreds of test-driven coding dojos using cyber-dojo.
I see the same test anti-patterns time after time after time.
Do some of your tests exhibit the same same anti-patterns?
Showing posts with label abstraction. Show all posts
Showing posts with label abstraction. Show all posts
security scan kanban
How is this board supposed to be used? I count the number of blue stories currently being Wibbled - 1,2,3. I look up to the top of the Wibbling column and see its WIP limit is 5. I know 3 is less than 5 so I know there is some spare Wibbling capacity. If I want to know how much I can do the math, 5-3==2. And repeat... I count the number number of blue stories currently being Fubar'd - 1,2,3,4. I look up to the top of the Fubaring column and see its WIP limit is 4. I know 4 is the same as 4 so there is currently no spare Fubaring capacity.
It seems odd to have two columns that look exactly the same when one has spare capacity and the other does not!
My problem is that 5 and 4 are abstractions!
5 is not the same as 5 trays.
5 is just 5.
But 5 trays is [Tray Tray Tray Tray Tray].
What's important about the 5 isn't the 5.
What's important about the 5 is that it represents 5 trays.
What's important about the 5 is the relationship between the 5 and the number of blue stories it limits. The Wibbling blue story WIP limit is abstracted to a 5 but the blue stories are not. I have 1,2,3 blue stories in the Wibbling column but I'm not representing them with a 3, I'm representing them with 1,2,3 actual blue story cards.
I remove the 5 WIP limit from the top of the Wibbling column. Instead I put in 1,2,3,4,5 orange Tray s.
I remove the 4 WIP limit from the top of the Fubaring column. Instead I put in 1,2,3,4 red Tray s.
I add the rule that every Story must always be in a tray.
Now I can instantly see the Wibbling column has some spare capacity because there are some Tray s with no Story in them. I can count the spare capacity if I want - 1 , 2 . No math needed. Easy. I can instantly see the Fubaring column has no spare capacity because all the Tray s contain a Story .
Now I have a kanban board that actually has kanban on it!
Each tray is a kanban.
Now I can experience pull.
Suppose the workers to the right of the Fubaring column use green Tray s. They send an empty Tray to the Fubaring column to signal they're ready to pull a Story .
The workers in the Fubaring column move a done Story from their Tray into the empty Tray .
The workers to the right of the Fubaring column pull the Story in its Tray back into their column.
The workers in the Fubaring column now have an empty Tray .
The workers in the Fubaring column can send their empty Tray to the Wibbling column to signal they're ready to pull a Story .
Thus each Story flows left to right while each empty Tray Tray Tray moves right to left.
Once we have a system under control we can try to improve. For example, we can see what happens when we reduce the WIP limit by 1 in the Wibbling column. We simply remove a Tray .
Caveat Emptor: I've never seen this style of kanban in actual use. It's just an idea so far. As always, if anyone has any feedback it would be much appreciated.
Update! Here's a follow up blog entry.
how to use conscious purpose without wrecking everything
Maximizing efficiency is the error of having a single goal, what William Blake once called “Newton’s sleep.”
Always the more beautiful answer who asks the more difficult question [e.e.cummings]
Evolution always means Co-evolution. The horse eats the grass, the grass grows stronger roots, the horse grows stronger jaws.
What is involved is not simply survival of the fittest, but survival of the fitting-in-est.
The amount of feedback that is built into living organisms differs by many orders of magnitude from the amount that we build into manmade systems.
Flexibility means the willingness to act in response to the feedback message by actually changing how the system works.
There are many Potemkin Villages in operation today, hiding and distracting us from awareness of what’s really going on.
Ignoring feedback merely means that the system will eventually experience a massive unpleasant surprise rather than a small unpleasant surprise.
As Bradford Keeney pointed out, stability is not homeostasis, it’s homeodynamics.
Once you get above that first level, the level of material things and forces, you are dealing with abstractions. In place of physical forces, you have communication—messages, signals. And in place of material things, you have relationships—which are abstractions.
Once we get above the level of physical objects and forces, we are dealing with patterns of interaction, that is, with abstractions.
Abstractions — that is, ideas — don’t die. They can’t be killed. They can’t be exterminated. They just keep coming back, over and over and over. This problem can never be solved if one continues to believe that the so-called "real" world of physical objects and forces is all there is. The Chinese have a word for this. They call it "being stuck in the ten thousand things."
In order to become birds, dinosaurs had to give up being dinosaurs.
If I design a system with no regard for the universe that surrounds it, I will have scanty knowledge of what can impact it.
Synectics
is an excellent book by William J.J. Gordon (isbn 978-0060324308). As usual I'm going to quote from a few pages:
The word Synectics, from the Greek, means the joining together of different and apparently irrelevant elements.
Abstraction breeds more abstraction and more generality instead of leading to tough yes-no tests.
Words like intuition, empathy, and play are merely names put to complex activities in the hope that the naming of the activity will in fact describe it.
Human beings are heir to a legacy of frozen words and ways of perceiving which wrap their world in comfortable familiarity.
Synectics theory agrees with the conviction that a man does not know even his own science if he knows only it.
"All the crappy solutions in the world have been rationalized by deadlines."
He refused to recognize the fact that his search for the perfect problem was a way of avoiding failure in solving a less perfect one.
Invention is akin to painting for in practice, the element being constructed has the capacity to tell the builder what the next step should be. In invention this is much more critical than in engineering because the inventor is always attempting to do something new.
When this forgetfulness is formalized into a methodology, it reinforces the rejection of the commonplace.
Organic functions are unfinished, cylical, and self-reproductive... Synthetic functions are complete and more obviously subject to decay.
Conventions as abstractions from reality constitute a virtually complete and unassailable pattern, whereas the commonplace is infinitely repatternable.
The child who asks: "What's that funny noise?" is told the noise is thunder in such a way that speculation is supposed to stop... But naming the noise does not describe it. It does not answer the question, it kills it.
Forward declaring std::string in C++
One type you can't forward declare in C++ is std::string
class string; // Computer says noThis does not say a lot but what it does say is that string is a class and unfortunately it's not - it's a typedef of basic_string<...>
If I compile a file containing just the line
#include <string>then g++'s -H flag tells me that
#ifndef EG_HPP #define EG_HPP class fwd_string; // instead of #include <string> void eg(const fwd_string &); // instead of // void eg(const std::string &); #endifWhere fwd_string.hpp looks like this:
#ifndef FWD_STRING_HPP
#define FWD_STRING_HPP
#include <string>
class fwd_string
{
public:
fwd_string();
fwd_string(const char *);
fwd_string(const std::string &);
std::string string;
...
};
#endif
You can also use this "type-tunneling" technique to
forward declare enums in C.
I've never seen this used in anger in an actual codebase.
It's just an idea. Caveat emptor.
Understanding comics - the invisible art
is an excellent book by Scott McCloud (isbn 0-06-097625-X). As usual I'm going to quote from a few pages:
Do you hear what I'm saying? If you do, have your ears checked, because no one said a word.
For now I'm going to examine cartooning as a form of amplification through simplification.
Since cartoons already exists as concepts for the reader, they tend to flow easily through the conceptual territory between panels. Ideas flowing into one another seamlessly.
These first symbols - cartoons really - gradually evolved away from any resemblance to their subject, toward the highly abstracted forms of modern languages… and eventually to our totally abstract sound-based system.
The longer any form of art or communication exists, the more symbols it accumulates.
In this chapter, we've dealt with the invisible worlds of senses and emotions, but in fact all aspects of comics show it to be an art of the invisible.
The more an artist devotes him/herself to either of these two focal points (form and idea/purpose), the more dramatic the change if he/she decides to switch.
Symbols are the stuff of which gods are made.
All media of communication are a by-product of our sad inability to communicate directly from mind to mind. Sad, of course, because nearly all problems in human history stem from that inability.
The wall of ignorance that prevents so many human beings from seeing each other clearly can only be breached by communication. And communication is only effective when we understand the forms that communication can take.
The Pragmatic Programmer
is an excellent book by Andrew Hunt and Dave Thomas (isbn 0-201-61622-X). As usual I'm going to quote from a few pages:
A very simple but particularly effective technique for finding the cause of a problem is simply to explain it to someone else.
Fred doesn't know why the code is failing because he didn't know why it worked in the first place.
Chips are designed to be tested.
Design to Test.
Abstractions live longer than details.
Some things are better done than described.
Don't give in to the false authority of a method.
Test Early. Test Often. Test Automatically.
Organize around functionality, not job functions.
When woodworkers begin a project, they cut the longest pieces first, then cut the smaller pieces out of the remaining wood.
The Law of Demeter for functions states that any method of an object should call only methods belonging to: itself, any parameters that were passed in to the method, any objects it creates, and directly held component objects.
No abstraction
Years ago, when my children were still at primary school I was asked to dress up as santa. I stuffed a few cushions under a big red santa costume, lalloped into the room where all the bright eyed kids were waiting, and after a few ho ho ho's I said
Do you know who I am?
to which one of the kids at the back shouted
You're Patrick's Dad.
Kids are great like that.
They're so direct.
They say it exactly as they see it.
The Label Law
The Label Law is one of Jerry Weinberg's laws from
The Secrets of Consulting
The name of the thing is not the thing.I was reminded of the Label Law recently when watching, you've guessed it, The Princess Bride (one of my favourite films) with my son Patrick. There's a scene where the Man in Black (aka Westley) explains to Buttercup how he has become the Dread Pirate Roberts:
...our tendency to attach a name - a label - to every new thing we see, and then to treat that thing as if the label were a true and total description...
Once the stereotyped label is attached, the problem becomes much harder to solve.
The name was the important thing for inspiring the necessary fear. You see, no one would surrender to the Dread Pirate Westley.The Label Law also reminds me of the apocryphal story of the man who, lacking any cheese, baited his mousetrap with a picture of some cheese, only to find the next day he had caught a picture of a mouse.
Richard Feynman knew the difference between something and its name. In his book the pleasure of finding things out he recalls spending time with his Dad in the woods:
Looking at the bird he says, "Do you know what that bird is? It's a brown throated thrush; but in Portuguese it's a … in Italian a …, " he says "in Chinese it's a …, in Japanese a …," etcetera. "Now," he says, "you know in all the languages you want to know what the name of the bird is and when you've finished with all that," he says, "you'll know absolutely nothing whatever about the bird. You only know about humans in different places and what they call the bird. Now," he says, "let's look at the bird."If you like The Label Law you might also like:
The ACM Turing Award Lectures
is an excellent book (isbn 0-201-54885-2). As usual I'm going to quote from a few pages:
The effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer. [Edsger Dijkstra]
If we go back to Latin roots, we find ars, artis meaning "skill." It is perhaps significant that the corresponding Greek word was Τέχνη, the root of both "technology" and "technique". [Donald Knuth]
The assignment statement is the von Neumann bottleneck of programming languages. [John Backus]
I find a certain technique most helpful in expanding my own capabilities. After solving a challenging problem, I solve it again from scratch, retracing only the insight of the earlier solution. I repeat this until the solution is as clear and direct as I can hope for. Then I look for a general rule for attacking similar problems in the most efficient way the first time. Often, such a rule is of permanent value…
To sum up, my message to the serious programmer is: spend a part of your working day examining and refining your own methods. [Robert Floyd]
I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. [C.A.R. Hoare]
UNIX is a simple coherent system that pushes a few good ideas and models to the limit. [Dennis Ritchie] I am a programmer. I write programs. [Ken Thompson]
To parody our current methods of teaching programming, we give beginners a grammar and a dictionary and tell them that they are now great writers. We seldom, if ever, given them any serious training in style… Like writing, programming is a difficult and complex art. [R.W. Hamming]
There are many ways to formulate things and it is risky to become too attached to one particular form or law and come to believe that it is the real basic principle. [Marvin Minsky]
To state a problem is to designate (1) a test for a class of symbol structures (solutions of the problem), and (2) a generator of symbol structures (potential solutions). To solve a problem is to generate a structure, using (2), that satisfies the test of (1). [Allen Newell and Herbert Simon]
The utility of a language as a tool of thought increases with the range of topics it can treat, but decreases with the amount of vocabulary and the complexity of grammatical rules which the user must keep in mind. Economy of notation is therefore important. [Kenneth Iverson]
Through the language glass
is an excellent book edited by Guy Deutscher (isbn 978-0-8050-8195-4). As usual I'm going to quote from a few pages:
In the meantime, The Origin of Species had appeared and Darwinism had conquered the collective psyche. As George Bernard Shaw later wrote, "Everyone who had a mind to change changed it."
The Lamarckian nature of Magnus's model now emerged as just one of the gaping holes in his Emmental of a theory.
People find names for the things they feel the need to talk about.
In a large society of strangers there will be many more occasions where elaborate information has to be conveyed without reliance on shared background and knowledge.
"What fetters the mind and benumbs the spirit… the dogged acceptance of absolutes." [Edward Sapir]
Languages differ essentially in what they must convey and not in what they may convey.
The Matses… have to be master epistemologists. There are separate verbal forms depending on whether you are reporting direct experience (you saw someone passing by with your own eyes), something inferred from evidence (you saw footprints on the sand), conjecture (people always pass by at that time of day), or hearsay (your neighbour told you he had seen someone passing by).
Guugu Yimithirr… does not make any use of egocentric coordinates [eg left, right] at all! … Whenever we would use the egocentric system, the Guugu Yimithirr use the four cardinal directions: gungga (North), jiba (South), guwa (West), and naga (East. … They maintain their orientation with respect to the fixed cardinal directions at all times. Regardless of visibility conditions, regardless of whether they are in a thick forest or on an open plain, whether indoors or outside, whether stationary or moving, they have a spot-on sense of direction.
John Haviland estimates that as many as one word in ten (!) in a normal Guugu Yimithirr conversation is north, south, west, or east, often accompanied by very precise hand gestures. … Everyday communication in Guuga Yimithirr provides the most intense drilling in geographic orientation from the earliest imaginable age.
The conventional predictions are that within two or three generations at least half the world's six thousand languages will have disappeared.
Each step in this chain was natural and made perfect sense in its own local context. But the end result seems entirely arbitrary.
The worst thing about this loss of transparency is that it is a self propelling process: the less consistent the system becomes, the easier it is to mess it up even further.
Until the eleventh century, English had a full-blown three-gender system just like German.
Two bowls
My I Ching for today was no 41 Reduction. Part of it reads:
Two bowls can be used for presentation. The two bowls must be timed appropriately: reduction of firmness and increase of flexibility have their times...
I was struck by how apt that is for software development. A reduction of firmness mirrors abstraction. But abstractions must be grounded by experience. By reality. It conjurs an image of periodically moving the work back and forth - between firmness and flexibility. It's also reminiscent of how code and its tests help to shape and form each other.
Models and abstraction
My friend Kevlin Henney recently did a in-the-brain-of talk for
SkillsMatter. He talked about Models and Modelling. Here's part of Kevlin's definition of a model
A model is an abstraction ... for a purpose.
There is an intimate relationship between a model and its abstraction.
I think it was Andrew Koenig who said
Abstraction is selective ignorance.
The word 'ignorance' might mislead you here. It doesn't mean ignorance in the sense of an ignorant person - a person who should know something important but doesn't. It means ignorance in the sense of not being aware of something unimportant. The word 'selective' relates to the word 'purpose'. A clear purpose helps you decide what's important and what's not. Things that aren't important can then be omitted. Modelling is primarily an act of omission. The less clarity of purpose in your mind, the less omission is likely to take place, the more the model will try to say, without clarity.
The London underground tube map as a nice example of modelling. The usefulness of the tube map is directly related to the massive amount of omission in it (or should that be not in it?)
- it's not scaled - 5cm on the map does not relate to 5 miles on the ground.
- it's not proportional - stations 8cm apart on the map are not twice as far apart (on the ground) as stations 4cm on the map.
- it's not relative - stations north/south of each other on the map are not necessarily north/south of each other on the ground.
- it doesn't tell you street names.
- it doesn't tell you where the bus stops are.
- the turns in the River Thames are not all 90 degrees or 45 degrees.
- the little nicks on one side of the line don't tell you if the platform will be on the left or right.
- ...
Things that a model does say should be said consistently. For example, the station names are all written horizontally, in the same font and style. If some were horizontal, some vertical, some slanted, some in black, some in blue, some large, some small, we might start wonder if the differences were significant. The model would be leading us astray. If the tube map was trying to be geographically accurate some parts of the map would get very squashed. The squashed stations might get a smaller font. And some might be angled into the only available space.
There is also at least one act of addition in the tube map. The lines have different colours. The circle line is yellow for example. Kevlin promises me that the steel tracks on the circle line are not yellow! But then again, perhaps the colour refers to the trains running on the track. Are the trains on the circle line noticeably yellow? If so, which had the yellowness first, the tube lines on the map or the actual trains? Or, over time, have they influenced each other?
It's easy to ignore just how much effort went into figuring out what to remove from the tube map. Wikipedia has a nice entry on how the tube map has changed over time. I remember once reading how Harry Beck spent years working on an early version of the tube map which his supervisor rejected because it needed to contain less.
A powerful abstraction helps you create a model of clarity, but the clarity can easily seduce you - you're not aware of all that has been omitted, and the effort needed to omit it. When you look at the tube map you just see the tube map.
The Humble Programmer
by Edsger W. Dijkstra, 1972 ACM Turing Award Lecture:
We all know that the only mental tool by means of which a very finite piece of reasoning can cover a myriad of cases is called "abstraction"; as a result the effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer. In this connection it might be worthwhile to point out that the purpose of abstraction is not to be vague, but to create a new semantic field in which one can be absolutely precise.
The tools we are trying to use and the language or notation we are using to express or record our thought are the major factors determining that we can think or express them at all!
The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility.
Programming will remain very difficult.
The best way to learn to live with our limitation is to know them.
In computer programming our basic building block has an associated time grain of less than a microsecond, but a program may take hours of computation time. I do not know of any other technology covering a ratio of 10^10 or more.
This challenge, viz. the confrontation with the programming task, is so unique that this novel experience can teach us a lot about ourselves. It should deepen our understanding of the processes of design and creation; it should give us better control over the task of organizing our thoughts. If it did not do so, to my taste we should not deserve the computer at all!
Slack
is the title of an excellent book by Tom DeMarco (one of the authors of
Peopleware). As usual I'm going to quote from a few pages:
The more efficient you get, the harder it is to change.
The survival tactic that Harry and others like him hit upon when their buffers begin to empty is to slow down.
The principal resource needed for invention is slack.
The purpose of the schedule was planning, not goal-setting.
You need to understand that management is utterly essential. It is.
Quality takes time.
You're efficient when you do something with minimum waste. And you're effective when you're doing the right thing.
When you're not safe, you feel afraid. And fear can inhibit change.
I see one pattern common to all winners. They acquire trust by giving trust.
We tend to resist learning things that really matter.
Most of us don't learn well from abstraction. We learn from example.
Training - practice by doing a new task much more slowly than an expert would do it.
Managing your risks requires that you go at some slower speed.
Systems Thinking
Human beings have evolved a very strong association that cause and effect are simple and linear; that cause and effect are local in space and local in time. We are not good at seeing the non-linear effects of our actions; the effects over distance and especially over time. Here's something a friend emailed me:
Last week I found that the guys had not checked in a big chunk of library code. It was in the place where common tool-chains were getting invoked, so no one noticed. They changed something and I was seeing sudden breakages even with older versions of the code which was working. They didn't even bother to put it into the version control. I just couldn't believe, people can work this way. I immediately put it into the svn.
Here's what I replied...
Why do you think they hadn't put it in svn? Because at some level they associate the pain of putting it into svn as being greater than the pain of not putting it into svn plus living in a messy kitchen! And who can blame them when helpful people like yourself come along and clean up their mess! They are probably hardly even consciously aware that have the option of putting it into svn themselves. If they haven't put it into svn for N weeks then that is the behaviour that has become their norm. And things that don't change are not seen or thought about. That's abstraction! You are part of the vicious cycle! You are helping to perpetuate their behaviour. You are part of the system!
A more congruent approach would have been to sit next to them and talk them through what they needed to do to tidy up the mess. You absolutely mustn't do any part of it for them. Only if they actually do it themselves will they have a chance of not repeating the situation. They have to type the keystrokes at the keyboard. Remember, it is trivial for you to do it, but for them it will mean working through some level of associated pain. That is not their "fault". They are simply, in this case, less experienced than you. You simply sit next to them and calmly, patiently, ever so patiently, talk them through it in a non threatening, non blaming way.
Abstraction
I came across this astonishing illusion the other day (via a Jerry Weinberg twitter). The blue spirals and the green spirals are the same colour. Really. Jerry said he'd looked at the pixels in Photoshop to confirm it. It's not that I don't trust Jerry but I looked too (in Gimp) because it's just so counter-intuitive. It is true. It's amazing isn't it.
The way we perceive the world is based partly on what information it's sending us but mainly on how we interpret that information. If we consciously saw everything we just couldn't cope with the information overload so our minds filter it for us. That's abstraction. We don't see what's there and we see what's not there.
10,000+ warnings is people problem
When you read "there is always a problem and it's always a people problem" it's easy to get the wrong idea. Technical problems and people problems are almost always deeply intertwined.
For example, suppose you're part of a team that has let warnings accumulate one on top the other over many months until they now number 10,000+. Aside from the obvious technical problem of having 10,000+ warnings the team has a much deeper people problem.
Ask yourself the question - why does the team continue to live with the pain of 10,000+ warnings? Their answer is "that's how its always been". Sure the number creeps ever upward, but what does that matter when they're up to 10,000+? The team have had so many warnings for so long that they no longer even think about them! That's abstraction!
And given that the team don't even see 10,000+ warnings as a problem will they be motivated to get rid of them? Unlikely. They've lived with 10,000+ warnings for so long they've become comfortable with the discomfort they cause! Of course they claim they're not in discomfort. Abstraction again!
They are caught in a vicious circle of blindness and numbness. To solve this people problem the first step is to somehow get the team to see and feel the pain 10,000+ warnings are causing. That will be difficult. Then you'll have to get individuals to change their behaviour. That will be difficult too.
Oh, and you also have a smaller problem - 10,000+ warnings.
Subscribe to:
Posts (Atom)