Transfer Learning for Humans: Why Engineering Leaders Must Stay Technical

As an individual contributor, I used to think "Those who can, do; those who can't become managers"...or consultants. Cynical of me, yes, but just being honest. The moment you stop being curious about engineering, you stop being effective at leading the people who practice it daily.

I haven't written production code professionally in years, but I still code in my spare time. I expect continuous learning from my team, so how do I hold up my end of that bargain?

In my line of work, I'm often assumed to be LESS hands-on. There's this notion that getting into the weeds makes you less strategic, that knowing implementation details clouds your judgment.

I call bullsh1t!

While I certainly don't trust myself to write production code anymore, I have yet to find a better way to grasp certain concepts than with hands-on experimentation. As an engineer when I have to take things on faith, I struggle. And if I struggle to understand what my teams are building, how can I effectively support them?

You'll hear about "transfer learning" in the AI space—it's when a model trained on one task applies that knowledge to a related but different task. Humans do this naturally. The mathematical concepts I learned decades ago didn't disappear when I became a manager; they became the foundation for understanding today's technologies.

Here's a concrete example from the current AI hype cycle: “Embeddings” (this post has been sitting in my drafts so maybe not as fresh as it was a year ago).

ChatGPT tells us "An embedding is a vector representation of data in a high-dimensional space. They serve as a bridge between the raw, often non-numeric data and the numerical algorithms that process them."

Said simply: an embedding is a mathematical way to represent any kind of data—words, sentences, emotions, images, sounds, concepts—as numbers that computers can work with.

But understanding the definition isn't the same as truly grasping the concept. Let’s dig deeper.

A long time ago, before embeddings were even a thing, I built a photo mosaic generator that creates images from thousands of other images. This project is a great example of a concept, in this case Embeddings, being used outside its context.

Now enjoy this Rainbow…

Here's how I built it:

1. Source Material: I wrote a web crawler and let it cook for a few hours. The result; hundreds of thousands of random photos.

2. Target Division: Took my target image (the rainbow) and divided it into tiles.

3. Feature Extraction: For each source image in (1) I transformed it from a 2D image into a 1D vector using the Discrete Cosine Transform. This index became the raw material I’d use to create the final mosaic image.

4. Matching: For each tile in the target, I computed the DCT of that tile, then found the source photo with the closest vector match using Root Mean Square comparison.

5. Assembly: Repeated until every part of the rainbow was matched with an appropriate photo.

This process—extracting features, creating vector representations, and finding similarities—is exactly what embeddings do in modern AI systems.

The technical details matter here. I extracted "features" like contrast, color histograms, and areas of high/low detail. These became the dimensions of my vector space. The Discrete Cosine Transform (similar to the Fast Fourier Transform you might know from audio processing) broke down each image into its essential mathematical components. DCT is great for identifying areas of high-contrast in an image and to the human eye contrast is the key piece in making the new image appear like the original.

In today's ML/AI world, the model does this comparison automatically e.g. Inference, but the fundamental concept remains: transform complex data into vectors, then use mathematical methods to find relationships and similarities.

Cool so you can draw a Rainbow or anything from a source image. Why does that matter? Understanding embeddings at this level isn't just intellectual curiosity—it's strategic advantage. When my team talks about:

• Recommendation systems that understand user preferences across multiple dimensions.

• Semantic search that grasps meaning beyond keywords.

• Translation services that map relationships between languages.

• Computer vision that recognizes patterns and objects.

If I understand the foundations I can ask better questions, spot potential issues, and make more informed decisions.

The Gemba walk is a Japanese practice of going to the actual place where work happens to understand it deeply. For engineering leaders, the "actual place" isn't the office; it's the development experience itself.

I'm not trying to be the strongest individual contributor on my team, that’s not my job. I’m looking for balance and aiming to maintain enough technical depth to:

• Ask informed questions and understand trade-offs when teams propose solutions.

• Spot patterns across different technologies and domains.

• Translate between business needs and technical constraints.

• Credibly advocate for technical investments to non-technical stakeholders.

The beautiful thing about mathematics and computer science fundamentals is their longevity. The matrix/vector math I learned in high school still applies to modern embeddings. The signal processing concepts from my early career transfer directly to audio/video ML applications. The statistical methods I picked up along the way inform how I think about A/B testing and data analysis.

Technology stacks come and go, but the underlying concepts persist. An understanding of these fundamentals becomes more valuable over time, not less.

If it’s not already obvious this approach has concrete advantages:

• Better Hiring: I can evaluate candidates more effectively when I understand what they'll actually be doing.

• Improved Planning: Technical complexity estimates become more accurate when you grasp the underlying challenges.

• Enhanced Communication: I can translate between technical teams and business stakeholders.

• Risk Assessment: Potential architectural problems become visible earlier when you understand the foundations.

• Team Credibility: Engineers respect leaders who understand their craft, even if those leaders aren't writing production code.

When did you last write code? When did you last dig into a new technology because you were curious? When did you last experience the developer workflow your teams navigate daily?

The excuse that you're "too strategic" for implementation details is exactly that—an excuse. Strategy without implementation understanding is just expensive guessing.

You don't need to become the strongest coder on your team. You don't need to commit to production, but you do need to stay curious, stay learning, and stay close enough to understand what you're asking your teams to accomplish.

Pick a side project. Learn a new language. Build something small but complete. Experience the friction your developers feel. Understand the abstractions they work with. Feel the satisfaction of solving a tricky problem with elegant code.

Your teams will notice. Your decision-making will improve. Your strategic thinking will be grounded in reality rather than Powerpoints and sales pitches.

There’s a lot of talk about Vibe Coding and some of it derogatory but we're all just trying to solve problems with technology. The moment we stop understanding how that technology actually works is the moment we stop being effective leaders in this field.

The best engineering leaders I know aren't the ones who've transcended the technical details—they're the ones who've never stopped being fascinated by them.

Have you found ways to stay technical while leading teams? What concepts from your past have transferred to new technologies in unexpected ways? Share your stories—I'd love to hear how other leaders navigate this balance.