Team Communication Across Disciplines

The question “What strategies help you handle complex or impossible test designs?” sparked interesting discussions among developers at Experimentation Elite (Dec 2025).

We discuss it in this video, but here’s a summary of insights:

Developers are typically excluded from the planning stages of experimentation. Product managers and designers, therefore, create test designs without technical consultation. And this results in those “impossible” test requirements that are technically unfeasible or difficult to implement

The problem is not just communication across different skill sets; it’s a lack of understanding of those skills. People from different roles don’t understand constraints/possibilities in other areas. For instance, some product managers lack a deep understanding of design and development, designers lack the required technical knowledge, etc.

Developers closest to the code understand what’s actually feasible to achieve with a test variant. They can identify solutions others might miss, so excluding their early input can lead to unrealistic or inefficient solutions, such as tests that take a long time to develop.

It’s not just developers either. Experiments can launch without proper metrics, which can affect the insights gained. Risky tests, in particular, need comprehensive measurement strategies. So, analysts should also be involved earlier in the process

What’s more, the larger the organisation, the more this collaboration problem is amplified. The communication problem can lead to longer gaps between design → development → analysis stages, where “Relevance bleed” can occur. This is where experiments lose information due to entropy.

In general, we support Lucia van den Brink’s excellent talk on the topic at Experimentation Elite. In that talk, she coined the term “the collaborative bonus”. This is the benefit you get from mixed skill sets being involved in experimentation.

Lucia also highlighted some data to support the thesis. Teams that collaborate early achieve better results overall and deliver a better experimentation program.

Early involvement of all three disciplines (product, design, development) improves ideation and prevents problems. Being flexible to minor design tweaks during the experimentation process can significantly reduce implementation time (as long as the updates are relevant to the hypotheses).