Reading · Andreas Hohmann

This site took too long to answer. You can still read it on the original site — the toolbar below keeps your place in the directory.
Envoy is an incredibly flexible proxy server. Almost every aspect of the processing chain from the listeners all the way to the management of the upstream connections is configurable. This is even more impressive given that the configuration can be changed on the fly (by reloading a file or via the xDS API) without restarting the server. Moreover, we are not restricted to Envoy's built-in…
How does Envoy pull this off given that C++ is not known as a particular "dynamic" language? This post tries to shed some light on Envoy's factory registries that form the foundation of this mechanism.</p>
Let's start with the caller side. In various places, Envoy has to construct objects implementing some interface using a given configuration. Here is an example from the code handling the upstream connections.</p>
 1</span>Network</span>::</span>UpstreamTransportSocketFactoryPtr</span> createTransportSocketFactory</span>(</span></span>
 2</span>    const</span> envoy</span>::</span>config</span>::</span>cluster</span>::</span>v3</span>::</span>Cluster</span>&</span> config</span>,</span></span>
 3</span>    Server</span>::</span>Configuration</span>::</span>TransportSocketFactoryContext</span>&</span> factory_context</span>)</span> {</span></span>
 4</span>  auto</span> transport_socket </span>=</span> config</span>.</span>transport_socket</span>(</span>)</span>;</span></span>
 5</span>  ...</span></span>
 6</span>  auto</span>&</span> config_factory </span>=</span> Config</span>::</span>Utility</span>::</span>getAndCheckFactory</span><</span></span>
 7</span>      Server</span>::</span>Configuration</span>::</span>UpstreamTransportSocketConfigFactory</span>></span>(</span>transport_socket</span>)</span>;</span></span>
 8</span>  ProtobufTypes</span>::</span>MessagePtr message </span>=</span> Config</span>::</span>Utility</span>::</span>translateToFactoryConfig</span>(</span></span>
 9</span>      transport_socket</span>,</span> factory_context</span>.</span>messageValidationVisitor</span>(</span>)</span>,</span> config_factory</span>)</span>;</span></span>
10</span>  return</span> config_factory</span>.</span>createTransportSocketFactory</span>(</span>*</span>message</span>,</span> factory_context</span>)</span>;</span></span>
11</span>}</span></span></code></pre>
The createTransportSocketFactory</a> method creates
a socket factory based on the cluster configuration. The key is the
getAndCheckFactory</a> method that takes the
transport_socket</code> protobuf configuration and returns a factory object of type
UpstreamTransportSocketConfigFactory</a>.
TransportSocket</a> is one of Envoy's fully dynamic
configuration objects using a
TransportSocket.typed_config</a>, a protobuf
Any</a> message. The getAndCheckFactory</a> method calls
getFactoryByType</a> with the typed config. If this does not
result in a factory, the method tries a lookup by name instead.</p>
 1</span>template</span> <</span>class</span> Factory</span>,</span> class</span> ProtoMessage</span>></span></span>
 2</span>static</span> Factory</span>*</span> getAndCheckFactory</span>(</span>const</span> ProtoMessage</span>&</span> message</span>,</span> bool</span> is_optional</span>)</span> {</span></span>
 3</span>  Factory</span>*</span> factory </span>=</span> Utility</span>::</span>getFactoryByType</span><</span>Factory</span>></span>(</span>message</span>.</span>typed_config</span>(</span>)</span>)</span>;</span></span>
 4</span>  ...</span></span>
 5</span>  if</span> (</span>factory </span>!=</span> nullptr</span>)</span> {</span></span>
 6</span>    return</span> factory</span>;</span></span>
 7</span>  }</span></span>
 8</span>  return</span> Utility</span>::</span>getAndCheckFactoryByName</span><</span>Factory</span>></span>(</span>message</span>.</span>name</span>(</span>)</span>,</span> is_optional</span>)</span>;</span></span>
 9</span>}</span></span></code></pre>
Now we have to understand the lookup of a factory by type (typed config) and by name.
Let's start with the lookup by name.</p>
 1</span>template</span> <</span>class</span> Factory</span>></span></span>
 2</span>static</span> Factory</span>*</span> getAndCheckFactoryByName</span>(</span>const</span> std</span>::</span>string</span>&</span> name</span>,</span> bool</span> is_optional</span>)</span> {</span></span>
 3</span>  ...</span></span>
 4</span>  Factory</span>*</span> factory </span>=</span> Registry</span>::</span>FactoryRegistry</span><</span>Factory</span>></span>::</span>getFactory</span>(</span>name</span>)</span>;</span></span>
 5</span>  if</span> (</span>factory </span>==</span> nullptr</span> &&</span> !</span>is_optional</span>)</span> {</span></span>
 6</span>    ExceptionUtil</span>::</span>throwEnvoyException</span>(</span></span>
 7</span>        fmt</span>::</span>format</span>(</span>"</span>Didn't find a registered implementation for name: '{}'</span>"</span>,</span> name</span>)</span>)</span>;</span></span>
 8</span>  }</span></span>
 9</span>  return</span> factory</span>;</span></span>
10</span>}</span></span></code></pre>
getAndCheckFactoryByName</code></a> calls a static method of
a "factory registry" that is parameterized by the type of the factories we are
interested. There must be a static registry object per Factory</code> type that
contains all the factories that implement this type, and Envoy must somehow
register all the available factories in this registry. The
FactoryRegistry</a> is indeed just a collection of static
methods, that is, a singleton similar to a Scala or Kotlin object</code>.</p>
Static object in C++ are notorious for there unpredictable construction and
destruction order known as the "static initialization order
fiasco"</a> or SIOF for short.
That's why many C++ style guides disallow static objects altogether. How does
Envoy get around these problems? A "registry" that keeps factory objects in
some map is definitely a rich object with non-trivial constructor and
destructor.</p>
The first step is to apply "initialization on first
use"</a> and place the static variable in of a method
instead of the top-level. The variable will get initialized when the method is
called for the first time.</p>
class</span> Foo</span> {</span></span>
public</span>:</span></span>
  static</span> Foo</span>&</span> foo</span>(</span>)</span> {</span></span>
    static</span> Foo</span>*</span> foo </span>=</span> new</span> Foo</span>(</span>)</span>;</span></span>
    return</span> *</span>foo</span>;</span></span>
  }</span></span>
}</span>;</span></span></code></pre>
This technique is also known as Meyer's Singleton</a>. While
using static local objects solves the initialization problem, they may still
cause trouble during destruction because of dependencies between these static
objects. That's why the "initialization on first
use"</a> pattern recommends using pointers, allocating
the objects on the heap, and never deallocating them. While this theoretically
creates a memory leak, the objects live for the duration of the program, and
the operating system will release the memory at the end of the process. The
destructors will never be called, however, so that those objects must not have
destructors doing anything meaningful besides freeing memory.</p>
FactoryRegistry</a> defines the static map of factories (per
factory type parameter Base</code>) in the factories</a>
method. Note how the definition as a template gives us a new registry for a
Base</code> factory type by just calling one of the static methods.</p>
 1</span>template</span> <</span>class</span> Base</span>></span></span>
 2</span>class</span> FactoryRegistry</span> ... </span>{</span></span>
 3</span>public</span>:</span></span>
 4</span>  ...</span></span>
 5</span>  static</span> absl</span>::</span>flat_hash_map</span><</span>std</span>::</span>string</span>,</span> Base</span>*</span>></span>&</span> factories</span>(</span>)</span> {</span></span>
 6</span>    static</span> auto</span>*</span> factories </span>=</span> new</span> absl</span>::</span>flat_hash_map</span><</span>std</span>::</span>string</span>,</span> Base</span>*</span>></span>;</span></span>
 7</span>    return</span> *</span>factories</span>;</span></span>
 8</span>  }</span></span>
 9</span>  ...</span></span>
10</span>  static</span> Base</span>*</span> getFactory</span>(</span>absl</span>::</span>string_view</span> name</span>)</span> {</span></span>
11</span>    auto</span> it </span>=</span> factories</span>(</span>)</span>.</span>find</span>(</span>name</span>)</span>;</span></span>
12</span>    if</span> (</span>it </span>==</span> factories</span>(</span>)</span>.</span>end</span>(</span>)</span>)</span> {</span></span>
13</span>      return</span> nullptr</span>;</span></span>
14</span>    }</span></span>
15</span>    return</span> it</span>-></span>second</span>;</span></span>
16</span>  }</span></span>
17</span>  ...</span></span>
18</span>}</span>;</span></span></code></pre>
Now that we know where the factories are kept and how the lookup by name is
performed, let's figure out how the factories are registered. The
registerFactory</a> method stores the pointer to the given
factory in the map under the given name.</p>
 1</span>template</span> <</span>class</span> Base</span>></span></span>
 2</span>class</span> FactoryRegistry</span> ... </span>{</span></span>
 3</span>public</span>:</span></span>
 4</span>  ...</span></span>
 5</span>  static</span> void</span> registerFactory</span>(</span>Base</span>&</span> factory</span>,</span> absl</span>::</span>string_view</span> name</span>,</span> ...</span>)</span> {</span></span>
 6</span>    auto</span> result </span>=</span> factories</span>(</span>)</span>.</span>emplace</span>(</span>std</span>::</span>make_pair</span>(</span>name</span>,</span> &</span>factory</span>)</span>)</span>;</span></span>
 7</span>    if</span> (</span>!</span>result</span>.</span>second</span>)</span> {</span></span>
 8</span>      ExceptionUtil</span>::</span>throwEnvoyException</span>(</span></span>
 9</span>          fmt</span>::</span>format</span>(</span>"</span>Double registration for name: '{}'</span>"</span>,</span> factory</span>.</span>name</span>(</span>)</span>)</span>)</span>;</span></span>
10</span>    }</span></span>
11</span>    ...</span></span>
12</span>  }</span></span>
13</span>  ...</span></span>
14</span>}</span>;</span></span></code></pre>
Note that this method is not threadsafe. We have to make sure that all factories are
registered before they are used from other threads. Envoy provides a coupld of helper
classes and macros to encourage this. The helper class RegisterFactory</a>
captures the registration of a single factory. The factory is constructed as a field
(using the factory's default constructor) and registered in in the constructor.</p>
 1</span>template</span> <</span>class</span> T</span>,</span> class</span> Base</span>></span> class</span> RegisterFactory</span> {</span></span>
 2</span>public</span>:</span></span>
 3</span>  RegisterFactory</span>(</span>)</span> {</span></span>
 4</span>    ASSERT</span>(</span>!</span>instance_</span>.</span>name</span>(</span>)</span>.</span>empty</span>(</span>)</span>)</span>;</span></span>
 5</span>    FactoryRegistry</span><</span>Base</span>></span>::</span>registerFactory</span>(</span>instance_</span>,</span> instance_</span>.</span>name</span>(</span>)</span>)</span>;</span></span>
 6</span>  }</span></span>
 7</span>private</span>:</span></span>
 8</span>  T instance_</span>{</span>}</span>;</span></span>
 9</span>}</span>;</span></span></code></pre>
The factory class T</code> is implicitly assumed to derive from Base</code>, to be
default-constructible, and to have a name</code> method returning a string. With
C++20 we can require this concept explicitly:</p>
 1</span>template</span><</span>typename</span> F</span>,</span> typename</span> Base</span>></span></span>
 2</span>concept</span> Factory </span>=</span> std</span>::</span>default_initializable</span><</span>F</span>></span></span>
 3</span>  &&</span> std</span>::</span>derived_from</span><</span>F</span>,</span> Base</span>></span></span>
 4</span>  &&</span> requires</span> (</span>const</span> F</span>&</span> factory</span>)</span> {</span></span>
 5</span>    {</span> factory</span>.</span>name</span>(</span>)</span> }</span> -</span>></span> std</span>::</span>convertible_to</span><</span>std</span>::</span>string</span>></span>;</span></span>
 6</span>  }</span>;</span></span>
 7</span></span>
 8</span>template</span><</span>typename</span> F</span>,</span> typename</span> Base</span>></span> requires Factory</span><</span>F</span>,</span> Base</span>></span></span>
 9</span>class</span> RegisterFactory</span> {</span></span>
10</span>public</span>:</span></span>
11</span>  RegisterFactory</span>(</span>)</span> {</span></span>
12</span>    FactoryRegistry</span><</span>Base</span>></span>::</span>registerFactory</span>(</span>factory_</span>,</span> factory_</span>.</span>name</span>(</span>)</span>)</span>;</span></span>
13</span>  }</span></span>
14</span>private</span>:</span></span>
15</span>  F factory_</span>{</span>}</span>;</span></span>
16</span>}</span>;</span></span></code></pre>
Now we have a registration class, but we still need to instantiate this class
for a concrete factory type. To this end, Envoy uses the static local variable
trick once more. The REGISTER_FACTORY</a> macro defines a
top-level forceRegister</code> function containing the static pointer to the
RegisterFactory</code> object:</p>
1</span>#</span>define</span> REGISTER_FACTORY</span>(</span>FACTORY</span>,</span> BASE</span>)</span>                              \</span></span>
2</span>  ABSL_ATTRIBUTE_UNUSED</span> void</span> forceRegister</span>##FACTORY</span>(</span>)</span> {</span>              \</span></span>
3</span>    ABSL_ATTRIBUTE_UNUSED</span> static</span> auto</span> registered </span>=</span>                   \</span></span>
4</span>        new</span> Envoy</span>::</span>Registry</span>::</span>RegisterFactory</span><</span>FACTORY</span>,</span> BASE</span>></span>(</span>)</span>;</span>       \</span></span>
5</span>  }</span></span></code></pre>
Here is the macro call for the DecompressorFilterFactory</a>
as an example:</p>
REGISTER_FACTORY</span>(</span>DecompressorFilterFactory</span>,</span> Server</span>::</span>Configuration</span>::</span>NamedHttpFilterConfigFactory</span>)</span>;</span></span></code></pre>
The registration functions are then called explicitly:</p>
1</span>void</span> ExtensionRegistry</span>::</span>registerFactories</span>(</span>)</span> {</span></span>
2</span>  Common</span>::</span>Http</span>::</span>MatchDelegate</span>::</span>Factory</span>::</span>forceRegisterSkipActionFactory</span>(</span>)</span>;</span></span>
3</span>  Common</span>::</span>Http</span>::</span>MatchDelegate</span>::</span>forceRegisterMatchDelegateConfig</span>(</span>)</span>;</span></span>
4</span>  ...</span></span>
5</span>  Extensions</span>::</span>HttpFilters</span>::</span>Decompressor</span>::</span>forceRegisterDecompressorFilterFactory</span>(</span>)</span>;</span></span>
6</span>  ...</span></span>
7</span>}</span></span></code></pre>
Note that the definition of the macro depends on the
ENVOY_STATIC_EXTENSION_REGISTRATION</a> flag. If set,
the factory registration is a plain static object and the forceRegister</code>
function is empty.</p>
We followed the factory lookup by name all the way to the registration. This
leaves the lookup by configuration (protobuf) type that we noticed at the
very beginning in the getAndCheckFactory</a> method</p>
 1</span>template</span> <</span>class</span> Factory</span>,</span> class</span> ProtoMessage</span>></span></span>
 2</span>static</span> Factory</span>*</span> getAndCheckFactory</span>(</span>const</span> ProtoMessage</span>&</span> message</span>,</span> bool</span> is_optional</span>)</span> {</span></span>
 3</span>  Factory</span>*</span> factory </span>=</span> Utility</span>::</span>getFactoryByType</span><</span>Factory</span>></span>(</span>message</span>.</span>typed_config</span>(</span>)</span>)</span>;</span></span>
 4</span>  ...</span></span>
 5</span>  if</span> (</span>factory </span>!=</span> nullptr</span>)</span> {</span></span>
 6</span>    return</span> factory</span>;</span></span>
 7</span>  }</span></span>
 8</span>  return</span> Utility</span>::</span>getAndCheckFactoryByName</span><</span>Factory</span>></span>(</span>message</span>.</span>name</span>(</span>)</span>,</span> is_optional</span>)</span>;</span></span>
 9</span>}</span></span></code></pre>
The getFactoryByType</a> method delegates to the static
FactoryRegistry</a> method of the same name. This
getFactoryByType</a> follows the same patterns
as the getFactory</a> method:</p>
1</span>static</span> Base</span>*</span> getFactoryByType</span>(</span>absl</span>::</span>string_view</span> type</span>)</span> {</span></span>
2</span>  auto</span> it </span>=</span> factoriesByType</span>(</span>)</span>.</span>find</span>(</span>type</span>)</span>;</span></span>
3</span>  if</span> (</span>it </span>==</span> factoriesByType</span>(</span>)</span>.</span>end</span>(</span>)</span>)</span> {</span></span>
4</span>    return</span> nullptr</span>;</span></span>
5</span>  }</span></span>
6</span>  return</span> it</span>-></span>second</span>;</span></span>
7</span>}</span></span></code></pre>
The only difference is the factoriesByType</code> call instead of the factories</code>
call. The registration by type is not performed when a factory is registered.
Instead, the map from type name to factory is created lazily by the
buildFactoriesByType</a> method and stored
in yet another static pointer variable in
factoriesByType</a>. To be threadsafe, this
method has to be called once in the main thread after all factories have been
registered for a given factory type but before
getFactoryByType</a> is called from another
thread.</p>
The buildFactoriesByType</a> implicitly
assumes that the Base</code> factory interface has a configTypes</code> method returning
the set of type strings under which to register the factory.</p>
 1</span>static</span> std</span>::</span>unique_ptr</span><</span>absl</span>::</span>flat_hash_map</span><</span>std</span>::</span>string</span>,</span> Base</span>*</span>></span>></span> buildFactoriesByType</span>(</span>)</span> {</span></span>
 2</span>  auto</span> mapping </span>=</span> std</span>::</span>make_unique</span><</span>absl</span>::</span>flat_hash_map</span><</span>std</span>::</span>string</span>,</span> Base</span>*</span>></span>></span>(</span>)</span>;</span></span>
 3</span></span>
 4</span>  for</span> (</span>const</span> auto</span>&</span> [</span>factory_name</span>,</span> factory</span>]</span> :</span> factories</span>(</span>)</span>)</span> {</span></span>
 5</span>    ...</span></span>
 6</span>    for</span> (</span>const</span> auto</span>&</span> config_type </span>:</span> factory</span>-></span>configTypes</span>(</span>)</span>)</span> {</span></span>
 7</span>      ...</span></span>
 8</span>      mapping</span>-></span>emplace</span>(</span>std</span>::</span>make_pair</span>(</span>config_type</span>,</span> factory</span>)</span>)</span>;</span></span>
 9</span>    }</span></span>
10</span>  }</span></span>
11</span>  return</span> mapping</span>;</span></span>
12</span>}</span></span></code></pre>
As a C++20 concept, this would read:</p>
1</span>template</span><</span>typename</span> T</span>></span></span>
2</span>concept</span> FactoryBase </span>=</span> requires</span> (</span>const</span> T</span>&</span> factory</span>)</span> {</span></span>
3</span>  {</span> factory</span>.</span>configTypes</span>(</span>)</span> }</span> -</span>></span> std</span>::</span>convertible_to</span><</span>std</span>::</span>set</span><</span>std</span>::</span>string</span>>></span>;</span></span>
4</span>}</span>;</span></span></code></pre>
So, in the end both the name and the type names are defined by the factory
itself through the name</code> and configTypes</code> method. The name</code> method is called
on the concrete factory objects whereas the configTypes</code> method must exist in
the base factory interface. It is therefore a virtual method in the
UntypedFactory</a> that all factories derive from. The default
implementation returns an empty type name set. The name</code> method is also a
virtual method in this class, but does not strictly have to be in the base
factory interface.</p>
 1</span>class</span> UntypedFactory</span> {</span></span>
 2</span>public</span>:</span></span>
 3</span>  virtual</span> ~UntypedFactory</span>(</span>)</span> =</span> default</span>;</span></span>
 4</span></span>
 5</span>  virtual</span> std</span>::</span>string</span> name</span>(</span>)</span> const</span> PURE</span>;</span></span>
 6</span>  ...</span></span>
 7</span>  virtual</span> std</span>::</span>set</span><</span>std</span>::</span>string</span>></span> configTypes</span>(</span>)</span> {</span> return</span> {</span>}</span>;</span> }</span></span>
 8</span>}</span>;</span></span></code></pre>
This completes our little tour through the Envoy factory registration implementation.</p>
Update 2024-03-22: After writing this post I stumbled upon Abseil's
NoDestructor</a> class that solves the destruction order
problem of static objects by not running the destructor of the wrapped object.
In contrast to the static pointer to a heap-allocated object that is never
freed, NoDestructor</a> lets us keep the object in static
storage and save one pointer indirection:</p>
1</span>class</span> Foo</span> {</span></span>
2</span>public</span>:</span></span>
3</span>  static</span> Foo</span>&</span> foo</span>(</span>)</span> {</span></span>
4</span>    static</span> absl</span>::</span>NoDestructor</span><</span>Foo</span>></span> foo</span>;</span></span>
5</span>    return</span> *</span>foo</span>;</span></span>
6</span>  }</span></span>
7</span>}</span>;</span></span></code></pre>
In case of Envoy's FactoryRegistry</code>, we could wrap the static factory hash map in
a NoDestructor</a>:</p>
 1</span>template</span> <</span>class</span> Base</span>></span></span>
 2</span>class</span> FactoryRegistry</span> ... </span>{</span></span>
 3</span>public</span>:</span></span>
 4</span>  ...</span></span>
 5</span>  static</span> absl</span>::</span>flat_hash_map</span><</span>std</span>::</span>string</span>,</span> Base</span>*</span>></span>&</span> factories</span>(</span>)</span> {</span></span>
 6</span>    static</span> absl</span>::</span>NoDestructor</span><</span>absl</span>::</span>flat_hash_map</span><</span>std</span>::</span>string</span>,</span> Base</span>*</span>>></span> factories</span>;</span></span>
 7</span>    return</span> *</span>factories</span>;</span></span>
 8</span>  }</span></span>
 9</span>  ...</span></span>
10</span>}</span>;</span></span></code></pre>
How does NoDestructor</a> work? It's mainly a wrapper around
placement new, constructing the wrapped object in a plain char array and never
calling the destructor. Fortunately, template argument packs and perfect
forwarding are tailor-made for such a wrapper.</p>
 1</span>template</span> <</span>typename</span> T</span>></span></span>
 2</span>class</span> PlacementImpl</span> {</span></span>
 3</span>public</span>:</span></span>
 4</span>  template</span> <</span>typename</span>...</span> Args</span>></span></span>
 5</span>  explicit</span> PlacementImpl</span>(</span>Args</span>&</span>&</span>...</span> args</span>)</span> {</span></span>
 6</span>    new</span> (</span>&</span>space_</span>)</span> T</span>(</span>std</span>::</span>forward</span><</span>Args</span>></span>(</span>args</span>)</span>...</span>)</span>;</span></span>
 7</span>  }</span></span>
 8</span>  absl</span>::</span>Nonnull</span><</span>const</span> T</span>*</span>></span> get</span>(</span>)</span> const</span> {</span></span>
 9</span>    return</span> Launder</span>(</span>reinterpret_cast</span><</span>const</span> T</span>*</span>></span>(</span>&</span>space_</span>)</span>)</span>;</span></span>
10</span>  }</span></span>
11</span>  ...</span></span>
12</span>  absl</span>::</span>Nonnull</span><</span>T</span>*</span>></span> get</span>(</span>)</span> {</span> return</span> Launder</span>(</span>reinterpret_cast</span><</span>T</span>*</span>></span>(</span>&</span>space_</span>)</span>)</span>;</span> }</span></span>
13</span>private</span>:</span></span>
14</span>  alignas</span>(</span>T</span>)</span> unsigned</span> char</span> space_</span>[</span>sizeof</span>(</span>T</span>)</span>]</span>;</span></span>
15</span>}</span>;</span></span></code></pre>
The actual NoDestructor</a> template contains this
implementation and adds the operators that make the
NoDestructor</a> wrapper look like a pointer.</p>
That's the best solution for static objects in c++ that I'm aware of (besides
not using static objects to begin with, see, for example LLVM's
rule</a>).</p>
Envoy factory registry

Comments
