This page cannot be shown here. You can still read it on the original site — the toolbar below keeps your place in the directory.
Today, I wanted to create a basic locking mechanism in Python to prevent certain commands/functions from running concurrently. I have an script that runs inside of a CI/CD pipeline but it can also be triggered manually. The problem is that certain commands within the script shouldn't be executed in parallel and I would like to handle that gracefully. use case Imagine a Python script…
Today, I wanted to create a basic locking mechanism in Python to prevent certain commands/functions from running concurrently.</p>
I have an script that runs inside of a CI/CD pipeline but it can also be triggered manually. The problem is that certain commands within the script shouldn't be executed in parallel and I would like to handle that gracefully.</p>
use case</h2>
Imagine a Python script that manages the compilation and distribution of software builds with two main command modes: build</code> and deploy</code>.</p>
-
python3 script.py build</code></strong>: Compiles source code into executables and libraries, creating compilation artifacts.</p>
</li>
-
python3 script.py deploy</code></strong>: Tags, packages, and deploys compiled artifacts to an environment.</p>
</li>
</ol>
It's easy to notice that it would be a bad idea to run these commands in
parallel. If a deploy</code> operation starts while a build</code> is still processing,
the deployment might catch incomplete or outdated artifacts.</p>
There are many things that could go wrong in this scenario.</p>
first try: initial implementation</h2>
I thought that this would be pretty easy to implement, I reached the fcntl</a> documentation and came up with this:</p>
import</span> fcntl</span></span>
import</span> sys</span></span>
import</span> time</span></span>
from</span> contextlib</span> import</span> contextmanager</span></span>
from</span> pathlib</span> import</span> Path</span></span>
</span>
</span>
@</span>contextmanager</span></span>
def</span> lock_execution</span>(</span>lock_file_path</span>:</span> Path</span> =</span> Path</span>(</span>"</span>/tmp/.example.lock</span>"</span>)</span>)</span>:</span></span>
"""</span>Prevents parallel execution.</span>"""</span></span>
</span>
with</span> open</span>(</span>lock_file_path</span>,</span> "</span>w</span>"</span>)</span> as</span> f</span>:</span></span>
try</span>:</span></span>
fcntl</span>.</span>flock</span>(</span>f</span>,</span> fcntl</span>.</span>LOCK_EX</span> |</span> fcntl</span>.</span>LOCK_NB</span>)</span></span>
yield</span></span>
except</span> IOError</span>:</span></span>
print</span>(</span></span>
f</span>"</span>Another instance using the lock </span>'</span>{</span>lock_file_path</span>}</span>' is already running.</span>"</span>,</span></span>
file</span>=</span>sys</span>.</span>stderr</span>,</span></span>
)</span></span>
sys</span>.</span>exit</span>(</span>0</span>)</span></span>
finally</span>:</span></span>
fcntl</span>.</span>flock</span>(</span>f</span>,</span> fcntl</span>.</span>LOCK_UN</span>)</span></span>
</span>
</span>
def</span> main</span>(</span>)</span>:</span></span>
print</span>(</span>"</span>Running main ...</span>"</span>)</span></span>
with</span> lock_execution</span>(</span>)</span>:</span></span>
print</span>(</span>"</span>Lock acquired, running ...</span>"</span>)</span></span>
time</span>.</span>sleep</span>(</span>5</span>)</span></span>
print</span>(</span>"</span>Done.</span>"</span>)</span></span>
</span>
</span>
if</span> __name__</span> ==</span> "</span>__main__</span>"</span>:</span></span>
main</span>(</span>)</span></span></code></pre>
The idea is to wrap the code that shouldn't be parallelized with the context manager lock_execution</code> function. Now, when I try to run this script from two terminal windows I get the following:</p>
#</span> Terminal 1</span></span>
[</span>aorith@arcadia:</span>~</span>]</span> $ python3 /tmp/first.py</span></span>
Running</span> main</span> ...</span></span>
Lock</span> acquired,</span> running</span> ...</span></span>
</span>
#</span> Terminal 2</span></span>
[</span>aorith@arcadia:</span>~</span>]</span> $ python3 /tmp/first.py</span></span>
Running</span> main</span> ...</span></span>
Another</span> instance</span> using</span> the</span> lock</span> '</span>/tmp/.example.lock</span>'</span> is</span> already</span> running.</span></span>
</span>
