Python has become a go-to language for scientific and data-intensive work. Its clean syntax, strong community support, and rich set of libraries – like NumPy, SciPy, and TensorFlow – make it well-suited for research and quick experimentation. In the life sciences in particular, Python shows up in everything from lab automation to data pipelines and AI models for drug discovery. Still, despite its strengths, Python isn’t without its challenges – especially when it comes to running performance-heavy tasks in parallel. Many of these limitations stem from how Python handles multiprocessing and memory management. 

These issues were the focus of a presentation by Karol Bubała, a software developer at Holisticon Connect. Speaking at a Cambridge Tech Week event aimed at professionals in health and life sciences, he shared insights from real-world biotech projects and offered practical ways to work around Python’s known technical bottlenecks. 

Why python became a favourite in science 

Python didn’t become the standard in scientific work by chance. It’s packed with tools made specifically for researchers – whether it’s NumPy and SciPy for number crunching, RDKit for chemistry-related tasks, or TensorFlow for building machine learning models. This ecosystem lets teams move fast, experiment often, and build real solutions without reinventing the wheel. The language itself is easy to read and forgiving, which helps when you’re prototyping or writing small scripts to solve a specific problem. 

As Karol pointed out during his presentation, Python’s simplicity is what makes it so approachable – especially for scientists who aren’t professional programmers. He also noted that many of the day-to-day technical hurdles, like file handling or text parsing, are already solved thanks to Python’s standard tools. And when performance becomes critical, it’s possible to plug in low-level components written in C or C++ without giving up the benefits of Python’s simplicity. 

The GIL problem (and what to do about it) 

One of the main technical roadblocks in Python is something called the Global Interpreter Lock, or GIL. It’s a built-in feature that prevents more than one thread from running Python code at the same time, even if you’re working on a machine with multiple cores. While the GIL helps Python manage memory more safely, it also makes true parallel processing difficult – especially in tasks that rely on heavy computation. 

Karol addressed this directly in his talk, saying: “Python threads do not offer parallelism because of GIL. But it can be achieved using processes.” What this means in practice is that multithreading still works well for tasks that involve a lot of waiting – like reading from a database or calling an external API – but falls short when it comes to intensive operations such as protein folding or large-scale simulations. 

To get around this, many teams use multiprocessing instead. That allows real parallel execution by running tasks in separate processes. But it comes with its own trade-offs: more memory usage, slower startup, and more complicated data handling between processes. In the end, the best approach depends on what exactly you’re trying to do. 

Performance challenges in Python (especially with multiprocessing) 

Drawing from his work on life sciences projects, Karol outlined four key types of performance issues that often come up when using Python: multithreading, multiprocessing, and two common subtypes of multiprocessing issues – memory-bound tasks and serialization bottlenecks. 

Multithreading tends to work well when most of the program’s time is spent waiting—for instance, on file operations or network requests. As Karol put it, “It’s designed for input-output operations where most of the heavy lifting is done somewhere else – like a database or microservice.” It’s lightweight and easy to scale in those scenarios. 

Multiprocessing uses separate processes instead of threads, which allows for true parallelism and helps with CPU-heavy workloads. However, because each process has its own memory space, it comes with trade-offs: greater memory consumption, longer startup times, and the need for careful orchestration to prevent inefficiencies. Techniques like batching or using job queues can help balance the load between processes. 

Two frequent issues that arise under the multiprocessing model are memory-bound workloads and serialization problems. 

Memory-bound tasks happen when each process has to load a large amount of static data – such as a machine learning model, in-memory cache, or a large database snapshot. If 100 processes each load the same data, memory usage can balloon and quickly become a limiting factor. One practical approach is to isolate memory-heavy steps or split large models into smaller components that can scale independently. 

Serialization bottlenecks emerge when complex Python objects must be passed between processes. If objects are difficult or slow to serialize, the overhead of moving them between the main process and workers can cancel out the benefits of parallelism. In one of Karol’s projects, this was solved by offloading part of the logic to a database. In other cases, switching to binary formats or using lower-level C++ integrations may be the better solution. 

What to do when python’s not enough 

In some situations, the most practical move is to step outside of Python altogether. As flexible as the language is, it’s not always the best choice – especially when your project needs to scale or squeeze out every bit of performance. As Karol put it, “when everything else fails, try thinking outside of Python.” 

There are a few ways to do that. One is to switch to alternative Python runtimes like Jython or IronPython, which don’t suffer from the Global Interpreter Lock. That said, these options only work if the libraries you depend on are compatible. A more common approach is to rewrite the most performance-critical parts of the codebase in a faster language like C++ or Java, and then connect them to the main Python application. This lets you keep Python where it makes sense – like in orchestration or scripting – while offloading the heavy lifting to faster technologies. 

At the end of the day, it’s about using the right tool for the job. In life sciences projects, mixing Python with lower-level components often gives teams the flexibility they want without sacrificing performance. 

Conclusion 

Python continues to be a strong choice in scientific and biotech projects – for good reason. It’s easy to work with, well-supported, and has a huge library ecosystem behind it. But as Karol’s presentation showed, it’s not without trade-offs. When performance and scalability start to matter, Python’s limitations – like the Global Interpreter Lock – can’t be ignored. 

The key message? Don’t fight the language – work with it. If you know where Python struggles, you can design around those pain points. With the right mix of tools and a bit of strategic thinking, even complex data processing tasks can be handled effectively. Especially in life sciences, where both speed and flexibility matter, planning your architecture with Python’s strengths and weaknesses in mind is the way to stay ahead. 

Passion And Execution

Who We Are

At Holisticon Connect, our core values of Passion and Execution drive us toward a Promising Future. We are a hands-on tech company that places people at the centre of everything we do. Specializing in Custom Software Development, Cloud and Operations, Bespoke Data Visualisations, Engineering & Embedded services, we build trust through our promise to deliver and a no-drama approach. We are committed to delivering reliable and effective solutions, ensuring our clients can count on us to meet their needs with integrity and excellence.

more about Holisticon Connect