Heisenbugs: bugs that change behavior when we observe them
Some bugs have a particular talent: they hide when we look for them, appear only in production, mysteriously disappear locally, or vanish as soon as a developer adds a few logs to understand what is happening. Worse, sometimes simply opening a debugger is enough to make them disappear.
These elusive bugs have a name: Heisenbugs.
The term refers to Heisenberg's uncertainty principle: in quantum mechanics, the act of measurement influences the observed system. In computer science, the idea is similar: adding logs, using a debugger, or changing the environment can be enough to alter the program's behavior and hide the bug.
Heisenbugs are among the hardest bugs to fix because the tool used to understand the problem can itself change the behavior of the system.
What is a Heisenbug?
A Heisenbug is a bug whose behavior changes when we try to observe it. For example:
- the bug disappears when debug mode is enabled;
- it no longer appears when adding a `console.log()`;
- it cannot be reproduced with a debugger;
- it only appears on certain servers.
Unlike a classic bug, the problem is not necessarily in the visible logic of the code. It is often related to interactions that are difficult to predict, such as concurrency, timing, memory state, or the execution environment.
Simple example: a timing problem
Imagine two tasks using the same variable:
let value = 0;
async function increment() {
const current = value;
await processData();
value = current + 1;
}
increment();
increment();The developer may expect: value = 2
But depending on when the operations are executed, the result may be: value = 1
Both functions read the same value before incrementing it.
This problem can be difficult to reproduce:
- on a fast machine, it happens rarely;
- adding logs changes the timing;
- under production load, it becomes frequent.
Simply adding:
console.log(current);can change the execution order and make the problem disappear.
The main causes of Heisenbugs
Race conditions
When we write code and need to execute many operations, we often do it in parallel using threads, workers, simultaneous requests, or asynchronous tasks. Two operations can access the same data in a different order.
A classic example:
1. A user clicks twice quickly.
2. Two requests are sent.
3. Both modifications overlap.
4. The final state depends on the timing.
This type of problem is often invisible during testing because test suites rarely execute multiple actions truly simultaneously. And I’ll be honest: writing this kind of test is often tedious. You have to orchestrate threads, synchronize tasks, manage delays… for a result that can sometimes feel irrelevant as long as the bug has never appeared in real conditions.
Memory-related problems
Some bugs depend on how memory is organized. And these ones, I absolutely hate them!
Here are some examples:
- reading an uninitialized variable;
- buffer overflow;
- use-after-free;
- memory corruption.
A seemingly insignificant change can modify the memory layout, such as adding a variable, enabling debug mode, or changing the compiler version.
The bug then appears to disappear. And at that point, you can spend hours searching for the cause… really frustrating.
Differences between environments
A large number of Heisenbugs come from differences between development and production environments.
Locally, we usually have a single machine, little data, and a cache that is either empty (or full).
In production, we may have several servers, many requests, historical data (I have already had some great surprises with data from old versions that were several years old...), or an unstable network.
Code can work perfectly for months during development and then fail only in production.
Logs that change behavior
Logs are essential for diagnosing problems, but they can sometimes modify the system.
Adding:
error_log("Debug value: " . $value);
can slightly slow down execution or change timing. And in an application with concurrency issues, a few milliseconds can be enough to change the result.
Caches
Caches are a common source of bugs that are difficult to reproduce.
For example:
- different data between two servers;
- a cache invalidated too late;
- an API response kept for too long.
A developer may only reproduce a bug with a specific browser or after several hours.
Why a debugger can hide a bug
We often associate debuggers with timing problems: adding a breakpoint slows execution and can change the behavior of a concurrent program.
But this is not the only reason.
In compiled languages such as C or C++, a program running with a debugger is often different from the one used in production.
A debug build may disable some optimizations, while a compiler in production may:
- reorder certain instructions;
- change how registers and memory are used.
Bugs related to undefined behavior, such as invalid memory access or uninitialized variables, may therefore only appear in production.
Memory management can also change: debug environments sometimes add protections or specific patterns in memory regions, which can hide or reveal certain problems.
So, a bug that disappears under a debugger is not necessarily fixed: the observed program is simply no longer running under exactly the same conditions.
How to diagnose a Heisenbug?
With a Heisenbug, the hardest part is often observing it without making it disappear. I therefore try to modify the system as little as possible during the analysis.
I prefer structured logs with a request identifier and contextual information rather than a flood of scattered console.log() statements. I also add metrics (response time, errors, memory usage, queues) that allow me to observe the problem without disturbing its behavior too much.
When the bug only appears in production, I try to reproduce it with some load or multiple simultaneous requests. Finally, I always question assumptions such as: "This function will definitely be called before that one."
In an asynchronous or concurrent system, this kind of certainty is often the first thing to challenge.
Let’s be honest: there is no magic solution for diagnosing these problems. We spend time scratching our heads and investigating. With some luck, your generative AI can give you a few leads, and sometimes, a little luck is also part of the process.
How to avoid Heisenbugs?
I don't think we can completely eliminate Heisenbugs, especially in concurrent or distributed systems. However, some practices can greatly reduce the risks.
I try to limit shared state as much as possible. Each component should ideally own its own data and responsibilities, rather than allowing multiple parts of the system to modify the same information. The fewer dependencies between components, the less unpredictable the interactions become.
I also try to make critical operations atomic. For example, an SQL query like:
UPDATE accounts
SET balance = balance - 100
WHERE id = 10;
is generally safer than a sequence of steps like "read → calculate → rewrite" executed in the application layer. The example is intentionally simplified, but the idea is to prevent another operation from modifying the data between two steps.
Finally, I rely on database transactions to avoid inconsistent intermediate states. When an operation modifies several pieces of data, it is often better to treat it as a single unit: either all changes are committed, or none of them are.
In distributed systems, I also prefer idempotent operations. A request received twice should not necessarily trigger two different actions. This is particularly important with message queues or systems using retry mechanisms.
Conclusion
Heisenbugs are among the hardest bugs to solve and are often a real headache for us developers. Unfortunately, they are part of our daily lives.
But there is also a small satisfaction at the end: once the mystery is solved, you get that little feeling of victory… or, at least for me, a certain sense of fulfillment after finally understanding what was happening.
Good luck with your next Heisenbugs. You will definitely encounter a few of them 🙂
