Got bugs? Here’s how to catch the errors in your scientific software

Science is becoming increasingly computational. Experimental data must be logged, cleaned, checked and analysed. Data analysis often involves iterative trial and error using ‘scripting’ programming languages such as Python and R. The outputs of such programs are then included in papers, presentations and grant applications.

A typical piece of professional software contains up to 50 errors per 1,000 lines of code (D. A. W. Soergel F1000Research 3, 303; 2015). But scientific code, which is written mainly by graduate students and postdocs who have little to no training in software development, is even more error-prone. Self-taught coders — and the artificial-intelligence-driven assistants they sometimes use — can create programs that seem to work yet generate nonsense, says computer scientist Amy Ko at the Information School at the University of Washington in Seattle. “If you have a program that computes something, it doesn’t mean that it’s correct.”

How to fix your scientific coding errors

Sometimes code fails to run altogether — because of a syntax error, for instance. This, “is annoying, but not the end of the world”, says ecologist and programmer Ethan White at the University of Florida at Gainesville. It’s easily fixed, he says. “The worst kind of code is code that executes but is wrong”.

Enter debugging, a crucial skill for software developers that is rarely taught to scientist-coders. Debugging “is like a detective story where you are both the investigator and the murderer”, says Andreas Zeller, a software engineer at the CISPA Helmholtz Center for Information Security in Saarbrücken, Germany, and author of The Debugging Book).

Debugging basics

Nature asked computing specialists to share their tips for debugging and ensuring that code does what it is supposed to do.

Document the conditions that cause the bug to appear. Is there a problematic input, for instance? If possible, identify a minimal working example (using stripped-down data or code, plus locked-down variables such as seeds for random-number generators) to replicate the problem easily. Then, iron the bugs out.

Use print statements. The simplest approach to debugging is to litter your code with ‘print’ commands that reveal a program’s internal state as it runs. While iterating over a collection of files to compute a number, for instance, you can ask your code to output the current file, current value and running tally.

Python’s ‘logging’ library provides a mechanism for doing this with varying degrees of verbosity, says Toby Hodges, who is based in Heidelberg, Germany, and is the curriculum director at The Carpentries, a global non-profit organization that teaches computational skills to researchers.

A toolkit for data transparency takes shape

Zeller is a fan of the print-statement method, because it produces a detailed, searchable event log. But he stresses that coders must approach the problem scientifically. “Are you just randomly poking in various directions without having a clear idea of what and where the bug could be? Or are you proceeding in a systematic fashion as a scientist would?”

Spin up a debugger. Integrated into popular coding environments such as VS Code, RStudio and Jupyter computational notebooks, debuggers are interactive tools that allow programmers to interrupt the flow of a program at any point (called setting a breakpoint), step from instruction to instruction and interrogate the system.

“You’re sort of like a detective, right?” says Katy Huff, a nuclear engineer at the University of Illinois at Urbana-Champaign and co-author of the 2015 book Effective Computation in Physics. “You set a breakpoint somewhere vaguely in the realm of where you think the lost diamond is,” Huff says. Then you can execute the code line by line and monitor how the computing environment changes. “You can pause the program while it’s running and say, what is the value of that number at this point? Oh, it’s still an integer? OK, great. Continue one more step. Is it still an integer or is it a NaN [not a number]?”

Crucially, debuggers allow users to change values on the fly and see what that does. But they are often incompatible with the “branching patterns of exploration” that characterize data analysis, warns Tracy Teal, chief executive of openRxiv, the organization encompassing the bioRxiv and medRxiv preprint servers, who is based in Davis, California. In that case, says Teal, print statements might be more fruitful.

Talk to the duck. Alternatively, you can talk through your problem out loud. “The simple act of verbalizing what you think could be the cause is tremendously effective when you’re debugging,” Zeller says. The recipient of your musings could be a colleague, but conventionally, it’s a rubber duck. “I have an office next door where the students have literally dozens of rubber ducks,” Zeller says.

AI chatbots can be sounding boards, too, Ko says — even if what they tell you is nonsensical. For example, Ko tried to use Claude Code, a coding tool by AI firm Anthropic in San Francisco, California, to debug a software system that she maintains. “As I wrote a detailed specification of the problem, I thought of the likely cause, and decided to investigate it while Claude spun its wheels to debug on my behalf. It came up with a very confident but incorrect cause. I found the actual cause before it was done.”

Trust, but verify

Once your code is working, ensure that it does what you expect.

One option is ‘unit testing’. Suppose you write a function to find the smaller of two numbers. Your test suite might feed it a range of values — positive and negative numbers, strings, extreme values and so on. The ‘testthat’ library in R and ‘pytest’ in Python, for example, provide the functionality to create and run such tests. By pairing these tests with automation systems (using ‘continuous integration’ technologies such as GitHub Actions), you can ensure that code changes do not break your algorithms.

Why Jupyter is data scientists’ computational notebook of choice