Python parallel debugging comes to HPC thanks to TotalView upgrade
At the SC26 supercomputing show this week (booth #2829), Perforce Software is demonstrating how its TotalView parallel debugging tool now understands Python, easing the path for using the language on large, distributed apps, particularly those in the high-performance computing (HPC) space.
- A view into parallel Python Under the hood, TotalView uses Microsoft's debugpy engine (via the Debug Adapter Protocol) to power its own hierarchical tree server.
- To kick off a debugging session, a developer can launch a single-process or parallel program under TotalView’s control.
- For now, developers have to choose between debugging the C/C++ side (while viewing the Python stack) or debugging Python on its own — though Burns said Perforce plans to merge both modes into a single workflow in an upcoming release.
- And though TotalView supports mpi4py and Python’s own multiprocessing module out of the box, future releases will support other Python concurrency frameworks, such as Dask and PyTorch’s torchrun.
- In the HPC community, "Python has grown in popularity,” said Bill Burns, Perforce vice president of software engineering, in an interview with The Register.
- The package doesn’t offer full cross-language debugging, which would be truly helpful for Python apps that call C/C++ libraries and modules.
- The OG debugger for supercomputer iron has buddied up with the data-science darling now muscling in on HPC: Python.
- Of course, with 500 or 1,000 interpreters all running in parallel, debugging will also get more difficult.
How this summary was made: extractive summary — built from the sources' own reporting. It is drawn only from the reporting of the sources listed below — read the originals for full detail. How we summarize →