Pawsey Scheduled Maintenance (November)

Scheduled Maintenance Report for Pawsey Supercomputing Research Centre

Completed

Good afternoon everyone,

Today’s maintenance work is complete, and the following services have been returned to service.

Setonix
HPE successfully applied the required Slingshot configuration change. The Cray Programming Environment (CPE) has been upgraded to version 26.03, and the 2026.11 software stack is now the default. The latest SLES 15 bug fixes and security updates have also been applied.

Important notes:
Pytorch
We have also identified that the new PyTorch software module pytorch/2.13.0-rocm7.2.4 is failing test with an illegal shader instruction error. This is being investigated, researchers that used PyTorch module on the system will be notified about resolution.

Scratch filesystem
During our return to service testing we have noted some minor degradation in performance of the flash partition of the scratch file system. Pawsey staff will continue to monitor and investigate the cause.

LD_LIBRARY_PATH
A new Lmod CVE fix requires upgrading to Lmod 9.4.3, which changes how LD_LIBRARY_PATH is handled. Pawsey Singularity modules have been updated accordingly and testing was successful, but any custom modules that directly read LD_LIBRARY_PATH may need review and modification. Please raise Help Desk ticket if you encounter any issues.

For details of the software stack changes and recommended user actions, please see November 2026 Software Update – Important Information.

ASKAP ingest
ASKAP Ingest has been returned to service following installation of the latest SLES 15 bug fixes and security updates.

Core services
General patching of all core services has been completed.

Acacia object Storage
Acacia has been returned to service following completion of security patching.

Banksia and tape storage
Banksia and tape storage have been returned to service. Banksia’s servers have received the latest bug fixes and security updates, ScoutAM has been upgraded to the latest version, and the Fibre Channel switches have received the latest firmware updates.

The upgrade from ScoutGW to VersityGW was not implemented and will be re-visited next month. VSS-6 will remain out of service.

Visualisation services
Visualisation services have been returned to service following patching.

Mediaflux
Mediaflux has been returned to service following an upgrade and patching in accordance with vendor recommendations.

Data Portal
The Data Portal has been returned to service. Data Portal now supports Acacia allocations on the Projects cluster.

Thank you for your patience during today’s maintenance. If you have any questions or encounter any issues, please contact help@pawsey.org.au.
Posted Nov 03, 2026 - 15:27 AWST

In progress

Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Nov 03, 2026 - 07:00 AWST

Scheduled

Maintenance on Pawsey systems will be carried out on Tuesday the 3rd November to apply required patches and updates to improve the systems stability, security, and performance. This maintenance window will also be used to undertake other tasks which require down-time to achieve.

HPE will be reapplying the Slingshot configuration change they made in October to perform troubleshooting in conjunction with resources in the US.

Planned work for this window includes:
• Patching of visualisation services will be undertaken
• Mediaflux server will have the latest bug and security fixes from Rocky Linux
• Mediaflux will be upgraded
• Data Portal will be upgraded
• Data Portal support for Acacia allocations will be rolled out to all projects with allocations on the Acacia
• Acacia will have some insecure packages removed
• Banksia's servers will have the latest bug and security fixes from Rocky Linux
• Banksia will be upgraded to the latest version of ScoutAM
• Banksia fibre switches will have firmware updates
• Patching of core Pawsey services will be undertaken
• Setonix will be upgraded to the Cray Programming Environment (CPE) version 26.03
• New software stack 2026.11 will be set as default on Setonix
• Setonix will have the latest bug and security fixes from SLES 15 applied
• ASKAP Ingest will have the latest bug and security fixes from SLES 15 applied

For more information about the software stack changes on Setonix, please see https://pawsey.atlassian.net/wiki/spaces/US/pages/2132049952/November+2026+Software+Update+-+Important+Information.

If you have any questions, please contact help@pawsey.org.au.
Posted Oct 27, 2026 - 09:02 AWST
This scheduled maintenance affected: ASKAP (ASKAP ingest nodes, ASKAP service nodes), Central Services (Authentication and Authorization, Service Desk, License Server, Application Portal, Origin, /home filesystem, /pawsey filesystem, Central Slurm Database, Documentation), The Australian Biocommons (Fgenesh++), Storage Systems (Acacia Ingest, Acacia MWA, Acacia Projects, Banksia, Data Portal Systems, MWA ASVO), Lustre filesystems (/scratch filesystem, /software filesystem, /askapbuffer filesystem, /askapingest filesystem), Setonix (Login nodes, Data-mover nodes, Slurm scheduler, Setonix work partition, Setonix debug partition, Setonix long partition, Setonix copy partition, Setonix askaprt partition, Setonix highmem partition, Setonix gpu partition, Setonix gpu high mem partition, Setonix gpu debug partition), and Visualisation Services (Remote Vis, Vis scheduler, Setonix vis nodes, Nebula vis nodes, Visualisation Lab, Reservation, CARTA - Stable, CARTA - Test, Pawsey Remote VR).