You are here

High speed data transfer at the University of Otago

The University of Otago, a research intensive organization located in the lower South Island of New Zealand, with researchers also based at campuses in Christchurch and Wellington, needed to transfer over half a petabyte of Genomics and Physics data to remote NeSI HPC facilities in 2016. In order to do that they needed a fast, scalable approach for transferring data.

The University of Otago currently uses REANNZ as the provider for private WAN, R&E and commodity internet. NeSI is currently hosted within the campus to provide High Performance Computing (HPC) facilities to New Zealand, supporting researchers across the public research sector and private industry. However, NeSI was unable to fully utilise the capability provided by the current 10Gbps circuit so REANNZ proposed a Science DMZ which lets normal campus traffic go through firewalls and Exinda for monitoring, traffic shaping and policy enforcement while large file transfers using multiple simultaneous tcp streams used by applications such as GridFTP/Globus Toolkit goes through a clean path, without any shaping and firewalling in between the transfer nodes.

This presentation takes a look at what a Science DMZ is, how it works, and how it is applied at the University of Otago, and at research institutes all over the world.
From the University of Otago’s perspective, we will look at technical and organisational challenges encountered during deployment of the new data transfer service, as well as the capabilities of the new service, both in terms of the performance benchmarks achieved, utilisation to date, and the direct benefits to research teams.