Scaling up scientific data management infrastructure

This abstract briefly highlights the data management challenges brought by the advent of modern computational methods and rapidly growing range of high through scientific equipments in the domain of structural sciences. Our requirement gathering exercises have revealed a significant gap between state of the art technologies and current data management practice. There are significant variations in data management requirements between individual researchers and facility service providers. Highly isolated technological solutions have been adopted by different stakeholders, making it hard for researchers to manage their experimental data which can be generated, collected, and analysed over a period of time at places across different collaborations. We also describe our approach to address this problem by presenting a loosely coupled architectural framework for managing scientific data lifecycles. We expect that the support for overlapping investigations and datasets will open up a whole range of possibilities to cross-examine datasets from different angles over time and space, ultimately, enabling existing isolated data management solutions to scale up to embrace the excitements brought by open research era.