Following a disaster, the majority of families rebuild their homes themselves. In this paper, we consider how the physical environment influences such ‘self-recovery’ by investigating disasters in the Philippines (typhoons Haiyan in 2013 and Haima in 2016) and Nepal (the Gorkha earthquake - 2015). Despite the many differences in the disaster contexts, there are some common barriers to self-recovery (and building back better) in a substantially changed and dynamic multi-hazard, post-disaster environment. These are related to changes in water supply (shortage or surplus), impacts of post-disaster geohazard events on infrastructure (particularly affecting transport) and the availability of technical advice. People face a broad spectrum of challenges as they recover and tackling these ‘geo-barriers’ may help to create a more enabling environment for self-recovery. The findings point to what needs to be in place to support self-recovery in dynamic physical environments, including geoscience information and advice, and restoration of infrastructure damaged by natural hazard events. Further research is necessary to understand the issues this raises for the shelter and geoscience communities, particularly around availability of geoscience expertise, capacity and information at a local scale.