We study the quantum dynamics of a single qubit coupled to a bath of interacting spins as a model for decoherence in solid state quantum memories. The spin bath is described by the Lipkin-Meshkov-Glick model and the bath spins are subjected to a transverse magnetic field. We investigate the qubit interacting via either an Ising- or an XY-type coupling term to subsets of bath spins of differing size. The large degree of symmetry of the bath allows us to find parameter regimes where the initial qubit state is revived at well defined times after the qubit preparation. These times may become independent of the bath size for large baths and thus enable faithful qubit storage even in the presence of strong coupling to a bath. We analyze a large range of parameters and identify those which are best suited for quantum memories. In general we find that a small number of links between qubit and bath spins leads to less decoherence and that systems with Ising coupling between qubit and bath spins are preferable.