We present optical spectroscopy, ultraviolet-to-infrared imaging, and X-ray observations of the intermediate luminosity optical transient (ILOT) SN 2010da in NGC 300 (d = 1.86 Mpc) spanning from −6 to +6 years relative to the time of outburst in 2010. Based on the light-curve and multi-epoch spectral energy distributions of SN 2010da, we conclude that the progenitor of SN 2010da is a ≈10–12 M ⊙ yellow supergiant possibly transitioning into a blue-loop phase. During outburst, SN 2010da had a peak absolute magnitude of M bol lesssim −10.4 mag, dimmer than other ILOTs and supernova impostors. We detect multi-component hydrogen Balmer, Paschen, and Ca ii emission lines in our high-resolution spectra, which indicate a dusty and complex circumstellar environment. Since the 2010 eruption, the star has brightened by a factor of ≈5 and remains highly variable in the optical. Furthermore, we detect SN 2010da in archival Swift and Chandra observations as an ultraluminous X-ray source (L X ≈ 6 × 1039 erg s−1). We additionally attribute He ii 4686 Å and coronal Fe emission lines in addition to a steady X-ray luminosity of ≈1037 erg s−1 to the presence of a compact companion.