Drought management frameworks are dependent on methods for monitoring and prediction, but quantifying the hazard alone is arguably not sufficient; the negative consequences that may arise from a lack of precipitation must also be predicted if droughts are to be better managed. However, the link between drought intensity, expressed by some hydrometeorological indicator, and the occurrence of drought impacts has only recently begun to be addressed. One challenge is the paucity of information on ecological and socioeconomic consequences of drought. This study tests the potential for developing empirical drought impact functions based on drought indicators (Standardized Precipitation and Standardized Precipitation Evaporation Index) as predictors and text-based reports on drought impacts as a surrogate variable for drought damage. While there have been studies exploiting textual evidence of drought impacts, a systematic assessment of the effect of impact quantification method and different functional relationships for modeling drought impacts is missing. Using Southeast England as a case study we tested the potential of three different data-driven models for predicting drought impacts quantified from text-based reports: logistic regression, zero-altered negative binomial regression (hurdle model), and an ensemble regression tree approach (random forest). The logistic regression model can only be applied to a binary impact/no impact time series, whereas the other two models can additionally predict the full counts of impact occurrence at each time point. While modeling binary data results in the lowest prediction uncertainty, modeling the full counts has the advantage of also providing a measure of impact severity, and the counts were found to be reasonably predictable. However, there were noticeable differences in skill between modeling methodologies. For binary data the logistic regression and the random forest model performed similarly well based on leave-one-out cross validation. For count data the random forest outperformed the hurdle model. The between-model differences occurred for total drought impacts and for two subsets of impact categories (water supply and freshwater ecosystem impacts). In addition, different ways of defining the impact counts were investigated and were found to have little influence on the prediction skill. For all models we found a positive effect of including impact information of the preceding month as a predictor in addition to the hydrometeorological indicators. We conclude that, although having some limitations, text-based reports on drought impacts can provide useful information for drought risk management, and our study showcases different methodological approaches to developing drought impact functions based on text-based data.