Gamma-ray bursts (GRBs) come in two classes1: long (> 2 s), soft-spectrum bursts and short, hard events. Most progress has been made on understanding the long GRBs, which are typically observed at high redshift (z ≈ 1) and found in subluminous star-forming host galaxies. They are likely to be produced in core-collapse explosions of massive stars2. In contrast, no short GRB had been accurately (< 10″) and rapidly (minutes) located. Here we report the detection of the X-ray afterglow from—and the localization of—the short burst GRB 050509B. Its position on the sky is near a luminous, non-star-forming elliptical galaxy at a redshift of 0.225, which is the location one would expect3,4 if the origin of this GRB is through the merger of neutron-star or black-hole binaries. The X-ray afterglow was weak and faded below the detection limit within a few hours; no optical afterglow was detected to stringent limits, explaining the past difficulty in localizing short GRBs.