The name came from a folk tale from the seventeenth century.
"The Devil bought the stones from a woman in Ireland, wrapped them up, and brought them to Salisbury plain. One of the stones fell into the Avon, the rest were carried to the plain. The Devil then cried out, "No-one will ever find out how these stones came here!" A friar replied, "That’s what you think!," whereupon the Devil threw one of the stones at him and struck him on the heel. The stone stuck in the ground and is still there."