Captain Kidd supposedly buried his treasure in this swamp. Specifically, it is located under an large, old oak tree on a ridge that rises abruptly near the water's edge.
Kidd was a pirate and/or privateer (accounts differ) who was hanged in the late 1700s, leaving behind a legend that he had buried a significant treasure somewhere on the northeast coast of North America. In "The Devil and Tom Walker", this treasure is the one buried in the swamp, although the details are almost entirely fabricated by Irving for the sake of the story. Kidd himself is not really relevant to the story; he was probably chosen in order to provide an additional hook for the reader; "Kidd's treasure" as opposed to plain old "pirate treasure".
Captain Kidd Buried the treasure in the swamp.