Milton Harbor, one of the most beautiful areas in Rye, has drawn people for thousands of years, since its early days as summer hunting and fishing grounds for the Lenape Native Americans.