There were no cotton fields or sprawling plantations worked by the enslaved here in Rye. But there was a time, in the late 18th and early 19th century, when Westchester County was largely agricultural — and, as in the American South, that agriculture was enabled by the institution of slavery.
One of the towns with a high percentage of the enslaved and households owning them was Rye. Put another way, there was slavery in my neighborhood, indeed on the very land on which my own small house now stands.
Slave labor made it possible for families like the famous Jays to accumulate large land holdings. And the slaveholders of Rye — in contrast to Southerners who saw their plantations burned and ruined — never ceased to profit from what slavery allowed them to build, even after its abolition. The significant acreage of farmland that slavery helped them buy and till became, following the coming of the New York & New Haven railroad in 1849, a source of wealth from the sale of real estate, once Rye and Westchester’s Sound Shore generally turned from agriculture to commuter suburb.
A handful of the largest two dozen properties in Rye in the early 19th century accounted for a large proportion of all those enslaved. Of 130 slaves, 68 were owned by just six families, including the Jay family, which included the nation’s first Supreme Court Justice John Jay. In 1790, his brother Peter Jay, who managed the family’s estate in Rye — of which John Jay was half-owner — owned nine slaves and the largest and highest-valued property in town. John and Peter’s father and grandfather had not only owned slaves but had helped to finance the slave trade.
What’s more, in Rye the economic windfall of slavery continued even after New York State voted to end slavery in 1797; the law allowed slavery to continue for 30 more years, permitting owners to continue to profit from it and even to sell the enslaved during the interim.
But the slaveowners of Rye were to enjoy an even greater return on their investment long after slavery — and local agriculture — were gone. The large landholdings they had been able to acquire and support with the help of enslaved labor became valuable for real estate development.
The former slave-owning families both sold rights-of-way to the railroad — which helped turn the town into a commuter suburb of New York City — and to developers who created housing subdivisions.
The Halstead family, for instance, sold a significant number of land parcels to the new railroad. Land sale records indicate that sellers with the surname Halstead sold 23 parcels to the New York and New Haven Railroad. Just a single sale of one property, a portion of land owned by the slave-owning Theall family (for whom a street in town is named today, as are streets named for the slave-owning Purdy, Guion, and Halstead families) netted $12,000 in 1865 — the equivalent of $235,000 today. An 1881 sale of a portion of one Halstead property netted $14,000 — or $426,000 today.
Property records show that such sales were common, even into the 20th century, when the Halstead family established a realty trust to subdivide the neighborhood in which I now live. Slavery helped slaveowners support the largest farms; suburbanization helped them realize financial gain on the eventual sale of that real estate.
It is worth noting, however, that the majority of Rye’s white households were not slaveowners. Of 205 households, only 43 were slaveowners. What’s more, 77 were listed as owning only “house and lot” — tiny homesteads worth less than $500 — and in some cases as little as $60. These were poor whites, some of them “watermen” gathering shellfish on the Long Island Sound or tradesmen, such as carpenters.
Should those owners of small and medium-sized farms — who typically did not use any slave labor — have wanted to expand, they would have found the large portions of the town’s real estate tied up in farms owned by slaveholders
It is a history relevant to the New York State Community Commission on Reparations Remedies, charged in 2024 with investigating the long-term economic effects of slavery on African-Americans. So it is that the story of slavery in my neighborhood is of more than historical interest.
It turns out to be a story of nuance and complexity — not as it involved Blacks, who were unconscionably bought, sold, and uncompensated for their labor — but as it involved whites, most of whom did not participate in slavery and some of whom were likely disadvantaged by it, or chose not to take advantage of it. Rye provides a useful window into a complex history.
This is an excerpt from “Slavery in My Neighborhood,” published in the spring issue of the Westchester Historian, a quarterly publication of the Westchester Historical Society. The full essay can be found on the Rye Historical Society website.
