Given its coastal location, it should come as no surprise that the sea plays a massive factor in Sag Harbor’s history. In particular, the village became the capital of the nascent nation’s whaling industry, attaining a status so revered it garnered multiple mentions in “Moby Dick.” Echoes of that past remain, in one sense with the so-called Old Whaler’s Church and in a way more literal sense with the local Whaling Museum. In general, Sag Harbor became the port du jour in 1789 when Congress declared it the official port of entry into the U.S., the first stop for any vessel entering the country. Such a distinction enabled the area to emerge as a hotbed of social vitality where all kinds of people—sailors, merchants, artists—freely mixed. But even though the last whaling ship set sail centuries ago, Sag Harbor remains a bustling hub of activity today.

There are no two ways about it: Sag Harbor has a thriving downtown—even though downtown is, geographically speaking, in the upper part of town. Eating options dot Main Street and the surrounding area by the water, serving up delicious, often locally-sourced meals for breakfast, lunch, dinner, brunch, and whatever the term is for buying a doughnut in the late afternoon even though you’ve got reservations in a few hours. For drinks, the bars perpetuate Sag Harbor’s throwback feel, trending toward the kind of local haunt where everyone is liable to know your name. Sag Harbor’s shops are homegrown favorites, with the village providing much in the way of vintage, antiques, and opportunities to purchase handmade products like jewelry or pottery. Stop in at a weekend farmer’s market to stock up on produce, pastries, cheese, and otherwise artisanal goods.