NEW YORK (CBSNewYork/AP) — People have worked hard, and mostly successfully, over the past four years to restore the New York and New Jersey coastline to what it was before Superstorm Sandy crashed ashore, causing widespread devastation. But some areas have not recovered, nor will they ever.

Oakwood Beach on Staten Island, improbably built on a salt marsh, is slowly being returned to nature after state officials concluded it would be foolish to rebuild in a place with so little protection from the sea.

Under a state buyout program, 196 homes have been demolished so far. Another 103 will soon meet the same fate.

PHOTOS: Sandy Slams Tri-State | Aftermath From Above

A few residents remain. But on one recent visit, there was no one on the streets, no children playing and few cars going by. Many of the houses that are still standing are boarded up and plastered with signs that say “no trespassing.”

“We’re going to stay here as long as we can,” said Connie Martinez, 61. “My husband is a very sick man. He had a stroke. He loves his yard, he loves his garden.”

MORE: Full Superstorm Sandy Coverage

Yet, a buyout may still be in her future, said Martinez, who had three neighbors drown during the Oct. 29, 2012 storm.

“The consensus of all my neighbors is they never should have built houses here,” she said. “It should have been left wetlands.”

Oakwood Beach is one of several places along the coast that have seen permanent changes wrought by the storm, some big, some literally hidden in the sand.

Over four years, New Jersey has doled out $2.2 billion in federal aid for rebuilding after the storm.

In Mantoloking, a shore town that saw nearly all of its 521 homes damaged or destroyed, a huge steel wall has been erected on the beach and then buried, in most places, under man-made dunes to keep the surf from reaching oceanfront homes during future storms.

In Seaside Heights, made infamous by the MTV reality show “Jersey Shore,” authorities still plan to rebuild an amusement park that had its roller coaster swept into the surf by the storm. But the town gave up on returning the park to a pier that jutted out into the water, opting instead for a new location on the beach.

In Toms River, the storm surge damaged or destroyed 10,000 homes. Mayor Tom Keller said about a quarter of those families are still trying to get back and they’ve hired a retired insurance executive to help.

“He’s the go-to person for anybody that’s got problems. He takes them and walks them through the application and the permitting processes,” Keller said.

The next step is to build protective dunes. Nearly all of the property owners have signed easements, WCBS 880’s Sean Adams reported.

“I am hoping that right after the first of the year, weather permitting, they’ll get started,” Keller said.

On Long Island, the barrier island city of Long Beach had its only full-service hospital destroyed by flooding. The medical center was replaced in 2015 by a smaller emergency department operated by an off-island community hospital.

But after four years of rebuilding, few places have as stark a then-and-now look as Oakwood Beach.

Not even the foundations remain of the seven homes that once stood on either side of Connie Martinez’ house. All that is there now are empty patches of green grass, which the state is still paying to have mown until the last residents leave.

Signs of former life can still be glimpsed here and there: exterior Christmas lights on one house, a children’s playset and collapsed trampoline at another. But they’re marked for demolition with splashes of red paint.

“As soon as there’s a buyout, they automatically throw up the boards, put the locks on the doors so there’s no trespassing and intruders,” said Joe Giordano, an engineer with NorthStar, the state contractor overseeing the demolition. “All those that are boarded up will come down in the next year to year and a half.”

Leonard Desisto, a retired city bus operator who hasn’t moved out, said the neighborhood’s state now reminds him of when he bought his house in 1982, when it wasn’t as built-up.

“It’s peaceful again,” he said. “To tell the truth, I like it better.”

Changes on other parts of the coastline are harder to see, unless you know what it was like before the storm.

On Fire Island, a popular summertime vacation spot, Sandy’s storm surge blasted clean through the narrow strip of land and left a new permanent inlet, between 1,475 and 2,345 feet across.

The breach between the Atlantic Ocean and the Great South Bay initially raised concerns about flooding on Long Island’s south shore. Experts say that hasn’t happened, and now officials at the National Park Service are examining whether to leave the new waterway open.

One argument in favor of doing so: The hole has helped flush pollution out of the bay, leading to a resurgence of clams and bass, according to Charles Flagg, a marine science professor at Stony Brook University.

At the sandy, wind-swept tip of Rockaway peninsula, the beachfront enclave of Breezy Point has made a remarkable recovery. More than 300 homes destroyed by flooding and a raging fire have been rebuilt.

But many of the new houses have a different look. Once a community of low bungalows, its new houses stand on concrete pillars or ultra-high foundations. A few have been rebuilt entirely of concrete, rather than wood, to better withstand pounding surf.

Arthur Lighthall, the general manager of the Breezy Point Cooperative, said some residents now have to climb a dozen steps to reach decks that formerly sat right on the sand. In addition to being taller, the new houses are shielded from the ocean by a new wall of protective dunes.

Former U.S. Rep. Bob Turner, a Breezy Point resident, said one important thing hasn’t changed by a storm that left much of the community a charred ruin.

“The people are the same,” he said. “There was some dislocation but not widespread.”

(TM and © Copyright 2016 CBS Radio Inc. and its relevant subsidiaries. CBS RADIO and EYE Logo TM and Copyright 2016 CBS Broadcasting Inc. Used under license. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed. The Associated Press contributed to this report.)