NEW YORK (CBSNewYork/AP) — Two of the Confederate Army’s best-known leaders have streets named for them in a place not normally associated with the Southern side of the Civil War — New York City. Now some elected officials are trying to undo it.

They say it’s high time Stonewall Jackson Drive and General Lee Avenue in Brooklyn are renamed, pushing to join a number of Southern cities that have removed or are considering taking down Confederate statues and other memorials in public places.

READ MORE: FBI Executes Search Warrant At Home Of Gabby Petito's Fiancé Brian Laundrie

“To honor these men who believed in the ideology of white supremacy and fought to maintain the institution of slavery constitutes a grievous insult to the many thousands of people in Brooklyn who are descendants of the slaves held in bondage,” says a letter sent to Army Secretary Robert Speer recently by Reps. Yvette Clarke, Jerrold Nadler, Nydia Velazquez and Hakeem Jeffries, members of Congress who all represent parts of the borough.

The roads aren’t readily accessible by the general public; they run through Fort Hamilton, an active military base in southwestern Brooklyn next to the Bay Ridge and Dyker Heights neighborhoods. As part of their U.S. Army careers, both Robert E. Lee and Thomas “Stonewall” Jackson spent time at the fort — Lee in the early part of the 1840s and Jackson toward the end of that decade, well before the Civil War started in 1861.

They aren’t the only military figures with street names at the fort — other roads are named for figures including World War I Gen. John Pershing and World War II Gen. George Marshall.

Most people who spoke with 1010 WINS’ Roger Stern said don’t change a thing.

“Both were American Generals first,” one man said.

“They were serving their part of the country during the Civil War and uh, they were doing their duty,” said another.

Army spokesman Major General Malcolm Frost issued a statement to The Associated Press reiterating the stance that “every Army installation is named for a soldier who holds a place in our military history. Accordingly, these historic names represent individuals, not causes or ideologies. It should be noted that the naming occurred in the spirit of reconciliation, not division.”

READ MORE: Yonkers Police: 2 Men Dead After One Jumps Off Building And Hits The Other 12 Stories Below

The Army made that same point in 2015, after a deadly church shooting in Charleston, South Carolina, of black worshippers by a white man increased the volume of debate over Confederate symbols. A number of U.S. military installations are named after Confederate figures, such as Forts Lee, Hood, Benning, Gordon, Bragg, Polk, Picket, A.P. Hill and Rucker, as well as Camp Beauregard.

But the Army has also made changes, as it did in 2000, when it renamed a road at Fort Bliss in El Paso, Texas, from Forrest Road to Cassidy Road. The first name was after Confederate Gen. Nathan Bedford Forrest, a Civil War commander and a leader of the Ku Klux Klan. At the time, an Army official said complaints about the name didn’t drive the change but didn’t rule out that they were a consideration.

The issue has come up elsewhere. In Florida, five people were recently arrested when a city council meeting in Hollywood ended with a clash over three streets named for Confederate generals.

Throughout the South, state and city governments are weighing what to do with the statues. New Orleans recently removed three Confederate statues and a monument to white supremacy, something the Brooklyn legislators referenced in their letter.

“We have evolved beyond the Confederacy in the United States, and for people of color who have to utilize that base, it’s a constant reminder of a very painful period of time,” Rep. Clarke told the AP.

Bay Ridge resident Joe Conly said he doesn’t see a change as necessary, stressing that Lee was a loyal soldier during the time he was at Fort Hamilton. “He served his country, the United States, well when he was in New York,” said Conly, 75, who is white.

But Marva Harris Small, 58, a black woman who works in the neighborhood near Fort Hamilton, said that whatever good the men might have done while at the base was subsumed by their serving as Confederate generals.

“The end product is what counts,” she said.

MORE NEWS: New York City Public Schools To Increase COVID Testing, Relax Quarantine Rules

(© Copyright 2017 CBS Broadcasting Inc. All Rights Reserved. The Associated Press contributed to this report.)