NEW YORK (CBSNewYork) – Residents of Lenox Terrace in Central Harlem got their first win against a developer with big plans to change their apartment complex.

The community board voted against the rezoning proposal to build new aprartment buildings with larger stores and restaurants.

READ MORE: Gov. Lamont: Connecticut Man Who Tested Positive For Omicron Variant Has Mild Symptoms, Resting At Home

A passionate crowds of Lenox Terrace and Harlem residents filled the community board meeting in protest, reported CBS2’s Aundrea Cline-Thomas.

“Once they take Lenox Terrace, the community is over with,” one person said.

(credit: Olnick Organization)

The Olnick Organization’s rezoning proposal would erect five new buildings with more expensive apartments, bigger stores and restaurants on the property. Olnick has owned Lenox Terrace since the beginning.

“These type of actions are going to cause a real racial displacement,” said Harlem District Leader Cordell Cleare.

The community board agreed and voted against the plan.

“One day waking up and Harlem is white… that’s a reality,” one speaker said at the meeting.

“The tax implications, how it will affect sanitation, transportation, commercial, all of those things. With this rezoning it will change the face of Harlem as it is,” said Community Board 10 chair Cicely Harris.

Seth Schochet, Olnick’s president, addressed the crowd briefly.

READ MORE: Bob Dole, Former Senate GOP Leader And Presidential Nominee, Dies At 98

But when Cline-Thomas tried to ask him questions – especially about claims of crumbling units and withholding overdue apartment upgrades, Shochet refused to talk.

Wednesday’s community board vote will serve as a recommendation for the City Council and Mayor Bill de Blasio to consider moving forward.

The sprawling complex from 132nd to 135th streets between Lenox and Fifth avenues was an oasis for African-Americans during a time when housing discrimination was the norm. In the late 1950s, ads called it Harlem’s first community with Park Avenue elegance, complete with 24-hour doormen.

“There is a good case for being a cultural landmark,” said Peg Breen of New York Landmarks and Conservancy.

Breen says landmarking wouldn’t necessarily stop future development.

“You would hopefully get a smaller building, a better building, a more appropriate building but it wouldn’t necessarily answer what the residents are looking for,” Breen said.

Prior landmarking attempts were denied, most recently in April, when tenants were told the buildings that date back to the ’50s were not architecturally significant.

“African Americans didn’t have the ability to have a house or apartment as grand as Frank Sinatra,” said Harlem resident and historian Michael Henry Adams.

Opponents of the development plan know the community vote is not enough.

“They underestimate how ruthless, clever and resources these developers have at their disposal,” Adams said.

MORE NEWS: Zayid Muthana Has Successful Surgery To Remove Bullet From His Head, Family Says

Residents are preparing for a long fight ahead.