WOODCLIFF LAKE, N.J. (CBSNewYork) — A New Jersey borough is being sued by the U.S. Department of Justice for religious discrimination.

The federal government says Woodcliffe Lake is unfairly targeting a Jewish organization by not letting it expand. But the borough says its decision has nothing to do with religion.

CBS2’s Jessica Layton was on the story Wednesday night and continued demanding answers Thursday.

In the quiet area of Woodcliffe Lake, there are whispers about the federal government suing the small borough for religious discrimination, with a Jewish house of worship at the center of the debate.

“We have not been allowed to observe and to celebrate with a proper facility as we need to,” Valley Chabad Rabbi Dov Drizin told Layton.

More: Feds Sue New Jersey Borough Over Its Treatment Of Jewish Group

Rabbi Drizin has run Valley Chabad out of his home on Overlook Drive since 2000. With no formal membership or dues and a policy of acceptance, people flock by the hundreds to take part in the Orthodox Jewish organization’s synagogue, Hebrew school and social events.

“It’s unique, it’s different,” he said.

Now, they want to be even bigger. But several times now, the borough has rejected the application to build a larger facility on alternate sites — most recently denying an application to expand on the property it already owns.

The longstanding dispute prompted the U.S. Attorney General to sue Woodcliffe Lake under the Religious Land Use and Institutionalized Persons Act of 2000.

“We alleged that over an eight-year period the town stopped every effort by the group to purchase an alternative worship site and then denied it permission to expand on its property,” said U.S. Attorney General Jeff Sessions.

But why? CBS2 went to borough hall to get answers.

At first, the administrator would only say lawyers are reviewing the federal lawsuit and would issue a statement soon. Layton pressed for more and later received a statement from borough’s attorney, reading “Valley Chabad continues to demand the construction of a 17,000 square foot facility with seating for 400 plus congregants. The Borough’s Zoning Ordinance requires a three acre minimum for such a facility,” going on to say the borough has identified other larger plots that could accommodate them.

CBS2 still wants to know what other plots the borough offered as suggestions. So far, no response.

The attorney for Valley Chabad said there were no three-acre properties available.

The years-long fight between the Jewish organization and the borough has become so controversial in town, most who live there were not willing to say where they stand on the issue – at least not on camera.

“In general, people today are afraid to say anything,” one man said.

“I’m supporting whatever they want to do, as long as they don’t do any harm or anything wrong. They should be allowed,” said a woman.

As for what’s next, Rabbi Drizin told Layton his wish is “that we can find a way to work together.”

That will depend on how the complicated and now national legal game plays out. In addition to the federal lawsuit, Valley Chabad has taken its own legal action against the borough.