ALBANY, N.Y. (AP/WCBS 880) — Gov. David Paterson said Wednesday that developers of an Islamic cultural center that would include a mosque near ground zero have rejected his offer to help them find a different site.

While in Manhattan, Paterson said the group apparently wants to remain with its current plans to build near the site of the Sept. 11, 2001, terrorist attacks.

READ MORE: New York Weather: CBS2’s 1/20 Thursday Morning Forecast

“I think they would like to stay where they are, and I certainly respect that and I certainly respect them,” Paterson said after the group spoke with one of his staff members. “Having said that, how much more foresighted would it have been if the Imam who is the developer of the project had been willing to hear what we are actually talking about?”

The mosque, to be located two blocks from ground zero, would be part of a 13-story, $100 million Islamic center that would feature a 500-seat auditorium, a swimming pool and a gym. It’s a project of the Cordoba Initiative, an advocacy group that promotes improved relations between Islam and the West.

Critics are suspicious of who will fund the project, and developers haven’t released their sources of capital. Opponents also say the religion that fueled the hatred in the terrorists shouldn’t be displayed so near the site.

Wednesday, Paterson described some opponents of the project as “shrill” and said they were acting out of political self-interest. But most, he said, appreciate the Constitution’s guarantee of freedom of religion while feeling “badgered” by pain following the terrorist attacks on the World Trade Center.

READ MORE: Reaves, Fox Score 2 Each As Rangers Beat Maple Leafs

“I wanted to talk to them (the developers) about what was the possibility of just having a conversation about would there be a way to provide a center for people of the Muslim faith in lower Manhattan, and it not interfere with what is still a memorial to what happened on September 11,” Paterson said.

“I hope that the type of cultural understanding that they are trying to promote when they build this center could be practiced right now,” he added.

New York City Mayor Michael Bloomberg, who accompanied Paterson, again defended the project.

“I always believed that government should not be involved in deciding who you pray to, what you say or where you say it,” Bloomberg said.

MORE NEWS: Nets Hold Off Wizards' Rally, Beat Washington

(Copyright 2010 by The Associated Press. All Rights Reserved.)