NEWARK, N.J. (CBSNewYork) — New Jersey Governor-elect Phil Murphy plans to legalize marijuana within his first 100 days in office, but many who supported his campaign have urged him to step away from the controversial initiative.

On Monday, Newark Bishop Jethro James invited Murphy to his Paradise Baptist Church in Newark to celebrate Martin Luther King Day and advise him publically not to legalize pot for the sake of the state’s most vulnerable.

“You legalize marijuana, you’re going to gut the best and brightest in many neighborhoods,” James said.

The congregation got on its feet to cheer as Bishop James called Murphy’s plan a terrible policy that would only bring more drugs into Newark’s neighborhoods while big marijuana profits.

“Everything can’t be about money,” James said. “There’s got to be some morality.”

Murphy estimates legalizing weed will raise millions of dollars in revenue for the state. While listening to the bishop on Monday he mostly kept his head down. CBS2’s Meg Baker caught up with the governor-elect afterwards.

“I can understand some who look at this as something where they have to get their minds around and get used to,” he said.

James says we need to learn from past mistakes, pointing to the opioid crisis now affecting every corner of the state.

“Nobody cares about opioids until it got on a bus from Trenton and stopped off in Princeton,” he said. “Nobody cared about opioids, now it’s an epidemic but it’s been in our neighborhoods for too long.”

Murphy did his best to assuage concerns, insisting the state will go through with it and “do this right and protect our kids.”

After Colorado legalized marijuana, statistics show the number of young Hispanics and African Americans arrested for marijuana related offenses went up 58 percent. Despite this, Murphy insists just the opposite will happen.

The governor-elect officially steps into office on Tuesday, and expects the state will gain $300 million from legalizing and taxing marijuana.