Janavi Kramer had just enjoyed a swim with friends and was sitting on a bench taking in a wonderful Cornish view – headlands plunging into deep water, bobbing boats, a bustling harbour town.

“We’re lucky to have this space,” said Kramer, who is an artist and scientific illustrator. “We use it for work, recreation and pleasure and it’s so scary to think that this could all be harmed. I worry about what we may be living with soon and what we’ll be leaving for future generations.”

He is talking about a £150m plan to redevelop Falmouth Docks in south-west Cornwall. It includes expanding the docks to allow larger cruise ships to berth, which many people worry will exacerbate environmental problems and put pressure on the fabric of the town.

Hannah Cocks, a marine social scientist, who had been taking a dip with Kramer, said she wasn’t opposed to redevelopment of the docks, which have been here for more than 150 years.

“But I want it to be done in a way that is environmentally and socially just,” she said. “The community has not been adequately consulted about a decision that could have such adverse effects.

“Even now Cornish infrastructure can’t cope with the number of people who come in the summer. Do we really need these huge cruise ships here?”

The plans also involves dredging around the docks, which some fear could lead to harmful material being churned up from the seabed, such as heavy metals washed into the harbour when Cornwall was a mining superpower and toxic paints once used to protect boat hulls.

Fishers and environmentalists are warning that precious habitats could be damaged, including oyster grounds and rare maerl beds – created when rare types of red seaweeds grow into hard, twig-like lumps.

Gio Reale, a wildlife storyteller, said it had been a good summer for spotting marine wildlife. “I normally go out looking for sharks. We have seen small-spotted catsharks and nursehounds, quite a few octopuses and a lot of the usual lobsters and pipefish.

“We have maerl beds, seagrass meadows, kelp here. It’s a beautiful but fragile ecosystem and I’m concerned it could be really badly harmed.”

The UK government is supporting the redevelopment. Falmouth already hosts cruises, but ministers say the expansion will allow larger vessels – such as Excellence Class cruise ships, which are up to 345 metres (1,131ft) long, to berth.

This could help to attract up to 200,000 cruise visitors annually by 2030, it says, injecting millions of pounds into the local economy and supporting hundreds of tourism-related jobs.

Cornwall council will debate elements of the scheme in the coming weeks but much of it is going through the planning process under the government’s fast track system, designed to push complicated developments through and cut red tape.

It means developers work with a single, lead environmental body – in this case the Marine Management Organisation (MMO) – rather than what the UK government calls a “merry-go-round of overlapping regulators”.

But many in the town worry that fast-tracking may lead to a lack of scrutiny.

Jem Wallis, who runs a sauna on the waterfront, said he didn’t feel a clear case had been made for the project. And he had never had a cruise passenger visit his sauna. “It all seems a massive risk to me.”

Zoe Young, a tour guide, added: “A terminal for stinky great ships to meet stinky coaches to Land’s End – is that the best we can hope for?”

When the Guardian visited John Spargo’s souvenir shop Tall Ships Trading on Church Street, a passionate debate about the pros and cons broke out. Spargo said most cruise passengers who arrive in the town were taken by coach to other attractions such as the Eden Project. “They don’t bring as much money into the town itself as you think,” he said.

Bernie, a visitor from Kent, said she thought cruise ships were dreadful. “Other places in Europe are trying to get rid of them. Why do you want bigger ones? It’s not worth losing the character of this place for cruise ships.”

Susie, who lives in Falmouth, disagreed. “They do bring in money. They are good for many of the shops and the boat tours do well when they come. St Mawes Ferry is always packed when a cruise ships is here. The passengers say they don’t feel they are on a real boat when they’re on a cruise ship so love getting on a little one.”

The government has said the fast-track system does not mean corners are being cut. It said the MMO was reviewing information submitted during the consultation period to support a “robust” marine licensing decision.

A spokesperson said: “The lead environmental regulator model is about ensuring better coordination between developers and environmental regulators to prevent delays in the planning process without lowering environmental standards.”

Balaena, the maritime engineering and shipbuilding group that operates the docks, said the project would ensure the port remained fit for purpose, safeguard skilled jobs, create apprenticeship opportunities and support businesses across Cornwall and the wider south-west.

It said “targeted pocket dredging” was needed around the dock, which had been subject to a comprehensive environmental impact assessment. There would be no dredging in the main docks channel or wider estuary and material that is not suitable for disposal at sea would be taken to a facility inland.

Jason Pascoe, who has worked the oyster fishery for 40 years, was unconvinced. “The river systems here are very complex. Can they be sure if they dump waste out to sea it won’t find its way back here? Ironically, we’ve just had our best ever season. This feels like a horrendous threat to me.”