Nick Padwick hunches over a microscope, examining a sample of compost he has made on his Norfolk farm. “Look at that bad boy! That’s a bacteria-feeding nematode!” he exclaims. “Stunning fungal hyphae.”

Padwick, the farm manager at Wild Ken Hill since 2018, is part of a growing movement of farmers taking a deep interest in the microscopic life forms upon which their livelihoods depend. Under this approach to regenerative farming, nurturing diverse soil communities – from bacteria and fungi to microscopic animals and worms – is seen as an essential prerequisite for growing healthy foods with minimal or no use of agrochemicals or soil-damaging machinery.

For Padwick, 59, this marks a dramatic shift after nearly four decades in conventional agriculture, the very systems that experts now blame for devastating soils worldwide. “I really have been a part of it,” he admits. “I cringe every time I think of it.”

The stakes could not be higher. Recent estimates say more than 60% of EU agricultural soils are degraded, with about 40% of UK soils similarly damaged. Globally, the UN Food and Agriculture Organization says 90% of the world’s topsoils could be at risk by 2050, a crisis intensified by accelerating global heating.

All this has implications for not just food security but biodiversity, water quality, flood mitigation, climate resilience and greenhouse gas emissions. As Prof Richard Bardgett, a Lancaster University soil ecologist and author, puts it: “Few things matter more to humans than their relationship with the soil.”

A key inspiration for Padwick’s mission to revive his land was Dr Elaine Ingham, a soil microbiologist. Through her Soil Food Web School (SFWS), he learned that a fully functional soil food web is needed to build structure, nourish plants and fight pests naturally. That web includes a vast array of bacteria, fungi, protists, and tiny animals such as nematodes and springtails.

Ingham’s approach is based on three key actions:

1. Use microscopy to identify missing or imbalanced soil organisms.

2. Create nutrient-rich compost from farm waste, such as straw and wood chips.

3. Put this compost in mesh bags and steep them in water, like giant teabags, to make extracts that can reintroduce beneficial microbes to depleted soils.

Farmers must then adopt management practices that protect these newly introduced organisms, allowing them to thrive and multiply.

Scientists broadly accept the value of diverse soil ecosystems. “Having worked on this topic for decades,” says Bardgett, “I would say a more complex soil food web can only be a good thing for plant growth and resilience.”

His scepticism, however, centres not on the principle but the practice; specifically, whether organisms from dilute compost extracts actually survive and thrive when introduced to new soils.

Prof Duncan Cameron, a soil biologist at the University of Manchester, said he shared those doubts initially. His Manchester colleague, Dr Janice Lake, an independent research fellow, has been testing the hypothesis that beneficial soil ecosystems can be transplanted from compost extracts into soils. She partnered with Daniel Tyrkiel, an SFWS graduate who founded the Soil Ecology Lab in Hampshire, which sells compost products and soil analysis and consultancy services to a growing network of regenerative farmers.

Lake’s recent preliminary greenhouse tests yielded promising results, not yet published. Barley and wheat treated with Tyrkiel’s compost extracts outperformed control crops. Field-scale trials this year will measure not just yields but several soil health indicators.

Bardgett, however, warns against viewing compost extracts as a quick fix. “My own view would be that it’s better to change the soil environment first, to make it more amenable for the organisms already in the soil to actually flourish,” he says. Otherwise, “most [of them] will simply die.”

Padwick has embraced this holistic approach. While he uses compost extracts, he recognises they are just one element in a comprehensive system overhaul. He minimises mechanical cultivation, which disrupts soil fungi and larger organisms; plants diverse cover crops to nourish soil biology year-round; and maintains unusually wide field margins for biodiversity. These wild areas harbour beneficial predators that control agricultural pests. Recent evidence suggests they can also help sustain vital mycorrhizal fungi networks.

A critical challenge for many farms transitioning to regenerative farming is that yields can drop sharply, at least in the short term. After eliminating chemical inputs, Padwick’s wheat production plunged from more than 7 tonnes per hectare to 1.5 tonnes/ha, before rising again to more than 3 tonnes/ha. Yet his business remains profitable because his input costs have been slashed and he sells premium products under the Wildfarmed brand, launched by the Groove Armada musician turned farmer, Andy Cato. And he is convinced yields will continue to rise as soil health improves: “It’s going to be another five years before we really start seeing how things can change.”

Perhaps the strongest evidence of the soil food web approach gaining traction is its adoption by G’s Fresh Ltd, an intensive horticulture company headquartered in Cambridgeshire. “We are set on transitioning to regenerative farming practices entirely across the organisation by 2030,” says the product director, Julius Joel.

G’s Fresh produces 1.2bn packs of salad annually, supplying two-thirds of the UK’s lettuce and celery demand as well as supermarkets across Europe. Its retail contracts demand consistently high-quality crops with precise delivery schedules. Yield reductions are not an option.

“We want to have our cake and eat it,” Joel says. “We’re aiming to improve resilience, enhance water infiltration, cut input costs, reduce emissions and grow profits. We think regenerative agriculture is going to do positive things in all these areas.”

The soil food web approach has even reached large-scale dairy production. Yeo Valley, the UK’s largest organic dairy business, has incorporated key principles into its established system. The Soil Ecology Lab has undertaken microscopy and soil analysis, andseeds have been treated with compost extracts before drilling.

“Putting biology and food right in the seed bed … it has really helped us with the establishment of roots,” says Will Mayor, the farms development manager. “That’s got to be a benefit, and particularly when you’re going into a tough season like this one … it can cope with weather extremes.”

Although, he notes: “You can’t reinvent agriculture. There is science to it. You have to fertilise [the soil], you have to make sure it’s healthy, and you have to suppress weeds in order to grow crops.”

This tension between cutting-edge practice and scientific validation is something Bardgett recognises. Many regenerative farming practitioners are “racing ahead of the actual evidence base”, he says. “In some ways this is a good thing, because farmers are actually just going ahead and doing stuff.” He emphasises that scientific research must catch up in order to inform policies that can effectively promote soil health.

Cameron goes further. “The government needs to actually wake up and realise that … the money that they’ve thrown at [improving soil health] is not even a sticking plaster.” Rather than being seen as a tangential problem, he believes boosting soil health is “the absolute front and centre of battling climate change”.

“This needs to be seen as a national security issue,” he says, “every bit as much as dealing with Putin.”