The workers, escorted at first by police, started work at the Cumberland Avenue site last week.
The landowners had not complied with an official tree replacement notice, so workers representing Argyll and Bute Council moved onto the site to implement the notice directly.
The trees were initially felled nearly three years ago, and the council imposed a notice requiring them to be replanted; this was upheld by a Scottish Government Reporter, although the number of trees was reduced to 28 – this should have been implemented five months ago.
An enforcement notice applying to a fence on the site has been withdrawn by the council, while a retrospective planning application is considered.
Meanwhile building materials which have been stored on the site have advertised for sale here.
Helensburgh Community Woodlands group, which has been campaigning for the restoration of the site, is holding its AGM on Monday at Helensburgh Tennis Club from 7pm to 9.30pm.
Archaeologist Fiona Baker will deliver an illustrated talk about Ardencaple Castle before subjects including the Cumberland Avenue site, meeting with planners and the Local Development Plan are considered.