I totally know what you're talking about! Such a cool concept for a show. Been years since I saw it though, no clue where to even begin looking for it now. Maybe try hitting up some of the retro shops that stock old series on tape? Long shot, but someone …

Honestly, your description immediately made me think of 'The Herbs'. I know it's not super spooky, but the puppets were kinda quirky and the show is definitely old enough to fit your timeframe.

If you’re looking for a show with puppets and a dragon, Eureeka's Castle is most likely what you’re after. The dragon’s name was Magellan and he’d often bump into things with his tail, causing chaos. What a classic!