skip empty folders in copy

Hello, 

I'm copying a folder with lots of subfolders and sub-subfolders.    I only want to copy folders with visio files.    (and copy the visio files as well)    I've got the filter set so that item name must contain "vsd" which gets me just the visio files.     However all the folders copy over most of them being empty.     Can I exclude copying empty folders somehow?

Thanks,

Brad