Bacteria in your filter is the same growing on the tank walls, substrate, decore etc. If your worried about removing too much bacteria at one time, take your old filter pad/media and place it somewhere in the tank. Make your mods, get it running. Wait a week remove old filter pad/media. Little ugly for a week but stress free