Banshees will spawn over bones, during nighttime.
If you see a dim light in the darkness, or a woman crying, you better walk back: banshees kill on sight.
If you hear her scream then you are dead.
During daytime they will disappear, save where light can't reach them.
If you want them to be removed regardless of the light level (e.g. underground), you can do it through
Settings>Advanced Settings>Mods>mobs_banshee