Well, your loading down the on chip memory controller with more with capacitance and more current draw.
Still, I don't think I'll be running out to buy 2x8GB DDR3 sticks to replace the 4x4GB setup. Heck, I'm not even sure if my P67 board supports them.