I think this would have fit the whole idea of power being a result of perception theme better. Jon Arryn, Ned and Stanis could still have thought that they were actually Jamie's. The affair could have still been real. Hell, even Jamie could have thought that he was the father. The whole war could still have worked out in exactly the same way. But what if it later turned out that Jamie was infertile the whole time?