I don’t think there is a way, other than making a final version of the document and removing the endnote fields and selecting it and changing the final bibliography font size. You can try to do it via the style menu (“Endnote Bibliography” is a style created by endnote) but it doesn’t “stick” when you update the citations and bibliography.