Chicago 15th A style depends on footnotes and not citations. Hence, you need to use Word to insert a footnote and then insert the endnote reference in the footnote. It then produces a bibliography at the end. Inserting a “citation” in the text will just produce what you are seeing, the inclusion of the reference in the bibliography at then end.
Chicago 15th B style has in text (Author, Year) citations and isn’t numbered. Thus if you look at the “citation” template, you will see it is empty in the 15th A style and has (Author Year|, Cited Pages|).
If you want a numbered bibliography list, inserted as an in text citation, numbered in the order of use (and numbered in the bibliography), you should probably choose a different output style. Try the default “numbered” style to see if it gives you the solution you are looking for?