duplicates not showing up

As you can see from the attached image, I’ve got quite a few duplicates in my library. But when I ask EndNote to search for duplicates, it doesn’t turn any up. Is there any reason? And is there any way to fix the duplicate entries other than one by one? I have over 8000 entries, so I’m not looking forward to that. Any advice would be appreciated!

Did you preselect “All References” (see attached image) before using the Find Duplicates function? This enables searching through all your records. In the Endnote library.

all references setting.gif

I did! It doesn’t make a difference.

Could you check the “Duplicates” settings in Endnote Preferences? (Go to the Endnote toolbar and select Edit >Preferences. In the Endnote Preferences pop-up dialog box click Duplicates.)

  1. Endnote is usually set to look for duplicates based on the Author, Year, and Title fields so confirm that the 3 boxes have been checked (refer to attached image). Note: You might consider deactivating Author if you’ve been accumulating references from multiple online databases such that some references may include middle names, initials, or other variations of the author names, which in turn would affect search results if incorporating the Author field.

  2. Verify that the Criteria is set to “Ignore spacing and punctuation”. (Selecting “Exact match” may impede the search results.)

  3. Also, to minimize duplicates when downloading from online databases you can also check the “Automatically discard duplicates” option.

Click the OK button to exit Endnote Preferences and try running the Find Duplicates command. Refer to attached image showing results in finding duplicates. For further assistance refer to the Endnote training video “Finding Duplicates”.

Note that when duplicates are found , you can either choose to a) review each pair of duplicates or b) or delete the duplicates by dragging them to the trash. If you have been using this library for generating citations and references in assorted papers. note that deleting the duplicates may be a problem . Since Endnote assigns a unique record number to each reference as it’s added to the library you may accidentally delete records that were used in prior papers – and without those records the links for the in-text citations and bibliography will be broken. You can either choose to check the record numbers before deleting or save the old library and generate a new Endnote library (without the duplicate records) and use the new library for subsequent papers.


As CG points out, endnote looks for exact duplicates in all three fields, and if you have entered by hand or from different databases, then minute differences in one or the other field will not flag them as duplicates.  For example one of the duplicates you show has a middle initial while the other one doesn’t .  When confronted with things like this, I switch to using other fields to identify duplicates and review them one by one.  So once, I will do it without considering the authors, clean those up, then I will repeat, changing the preferences to match author but not title, or try author or title with year and use page numbers.  (removing the author from the set is nicely explained in the tutorial that CG gave a link to.)

 

 

The preference to “automatically discard duplicates” only works if you are working in the combined library+search mode, which automatically imports any search of online database results into the library, which is rarely what I want (albeit without the duplicates).  I am more much selective on what I will copy into my permanent library. Also,  it probably won’t screen out duplicates that aren’t exact either?  That setting has no effect on copying records from the online mode into your library, anyway.  

 

To get around the duplicate problem when importing large numbers of references, I import them into a new temporary library and then import those into my main library with discard duplicates on.  But I do that with the Year, Author and Title options selected, as I am usually importing from the same database.  But if I have been OCD, and put corrected diacritics in author names or italicized gene names as they appear in the print version, then the duplicate search screening doesn’t work either.  

 

Thanks to you both! I managed, through checking off various combinations of fields, to locate most duplicates. I’ll try the temporary library solution next time!