Very large Endnote library - how can we fix this?

I have been using Endnote for a number of years, presently I like X9, and my current build is X9.3.3.(Bld 13966) it’s a license through my job Previously I used X7 at a university, and have used previous versions to a lesser extent.

I have a lot of references (who doesn’t?) I know libraries can be compressed, but what I’ve seen is this is almost exactly like WinRAR or an archiving tool good for compressing and moving large libraries. what concerns me more is using my existing library. With the ongoing memory leaks, and misbehaviours of Windows software in general, this would be a nice fix for Endnote, which has existed as long as I have used it.

When adding attachments, and creating new citations, the libary grows in size. I add a lot of research notes and other things (byte-sized data) and there is the occasional large PDF file like this 299 page sodium battery paper I have. The library grows in size. In my last job, I had just over 1,000 refererences in my library, and the file size was something like 3.5GB. Things slow down a lot, especially in windows 10 and up.

I had this idea (it was a slow day) I created a new library, blank. I created groups and folders identical to my really large library. I copied/pasted all of my references to the new library. The new library is 1.1GB and has exactly the same content.

I am thinking there is a lot of junk and a lot of meta data that accumulates when tossing references in the bin, and when I move a file (or reference or both) from one folder to another, it creates a new file, and deletes the old one. I’ve used computers a long time, I know when the computer moves a file it first copies the original, pastes it then deletes the original. One can compare this to the transporter on Star Trek, the original is now gone, it’s not “moved” in the sense we think it is. This large library file size is again raising it’s head to me as my new library has breached 1,000 citations (new job) and has grown beyond 1.6 GB in size, I’m starting to feel the drag.

Is there some plan in the future to “repair” or defrag the End Note library? I will in time do what I did before, and just create a new (and smaller) library. It took about 2 hours last time, but it’s annoying to have to do this. It would be nice if software were packed tight like in the old days instead of sprawling into any available spaces.

I’m just putting this out there.

4 Likes

Damned good idea! A library “defrag”

This sounded very interesting and I tested this on my 3500 refs (and many pdfs) library. I did skip the groups part as the few groups I have in my main library weren’t important. The original Data-folder was 6,7 GB. The new copy was 6.5 GB. So in my case the copying into a new library did not shrink the size very much. Has anyone else tested the “defrag” method?

1 Like

Interesting. I am at 4.0GB again (1098 references). I did a cleanout of duplicate files, and saw the library size grow a little bit. When I move references around within a library, I also see very small increases in size.

In your case, I think maybe you are neater than I am. :slight_smile:

I have my own “system” where I have many subfolders, and this could be where the metadata and junk accumulation begins. I’m not an expert on filing system, but there are small amount of data associated with that in an operating system. If one goes from Windows to Linux for example, the number of files and folders in Linux can be overwhelming. This isn’t a problem for Linux because it just works differently than windows. I figure the container that is the Endnote library has it’s own filing system it uses, and it gets more inefficient when there are more groups and folders? One possibility.

I re-discovered my old post this morning because my library has once more grown beyond my liking. My new 12 processor laptop doesn’t struggle, so maybe I should leave it be for now.