<commit_before># Adios Mendeley #

*Move from Mendeley to Zotero preserving notes, dates,
 folders/collections, and PDF annotations.*

I have used the code here (and code from
https://github.com/flinz/mendeley2zotero) to move from Mendeley to Zotero. In this moving
I haven't lost any of the Mendeley annotations for entries, the
annotations of Mendeley in the PDFs themselves, Mendeley's folder
structure (somewhat equivalent to Zotero's collections), or the date the
reference is added.

A few other files here allow me to automatically extract annotations and
highlights from PDFs (annotations and highlights in the PDF itself), as
well as prepare the BibTeX file for easy usage with Emacs and with a
tablet.

## Update (2018-06-29): READ THIS FIRST ##

Zotero has now a way to import directly from Mendeley. You most likely want to try that first (and second, and third, ... before playing around with these scripts, messing around with R, etc). These are two links:
https://www.zotero.org/support/kb/mendeley_import 
https://forums.zotero.org/discussion/72260/available-for-beta-testing-mendeley-import

(and, for the sake of historical records, you can look at https://github.com/rdiaz02/Adios_Mendeley/issues/4 where <NAME> alerted me to the issue, and the conversation that followed).

## Using it ##

1. Start R, make the needed changes in the first four variables defined in
**mend-to-bibtex.R**, and run it, paying attention to possible errors and
warnings. In particular:
	- Mendeley makes no provision to avoid repeated BibTeX keys. So go and fix
      them: the run will give you the titles of the files with repeated keys.

	- Mendeley will not export a BibTeX entry if there is no BibTeX
  key. You might have deleted the key, or if you set Mendeley to
  automatically create keys, no key will be created if there is no author
  and no title. Again, the program will tell you which entries have no
  key.

2. Once you have your new BibTeX file, before importing it into Zotero, open
it up in [JabRef](http://jabref.sourceforge.net/). Why? Because there can
be things that are not OK, like a keyword entry with newlines, or latex
code itself in the notes, and this will break the import into Zotero
(since this only happened to me in two entries, I did not change the code
to catch and fix these issues).

  If you first go through JabRef first, you will find the problem very
  quickly and easily (as easy as being told about the problematic line
  number). Fix any remaining issues.

3. When done, import into Zotero. Beware that imports of large bibliographies
can be slow, even more if you enable full PDF indexing ---I do the import
first without PDF indexing, by changing the settings to 0 pages and 0
characters, and only after I have my full bibliography in Zotero I enable
the indexing back.

  Beware that I've only imported into a Zotero that had Better BibTeX,
  [BBT](https://zotplus.github.io/better-bibtex/), as an extension. I am not
  sure, but this might add also some extra functionality in the import (it
  definitely does for the export of BibTeX and I find it absolutely
  essential).



4. Finally, when you import in Zotero, pay attention to the "not imported"
note you might see as one of your last references. In my case, two
references that imported fine in JabRef did not in Zotero (weird
characters in authors' names). You will see in the text of the note
something like "Better BibTeX could not import ...". Fix those entries if
they are just a few, and add them by themselves (possibly adding them to
their corresponding collections), or fix the entries in the complete
BibTeX and go back to step 3.

5. Fix the dates. My
[naive expectation](https://forums.zotero.org/discussion/50759) that the
timestamp field would be recognized does not work. But you can use Alex
Seeholzer's [mendeley2zotero](https://github.com/flinz/mendeley2zotero).
	1. Make a backup copy of your databases.
	2. Modify the script to have line `d_added =
	datetime.datetime.utcfromtimestamp(d_added)` be `d_added =
	datetime.datetime.utcfromtimestamp(d_added/1000)` (at least as of July
	2015 Mendeley is using time in miliseconds since 1970).  (This might have been fixed by
	the time you read this, since I filed an [issue](https://github.com/flinz/mendeley2zotero/issues/2)).
	3. Run the script with the `added_dates` option. You might get some warnings that, as far as I can
	tell, are inoquous.
	4. Copy the zot.sqlite as the new zotero.sqlite, start Zotero, and
       check.



Enjoy! Mendeley is now just a bad dream of days gone by. :-)


### Using it: what operating systems? ###

I've only used this with GNU Linux. I guess it should work in other Unixes
(Macs, for instance) but some things might not work with Windows directly,
and you'll need to modify the code.



## What and how: details  ##

I use both the BibTeX file exported from Mendeley and the sqlite
database. The BibTeX file that Mendeley exports is missing many things
that you probably want to keep, so I query the sqlite database,<commit_msg>Marked the bug in mendeley2zotero as fixed<commit_after># Adios Mendeley #

*Move from Mendeley to Zotero preserving notes, dates,
 folders/collections, and PDF annotations.*

I have used the code here (and code from
https://github.com/flinz/mendeley2zotero) to move from Mendeley to Zotero. In this moving
I haven't lost any of the Mendeley annotations for entries, the
annotations of Mendeley in the PDFs themselves, Mendeley's folder
structure (somewhat equivalent to Zotero's collections), or the date the
reference is added.

A few other files here allow me to automatically extract annotations and
highlights from PDFs (annotations and highlights in the PDF itself), as
well as prepare the BibTeX file for easy usage with Emacs and with a
tablet.

## Update (2018-06-29): READ THIS FIRST ##

Zotero has now a way to import directly from Mendeley. You most likely want to try that first (and second, and third, ... before playing around with these scripts, messing around with R, etc). These are two links:
https://www.zotero.org/support/kb/mendeley_import 
https://forums.zotero.org/discussion/72260/available-for-beta-testing-mendeley-import

(and, for the sake of historical records, you can look at https://github.com/rdiaz02/Adios_Mendeley/issues/4 where <NAME> alerted me to the issue, and the conversation that followed).

## Using it ##

1. Start R, make the needed changes in the first four variables defined in
**mend-to-bibtex.R**, and run it, paying attention to possible errors and
warnings. In particular:
	- Mendeley makes no provision to avoid repeated BibTeX keys. So go and fix
      them: the run will give you the titles of the files with repeated keys.

	- Mendeley will not export a BibTeX entry if there is no BibTeX
  key. You might have deleted the key, or if you set Mendeley to
  automatically create keys, no key will be created if there is no author
  and no title. Again, the program will tell you which entries have no
  key.

2. Once you have your new BibTeX file, before importing it into Zotero, open
it up in [JabRef](http://jabref.sourceforge.net/). Why? Because there can
be things that are not OK, like a keyword entry with newlines, or latex
code itself in the notes, and this will break the import into Zotero
(since this only happened to me in two entries, I did not change the code
to catch and fix these issues).

  If you first go through JabRef first, you will find the problem very
  quickly and easily (as easy as being told about the problematic line
  number). Fix any remaining issues.

3. When done, import into Zotero. Beware that imports of large bibliographies
can be slow, even more if you enable full PDF indexing ---I do the import
first without PDF indexing, by changing the settings to 0 pages and 0
characters, and only after I have my full bibliography in Zotero I enable
the indexing back.

  Beware that I've only imported into a Zotero that had Better BibTeX,
  [BBT](https://zotplus.github.io/better-bibtex/), as an extension. I am not
  sure, but this might add also some extra functionality in the import (it
  definitely does for the export of BibTeX and I find it absolutely
  essential).



4. Finally, when you import in Zotero, pay attention to the "not imported"
note you might see as one of your last references. In my case, two
references that imported fine in JabRef did not in Zotero (weird
characters in authors' names). You will see in the text of the note
something like "Better BibTeX could not import ...". Fix those entries if
they are just a few, and add them by themselves (possibly adding them to
their corresponding collections), or fix the entries in the complete
BibTeX and go back to step 3.

5. Fix the dates. My
[naive expectation](https://forums.zotero.org/discussion/50759) that the
timestamp field would be recognized does not work. But you can use Alex
Seeholzer's [mendeley2zotero](https://github.com/flinz/mendeley2zotero).
	1. Make a backup copy of your databases.
	2. ~Modify the script to have line `d_added =
	datetime.datetime.utcfromtimestamp(d_added)` be `d_added =
	datetime.datetime.utcfromtimestamp(d_added/1000)` (at least as of July
	2015 Mendeley is using time in miliseconds since 1970).~  (This was fixed by
	in the [issue](https://github.com/flinz/mendeley2zotero/issues/2)) I filed.~
	3. Run the script with the `added_dates` option. You might get some warnings that, as far as I can
	tell, are inoquous.
	4. Copy the zot.sqlite as the new zotero.sqlite, start Zotero, and
       check.



Enjoy! Mendeley is now just a bad dream of days gone by. :-)


### Using it: what operating systems? ###

I've only used this with GNU Linux. I guess it should work in other Unixes
(Macs, for instance) but some things might not work with Windows directly,
and you'll need to modify the code.



## What and how: details  ##

I use both the BibTeX file exported from Mendeley and the sqlite
database. The BibTeX file that Mendeley exports is missing many things
that you probably want to keep, so I query the sqlite database,
{'fraction_non_alphanumeric': 0.06055191582291046, 'fraction_numerical': 0.010025809013301568, 'mean_word_length': 4.0375, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 16, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2027375', 'n_tokens_mistral': 2998, 'n_tokens_neox': 2797, 'n_words': 1544}