A blog post on how to remove duplicate address book entries using sed, uniq and sort. It suggests converting all vcard entries to single lines, using sort then uniq and finally converting back. It removes absolute duplicates.
This is a simple perl script which *very crudely* removes duplicates in Evolution data. It deletes any contacts with the same name.