Why?
The RootsMagic Source List window has an AutoMerge function that can merge Master Sources that are identical in every respect, including the pointer to the Source Template and the name of the Master Source. Thus it fails to merge sources that are practically identical and that we would wish were merged. Source Templates – Merge Duplicates addresses the first barrier for Source AutoMerge. The script below offers an external Source AutoMerge that does not mind if:
- the Master Source names are different
- there are white space differences around or within Source Text or Comments but the non-blank texts are otherwise identical
It is more forgiving than the built-in Source AutoMerge yet does not corrupt the core properties of the final, common Master Sources.
Practically Identical Master Sources
Under what conditions can such practically identical sources arise that RootsMagic’s Source AutoMerge does not recognise them?
- Sources added to a person within FamilySearch Family Tree have their name prefixed by the person’s name. When imported into RootsMagic via its Share Data utility, that becomes the Master Source name. Thus, the same source assigned to two different persons will have different Master Source names.
- Sources that have outer white space following the Master Source Text or Comments lose that on export or transfer via drag’n’drop. Should these sources return to the database through collaboration with another party, SmartMerge will merge the persons but not the sources because of this differing white space.
- The Ultimate Splitting tool described on Sources – Adventures in Extreme Splitting adds person name(s) and event to the name of the split Master Source, which, in many instances, will be identical to other Master Sources.split from a common lumpy Master Source.
Outcomes
An example of a small database, starting with lumpy sources:
| State | Sources | Citations |
|---|---|---|
| Original | 13 | 58 |
| after Ultimate Split | 71 | 116 |
| after Delete Lumpies After Split | 58 | 58 |
| after RM Source AutoMerge | 58 | 58 |
| after SQLite Sources – Merge Duplicates | 28 | 58 |
Download
Sources-MergeDuplicates.sql Rev, 2013-08-08 now ignores differences in white space within Source Text and Comments
Usage
- Make a copy of your database on which you can run these procedures so that you can return to the original if you are not satisfied with the results.
- Open the database in your SQLite manager with a RMNOCASE extension.
- REINDEX the database using SQLite.
- Sources-MergeDuplicates.sql does require that the sources use the same Source Template and so Source Templates – Merge Duplicates should be run first if there is any thought that there may be duplicate source templates.
- It is also advisable to run the SQLite Delete Phantoms to clean up the database for faster execution of the following script but this may be more aggressive than you might like.
- Execute Sources-MergeDuplicates.sql to merge the practically identical sources; this is a very intensive series of SQL statements and may be rather slow.
- In RootsMagic, run File > Database tools > Rebuild Indexes on this database; not needed if its “Test database integrity” reports OK.