Death Year Mismatch #date #death

Lists individuals whose Death Year is missing from RootsMagic Explorer (and other views and reports where just the YEAR is outputted) or mismatches the value that has been stored in the Date field of the Death fact. Adapted from the query Birth Year Mismatch.

New: see Rebuild Indexes and Update Birth and Death Years for a batch procedure to match up the Birth and Death years with their facts.

Download: DeathYearMisMatch.sqlRMtrix_tiny_check.png

Listing:

-- DeathYearMisMatch.sql
-- Lists individuals whose Death Year is missing from the NameTable (and thus
-- missing from views and reports where just the YEAR is outputted) or
-- mismatches the value that has been stored in the Date field of the Death fact.
-- #1 2010-06-03 created by ve3meo, based on BirthYearMismatch.sql
-- #2 2012-01-27 corrected overcount of Death fact due to alternate names
 
SELECT
  RIN ,
  Surname ,
  Suffix ,
  Prefix ,
  Given AS 'Given Name(s)',
  DeathYear AS 'Death Year',
  DATE AS 'Death Fact Date',
  IsPrimary AS 'Primary?',
  DeathCount AS 'Death Facts'
FROM
  (
    SELECT
      N.Ownerid AS Rin ,
      N.Surname COLLATE Nocase ,
      N.Suffix COLLATE Nocase ,
      N.Prefix COLLATE Nocase ,
      N.Given COLLATE Nocase ,
      N.Deathyear ,
      E.DATE ,
      E.Isprimary ,
      LENGTH( N.Deathyear ) AS Bystrlen , -- Death year string length
      COUNT( 1 ) AS Deathcount            -- count up multiple Death facts
    FROM
      Nametable N ,
      Eventtable E
    WHERE
      N.Ownerid = E.Ownerid AND E.Eventtype = 2 AND E.Ownertype = 0 AND +N.IsPrimary
    GROUP BY
      1
  )
WHERE
  NOT LIKE( Deathyear , Substr( DATE , 8 - Bystrlen , Bystrlen ) )   -- the mis-match test,
  -- cannot find Death Year where it should be in the Date field
  AND DATE NOT LIKE '.' -- no date entered
  AND DATE NOT LIKE 'TUNKNOWN' -- example of suppressing a not bothersome Date value
ORDER BY
  RIN ;

Birth Year Mis-Match #date #birth

Lists individuals whose Birth Year is missing from the sidebar (and other views and reports where just the YEAR is outputted) or mismatches the value that has been stored in the Date field of the Birth fact. This is less of an issue as of RM 5.0.2.0, which incorporates an update of the BirthYear and DeathYear fields in the NameTable under the Rebuild Indexes tool in the menu File > Database Tools. However, the utility is still useful in drawing attention to where there are multiple Birth facts and where more than one is marked Primary.

New: see Rebuild Indexes and Update Birth and Death Years for a batch procedure to match up the Birth and Death years with their facts.
Download latest: BirthYearMisMatch.sqlRMtrix_tiny_check.png
Ver 4 corrects an overcount of Birth facts due to alternate names.
Ver 3 reports number of Birth facts per individual and suppresses facts with ‘UNKNOWN’ in the Date field.
Ver 2 added checking for years before 1000.

SQLiteDeveloperSQLeditor-BirthYearMisMatch.png
SQLite Developer screen shot from Ver 1.

This sql query uses some sqlite core functions to measure string length, extract a sub-string and compare to another string.

  1. In the above screenshot from the original version of the query, all the Birth Years shown are 0. The Date field could be worked on in RM4’s Edit Person dialog to change from free text (stored with a ‘T’ prefix) to a recognisable date format (stored with a ‘D’ prefix) and that should update the Birth Year.
  2. In the case of multiple Birth Facts for an individual, it is the last one saved that sets the Birth Year; any other Birth Fact not matching the Birth Year will be detected. The query could be modified to suppress reporting a mis-matched Birth Fact when another for that individual does match but it was thought to be useful to draw attention to the mis-match for review and possible correction or deletion.
  3. For RIN 681, a valid date format is stored for Jan 17 in the year 0! The mismatch that was detected compared ‘0’ to ‘0000’. The original version of the query reported a mismatch for all years <1000; later versions do not.

RMGC_Properties – Query #database

Description

Lists the properties of a RootsMagic 4 or 5 database except for dates, plus reports extra detail characterizing the database and identifying possible problems in it or in transfers to another other RootsMagic database via GEDCOM or drag’n’drop. Most SQLite managers can export the results to a .csv spreadsheet for tracking and comparison. Could be useful to compare before and after any major operations on the database.

Now has a parameter input dialog to select Summary level (matching RM’s File > Properties report) or default to the Detail level as in the sample below. Summary level works (again and only) with SQLite Developer except for some late (but not the latest) upgrades which require running the last Select after the query first runs. All others skip the parameter input and produce the Detail report. Does not yet report on the RM5 Research Manager feature.

New 2012-03-01: discovered and added class of Headless Citations for Alternate Names mispointing to a Primary Name
New 2012-02-20: corrected Orphaned Alternate Names; adapted Media Date+Description test for RM4/5 differences
Rev 2010-02-13: – Duplicate Citations
Rev 2010-07-05: – Duplicate Citations for families overstated due non-discrimination between different events – corrected.
Rev 2010-07-06: – Duplicate Citations for Individuals erroneously pointed to some family facts – corrected.
Rev 2010-07-14 – added Witness & Role Properties, using a new, unmodified blank database for detection of changes to role names and sentences.
N.B.: note the comment in the sql file about the requirement for a new, empty, unmodified database file for comparison to the file under test.
Rev 2010-08-09 – added Places and Place Details: Used, having Geo-coordinates; added FactType and SourceTemplate properties
Rev 2010-08-10 – added Version property as recorded in ConfigTable
Rev 2010-12-13 – added Multimedia duplicate filenames and Multimedia duplicate links properties

RMGC_Properties.sqlRMtrix_tiny_check.png
Also download or create a reference empty database and revise the path in the script to match its location on your system: Empty5021.rmgc

Sample Output

ValueVariableRemark
5008Versionvs Control version 5008
1157Peopleall records in PersonTable
0– Nameless Peopleno record in NameTable for that RIN
97– Unresolved Duplicate Name Pairspairs of Given and Surnames, not flagged as “Not a Problem”
16– Resolved* Duplicate Name Pairsflagged as “Not a Problem” – flags lost on transfer
2– Unresolved Duplicates with Media Linkssecondary persons’ links lost on merge
39Alternate namesall records in NameTable where IsPrimary=0
0– Orphaned Alternate names*no Primary name record found
389Familiesall records in FamilyTable
66Fact Typesno. of records from FactTypeTable
2– Custom Fact Typesno. of custom Fact Types
9– Customised Built-in Fact Typesno. of built-in Fact Types modified
35– Unused Fact Typesno. of Fact Types not used
0– Blank Fact Type NamesFactTypes must be named
0– Blank FactType SentencesFactType needing definition
2831Eventsall records of EventTable
0– Orphaned Eventsevents for which no person or family match in respective tables
135– Event WitnessesAll records in WitnessTable of persons sharing events
17— Nominal Witnessesnot Persons from database, but named in WitnessTable as sharing an event
0— Headless WitnessesPersonID (RIN) in WitnessTable missing from PersonTable
0— Witnesses to Lost EventsEventID in WitnessTable cannot be found in EventTable
0— Witnesses with blank Roleno role has been assigned from RoleTable or the RoleTable role is empty
3— Witnesses with Custom Sentencea custom sentence has been assigned, unique to this witness
27— Witnesses with Notea note has been entered for this witness to an event
81— Rolesno. of records from RoleTable
23– Custom Rolesno. of custom roles
1– Customised Built-in Rolesno. of built-in roles modified
54– Unused Rolesno. of roles not used
0– Blank Role NamesRoles needing definition
2– Blank Role SentencesRoles needing definition
775Total Placesall records in PlaceTable incl Places and Place Details (Sites)
146– System Placessystem supplied Places: LDS Temples
470– User Placesuser defined Places excl Sites
296— Used, having Geo-coordinatesnon-empty Lat or Long
27— Unused User Places*not used by EventTable, will be dropped in a transfer
159— User Place Detailsuser defined Sites
9– Used, having Place Detail Notes*Site Notes will be lost in a transfer
30– Used, having Geo-coordinatesnon-empty Lat or Long
1– Unused Place Details*Sites will be lost in a transfer
829Source Templates
  1. of records from SourceTemplateTable
416– Custom Source Templates
  1. of custom Source Templates
0– Unupdated Built-in SourceTemplates
  1. not matching reference database
393– Unused Custom SourceTemplates*lost on transfer
0– Incomplete Source Templatesmissing part of definition
139Total Sourcesall records from SourceTable
2– Unused Sources*SourceTable records unused by CitationTable
1697Total Citationsall records from CitationTable
0– Duplicate Citationsidentical in most respects, cluttering reports
0– Sourceless Citations*no SourceTable record for this CitationTable record
10– Headless Citations*CitationTable records for which no Person, Event, Family, AltName found; cleaned on transfer
12Repositoriesall records from AddressTable of type Repository
36To- do tasksall records from ResearchTable
123Multimedia itemsall records from MultimediaTable
12– lacking thumbnailprobably an imported reference to an image file that has yet to be found
0– duplicate multimedia filenamesprobably having different paths
158Multimedia linksall records from MediaLinkTable
0– with Date & Description*(RM4) if a record has both, the Description is lost in a transfer
10– with Date & Description*(RM5) if a record has both, the Description is lost in a transfer
5– duplicate multimedia linksimage appears multiple times for person, fact
20Addressesall records from AddressTable of type Address
0– blank namesName field of AddressTable record is blank
0Correspondenceall records from ResearchTable of type Correspondence
  • NOT TRANSFERABLE
via GEDCOM or Drag&Drop to another RM database

Discussions & comments from Wikispaces site


ve3meo

Tracing the “Headless Witnesses”

ve3meo
07 December 2011 17:14:19

Following Don Newcomb’s post at RootsMagic Forum, let’s look for a solution.

RMGC_Properties query reports a number of Headless citations. The core of that query gives a list of the RINs that were once witness to some event but are no longer persons in the database. Reviewing an earlier backup of the database may reveal these persons.
-- List of RINs of Headless Witnesses
SELECT PersonID FROM WitnessTable WHERE WitnessTable.PersonID > 0
EXCEPT SELECT PersonID FROM PersonTable
;

A more complex query could look up the persons in the database currently owning (the Principal role) or sharing the event.


ve3meo

Database Properties List – Query #databaseproperties

Problem

Currently under RootsMagic 4, the only way to obtain the Database Properties for a file is to go to File > Properties and use Alt+Prnt Screen (or Ctrl+Prnt Scrn) to copy the screen to the clipboard and use CTRL+V to paste the contents to a program capable of rendering the result.

Solution pt 1

In the meantime, the following code can be used to set up and populate a temporary table (outside of one’s database): (the full solution is integrated in RMtrix_tiny_check.png)

CREATE TEMP TABLE VariableTable
(
Variable TEXT,
VALUE INTEGER
);
 
INSERT INTO VariableTable
SELECT 'People', COUNT(1)
FROM PersonTable;
 
INSERT INTO VariableTable
SELECT 'Families', COUNT(1)
FROM FamilyTable;
 
INSERT INTO VariableTable
SELECT 'Events', COUNT(1)
FROM EventTable;
 
INSERT INTO VariableTable
SELECT 'Alternate names', COUNT(1)
FROM NameTable
WHERE IsPrimary = 0;
 
INSERT INTO VariableTable
SELECT 'Places', COUNT(1)
FROM PlaceTable
WHERE PlaceType = 0;
 
INSERT INTO VariableTable
SELECT 'Sources', COUNT(1)
FROM SourceTable;
 
INSERT INTO VariableTable
SELECT 'Citations', COUNT(1)
FROM CitationTable;
 
INSERT INTO VariableTable
SELECT 'Repositories', COUNT(1)
FROM AddressTable
WHERE AddressType = 1;
 
INSERT INTO VariableTable
SELECT 'To-do tasks', COUNT(1)
FROM ResearchTable
WHERE TaskType = 0;
 
INSERT INTO VariableTable
SELECT 'Multimedia items', COUNT(1)
FROM MultimediaTable;
 
INSERT INTO VariableTable
SELECT 'Multimedia links', COUNT(1)
FROM MediaLinkTable;
 
INSERT INTO VariableTable
SELECT 'Addresses', COUNT(1)
FROM AddressTable
WHERE AddressType = 0;
 
INSERT INTO VariableTable
SELECT 'Correspondence', COUNT(1)
FROM ResearchTable
WHERE TaskType = 1;

Solution pt 2

The following code can then be run to produce the results of the Database Properties List:

SELECT *
FROM VariableTable;

Finally, this code can be run to drop that temporary table (or the SQLite manager can just be closed):

DROP TABLE VariableTable;

The results of the query should be able to be copied or exported in some useful format to a file, depending on the capabilities of the SQLite manager used.

The two items not currently a part of the result set on the first pass of trying to reproduce it include Database and Created.

Discussions & comments from Wikispaces site


ve3meo

Good idea!

ve3meo
29 January 2010 14:02:56

Good idea, Romer! You could add to the page that the results may be copied or exported in some useful format, depending on the sqlite manager used, to a file.

Query Menu

Wikispaces lacked the Category feature of WordPress. This was an attempt, not updated since 2012, to provide a categorised list of pages and scripts and was the basis for the menu structure in RMtrix and the Categories used on this WordPress site. (Tom Holden 2019-01-26)

Addresses | Correspondence | Database | Edit | Facts | Fact Types | File | Functions | Groups | Media | People | Places | Reports | Repositories | Research | Search | Sources | To-Do | Tools


This page lists the tools sorted by category. Another way of finding any pages, not just query pages, is by the tag cloud at Find Pages by Tag. To view the most recent tools, have a look at RM SQLite Queries which is sorted by Date Added.


SubMenu1SubMenu2DescriptionPageQuery NameDate AddedRMtrix

Addresses

Blank NamesList Persons with Blank Names in the Address List — (a fault that may occur in a GEDCOM import).Four Little Queriesblankname_in_addresslist2009-12-30Y
How usedLists all addresses in AddressTable and reports how they are used.Addresses – How UsedAddressesHowUsed2012-08-17Y

Correspondence

Database

Copy without PeopleMakes an empty Master from current database, preserving custom fact types and source templates, Places, Master Sources and associated Repositories and Addresses, and Place/Source MediaDepopulate but keep Customs, Places, SourcesCustomDatabaseShell2011-12-10
Copy to vanilla DB3Clones a RM4/5 database without the RMNOCASE collation so that we’re free to do what we want to the data. Even works with RM but it might trip up on non-English alphabets.Convert Database to NOCASERM#_CREATE_as_DB3_NOCASE | RM#_Copy_Data_to_DB3_NOCASE2011-12-09
Database tools…
Integrity checkExplains why an integrity check is needed and how, using PRAGMA quick_check.Check RootsMagic Database IntegrityCheck RootsMagic Database Integrity2010-05-27
Integrity checkPRAGMA integrity_check(integer);Y
ReindexREINDEX. N.B., for as long as a fake RMNOCASE collation is used, this REINDEX is needed for external operations on the database and must be followed by RM’s Rebuild Indexes on returning to work the database with RM.Y
Update Birth and Death YearsSets Birth and Death years as seen in the sidebar index and various other reports and displays to match the corresponding facts. NOT for use on RM4; requires RM5+ Rebuild Indexes after completion.Rebuild Indexes and Update Birth and Death YearsUpdateBirthDeathYears2012-11-14Y
CleanDeletes unused rows from tables as reported by RMGC_Properties. May need checkbox options to select what tables are cleaned. RM4 & 5 are differentTBD
CleanDeletes citations of non-existent Sources from database that manifest themselves in the Edit Persons screen as a citation (counted and checkmarked) but return nothing when opened.Delete Phantom Citations – QueryDelete Phantom Citations2010-01-30
CleanDelete PhantomsCleans out unused records from most tables, including phantom citations. Follow by RM Rebuild Indexes and Compact.Delete PhantomsDeletePhantoms2012-10-27Y
CompactVACUUMY
Import
ImportAncestry.com and FTMProcedures involving Excel, text editor and SQLite queries to bring Ancestry hyperlinks, citations and images into RootsMagic that are otherwise lost on a straight GEDCOM import.Ancestry.com and RootsMagic 5MergeAncestryURLsToFTM
Shrinking Verbose Master Sources from Ancestry.com
2012-10-31
Exportpossible route to a control of Fact settings for GEDCOM exportTBD
Backup…
Backup database
Backup database + media to RARBacks up the database file along with all the media files referenced by it to one file. Uses the command versions of SQLite3 (free), WinRAR ($fee) and 7-Zip (free). The 7-Zip version produces a .rmgb file that RootsMagic can restore, media and database.Backup Media with Database – RARRMfullbackup.bat2011-02-02
Backup database + media to ZIPBacks up the database file along with all the media files referenced by it to one file. Uses the command versions of SQLite3 (free), WinRAR ($fee) and 7-Zip (free). The 7-Zip version produces a .rmgb file that RootsMagic can restore, media and database.Backup Media with Database – 7ZipRMfullbackup.bat2011-02-02
Properties-BasicReports in a list most of the values found in RM4’s File > Properties report; results can be exported to a file.Database Properties List – QueryDatabase Properties2010-01-29Y
Properties-ExtendedAs ‘Database Properties’ but with more detail and pointing out possible problem areasRMGC_Properties – QueryRMGC_Properties2010-01-31Y
Properties-Count rows in tablesLists count of rows in each table — If not 22 tables listed, database has corruptionRMGC Tables Row CountRMGC_TablesRowCount2010-05-26
Corrupt Database RecoveryAn outline of procedures that have been used with some success to recover a corrupted RootsMagic database.Corrupt Database RecoveryCorrupt Database Recovery2012-03-10
People View ErrorAn outline of procedures that have been used when People View throws error message “SQLite Error 1 – at most 64 tables in a join”.People View Error – at most 64 tables in a JoinPeople View Error2012-03-19

Edit

Person
PersonBirth YearLists Individuals whose Birth Year from NameTable does not match that of the date for their Birth fact.Birth Year Mis-MatchBirthYearMisMatch2010-01-17Y
PersonDeath YearLists Individuals whose Death Year from NameTable does not match that of the date for their Death fact.Death Year MismatchDeathYearMismatch2010-06-04Y
PersonLast EditedQueries to list and set Edit Dates.Date Last EditedUnnamed2013-01-25
PersonSet LivingAnother version of a Set Living query.Another version of a Set Living querySetLivingFlag2011-07-04
PersonSet LivingA discussion of queries that can modify the Living flag.Set Living FlagSetLivingFlag2011-07-02
PersonSame day sort orderRe-orders all same day events of limited types to a natural order, e.g., Birth before DeathDates – Same Day Sort OrderSortDateSameDayOrder | SortDateSameDayOrderCustom2011-12-19Y
PersonCopy fact to groupCopies a fact/event for a person to a Named Group of persons, along with the Sources but not the Media for the Fact.Copy Fact to GroupCopyFact2Group2011-04-05
PersonCopy RIN to REFNVariant of CopyFact2Group that copies a REFN fact to a group, substituting the target persons’ PersonID’s or RIN for that of the source person.Copy RIN to REFNCopyRINtoREFN2011-06-29
PersonConvert sharable factsList and convert events having key properties in common to shared events. Especially useful on imports from Legacy Family Tree in which events were copied to multiple persons.Sharable Events – Find and Convert to SharedSharableFacts2 | SharableEvents-Convert2011-12-12Y

Facts

Names – Add MarriedInserts an Alternate Name of type “Married” for female “Wife” spouses in a “Family” where no Alternate Surname matches the male “Husband’s” and a (family) Marriage fact exists.Names – Add MarriedName-Add_Married2012-11-26Y
Add Marriage to Couples WithoutAdds a Marriage event to any couple (family) in a database that does not have this fact already.Event – Add Marriage to Couples WithoutEvent-Add_Marriage2013-01-16
MergeDiscussion of techniques used to copy Description from one type of event to another and the beginning of a more complex merging of near duplicate eventsEvents – MergeUnnamed2013-02-13
Non-Privatized SSNsFinds SSN facts that have not been set to Private.WebTrees Website – Pre-Processing Tools using SQLite in Visual Basic et alRMSanity.exe2012-02-06
Shared Events with missing witnessesA fact may show that it is shared with someone whose name is missing. This query lists such facts with the names of the principals and the RINs of the missing witnesses.Shared Events With Missing WitnessesTraceHeadlessWitnesses2011-12-10Y
Sort Order ProblemsConecptual discussion and scripts to find undated or same date events that are reported by RM in an unnatural order.Facts – Sort Order ProblemsFactOrderTable, SortOrderProblems2013-01-07
Split SharedConvert shared events to individual events for compatibility on export to 3rd party software. Split | Unshare | Undo | Hide TracksFacts – Split Shared to IndividualFacts-SplitSharedToIndiv | Facts-Unshare |Facts-Split-Undo | Facts-Split-HideTracks2012-12-19Y

Fact Types

View current outputs;
Set all to GEDCOM;
Snapshot current;
View snapshot outputs;
Restore from snapshot;
Delete snapshot
Lists the Include settings for all fact types.

Sets all fact types to be included in GEDCOM and drag’n’drop.

Stores a snapshot of all settings for fact types in an extra table in the database.

Lists the include settings stored in the snapshot.

Restores all settings for fact types from the snapshot.

Deletes the extra table storing the snapshot from the database.

Fact Inclusion Controls2012-03-12Y

File

OpenOpen Windows Explorer browser on *.rmgc files.Y
Open Recent
Search for Files
Close
Copy
ExitY

Functions

Decodes most of the possible formats found in RM4 Date fields of the form Da+nnnnnnnn.x+nnnnnnnn.xDate DecoderDateDecoder2010-01-21
For encoding to SortDates, dates with optional RM modifiers and decoding same.Dates – SortDate AlgorithmSortDateDecodeDev2012-11-03

Groups

Top page for a collection of queries for refreshing Named GroupsNamed Group RefreshRefresh – various2011-11-26
AncestorsProvides a manual refresh for a specific Named Group, i.e., the ancestors of a specified person.Ancestors Named GroupAncestorsGroup2011-11-23
Ancestors + Collateral LinesThe RMSplit C# program below allows you to create a group in RootsMagic5 which contains a selected person, their ancestors, and a selected number of “leaves” (collateral lines us a given depth).SplitTreeRmsplit2011-01-03
Census neededBuilds or refreshes a group of persons whose lifetime probably spanned a user-defined Census Year and who had some event in the user-defined jurisdiction but not a Census fact for that year.Census Needed – Named GroupCensusNeededGroup | CensusNeededGroup22011-11-26
Mark/Unmark ListsMarks or Unmarks members of a group according to a list; equivalent to memorising the checkboxes in RootsMagic Explorer for re-use.Named Group – Mark or Unmark List refreshGroup Unmark List Refresh | Group Mark List Refresh2011-11-27

Media

Compare and AttachCompares RM MultiMedia report to contents of a folder for review and generates a GEDCOM which on import attaches all unlinked items to the Media Gallery. Excel workbook, not SQLIte.Media Comparison & Attach UtilityRM-Import-Unlinked-Media2013-01-19
Copy from Shared Facts to Sharee PersonalGenerates Personal mediatags from shared events’ media for non-Principal sharees.MediaTags – Copy Shared Facts Media To Sharee PersonalMediaTags-CopySharedFactsMediaToShareePersonal2012-12-27Y
Delete Personal having Fact DuplicatesDeletes media tags from Person where a duplicate exists in a fact.MediaTags – Delete Personal Having Fact DuplicatesMediaTags-DeletePersonalHavingFactDupes2012-12-28Y
Delete UnusedDeletes records from MultiMediaTable not tagged by any Person, Family, Fact, Place, Source or Citation.Media – Delete UnusedMediaDeleteUnused2013-01-21Y
DuplicatesSeries of queries to list duplicate media file names in the Media Gallery and list duplicate links to items in the Gallery. Repairs a specific case of duplicate file names as an example; repairs all cases of duplicate links.Media Repair QueriesMediaRepair2010-12-14
List usesComplete listing of all users of media in the Gallery, more complete and navigable than what is provided in RM 4.1.2.1Media Users List QueryMedia Users List2011-11-09Y
Files usageLists files under the RM Multimedia default folder and flags those used by the RM database; helps to ensure that files are used.Scrapbook Files StatusScrapBookFilesStatus2011-03-13
Update Media PathsA SQLite equivalent to RootsMagic’s own Search & Replace function on “Multimedia filenames” in the “Field to search” selection but with no programming constraintsUpdate Media PathsUpdate Media Paths2012-01-11
Missing MediaLists those image files that have not been opened by RM, hence nothing in the thumbnail field. With comments leading to other references.Missing Media2012-01-15
Preserve Captions et al from RM4 to RM5This series of queries and related Windows commands revises a RM4 database and makes copies of the multiply-linked files so that the update to RM5 will preserve all the metadata.Media – Preserve Captions et al from RM4 to RM5RM4-MultiLinkedMediaDuplicationPreRM52012-01-31
Sync Image File MetadataCompare EXIF, IPTC, XMP metadata stored in image files to the Caption, Description and other metadata stored in a RootsMagic 5 database for image files linked to its Media Gallery and copy between the files and the database.Media Metadata, Read, Write, Compare with PicasaRM5comparePicasa.bat
ImportPicasaDescriptions.sql
RM5copyMetadataToImageFileExifToolArgList.sql
RM5copyMetadataToImageFileExifToolArgList.bat
RM5copyMetadataToRM.sql
RM5copyMetadataToRM.bat
2012-01-17
Broken Links; Unused Items; Duplicate ItemsFinds broken media links using an extension in Visual Basic.
Finds media items in the Media Gallery that are not tagged to anything in the database.
Finds media items that have been attached more than once to the Media Gallery
WebTrees Website – Pre-Processing Tools using SQLite in Visual Basic et alRMSanity.exe2012-02-06
Media Type ResetSets the Media Type value for each media file to one of the four types: Image, File, Sound, Video, according to the file extension.Media Type ResetMediaTypeReset2012-12-04Y

People

MergeFast Duplicate Name SearchLists duplicate name pairs with a weighted score indicating degree of match. On a large database, produces results in 4.5 minutes similar to what RootsMagic 4’s Duplicate Search Merge tool does in 45 min.Duplicate Name Search – queryDuplicateNameSearch2010-02-23Y
MergeMark unmerged not a problemCopies all unmerged pairs from the results of RM4’s Duplicate Search Merge Tool to RM4’s ExclusionTable so that they will not reappear in subsequent runs of Duplicate Search Merge.Duplicate Search Merge DatabaseMarkNotProblem2010-02-21
Delete ManyDelete by Color Code

Delete by Named Group

Delete all people color-coded RED (or other color by editing the query). Follow with Delete Phantoms.

Delete all people belonging to a Named Group (edit the query to the group ID). Follow with Delete Phantoms.

Delete Many DeleteByColorCode

DeleteByNamedGroup

2012-10-27
Phantom Spouses – Unnamed & childlessList or delete phantom spouses (PersonID=0 in FamilyTable and childless) as from Ancestry.com via FTM2012.Phantom Spouses – Unnamed and childlessUnnamed2013-03-14

Places

County CheckThere are four Internet sites that RM5’s County Check is set to call for maps and other information.County Check2011-12-10
Unused PlacesList of unused PlacesFour Little QueriesUnusedPlaces2009-12-30Y
Place DetailsLists Places having Place DetailsFour Little QueriesPlacesDetails2009-12-30Y
Place Details with FactsReturns all Places with Place Details and the Persons or Families (couples) and their Facts, including Shared events, using the Place Detail.Facts Having Place Details – QueryFactsHavingPlaceDetails32010-01-16
Place Details without a PlaceOne query lists all Place Detail rows from PlaceTable for which there is no master Place. Another deletes such rows.Place Details without a Place Place Details without a Place2012-01-30Y
Convert to Place DetailsCombo of SQLite queries and RM4 edits of Place List beats having to edit every fact/event to split a Place into Place and Place detail; otherwise, wait for the RootsMagician!Places to Place Details ConversionPlaces to Place Details Conversion2011-11-16
Convert Place + Place Detail to PlaceThe opposite direction is much easier. Needed for export to GEDCOM for third party apps that do not support Place Detail.Convert Place + Place Detail to PlaceConvert Place + Place Detail to Place2012-01-22
Frequency of UseLists total number of events and the maximum user of each place.Places – Frequency of UsePlace_Frequency2013-03-25
Orphaned Place DetailsFinds Place Details that have no owner Place.WebTrees Website – Pre-Processing Tools using SQLite in Visual Basic et alRMSanity.exe2012-02-06
Parse and Recombine NamesCollection of queries to parse the 4-part comma-delimited Standardized place names and place the first 3 parts in the regular Name field and the first 2 parts in the Abbreviated field.Place Names – parse and recombinePlaceCommaParse
PlaceParse
PlaceAbbrevUpdate
PlaceNameUpdate
2013-02-17
Recombine Fractures from FTM2012Recombines places from Ancestry.com having the “/” in the name which get split in FTM2012 into Place Name and Event Description.Places – Recombine Fractures from FTM 2012Places-RecombineFTMfractures2013-03-22

Reports

Format
FormatParagraphingRemoves CR/LF characters around custom fact sentences and person/family/alt name/fact notes and adds CR/LF pairs to ends of notes except the last, for first cut batch paragraphing.ParagraphingParagraph-Strip | Paragraph-Add2011-12-22Y
FormatStrip footnote line feedsThis query strips extraneous Carriage Returns from the end of the Footnote sentence template for the Source Template “Vital Records (state-level, online derivatives)” which cause unwanted white space in reports.Source TemplatesFix Extra Line Feeds in Footnote2011-10-02
FormatNarrative, Jerry Bryan TrickCauses Narrative Descendant reports to output childless, spouseless children in their own generation, along with their married or parental siblings.Reports, Narrative, Jerry Bryan TrickDummyFamily-Add
DummyFamily-Delete
2012-01-15
FormatPlaces – first name exploit for improved narrativesTo mitigate repetition of full place names, changes the default sentence templates to use [Place:first] instead of [Place] and then goes on to generate custom sentence templates or modify existing ones for all first instances of a place in each person’s chronology to use [Place] instead of [Place:first]Places – first name exploit for improved narrativesPlaces-FirstNameExploit2013-03-30
My Lists
My ListsAhnentafel 64 generationsEvery direct line ancestral RIN list for one or all persons in the database with the Ahnentafel number of the last person in each line, to 64 generations, exceeding RM’s 32 gen limit on the number. Also shows all lines, not just those of parents selected for display in RM’s main Views.Ahnentafel 64 generationsAhnentafel-642012-01-13
My ListsAll factsList all the Individual, Family and Shared Facts/Events for all persons in a database. A complex query using UNION ALL, COUNT() and GROUP BY, constants and NULL to assemble multiple SELECTs in one big result.Pulling Together All the Events for An IndividualAllFacts4Persons2010-01-06
My ListsBegattersAn optional extension to AllFacts4Persons or standalone. Having a Child as an event for the Father and the Mother.Births of children as factsPersonsBegatChildren2010-01-14
My ListsDirect AncestorsLists all the ancestral lines for a given RINAncestors QueryAncestors2011-11-23
My ListsDNATwo queries that list the test results in a format suitable for easy review and copying and pasting into other applications. RM4.1.1.4 provides no DNA report.DNA Test results listDNA_mtDNA_locationslist | DNA_Y-STR_markerslist2011-06-18
My ListsFacts shared with persons in databaseLists people who share a fact with a Principal, as well as relevant fact and Principal information.People Who Share a Fact with a Principal List – QueryPeople Who Share a Fact with a Principal2010-03-10Y
My ListsFacts shared with persons NOT in databaseLists people who share a fact with a Principal, but who aren’t in a tree in the database file, as well as relevant fact and Principal information.People Who Share a Fact with a Principal, But Who Are Not in a Tree in the File List – QueryPeople Who Share a Fact with a Principal, But Who Are Not in a Tree in the File2010-02-26Y
My ListsGeo-LifelinesHave you ever wished to be able to look at all the facts in your family tree database that happened within a day’s horseride of a certain location? This adaptation of the LifeLines query helps you view your events for any geographic area in addition to looking at the lifeline of any person in your database.Geo-Lifelines QueryGeo-Lifelines2010-08-31
My ListsLifeLinesLists all events for all persons whether in a database tree or not, including shared facts, date, fact detail, site and place, MRIN, other parties, and duplication indicator. Sorted by RIN and Sort Date. With the right SQLite manager, can filter results for one person.LifeLines – QueryLifeLines2010-03-15
My ListsMap Events (KML)This query helps you plot events from your RootsMagic database on Google Maps, Google Earth, and Bing Maps, provided there are geo-coded Places and Place Details (sites) in your database.MapEvents-KML queryMapEvents-KML2010-09-03
My ListsMultiple spousesLists persons with multiple spouses in descending number – may flag a data problemMultiple Spouses queryMulti-spouses2010-08-23Y
My ListsNameless witnesses to shared factsReport on the role, event and owner or principal of the shared event for which a witness or sharee is no longer a person in the database.Shared Events With Missing WitnessesTraceHeadlessWitnesses2011-12-09Y
My ListsNon-ProvenList non-proven eventsReport Non Proven Proven2013-03-28
My ListsTo-Do GridA Query for a To Do List in a Grid Format – using the REFN to assign status for paragraphing and census. Versions for both SQLite directly and MS Access via SQLODBC.A Query for a To Do List in a Grid FormatMyToDoList2011-07-07Y

Repositories

Research

Search

NameProduces a list of names that match or sound like specified names, similar to RM4’s NameFind.Name Find queryNameFind2011-11-21
SurnameList Persons with specified Surnames. — Example of creating a SQL View or Virtual Table and the explicit use of COLLATE NOCASE to override the RMNOCASE collation defined for certain fields and embedded in the RootsMagic application.Four Little Queriesselected_surnames2009-12-30
Search & ReplaceDiscussion and examples of how you can search (filter) using any SQLite manager and replace found values with revised ones using SQLiteSpy.Search & ReplaceSearch&Replace2010-11-09

Sources

Source ListAbout as close as we can get to a RM4 Source List Report presented in tabular form with which the results can be sorted and filtered with relative ease.Source List QuerySource List2011-11-05Y
Source DetailsLists Source Names along with Source Detail field names; illustrates principles applicable to Master Source fields and other XML-like columns.Source Detail View (Parsing XML)Source Detail View (Parsing XML)2011-06-06
CitationsLists all citations in the database from which citations of non-existent sources (‘phantoms’) and citations for non-existent events or persons (‘headless’) can be found, along with other useful information such as all citations per source.All Citations – QueryAllCitations2010-01-30
CitationsLists all source names cited for a person in descending order of the count of duplicate citations – a help in finding and resolving duplicate citations after merging.All Citations & Dupes Count – QueryAllCitations+Dupes2010-07-08
Citations, InvisibleList

Convert

Lists citations for Primary Name as imported from Ancestry.com, FTM, et al but hidden in RM.
Converts such invisible citations to visible Person citations.
Citations Invisible RevealedCitations, Invisible – List
Citations, Invisible – Convert to Personals
2012-09-10Y
Sources by TemplateLists Sources associated with Source Templates, as well as an extension to Citation details. Free-Form templates are excluded since they’re not included in SourceTemplateTable.Source Template List – QuerySourceTemplateListWithCitationDetails2.sql2010-02-13
Make Built-in TemplatesEditableConverts Sources using uneditable, built-in templates to using editable copies.Source TemplatesSrcTmpltsConvert2011-10-02
Revert Editable Templates to Built-inReverts Sources modified by SrcTmpltsConvert to using uneditable, built-in templates.Source TemplatesSrcTmpltsRevert2011-10-02
Source Template Field CountLists the total number of fields used in a source template and the number of fields in its footnote sentence template as a measure of complexity.Source Templates – Field CountSourceTemplates-FieldCount2013-04-08

To-Do

Lists uncompleted To Do tasks for Individuals, Families and General; can be readily modified to filter or sort by surname, priority, repository, etc.RM4 To Do ListTo-Do (was To-Do4Persons)2010-01-18Y

Tools

WebTags – Make from Ancestry.com CommentsCreates a Citation WebTag for each citation having an URL beginning the Citation Comments field with “http://”. Not restricted to Ancestry.com downloads.WebTags – from Ancestry.com and FTMWebTags-MakeFromAncestryComments2012-12-10Y
WebTags – ConsolidateCreates Person WebTags from Citation WebTags to consolidate all WebTags pertaining to a person under the WebTags button on the Edit Person Screen.WebTags – ConsolidateWebTags-Consolidate2012-12-10Y
WebTags – Delete DuplicatesDeletes perfectly duplicate WebTags as will occur if WebTags-Consolidate is repeated or other possible cause.WebTags – ConsolidateWebTags-DeleteDuplicates2012-12-10Y
WebTags – Convert to Notes et alAppend Person, Source and Citation WebTags to Person Notes, Source Comments and Citation Comments for passing through GEDCOM to other software.WebTags – convert to Notes et alWebTags_to_Notes2013-03-03

Addresses | Correspondence | Database | Edit | Facts | Fact Types | File | Functions | Groups | Media | People | Places | Reports | Repositories | Research | Search | Sources | To-Do | Tools

Source Templates – Field Count #sourcetemplates

Update 2021-03-10: tested with #RM8 and minor rev for wider compatibility with SQLite managers (Pat Jones)

The complexity of a Source Template is somewhat proportional to the number of fields and their distribution between the Master Source and Source Details (citation-specific). It is probable that a source with a greater number of fields in total AND a greater number of them in Source Details will be more complex than one that has lesser numbers and will suffer more distortion and loss when exported to standard GEDCOM (see Source Templates, A Comparative Example.pdf). There are other factors that complicate the fidelity of export which could well result in some exceptions but they are harder to measure. These include:

  • fields in the RM Footnote sentence template out of order from the simple convention of Master fields first followed by Source Detail fields
  • words or strings that are within a sentence template switch that are conditional on a non-empty Source Detail field

This query does the easy task of counting the total number of fields and the number of those that are Source Detail fields.

SourceTemplates-FieldCount.PNG
Screenshot from SQLiteSpy showing results from some of the built-in Source Templates. You can sort by any column by clicking on it.

SourceTemplates-FieldCount.sql

SourceTemplates-FieldCount.sql

-- SourceTemplates-FieldCount.sql
-- 2013-04-08 Tom Holden ve3meo

-- works with RM8 PJ 2021-03-07
/*
Converts the FieldDefs blob from SourceTemplateTable to a 1-line string and
counts the number of occurrences of "<FieldName>" in it (the number of fields)
and the subset of same that are Source Detail fields "<CitationField>True".
Does so by measuring the lengths of the FieldDefsTxt strings with and without
the search string and dividing the difference by the length of the search string.
*/
DROP TABLE IF EXISTS xSrcTpltFldDefs;

CREATE TEMP TABLE IF NOT EXISTS xSrcTpltFldDefs AS
	SELECT TemplateID
		,NAME
		,REPLACE(REPLACE(CAST(FieldDefs AS TEXT), X'0A', ''), X'0D', '') AS FieldDefsTxt
	FROM SourceTemplateTable;

SELECT TemplateID
	,NAME
	,(LENGTH(FieldDefsTxt) - LENGTH(REPLACE(FieldDefsTxt, '<FieldName>', ''))) / LENGTH('<FieldName>') AS TotalFields
	,(LENGTH(FieldDefsTxt) - LENGTH(REPLACE(FieldDefsTxt, '<CitationField>True', ''))) / LENGTH('<CitationField>True') AS SrcDtlFields
	,FieldDefsTxt
FROM xSrcTpltFldDefs;

Census Needed – Named Group #namedgroup #census

Intro

Builds a group of persons whose lifetime probably spanned a user-defined Census Year and who had some event in the user-defined jurisdiction but not a Census fact for that year. The same script can be used for many named groups having different Census Years and target locations. The script includes examples of building lists of RINs that behave like RM’s manual marking and unmarking of persons for a group.

Download

2021-11-25 adapted for #RM8

CensusNeededGroup.sql (for RM4 to pre-RM8)
– initial query works on individual facts only.

CensusNeededGroup2.sql (for RM4 to pre-RM8)
rev. 2020-05-16 to correct misuse of LabelID
– this query works on all types of facts, individual, family and shared by re-using the LifeLines query – a bulldozer solution for a shovel sized pile, but, what the heck!, it was all ready to use.

Usage

  1. Backup your database before you do this every time until you are confident that it does no harm.
  2. In RM, create a named group having the words “Census needed” and “#”CensusYear and Jurisdiction (in that order and without the quotes), where CensusYear is the year of the census, Jurisdiction is the country, or province/state, or municipality.
    e.g., “*Census needed #1871 Canada”
  3. Open your RM database with SQLite Expert Personal and load CensusNeededGroup.sql or CensusNeededGroup2.sql for RM4-preRM8 or, for RM8 databases, CensusNeededGroup2-RM8.sql into the SQL editor.
  4. Execute the query (F5)
  5. You will be prompted to enter a value for @CensusYear. In this example, enter “1871” (without quotes and without the leading #).
  6. You will be prompted to enter a value for @Jurisdiction. In this example, enter “Canada”.
  7. When the query has finished, you will need to refresh the RM4 sidebar view of your group by selecting another group and then re-selecting it.

Discussion

The script looks for the group name that matches its search string. If it does not find a match, nothing happens. Otherwise, the current members list for the group is deleted and the script proceeds to build the new list. Each group has to have a unique name matching the rules above; the user of the script enters the variables year and jurisdiction for it to act on any one of the groups.

Persons having neither a Birth or Death date are ignored by the script as it cannot determine anything about their probability of being alive in the Census Year. Persons having both dates are readily evaluated against the Census Year while those having only one date are given a lifespan of 75 years – that number can be easily found in the script and changed. It could also be incorporated as a parameter specified at run time.

The @Jurisdiction parameter is used to search events with a place name containing the search string. This could be as broad as a country “Canada” as in the example; a state “, OH,” if you used abbreviations (and the leading and trailing punctuation would be needed to avoid picking up every place with “oh” in the name), or a town. If any event occurred in a matching place, that person will be selected unless they have a Census fact for that year.

Thus it is important that there be at least an estimated Birth date and/or an estimated Death date and a guess at the Place of one of these or any other individual event.

To add selected persons that are to be included or excluded from the group, look to the last two sql statements (commented out) in the CensusNeededGroup.sql script. They show how to build the lists and actually carry out the marking and unmarking of the same RINs (net change = 0).

Places – first name exploit for improved narratives #places #reports

Unlike the names of persons, RootsMagic 6 does not yet have any mechanism for automatically revolving through Place names in reports, i.e., using the full place name only if it has not been previously used for a person while taking advantage of the first or lowest level place name or the abbreviated place name for subsequent instances in the person’s narrative. It is up to the user to modify default sentences for Facts and Roles and to create custom sentences for events in order to take advantage of the :first and :short options for the Place field and thus reduce needless repetition of higher level place values such as county, state/province/shire, country. And exacerbating repetition of country is an effect of RootsMagic’s own County Checker and Gazetteer which want to populate the Place field with country.

To mitigate such repetition with a token effort, this script automatically edits the default sentence templates to use [Place:first] instead of [Place] and then goes on to generate custom sentence templates or modify existing ones for all first instances of a place in each person’s chronology to use [Place] instead of [Place:first]. That leaves all subsequent instances of the place for that person to be the default sentence having [Place:first] or to be so modified, which it also does.

For a person who was born, was schooled, worshipped, worked, married, resided, died and was buried in the same Place, his narrative will have but one instance of the full Place name (city, county, state, country) for his birth; all other instances will be just the city. More or less! A “less” is where the person appears as a child below his parent’s narratives as these phrases are not subject to any accessible template.

Places-FirstNameExploit.PNG
Screenshot from MS Word comparing the revised narrative to the original.

In the before/after example above, the Birth event was detected as the first use of Glasgow so a custom sentence for the event was generated that was the same as the original default sentence with the full Place. All the subsequent events in Glasgow, including the shared or witnessed census events, used the new default sentences having [Place:first] so “, Lanarkshire, Scotland, United Kingdom” appears deleted (red strikethrough). The one Quebec City event is a first and only so it received a custom sentence with full Place name. Then North Bay appears three times, the first with a full Place custom sentence, the rest using the first place value per the revised default sentence template.

This utility is not the be all and end all for narratives but it should be useful as a quick streamliner and a basis on which further customisation of sentence templates can deliver further improvement. It does not address all the combinations of options that can be added to [Place] because there could be many. SQLite does not have a regular expression search and replace function which would be necessary to deal with them efficiently and comprehensively.

Places-FirstNameExploit.sql

-- Places-FirstNameExploit.sql
/*
2013-03-29 Tom Holden ve3meo
rev 2013-03-30 added [Place:plain] update to [Place:plain:first] for defaults only.
    needs regexp search & replace to deal with all combinations of Place modifiers
 
Exploits the Place:first option for Place names in narratives by
setting default sentences to Place:first and customising to Place 
only for the first event for a person in any place. 
 
This makes the narrative less wordy and repetitious of the higher levels 
in a Place name. However, it has no effect on the phrases used in the 
Children list at the bottom of a person's narrative which continue 
to use the full place name.
*/
-- Set default Fact sentences to use the first Place name.
UPDATE FactTypeTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]');
UPDATE FactTypeTable
SET Sentence = REPLACE(Sentence, '[Place:plain]', '[Place:plain:first]');
 
-- set default role sentences to use the first Place name
UPDATE RoleTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]');
UPDATE RoleTable
SET Sentence = REPLACE(Sentence, '[Place:plain]', '[Place:plain:first]');
 
-- create table of first event in a place for a person including shared events
DROP TABLE
IF EXISTS xFirstPlaceEvents;
 
CREATE TEMP TABLE
IF NOT EXISTS xFirstPlaceEvents AS
    SELECT *
    FROM (
        -- INDIV events
        SELECT EventID
            ,0 AS isSharer
            ,PlaceID
            ,OwnerID
            ,SortDate
        FROM EventTable
        WHERE OwnerType = 0
            AND PlaceID > 0
 
        UNION
 
        -- Husband events
        SELECT EventID
            ,0 AS isSharer
            ,PlaceID
            ,FatherID AS OwnerID
            ,SortDate
        FROM EventTable
        INNER JOIN FamilyTable ON EventTable.OwnerID = FamilyTable.FamilyID
            AND OwnerType = 1
            AND PlaceID > 0
 
        UNION
 
        -- wife events
        SELECT EventID
            ,0 AS isSharer
            ,PlaceID
            ,MotherID AS OwnerID
            ,SortDate
        FROM EventTable
        INNER JOIN FamilyTable ON EventTable.OwnerID = FamilyTable.FamilyID
            AND OwnerType = 1
            AND PlaceID > 0
 
        UNION
 
        -- shared events
        SELECT WitnessID
            ,1 AS isSharer
            ,EventTable.PlaceID AS PlaceID
            ,WitnessTable.PersonID AS OwnerID
            ,EventTable.SortDate AS SortDate
        FROM WitnessTable NATURAL
        INNER JOIN EventTable
        WHERE EventTable.PlaceID > 0
        ORDER BY OwnerID
            ,SortDate DESC -- so next GROUP BY will pick up the smallest SortDate or first event in the group
        )
    GROUP BY OwnerID
        ,PlaceID
    ORDER BY EventID; -- so the IN() expression in the following queries will see an ordered list
 
-- set all first events for a person to use the default sentence customised with the full Place
-- except those already with custom sentences 
UPDATE EventTable
SET Sentence = (
        SELECT REPLACE(FactTypeTable.Sentence, '[Place:first]', '[Place]')
        FROM EventTable Events
        INNER JOIN FactTypeTable ON Events.EventType = FactTypeID
        WHERE EventTable.EventID = Events.EventID
        )
WHERE EventID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 0
        )
    AND EventTable.Sentence LIKE '' --change this to OR with a match to the default
    OR EventTable.Sentence LIKE (
        SELECT REPLACE(FactTypeTable.Sentence, '[Place:first]', '[Place]')
        FROM EventTable Events
        INNER JOIN FactTypeTable ON Events.EventType = FactTypeID
        WHERE EventTable.EventID = Events.EventID
        );
 
-- set all other custom sentences for first events to use the full name
UPDATE EventTable
SET Sentence = REPLACE(Sentence, '[Place:first]', '[Place]')
WHERE EventID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 0
        );
 
-- set all other custom sentences for non-first events to use the first name
UPDATE EventTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]')
WHERE EventID NOT IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 0
        );
 
--DO The Same steps for shared events
-- set all first witness for a person to use the default sentence customised with the full Place
-- except those already with custom sentences 
UPDATE WitnessTable
SET Sentence = (
        SELECT REPLACE(RoleTable.Sentence, '[Place:first]', '[Place]')
        FROM WitnessTable Witness
        INNER JOIN RoleTable ON Witness.ROLE = RoleTable.RoleID
        WHERE WitnessTable.WitnessID = Witness.WitnessID
        )
WHERE WitnessID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 1
        )
    AND WitnessTable.Sentence LIKE '' --change this to OR with a match to the default
    OR WitnessTable.Sentence LIKE (
        SELECT REPLACE(RoleTable.Sentence, '[Place:first]', '[Place]')
        FROM WitnessTable Witness
        INNER JOIN RoleTable ON Witness.ROLE = RoleTable.RoleID
        WHERE WitnessTable.WitnessID = Witness.WitnessID
        );
 
-- set all other custom sentences for first witness events to use the full name
UPDATE WitnessTable
SET Sentence = REPLACE(Sentence, '[Place:first]', '[Place]')
WHERE WitnessID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 1
        );
 
-- set all other custom sentences for non-first witness events to use the first name
UPDATE WitnessTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]')
WHERE WitnessID NOT IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 1
        );

ReportNon_proven #events

List non-proven Facts.

This SQL query will report non-proven facts. This is useful when going though a non-sourced acquired database so you can go though all of the events and find a source for them.

Download: Proven.sql

Discussions & comments from Wikispaces site


ve3meo

Does it depend on LDS support turned on?

ve3meo
06 April 2013 03:05:17

Ran the query but got no results. On closer inspection, I see that there is a JOIN to the LinkTable which is empty in my database. Removing this join does get results.

Is it your intention to restrict the query to only those events that belong to and were imported from a Family Search Family Tree?

I confess to knowing next to nothing about the LDS components of the RootsMagic database.

Tom

P.S. – delighted that you have made a start on contributing to the wiki!

Places – Recombine Fractures from FTM 2012 #ancestrycom #ftm2012 #places

This is a problem with an Ancestry.com tree synchronized with Family Tree Maker 2012 (version 21.0.0.723), and exported therefrom. Places containing a forward slash (“/”) in the name are split on import into FTM into two parts. Everything after the slash goes to the Place name and all before to the Event description. The FTM2012 GEDCOM exports the fracture while the Ancestry.com direct GEDCOM does not. As only FTM2012 can automatically download media and deliver the paths via GEDCOM to RootsMagic, I am forced to use its GEDCOM.

Here’s an example:

Ancestry.com place:Toronto (West/Ouest) (City/Cité) Ward/Quartier No 5, Toronto (west/ouest) (city/cité), Ontario, Canada
FTM2012 Residence description:Toronto (West/Ouest) (City/Cité) Ward/Quartier No 5, Toronto (west/ouest) (city
FTM2012 place:Cité), Ontario, Canada

FTM appears to parse on the last slash, which it drops.

[inline comment: “FTM appears to parse on the last slash, which it drops”
external image user_none_lg.jpg ve3meo Mar 24, 2013
Belatedly, I worked out a procedure within FTM 2012 to recombine the fractured places. It is a bit tedious but better than editing one fact at a time; doesn’t hold a candle to direct access to the database as we have with RM and a batch process as I did with SQLite. If interested, read http://boards.ancest…/9578.2/mb.ashx. The parsing of the place on the forward slash seems to be a holdover from much earlier versions of FTM and maybe even an old version of GEDCOM which combined a fact place and description on one line, separated by the slash.
]

This may be a problem solely with places originating from Ancestry’s databases for the Censuses of Canada with the complexities of bilingual English/French wording. Healing the fractures seemed a daunting task to do manually through RootsMagic so I worked up a series of SQLite queries that seem to have cured the patient. I’ll go on to merge places and/or split out Place Details within RM.

Places-RecombineFTMfractures.sql

-- Places-RecombineFTMfractures.sql
/*
2012-03-22 Tom Holden ve3meo
Investigation into fractured Place names from FTM2012 where part of the name
is exported in the Event description (Detail) and the balance in the
Place name. Typically, these are places from Ancestry Canada Census
databases with '/' between English/French words, e.g. "west/ouest".
 
Recombines the fractures and deletes the event description part.
 
Does not carry Place Details (Site) over to the recombined Place - a revised
version could by replacing the fractured Name in PlaceTable rather than
creating a new recombined Place as this script does. Don't know why I didn't
think of that.
 
N.B. This is a series of queries intended to be executed one at a time in sequence
although one might go for broke and fire them off as a batch. There would be no chance
to inspect the data.
*/
 
/*
Find Places with just ')' in name and not the balancing parenthesis, found in the event description,
as in incorrectly exported places from FTM 2012.
*/
SELECT PlaceID
    ,NAME
FROM PlaceTable
WHERE PlaceType = 0
    AND NAME LIKE '%)%'
    AND NAME NOT LIKE '%(%';
 
/* Make a Table of EventIDs using the split Place names
*/
DROP TABLE
 
IF EXISTS xEventPlace;
    CREATE TEMP TABLE
 
IF NOT EXISTS xEventPlace AS
    SELECT EventID
        ,Details
        ,Event.PlaceID AS PlaceID
        ,Place.NAME AS Place
        ,Event.SiteID AS SiteID
        ,Site.NAME AS Site
    FROM PlaceTable Place NATURAL
    INNER JOIN EventTable Event
    LEFT JOIN PlaceTable Site ON (Event.SiteID = Site.PlaceID)
        AND Site.PlaceType = 2
    WHERE Place.PlaceID IN (
            SELECT PlaceID
            FROM PlaceTable
            WHERE PlaceType = 0
                AND NAME LIKE '%)%'
                AND NAME NOT LIKE '%(%'
            )
        AND Event.Details LIKE '%(%';
 
-- Generate recombined Place names in PlaceTable
INSERT INTO PlaceTable
SELECT DISTINCT NULL AS PlaceID
    ,0 AS PlaceType
    ,Details || '/' || Place AS NAME
    ,'' AS Abbrev
    ,'' AS Normalized
    ,0 AS Latitude
    ,0 AS Longitude
    ,0 AS LatLongExact
    ,0 AS MasterID
    ,'Generated by SQLite query from fractured FTM 2012 export: ' || Details || '/' || Place
FROM xEventPlace;
 
-- Revise events to point to recombined Places
UPDATE EventTable
SET PlaceID = (
        SELECT PlaceTable.PlaceID
        FROM PlaceTable
            ,xEventPlace
        WHERE PlaceType = 0
            AND PlaceTable.NAME LIKE EventTable.Details || '/' || xEventPlace.Place
            AND EventTable.EventID = xEventPlace.EventID
        )
WHERE EventID IN (
        SELECT EventID
        FROM xEventPlace
        ORDER BY EventID ASC
        );
 
-- Verify event places
SELECT EventID
    ,xEventPlace.Details
    ,xEventPlace.Place
    ,PlaceTable.NAME
FROM xEventPlace
LEFT JOIN EventTable USING (EventID)
LEFT JOIN PlaceTable ON (EventTable.PlaceID = PlaceTable.PlaceID)
 
-- Erase event descriptions
UPDATE EventTable
SET Details = ''
WHERE EventID IN (
        SELECT EventID
        FROM xEventPlace
        ORDER BY EventID ASC
        );
    -- All done

Inline comments


ve3meo

Comment: Belatedly, I worked out a procedure w…

ve3meo
24 March 2013 21:23:21

Belatedly, I worked out a procedure within FTM 2012 to recombine the fractured places. It is a bit tedious but better than editing one fact at a time; doesn’t hold a candle to direct access to the database as we have with RM and a batch process as I did with SQLite. If interested, read http://boards.ancest…/9578.2/mb.ashx . The parsing of the place on the forward slash seems to be a holdover from much earlier versions of FTM and maybe even an old version of GEDCOM which combined a fact place and description on one line, separated by the slash.