MediaTags – Copy Shared Facts Media To Sharee Personal #media #sharedevent

This script responds as a workaround to a 13 Dec 2011 wish expressed by MarkVS, that RootsMagic’s Media Album/Gallery and other Media dialog windows that show MediaTags should also show the indirect tags to shared events for the Persons having non-Principal roles, sometimes called “sharees” and the shared event in their context a “sharee event”. To do so can only be done by programming changes; one year later, nothing has changed. A workaround is to generate Personal mediatags from the shared event’s media for these non-Principal sharees. That’s what this script does.

Considerations:

  1. Assuming the shared event’s mediatag is Primary for that event and Included in Scrapbook, so, too is each inherited mediatag for each sharee event. Therefore, the script generates a Primary-No, Scrapbook-No mediatag for the Person to avoid conflict with the truly Personal and other script-generated Personal mediatags and so that the Person Scrapbook does not receive duplicates.
  2. These workaround mediatags probably should have some unique property that would distinguish them from regular mediatags and the indirect ones that may appear if and when RootsMagic is enhanced to display them. The current script does so, perhaps too subtly, by adding a sentence to the Comment field in the Media Tag dialog window (Description field in MediaLinkTable). There are other techniques that might be better suited for future script operations – e.g., an apparently unused Note field in MediaLinkTable or wrapping the sentence in some distinct string sequence such as the “{ }” pairing used for distinctive privacy flags in Facts – Split Shared to Individual (note that they do not function as privacy flags in the MediaTag Comments/Description field because it is not outputted in any report anyway).
  3. If someone uses Facts – Split Shared to Individual in combination with this script, there certainly will be real duplicate mediatags for each person, one is this workaround Personal tag, the other is the tag for the newly created Individual event. A procedure is needed to delete the duplicate Personal ones. It could rely on a more distinctive branding as discussed in 2) or simply look for basic matches with mediatags for events as in MediaTags – Delete Personal Having Fact Duplicates.

2013-01-05 1.1 outputs media tag for sharees having no defined role in the event.
Sentence in tag description reads “Shared in the …”.
2012-12-28 V1 complete – tags for both family and non-family shared events
2012-12-27 V0 creates mediatags for shared individual events, not for shared family events

MediaTags-CopySharedFactsMediaToShareePersonal.sql RMtrix_tiny_check.png

-- MediaTags-CopySharedFactsMediaToShareePersonal.sql
/*
2012-12-27 Tom Holden ve3meo
2012-12-28 V1 complete
2013-01-05 1.1 outputs media tag for sharees having no defined role in the event.
           Sentence in tag description reads "Shared in the ...".
 
Creates a media tag, for media already tagged to a shared event,
 to the Persons sharing the event. The Description contains
 the original Description appended with the Person's role name,
 the event the Person shared in and the name(s) of the Principal(s)
 in the event.
 
Version 1 deals with tags for shared non-family and family events.
 
Use MediaTags-DeletePersonalHavingFactDupes.sql to delete tags created by this script.
*/
 
 
INSERT OR REPLACE INTO MediaLinkTable
 
-- tags for shared individual (non-family) events
SELECT
 NULL AS LinkID,
 MediaID,
 0 AS OwnerType,
 OwnerID,
 0 AS IsPrimary,
 0 AS Include1,
 0 AS Include2,
 0 AS Include3,
 0 AS Include4,
 0 AS SortOrder,
 0 AS RectLeft,
 0 AS RectTop,
 0 AS RectRight,
 0 AS RectBottom,
 '' AS Note,
 Caption,
 RefNumber,
 DATE,
 SortDate,
 Description
FROM
(
SELECT
 ML.MediaID AS MediaID,
 W.PersonID AS OwnerID,
 ML.Caption AS Caption,
 ML.RefNumber AS RefNumber,
 ML.DATE AS DATE,
 ML.SortDate AS SortDate,
 ML.Description
  || ' '  || ifnull(R.RoleName, 'Shared')  || ' in the '  || LOWER(F.Name)  ||  ' of '  || N.Given  || ' '  || N.Surname  || '-'  || N.OwnerID
  AS Description
FROM MediaLinkTable ML
INNER JOIN EventTable E
 ON ML.OwnerID = E.EventID
 AND ML.OwnerType = 2 -- Event
 AND E.OwnerType = 0 -- Person
INNER JOIN WitnessTable W
 USING(EventID)
INNER JOIN FactTypeTable F ON E.EventType = F.FactTypeID
LEFT JOIN RoleTable R ON W.ROLE = R.RoleID
INNER JOIN NameTable N ON E.OwnerID = N.OwnerID AND +N.IsPrimary
WHERE W.PersonID > 0
AND ML.MediaID || '.' || W.PersonID
NOT IN
(
 SELECT DISTINCT MediaID || '.' || OwnerID
 FROM MediaLinkTable
 WHERE OwnerType = 0
 ORDER BY MediaID, OwnerID
 )
 
UNION
-- tags for shared family events
SELECT
 ML.MediaID AS MediaID,
 W.PersonID AS OwnerID,
 ML.Caption AS Caption,
 ML.RefNumber AS RefNumber,
 ML.DATE AS DATE,
 ML.SortDate AS SortDate,
 ML.Description  || ' '  || ifnull(R.RoleName, 'Shared')  || ' in the '  || LOWER(F.Name)  ||  ' of '  || Husb.Given  || ' '  || Husb.Surname  || '-'  || Husb.OwnerID  || ' & '  || Wife.Given  || ' '  || Wife.Surname  || '-'  || Wife.OwnerID
 
  AS Description
FROM MediaLinkTable ML
INNER JOIN EventTable E
 ON ML.OwnerID = E.EventID
 AND ML.OwnerType = 2 -- Event
 AND E.OwnerType = 1 -- FamilyPerson
INNER JOIN FamilyTable Fam
 ON E.OwnerID = Fam.FamilyID
LEFT JOIN NameTable Husb
 ON Fam.FatherID = Husb.OwnerID AND +Husb.IsPrimary
LEFT JOIN NameTable Wife
 ON Fam.MotherID = Wife.OwnerID AND +Wife.IsPrimary
INNER JOIN WitnessTable W
 USING(EventID)
INNER JOIN FactTypeTable F ON E.EventType = F.FactTypeID
LEFT JOIN RoleTable R ON W.ROLE = R.RoleID
--INNER JOIN NameTable N ON E.OwnerID = N.OwnerID AND +N.IsPrimary
WHERE W.PersonID > 0
AND W.PersonID || '.' || ML.MediaID
NOT IN
(
 SELECT DISTINCT MediaID || '.' || OwnerID
 FROM MediaLinkTable
 WHERE OwnerType = 0
 ORDER BY MediaID, OwnerID
 )
)
 
;
 
-- USE MediaTags-DeletePersonalHavingFactDupes.sql to delete tags created by above
;

Fact Inclusion Controls #facttypes #events #gedcom #reports

2021-10-28: The following queries are compatible with #RM8 but, while included in RMtrix_tiny_check.png, the app itself is not.

RMtrix_tiny_check.png
If a user has set certain Fact Types to be excluded from GEDCOM export, those fact types are also not transferred to another database via Drag and Drop (DnD). This setting is controlled in the Edit Fact Type window, accessible through the menu path Lists > Fact Type List which opens the Fact Types window through which one can inspect the settings one fact at a time. The Edit button opens the Edit Fact Type window on the selected fact. To ensure that all facts or events for the selected persons are transferred via DnD, one needs to remember to inspect and change the ‘excluded’ facts to ‘included’; following the DnD, the settings wanted for GEDCOM, if different from those for DnD, must be restored. Given the one-at-a-time procedure, it is desirable to have more expeditious ways to change settings. This page offers some short queries that will satisfy some needs until such time as RootsMagic gains more efficient controls. These queries are included in RMtrix.

List Include Settings

This query puts the Include settings for all fact types in table form on one screen for easy inspection. Sorting the list on any column rapidly shows which fact types have been included or excluded from RootsMagic outputs.

FactTypeIncludeSettings.PNG
Screenshot from SQLiteSpy. 1 = included, 0 = excluded.
-- List FactType Include Settings
SELECT
 Name,
 Flags & 1 AS GEDCOM,
 Flags & 2 <> 0 AS WEB,
 Flags & 4 <> 0 AS FGS,
 Flags & 8 <> 0 AS NARR,
 Flags & 16 <> 0 AS INDIV,
 Flags & 32 <> 0 AS LISTS
FROM FactTypeTable
ORDER BY Name
;

N.B.: As written, the above query (and some of the others) requires SQLiteSpy to have the fake RMNOCASE collation as described at RMNOCASE – faking it in SQLiteSpy.

Save Current Fact Type Settings for Later Restoration

It would be nice to be able to save the current settings, make changes that are temporary, and restore them later. This query simply makes a copy of the FactTypeTable and saves it in the database file. Dead simple and does not affect RootsMagic operations.

-- Save current FactType settings for subsequent restoration
DROP TABLE IF EXISTS zFactTypeTable;
CREATE TABLE IF NOT EXISTS zFactTypeTable
AS SELECT * FROM FactTypeTable;

Set All Fact Types to be Included in GEDCOM and Drag and Drop

This query changes only the GEDCOM include setting so that all fact types are included in both Export and DnD. Similar UPDATE queries could be done for other include settings.

-- Set all FactTypes to be included in GEDCOM and Drag and Drop
UPDATE FactTypeTable
 SET Flags = Flags +1
WHERE NOT Flags & 1
;

Restore Saved Fact Type Settings

Regardless of what changes have been made or how they were made, this query returns the FactTypeTable to the way it was when its then current settings were saved to a backup table.

-- Restore saved FactTypeTable settings
INSERT OR REPLACE INTO FactTypeTable
SELECT * FROM zFactTypeTable;

Other Ideas

  1. Some may want to have more than one set of saved settings and be able to name or describe each set. That is possible; it would require another custom table and the addition of a SettingID column to the table that stores the multiple sets, or a different table for each set.
  2. Direct interaction with the settings from a tabular screen – possible using a high level programming language

Discussions & comments from Wikispaces site


Geraniums

Having problems

Geraniums
17 March 2012 21:40:50

I opened a database in SQLiteSpy.
I pasted this into the program:

— List FactType Include Settings
SELECT
Name,
Flags & 1 AS GEDCOM,
Flags & 2 <> 0 AS WEB,
Flags & 4 <> 0 AS FGS,
Flags & 8 <> 0 AS NARR,
Flags & 16 <> 0 AS INDIV,
Flags & 32 <> 0 AS LISTS
FROM FactTypeTable
ORDER BY Name
;

Then F9.
Error: no such collation sequence: RMNOCASE.

I get the same message when pasting this:
— Save current FactType settings for subsequent restoration
DROP TABLE IF EXISTS zFactTypeTable;
CREATE TABLE IF NOT EXISTS zFactTypeTable
AS SELECT * FROM FactTypeTable;

However, I was able to paste and run this:
— Set all FactTypes to be included in GEDCOM and Drag and Drop
UPDATE FactTypeTable
SET Flags = Flags +1
WHERE NOT Flags & 1
;

But then I couldn’t go back to the old setting because that command didn’t work.


Geraniums

Geraniums
17 March 2012 21:42:57

Now on to the next problem. After being able to change all the Fact Types to YES by running the command, after Dragging and Dropping into a new database, the properties numbers don’t equal.

In the new database, the numbers are lower in some cases, one being the Place List.

Shared Events With Missing Witnesses #sharedevent

HeadlessWitness.jpg#RM8 compatible

RootsMagic 4 & 5 fail to clean up all the tables after some entities are merged or deleted. The new Database Tools don’t either. Apart from wasted disk space, there are other more concerning consequences. One is what has been called “Headless Witnesses” – these are RINs found in the WitnessTable for whom there is no matching person in the PersonTable. A result is that a fact may show that it is shared with someone whose name is missing.

The RMGC_Properties query reports a count of the number of sharings for which the RIN of the witness does not exist in the database. Its core statement can be brought out to inspect the RINs. Perhaps, by reviewing a database backup, the missing persons can be identified and the matter resolved.

-- List of RINs of Headless Witnesses
SELECT PersonID FROM WitnessTable
WHERE WitnessTable.PersonID > 0
EXCEPT
SELECT PersonID FROM PersonTable
;

A more comprehensive query was developed to report on the role, event and owner or principal of the shared event. Download TraceHeadlessWitnesses.sql RMtrix_tiny_check.png. Here’s a sample result:
TraceHeadlessWitnessesScreenshot.PNG
Using these results the search can be quickly narrowed to two shared Census facts for RIN 829. The Headless Witness can be removed and,
if necessary, replaced. The query is designed to report on shared family events, too, but there were no Headless Witnesses in the subject database.

Names – Add Married #names #alternatenames #marriage #spouse

Update 2021-03-11: #RM8 version by Pat Jones added.

Having the married name of a person as an Alternate Name can be useful to those more familiar with it than the birth name as per LessTX's posting on the Wish List:
Add Married Name as Alternate Name and has been requested over the years in these postings found in the RootsMagic Forums. Or for looking up someone in the database from a source that cites a married name. While RootsMagic 6 currently does not index alternate names for reports, it does for its new Publish Online website generator. This query adds a Married name as an Alternate to those persons not already having one.

The Alternate Name fact generated fills the fields as follows:

Name-AddMarried-EditPersonCapture.PNG
Example of an Alternate Name fact created by the query.

Given Names: Husband's
Surname: Husband's
Name Type: Married
Prefix: Mrs.
Suffix: (Wife's_primary_surname, Wife's_Given)
Nickname: Wife's
Proof: empty
Date: MarriageDate
SortDate: After MarriageDate
Private: unchecked

The Name Index for RM Reports does not include Alternate Names but that for the Publish Online website of RM6 does; so Private is left unchecked.

The default sentence for this fact reads clumsily and adds nothing to a narrative; by setting the Custom sentence for each such fact to one space character " ", no sentence is outputted. However, the Alternate Name field values are outputted in Individual Summary and other table type reports.

Download

Name-Add_Married-RM8 2021-03-11 rev by Pat Jones for #RM8

Name-Add_Married-RM7 RMtrix_tiny_check.png compatible with RM5 to RM7

Requires SQLiteSpy with the fake RMNOCASE collation.
DO NOT USE ON A DATABASE EARLIER THAN RM5.
USE RootsMagic File > Database tools > Rebuild Indexes after executing the query.

-- Name-Add_Married.sql
/*
2012-11-24 Tom Holden ve3meo
 
Inserts an Alternate Name of type "Married" for female "Wife" spouses
in a "Family" where no Alternate Surname matches the male "Husband's" and
a (family) Marriage fact exists.
 
Prefixes with "Mrs.".
Suffixes the birth name in parentheses ().
 
Assigns the Marriage date and sortdate to the Alternate Name.
 
Sets the fact to Not Private and the fact sentence to ' ', assuming the user wishes to have
the Married Name show on the Publish Online website Name Index but not in narrative reports. It will
show in other reports but alternate names do not come out in report Name Indexes.
 
Because the NameTable uses the RMNOCASE collation, the query first REINDEXes
the database against the fake RMNOCASE collation. Following the query,
it is essential to use RootsMagic's File > Database tools > Rebuild Indexes, i.e.,
do not use this procedure on databases prior to RootsMagic 5.
*/
 
-- Following deletes all Alternate Names of type Married
-- DELETE FROM NameTable WHERE NameType = 5 AND NOT +IsPrimary;
 
-- Name-Add_Married
REINDEX -- because following procedure does string matches on columns indexed using RMNOCASE
;
 
INSERT OR REPLACE INTO NameTable
SELECT
 NULL AS NameID,
 Wife.OwnerID AS OwnerID,
 Husband.Surname AS Surname,
 Husband.Given AS Given,
 'Mrs.' AS Prefix,
 '(' || Wife.Surname || ', ' || Wife.Given || ')' AS Suffix,
 Wife.Nickname AS Nickname,
 5 AS NameType,
 Event.DATE AS DATE, -- set to ',' for undated, Event.Date to match Marriage date.
 CASE Event.SortDate & 1023
  WHEN 1023 THEN Event.SortDate --
  ELSE Event.SortDate + 1
 END AS SortDate,
-- Event.SortDate +1 AS SortDate, -- set to 9223372036854775807 to sort with undated Alt Names, Event.SortDate to match Marriage
 0 AS IsPrimary,
 0 AS IsPrivate, -- set to 1 to make Private
 0 AS Proof,
 0.0 AS EditDate,
-- 'After [person:hisher] marriage, [person] was also known as [Desc].' AS Sentence,
 ' ' AS Sentence, -- a blank space to prevent the default sentence from being outputted
 '' AS Note,
 Wife.BirthYear AS BirthYear,
 Wife.DeathYear AS DeathYear
FROM NameTable Wife
INNER JOIN FamilyTable Family ON Wife.OwnerID=MotherID AND +Wife.IsPrimary
INNER JOIN NameTable Husband ON FatherID = Husband.OwnerID AND +Husband.IsPrimary
INNER JOIN EventTable Event ON Family.FamilyID = Event.OwnerID AND Event.EventType = 300
INNER JOIN PersonTable Person2 ON Wife.OwnerID = Person2.PersonID AND Person2.Sex=1
INNER JOIN PersonTable Person1 ON Husband.OwnerID = Person1.PersonID AND Person1.Sex=0
WHERE
 Wife.Surname NOT LIKE Husband.Surname AND
 Wife.OwnerID NOT IN
(
-- Wives with Alternate Surnames matching Husband's Surname
SELECT Wife.OwnerID --, Wife.IsPrimary, Wife.Given, Wife.Surname, Husband.Surname
FROM NameTable Wife
INNER JOIN FamilyTable ON Wife.OwnerID=MotherID AND NOT +Wife.IsPrimary
INNER JOIN NameTable Husband ON FatherID = Husband.OwnerID AND +Husband.IsPrimary
WHERE Wife.Surname LIKE Husband.Surname
   OR Wife.Surname LIKE Wife.Surname || '-' || Husband.Surname
)
;
 
--- BE SURE TO REBUILD INDEXES IN ROOTSMAGIC --

Sharable Events – Find and Convert to Shared #sharedevent

Problem:

MarkVS posted:
I have still not tackled the task of going through my RM4 database to merge up events that were originally added separately for each individual (in RM3) but would have been added as a shared event in RM4, and RM5.

A Query that would show events that were likely to be suitable for merging would be very useful. In the vast majority of cases this would be a Census event – but it might apply to tohers.I was thinking along the lines of something that listed Census events where the following data was identical
a) the date of the event
b) Census type
c) identical address and place

Then sorted into date/address/place sequence so the likley identical events would be listed together for potential manual update in RM4/5.

Perhaps there would be a cleverer way to identify merge candidates across all Event types?

MVS.


Finding Sharable Events

SharableFacts2.sql RMtrix_tiny_check.png for RM4 to RM7
SharableFacts2-RM8.sql for #RM8 2022-02-21

Working on Lee Irons request, I came up with this adaptation of the SourceList.sql query that displays and counts events having citations. His database, or at least those events he wants merged as shared events, is rigorously and systematically cited so that was the easier query to start with. The “Sharable” column lists the number of other persons or events having the same EventDate, “Cited by”, “Description”, “Event Place”, “Event Site”, “Source Name”, “Cit Fields” values.


Convert Sharable Events to Shared – A Start

Lee Irons posed the problem coming from Legacy Family Tree 7.5 that its nearest equivalent to RM’s shared events is to copy facts to different people. When imported to RootsMagic, it would be advantageous to merge these to shared facts. He outlined his preferred merging as:

This is how the merge would work for shared individual facts. Facts associated with different persons that are exactly the same in terms of Fact Type, Date, Place, Place Details, Description, and Sources but have different information for each person in the Note field would be merged into a single shared individual fact. This new shared indivdual fact would have the same Fact Type, Date, Place, Place Details, Description, and Sources as before, and all of the persons would be listed in the People Who Share This Fact with each person’s unique data from the Note field of their indivudal fact being moved to the Role Note for that person in the People who Share This Fact list of the new shared individual fact. The Note field for the new shared individual fact would be left blank.

This is how the merge would work for shared family facts. This function would merge macthing individual facts and a matching family fact into a new shared family fact. When there is (1) the same situation as identified above for individual facts and (2) the persons that have these matching individual facts are part of the same family and (3) there is a family fact with exactly the same Fact Name as the individual fact type and exactly the same Date, Place, Place Details, and Description as the matching indivdual facts of the persons in the same family, then all of these would be merged into a single shared family fact. The new shared family fact would (1) get the Family Fact Type, Date, Place, Place Details, Description, and Note of the original family fact, and (2) would get the source information of the matching individual facts, and (3) would have all of the individuals in the family who have a matching individual fact added to the People Who Share This Fact list of the new family fact, with each person’s unique data from the Note field of their indivudal fact being moved to the Role Note for that person in the People who Share This Fact list of the new shared family fact.

After many hours of feverish brain activity, I think I have a solution for RootsMagic 4-7 in
SharableEvents-Convert.sql (requires a RMNOCASE collation – see RMNOCASE – faking it in SQLiteSpy).
This query has been used with both RM4 and 5 but it does not yet do anything with media that may be linked to an event which has been deleted in favour of sharing another event. Media handling may have to be different for the two versions. Nor does it do anything about assigning roles to the sharers of an event – that could be automated with more grey cells fried. Already shared events that become sharers of another event have their own sharers reassigned to their new shared event; this appears to work correctly with limited testing.

One challenge was choosing which individual was to be the Principal for a group of sharable individual events. The eldest male was a natural choice for 19th century events but what if there was an older female, e.g., the widow with children – the eldest male might be a child. The algorithm I developed results in the eldest female as Principal if she is more than 10 years older than the eldest male. Of course that might result in the elderly woman being the Principal for a census event when she is resident in her son’s house.

As written, there is some duplication of procedures between the two blocks of code corresponding to Lee’s two merge outlines; these can be split apart and run independently. I would welcome any feedback on how it works with any database for which sharable events figure significantly.

SharableEventsConvert-Properties.PNG
Sample database imported from a GEDCOM from Legacy Family Tree 7.5

In the sample database Lee sent, about 3200 events were converted to sharers, resulting in a reduction of about 3200 citations. Everything else on the properties report remained the same.

SharableEventsConvert-EditPersonBeforeAfter.png
Sample database Edit Person screen before and after conversion

As delivered, the GEDCOM resulted in identical Fact Names for both Person and Family facts for Cemetery, Occasion and Proximity. Renaming would distinguish them readily on the Edit Person screen but the Details give a clue. “Household of Irons, John …” is an Individual’s fact while “Household of Irons-Dunham, … – Henry A. Dunh…” is a Family fact. The highighted event in the Before window is an Indi fact that was converted in the After window to the highlighted share of a family fact. Hannah is one of 10 people sharing the family fact with the two Principals.

SharableEventsConvert-EditSharedEvent.PNG
Sample database after conversion: the Role Note contains the Note formerly in the event which was deleted and replaced by sharing a master event.

With no automatically assigned role, and no role defined for this fact type, the program offers the opportunity to define and assign a new role type. However, it may not be necessary to do so if there is no intent to generate a sentence in reports. It is conceivable that the procedure could automatically assign a pre-existing role or even generate a new role but that may be more confusing than no role at all.

Discussions & comments from Wikispaces site


arfeeld

Running in RM6

arfeeld
29 April 2013 01:53:52

Has anyone had any luck running this against RM6.
I tried and some date is put in the tmp tables but nothing else seems to happen.
Using a Legacy 7.5 imported file.

Ron


ve3meo

ve3meo
29 April 2013 04:27:21

I just ran it against a RM6 database created by importing the same Legacy 7.5 GEDCOM as in the example and it had the same dramatic effect on File Properties and produced shared events. Perhaps your database has highly similar events that differ just enough that the algorithm does not accept them as sharable.

Tom


ve3meo

Inline comment: “about 3200 events were converted to sharers, resulting in a reduction of about 3200 citations. Everything else on the properties report remained the same”

ve3meo
04 September 2018 02:57:50

ve3meo Dec 17, 2011

The first draft of this page compared the wrong pair of database files before and after conversion. It understated the numbers of conversions and had an initially puzzling discrepancy in the number of sources.

Inline comments


ve3meo

Comment: The first draft of this page compared…

ve3meo
18 December 2011 03:04:32

The first draft of this page compared the wrong pair of database files before and after conversion. It understated the numbers of conversions and had an initially puzzling discrepancy in the number of sources.

Death Year Mismatch #date #death

Lists individuals whose Death Year is missing from RootsMagic Explorer (and other views and reports where just the YEAR is outputted) or mismatches the value that has been stored in the Date field of the Death fact. Adapted from the query Birth Year Mismatch.

New: see Rebuild Indexes and Update Birth and Death Years for a batch procedure to match up the Birth and Death years with their facts.

Download: DeathYearMisMatch.sqlRMtrix_tiny_check.png

Listing:

-- DeathYearMisMatch.sql
-- Lists individuals whose Death Year is missing from the NameTable (and thus
-- missing from views and reports where just the YEAR is outputted) or
-- mismatches the value that has been stored in the Date field of the Death fact.
-- #1 2010-06-03 created by ve3meo, based on BirthYearMismatch.sql
-- #2 2012-01-27 corrected overcount of Death fact due to alternate names
 
SELECT
  RIN ,
  Surname ,
  Suffix ,
  Prefix ,
  Given AS 'Given Name(s)',
  DeathYear AS 'Death Year',
  DATE AS 'Death Fact Date',
  IsPrimary AS 'Primary?',
  DeathCount AS 'Death Facts'
FROM
  (
    SELECT
      N.Ownerid AS Rin ,
      N.Surname COLLATE Nocase ,
      N.Suffix COLLATE Nocase ,
      N.Prefix COLLATE Nocase ,
      N.Given COLLATE Nocase ,
      N.Deathyear ,
      E.DATE ,
      E.Isprimary ,
      LENGTH( N.Deathyear ) AS Bystrlen , -- Death year string length
      COUNT( 1 ) AS Deathcount            -- count up multiple Death facts
    FROM
      Nametable N ,
      Eventtable E
    WHERE
      N.Ownerid = E.Ownerid AND E.Eventtype = 2 AND E.Ownertype = 0 AND +N.IsPrimary
    GROUP BY
      1
  )
WHERE
  NOT LIKE( Deathyear , Substr( DATE , 8 - Bystrlen , Bystrlen ) )   -- the mis-match test,
  -- cannot find Death Year where it should be in the Date field
  AND DATE NOT LIKE '.' -- no date entered
  AND DATE NOT LIKE 'TUNKNOWN' -- example of suppressing a not bothersome Date value
ORDER BY
  RIN ;

Birth Year Mis-Match #date #birth

Lists individuals whose Birth Year is missing from the sidebar (and other views and reports where just the YEAR is outputted) or mismatches the value that has been stored in the Date field of the Birth fact. This is less of an issue as of RM 5.0.2.0, which incorporates an update of the BirthYear and DeathYear fields in the NameTable under the Rebuild Indexes tool in the menu File > Database Tools. However, the utility is still useful in drawing attention to where there are multiple Birth facts and where more than one is marked Primary.

New: see Rebuild Indexes and Update Birth and Death Years for a batch procedure to match up the Birth and Death years with their facts.
Download latest: BirthYearMisMatch.sqlRMtrix_tiny_check.png
Ver 4 corrects an overcount of Birth facts due to alternate names.
Ver 3 reports number of Birth facts per individual and suppresses facts with ‘UNKNOWN’ in the Date field.
Ver 2 added checking for years before 1000.

SQLiteDeveloperSQLeditor-BirthYearMisMatch.png
SQLite Developer screen shot from Ver 1.

This sql query uses some sqlite core functions to measure string length, extract a sub-string and compare to another string.

  1. In the above screenshot from the original version of the query, all the Birth Years shown are 0. The Date field could be worked on in RM4’s Edit Person dialog to change from free text (stored with a ‘T’ prefix) to a recognisable date format (stored with a ‘D’ prefix) and that should update the Birth Year.
  2. In the case of multiple Birth Facts for an individual, it is the last one saved that sets the Birth Year; any other Birth Fact not matching the Birth Year will be detected. The query could be modified to suppress reporting a mis-matched Birth Fact when another for that individual does match but it was thought to be useful to draw attention to the mis-match for review and possible correction or deletion.
  3. For RIN 681, a valid date format is stored for Jan 17 in the year 0! The mismatch that was detected compared ‘0’ to ‘0000’. The original version of the query reported a mismatch for all years <1000; later versions do not.

RMGC_Properties – Query #database

Description

Lists the properties of a RootsMagic 4 or 5 database except for dates, plus reports extra detail characterizing the database and identifying possible problems in it or in transfers to another other RootsMagic database via GEDCOM or drag’n’drop. Most SQLite managers can export the results to a .csv spreadsheet for tracking and comparison. Could be useful to compare before and after any major operations on the database.

Now has a parameter input dialog to select Summary level (matching RM’s File > Properties report) or default to the Detail level as in the sample below. Summary level works (again and only) with SQLite Developer except for some late (but not the latest) upgrades which require running the last Select after the query first runs. All others skip the parameter input and produce the Detail report. Does not yet report on the RM5 Research Manager feature.

New 2012-03-01: discovered and added class of Headless Citations for Alternate Names mispointing to a Primary Name
New 2012-02-20: corrected Orphaned Alternate Names; adapted Media Date+Description test for RM4/5 differences
Rev 2010-02-13: – Duplicate Citations
Rev 2010-07-05: – Duplicate Citations for families overstated due non-discrimination between different events – corrected.
Rev 2010-07-06: – Duplicate Citations for Individuals erroneously pointed to some family facts – corrected.
Rev 2010-07-14 – added Witness & Role Properties, using a new, unmodified blank database for detection of changes to role names and sentences.
N.B.: note the comment in the sql file about the requirement for a new, empty, unmodified database file for comparison to the file under test.
Rev 2010-08-09 – added Places and Place Details: Used, having Geo-coordinates; added FactType and SourceTemplate properties
Rev 2010-08-10 – added Version property as recorded in ConfigTable
Rev 2010-12-13 – added Multimedia duplicate filenames and Multimedia duplicate links properties

RMGC_Properties.sqlRMtrix_tiny_check.png
Also download or create a reference empty database and revise the path in the script to match its location on your system: Empty5021.rmgc

Sample Output

ValueVariableRemark
5008Versionvs Control version 5008
1157Peopleall records in PersonTable
0– Nameless Peopleno record in NameTable for that RIN
97– Unresolved Duplicate Name Pairspairs of Given and Surnames, not flagged as “Not a Problem”
16– Resolved* Duplicate Name Pairsflagged as “Not a Problem” – flags lost on transfer
2– Unresolved Duplicates with Media Linkssecondary persons’ links lost on merge
39Alternate namesall records in NameTable where IsPrimary=0
0– Orphaned Alternate names*no Primary name record found
389Familiesall records in FamilyTable
66Fact Typesno. of records from FactTypeTable
2– Custom Fact Typesno. of custom Fact Types
9– Customised Built-in Fact Typesno. of built-in Fact Types modified
35– Unused Fact Typesno. of Fact Types not used
0– Blank Fact Type NamesFactTypes must be named
0– Blank FactType SentencesFactType needing definition
2831Eventsall records of EventTable
0– Orphaned Eventsevents for which no person or family match in respective tables
135– Event WitnessesAll records in WitnessTable of persons sharing events
17— Nominal Witnessesnot Persons from database, but named in WitnessTable as sharing an event
0— Headless WitnessesPersonID (RIN) in WitnessTable missing from PersonTable
0— Witnesses to Lost EventsEventID in WitnessTable cannot be found in EventTable
0— Witnesses with blank Roleno role has been assigned from RoleTable or the RoleTable role is empty
3— Witnesses with Custom Sentencea custom sentence has been assigned, unique to this witness
27— Witnesses with Notea note has been entered for this witness to an event
81— Rolesno. of records from RoleTable
23– Custom Rolesno. of custom roles
1– Customised Built-in Rolesno. of built-in roles modified
54– Unused Rolesno. of roles not used
0– Blank Role NamesRoles needing definition
2– Blank Role SentencesRoles needing definition
775Total Placesall records in PlaceTable incl Places and Place Details (Sites)
146– System Placessystem supplied Places: LDS Temples
470– User Placesuser defined Places excl Sites
296— Used, having Geo-coordinatesnon-empty Lat or Long
27— Unused User Places*not used by EventTable, will be dropped in a transfer
159— User Place Detailsuser defined Sites
9– Used, having Place Detail Notes*Site Notes will be lost in a transfer
30– Used, having Geo-coordinatesnon-empty Lat or Long
1– Unused Place Details*Sites will be lost in a transfer
829Source Templates
  1. of records from SourceTemplateTable
416– Custom Source Templates
  1. of custom Source Templates
0– Unupdated Built-in SourceTemplates
  1. not matching reference database
393– Unused Custom SourceTemplates*lost on transfer
0– Incomplete Source Templatesmissing part of definition
139Total Sourcesall records from SourceTable
2– Unused Sources*SourceTable records unused by CitationTable
1697Total Citationsall records from CitationTable
0– Duplicate Citationsidentical in most respects, cluttering reports
0– Sourceless Citations*no SourceTable record for this CitationTable record
10– Headless Citations*CitationTable records for which no Person, Event, Family, AltName found; cleaned on transfer
12Repositoriesall records from AddressTable of type Repository
36To- do tasksall records from ResearchTable
123Multimedia itemsall records from MultimediaTable
12– lacking thumbnailprobably an imported reference to an image file that has yet to be found
0– duplicate multimedia filenamesprobably having different paths
158Multimedia linksall records from MediaLinkTable
0– with Date & Description*(RM4) if a record has both, the Description is lost in a transfer
10– with Date & Description*(RM5) if a record has both, the Description is lost in a transfer
5– duplicate multimedia linksimage appears multiple times for person, fact
20Addressesall records from AddressTable of type Address
0– blank namesName field of AddressTable record is blank
0Correspondenceall records from ResearchTable of type Correspondence
  • NOT TRANSFERABLE
via GEDCOM or Drag&Drop to another RM database

Discussions & comments from Wikispaces site


ve3meo

Tracing the “Headless Witnesses”

ve3meo
07 December 2011 17:14:19

Following Don Newcomb’s post at RootsMagic Forum, let’s look for a solution.

RMGC_Properties query reports a number of Headless citations. The core of that query gives a list of the RINs that were once witness to some event but are no longer persons in the database. Reviewing an earlier backup of the database may reveal these persons.
-- List of RINs of Headless Witnesses
SELECT PersonID FROM WitnessTable WHERE WitnessTable.PersonID > 0
EXCEPT SELECT PersonID FROM PersonTable
;

A more complex query could look up the persons in the database currently owning (the Principal role) or sharing the event.


ve3meo

Database Properties List – Query #databaseproperties

Problem

Currently under RootsMagic 4, the only way to obtain the Database Properties for a file is to go to File > Properties and use Alt+Prnt Screen (or Ctrl+Prnt Scrn) to copy the screen to the clipboard and use CTRL+V to paste the contents to a program capable of rendering the result.

Solution pt 1

In the meantime, the following code can be used to set up and populate a temporary table (outside of one’s database): (the full solution is integrated in RMtrix_tiny_check.png)

CREATE TEMP TABLE VariableTable
(
Variable TEXT,
VALUE INTEGER
);
 
INSERT INTO VariableTable
SELECT 'People', COUNT(1)
FROM PersonTable;
 
INSERT INTO VariableTable
SELECT 'Families', COUNT(1)
FROM FamilyTable;
 
INSERT INTO VariableTable
SELECT 'Events', COUNT(1)
FROM EventTable;
 
INSERT INTO VariableTable
SELECT 'Alternate names', COUNT(1)
FROM NameTable
WHERE IsPrimary = 0;
 
INSERT INTO VariableTable
SELECT 'Places', COUNT(1)
FROM PlaceTable
WHERE PlaceType = 0;
 
INSERT INTO VariableTable
SELECT 'Sources', COUNT(1)
FROM SourceTable;
 
INSERT INTO VariableTable
SELECT 'Citations', COUNT(1)
FROM CitationTable;
 
INSERT INTO VariableTable
SELECT 'Repositories', COUNT(1)
FROM AddressTable
WHERE AddressType = 1;
 
INSERT INTO VariableTable
SELECT 'To-do tasks', COUNT(1)
FROM ResearchTable
WHERE TaskType = 0;
 
INSERT INTO VariableTable
SELECT 'Multimedia items', COUNT(1)
FROM MultimediaTable;
 
INSERT INTO VariableTable
SELECT 'Multimedia links', COUNT(1)
FROM MediaLinkTable;
 
INSERT INTO VariableTable
SELECT 'Addresses', COUNT(1)
FROM AddressTable
WHERE AddressType = 0;
 
INSERT INTO VariableTable
SELECT 'Correspondence', COUNT(1)
FROM ResearchTable
WHERE TaskType = 1;

Solution pt 2

The following code can then be run to produce the results of the Database Properties List:

SELECT *
FROM VariableTable;

Finally, this code can be run to drop that temporary table (or the SQLite manager can just be closed):

DROP TABLE VariableTable;

The results of the query should be able to be copied or exported in some useful format to a file, depending on the capabilities of the SQLite manager used.

The two items not currently a part of the result set on the first pass of trying to reproduce it include Database and Created.

Discussions & comments from Wikispaces site


ve3meo

Good idea!

ve3meo
29 January 2010 14:02:56

Good idea, Romer! You could add to the page that the results may be copied or exported in some useful format, depending on the sqlite manager used, to a file.

Source Templates – Field Count #sourcetemplates

Update 2021-03-10: tested with #RM8 and minor rev for wider compatibility with SQLite managers (Pat Jones)

The complexity of a Source Template is somewhat proportional to the number of fields and their distribution between the Master Source and Source Details (citation-specific). It is probable that a source with a greater number of fields in total AND a greater number of them in Source Details will be more complex than one that has lesser numbers and will suffer more distortion and loss when exported to standard GEDCOM (see Source Templates, A Comparative Example.pdf). There are other factors that complicate the fidelity of export which could well result in some exceptions but they are harder to measure. These include:

  • fields in the RM Footnote sentence template out of order from the simple convention of Master fields first followed by Source Detail fields
  • words or strings that are within a sentence template switch that are conditional on a non-empty Source Detail field

This query does the easy task of counting the total number of fields and the number of those that are Source Detail fields.

SourceTemplates-FieldCount.PNG
Screenshot from SQLiteSpy showing results from some of the built-in Source Templates. You can sort by any column by clicking on it.

SourceTemplates-FieldCount.sql

SourceTemplates-FieldCount.sql

-- SourceTemplates-FieldCount.sql
-- 2013-04-08 Tom Holden ve3meo

-- works with RM8 PJ 2021-03-07
/*
Converts the FieldDefs blob from SourceTemplateTable to a 1-line string and
counts the number of occurrences of "<FieldName>" in it (the number of fields)
and the subset of same that are Source Detail fields "<CitationField>True".
Does so by measuring the lengths of the FieldDefsTxt strings with and without
the search string and dividing the difference by the length of the search string.
*/
DROP TABLE IF EXISTS xSrcTpltFldDefs;

CREATE TEMP TABLE IF NOT EXISTS xSrcTpltFldDefs AS
	SELECT TemplateID
		,NAME
		,REPLACE(REPLACE(CAST(FieldDefs AS TEXT), X'0A', ''), X'0D', '') AS FieldDefsTxt
	FROM SourceTemplateTable;

SELECT TemplateID
	,NAME
	,(LENGTH(FieldDefsTxt) - LENGTH(REPLACE(FieldDefsTxt, '<FieldName>', ''))) / LENGTH('<FieldName>') AS TotalFields
	,(LENGTH(FieldDefsTxt) - LENGTH(REPLACE(FieldDefsTxt, '<CitationField>True', ''))) / LENGTH('<CitationField>True') AS SrcDtlFields
	,FieldDefsTxt
FROM xSrcTpltFldDefs;