Shared Events With Missing Witnesses #sharedevent

HeadlessWitness.jpg#RM8 compatible

RootsMagic 4 & 5 fail to clean up all the tables after some entities are merged or deleted. The new Database Tools don’t either. Apart from wasted disk space, there are other more concerning consequences. One is what has been called “Headless Witnesses” – these are RINs found in the WitnessTable for whom there is no matching person in the PersonTable. A result is that a fact may show that it is shared with someone whose name is missing.

The RMGC_Properties query reports a count of the number of sharings for which the RIN of the witness does not exist in the database. Its core statement can be brought out to inspect the RINs. Perhaps, by reviewing a database backup, the missing persons can be identified and the matter resolved.

-- List of RINs of Headless Witnesses
SELECT PersonID FROM WitnessTable
WHERE WitnessTable.PersonID > 0
EXCEPT
SELECT PersonID FROM PersonTable
;

A more comprehensive query was developed to report on the role, event and owner or principal of the shared event. Download TraceHeadlessWitnesses.sql RMtrix_tiny_check.png. Here’s a sample result:
TraceHeadlessWitnessesScreenshot.PNG
Using these results the search can be quickly narrowed to two shared Census facts for RIN 829. The Headless Witness can be removed and,
if necessary, replaced. The query is designed to report on shared family events, too, but there were no Headless Witnesses in the subject database.

Names – Add Married #names #alternatenames #marriage #spouse

Update 2021-03-11: #RM8 version by Pat Jones added.

Having the married name of a person as an Alternate Name can be useful to those more familiar with it than the birth name as per LessTX's posting on the Wish List:
Add Married Name as Alternate Name and has been requested over the years in these postings found in the RootsMagic Forums. Or for looking up someone in the database from a source that cites a married name. While RootsMagic 6 currently does not index alternate names for reports, it does for its new Publish Online website generator. This query adds a Married name as an Alternate to those persons not already having one.

The Alternate Name fact generated fills the fields as follows:

Name-AddMarried-EditPersonCapture.PNG
Example of an Alternate Name fact created by the query.

Given Names: Husband's
Surname: Husband's
Name Type: Married
Prefix: Mrs.
Suffix: (Wife's_primary_surname, Wife's_Given)
Nickname: Wife's
Proof: empty
Date: MarriageDate
SortDate: After MarriageDate
Private: unchecked

The Name Index for RM Reports does not include Alternate Names but that for the Publish Online website of RM6 does; so Private is left unchecked.

The default sentence for this fact reads clumsily and adds nothing to a narrative; by setting the Custom sentence for each such fact to one space character " ", no sentence is outputted. However, the Alternate Name field values are outputted in Individual Summary and other table type reports.

Download

Name-Add_Married-RM8 2021-03-11 rev by Pat Jones for #RM8

Name-Add_Married-RM7 RMtrix_tiny_check.png compatible with RM5 to RM7

Requires SQLiteSpy with the fake RMNOCASE collation.
DO NOT USE ON A DATABASE EARLIER THAN RM5.
USE RootsMagic File > Database tools > Rebuild Indexes after executing the query.

-- Name-Add_Married.sql
/*
2012-11-24 Tom Holden ve3meo
 
Inserts an Alternate Name of type "Married" for female "Wife" spouses
in a "Family" where no Alternate Surname matches the male "Husband's" and
a (family) Marriage fact exists.
 
Prefixes with "Mrs.".
Suffixes the birth name in parentheses ().
 
Assigns the Marriage date and sortdate to the Alternate Name.
 
Sets the fact to Not Private and the fact sentence to ' ', assuming the user wishes to have
the Married Name show on the Publish Online website Name Index but not in narrative reports. It will
show in other reports but alternate names do not come out in report Name Indexes.
 
Because the NameTable uses the RMNOCASE collation, the query first REINDEXes
the database against the fake RMNOCASE collation. Following the query,
it is essential to use RootsMagic's File > Database tools > Rebuild Indexes, i.e.,
do not use this procedure on databases prior to RootsMagic 5.
*/
 
-- Following deletes all Alternate Names of type Married
-- DELETE FROM NameTable WHERE NameType = 5 AND NOT +IsPrimary;
 
-- Name-Add_Married
REINDEX -- because following procedure does string matches on columns indexed using RMNOCASE
;
 
INSERT OR REPLACE INTO NameTable
SELECT
 NULL AS NameID,
 Wife.OwnerID AS OwnerID,
 Husband.Surname AS Surname,
 Husband.Given AS Given,
 'Mrs.' AS Prefix,
 '(' || Wife.Surname || ', ' || Wife.Given || ')' AS Suffix,
 Wife.Nickname AS Nickname,
 5 AS NameType,
 Event.DATE AS DATE, -- set to ',' for undated, Event.Date to match Marriage date.
 CASE Event.SortDate & 1023
  WHEN 1023 THEN Event.SortDate --
  ELSE Event.SortDate + 1
 END AS SortDate,
-- Event.SortDate +1 AS SortDate, -- set to 9223372036854775807 to sort with undated Alt Names, Event.SortDate to match Marriage
 0 AS IsPrimary,
 0 AS IsPrivate, -- set to 1 to make Private
 0 AS Proof,
 0.0 AS EditDate,
-- 'After [person:hisher] marriage, [person] was also known as [Desc].' AS Sentence,
 ' ' AS Sentence, -- a blank space to prevent the default sentence from being outputted
 '' AS Note,
 Wife.BirthYear AS BirthYear,
 Wife.DeathYear AS DeathYear
FROM NameTable Wife
INNER JOIN FamilyTable Family ON Wife.OwnerID=MotherID AND +Wife.IsPrimary
INNER JOIN NameTable Husband ON FatherID = Husband.OwnerID AND +Husband.IsPrimary
INNER JOIN EventTable Event ON Family.FamilyID = Event.OwnerID AND Event.EventType = 300
INNER JOIN PersonTable Person2 ON Wife.OwnerID = Person2.PersonID AND Person2.Sex=1
INNER JOIN PersonTable Person1 ON Husband.OwnerID = Person1.PersonID AND Person1.Sex=0
WHERE
 Wife.Surname NOT LIKE Husband.Surname AND
 Wife.OwnerID NOT IN
(
-- Wives with Alternate Surnames matching Husband's Surname
SELECT Wife.OwnerID --, Wife.IsPrimary, Wife.Given, Wife.Surname, Husband.Surname
FROM NameTable Wife
INNER JOIN FamilyTable ON Wife.OwnerID=MotherID AND NOT +Wife.IsPrimary
INNER JOIN NameTable Husband ON FatherID = Husband.OwnerID AND +Husband.IsPrimary
WHERE Wife.Surname LIKE Husband.Surname
   OR Wife.Surname LIKE Wife.Surname || '-' || Husband.Surname
)
;
 
--- BE SURE TO REBUILD INDEXES IN ROOTSMAGIC --

Sharable Events – Find and Convert to Shared #sharedevent

Problem:

MarkVS posted:
I have still not tackled the task of going through my RM4 database to merge up events that were originally added separately for each individual (in RM3) but would have been added as a shared event in RM4, and RM5.

A Query that would show events that were likely to be suitable for merging would be very useful. In the vast majority of cases this would be a Census event – but it might apply to tohers.I was thinking along the lines of something that listed Census events where the following data was identical
a) the date of the event
b) Census type
c) identical address and place

Then sorted into date/address/place sequence so the likley identical events would be listed together for potential manual update in RM4/5.

Perhaps there would be a cleverer way to identify merge candidates across all Event types?

MVS.


Finding Sharable Events

SharableFacts2.sql RMtrix_tiny_check.png for RM4 to RM7
SharableFacts2-RM8.sql for #RM8 2022-02-21

Working on Lee Irons request, I came up with this adaptation of the SourceList.sql query that displays and counts events having citations. His database, or at least those events he wants merged as shared events, is rigorously and systematically cited so that was the easier query to start with. The “Sharable” column lists the number of other persons or events having the same EventDate, “Cited by”, “Description”, “Event Place”, “Event Site”, “Source Name”, “Cit Fields” values.


Convert Sharable Events to Shared – A Start

Lee Irons posed the problem coming from Legacy Family Tree 7.5 that its nearest equivalent to RM’s shared events is to copy facts to different people. When imported to RootsMagic, it would be advantageous to merge these to shared facts. He outlined his preferred merging as:

This is how the merge would work for shared individual facts. Facts associated with different persons that are exactly the same in terms of Fact Type, Date, Place, Place Details, Description, and Sources but have different information for each person in the Note field would be merged into a single shared individual fact. This new shared indivdual fact would have the same Fact Type, Date, Place, Place Details, Description, and Sources as before, and all of the persons would be listed in the People Who Share This Fact with each person’s unique data from the Note field of their indivudal fact being moved to the Role Note for that person in the People who Share This Fact list of the new shared individual fact. The Note field for the new shared individual fact would be left blank.

This is how the merge would work for shared family facts. This function would merge macthing individual facts and a matching family fact into a new shared family fact. When there is (1) the same situation as identified above for individual facts and (2) the persons that have these matching individual facts are part of the same family and (3) there is a family fact with exactly the same Fact Name as the individual fact type and exactly the same Date, Place, Place Details, and Description as the matching indivdual facts of the persons in the same family, then all of these would be merged into a single shared family fact. The new shared family fact would (1) get the Family Fact Type, Date, Place, Place Details, Description, and Note of the original family fact, and (2) would get the source information of the matching individual facts, and (3) would have all of the individuals in the family who have a matching individual fact added to the People Who Share This Fact list of the new family fact, with each person’s unique data from the Note field of their indivudal fact being moved to the Role Note for that person in the People who Share This Fact list of the new shared family fact.

After many hours of feverish brain activity, I think I have a solution for RootsMagic 4-7 in
SharableEvents-Convert.sql (requires a RMNOCASE collation – see RMNOCASE – faking it in SQLiteSpy).
This query has been used with both RM4 and 5 but it does not yet do anything with media that may be linked to an event which has been deleted in favour of sharing another event. Media handling may have to be different for the two versions. Nor does it do anything about assigning roles to the sharers of an event – that could be automated with more grey cells fried. Already shared events that become sharers of another event have their own sharers reassigned to their new shared event; this appears to work correctly with limited testing.

One challenge was choosing which individual was to be the Principal for a group of sharable individual events. The eldest male was a natural choice for 19th century events but what if there was an older female, e.g., the widow with children – the eldest male might be a child. The algorithm I developed results in the eldest female as Principal if she is more than 10 years older than the eldest male. Of course that might result in the elderly woman being the Principal for a census event when she is resident in her son’s house.

As written, there is some duplication of procedures between the two blocks of code corresponding to Lee’s two merge outlines; these can be split apart and run independently. I would welcome any feedback on how it works with any database for which sharable events figure significantly.

SharableEventsConvert-Properties.PNG
Sample database imported from a GEDCOM from Legacy Family Tree 7.5

In the sample database Lee sent, about 3200 events were converted to sharers, resulting in a reduction of about 3200 citations. Everything else on the properties report remained the same.

SharableEventsConvert-EditPersonBeforeAfter.png
Sample database Edit Person screen before and after conversion

As delivered, the GEDCOM resulted in identical Fact Names for both Person and Family facts for Cemetery, Occasion and Proximity. Renaming would distinguish them readily on the Edit Person screen but the Details give a clue. “Household of Irons, John …” is an Individual’s fact while “Household of Irons-Dunham, … – Henry A. Dunh…” is a Family fact. The highighted event in the Before window is an Indi fact that was converted in the After window to the highlighted share of a family fact. Hannah is one of 10 people sharing the family fact with the two Principals.

SharableEventsConvert-EditSharedEvent.PNG
Sample database after conversion: the Role Note contains the Note formerly in the event which was deleted and replaced by sharing a master event.

With no automatically assigned role, and no role defined for this fact type, the program offers the opportunity to define and assign a new role type. However, it may not be necessary to do so if there is no intent to generate a sentence in reports. It is conceivable that the procedure could automatically assign a pre-existing role or even generate a new role but that may be more confusing than no role at all.

Discussions & comments from Wikispaces site


arfeeld

Running in RM6

arfeeld
29 April 2013 01:53:52

Has anyone had any luck running this against RM6.
I tried and some date is put in the tmp tables but nothing else seems to happen.
Using a Legacy 7.5 imported file.

Ron


ve3meo

ve3meo
29 April 2013 04:27:21

I just ran it against a RM6 database created by importing the same Legacy 7.5 GEDCOM as in the example and it had the same dramatic effect on File Properties and produced shared events. Perhaps your database has highly similar events that differ just enough that the algorithm does not accept them as sharable.

Tom


ve3meo

Inline comment: “about 3200 events were converted to sharers, resulting in a reduction of about 3200 citations. Everything else on the properties report remained the same”

ve3meo
04 September 2018 02:57:50

ve3meo Dec 17, 2011

The first draft of this page compared the wrong pair of database files before and after conversion. It understated the numbers of conversions and had an initially puzzling discrepancy in the number of sources.

Inline comments


ve3meo

Comment: The first draft of this page compared…

ve3meo
18 December 2011 03:04:32

The first draft of this page compared the wrong pair of database files before and after conversion. It understated the numbers of conversions and had an initially puzzling discrepancy in the number of sources.

Death Year Mismatch #date #death

Lists individuals whose Death Year is missing from RootsMagic Explorer (and other views and reports where just the YEAR is outputted) or mismatches the value that has been stored in the Date field of the Death fact. Adapted from the query Birth Year Mismatch.

New: see Rebuild Indexes and Update Birth and Death Years for a batch procedure to match up the Birth and Death years with their facts.

Download: DeathYearMisMatch.sqlRMtrix_tiny_check.png

Listing:

-- DeathYearMisMatch.sql
-- Lists individuals whose Death Year is missing from the NameTable (and thus
-- missing from views and reports where just the YEAR is outputted) or
-- mismatches the value that has been stored in the Date field of the Death fact.
-- #1 2010-06-03 created by ve3meo, based on BirthYearMismatch.sql
-- #2 2012-01-27 corrected overcount of Death fact due to alternate names
 
SELECT
  RIN ,
  Surname ,
  Suffix ,
  Prefix ,
  Given AS 'Given Name(s)',
  DeathYear AS 'Death Year',
  DATE AS 'Death Fact Date',
  IsPrimary AS 'Primary?',
  DeathCount AS 'Death Facts'
FROM
  (
    SELECT
      N.Ownerid AS Rin ,
      N.Surname COLLATE Nocase ,
      N.Suffix COLLATE Nocase ,
      N.Prefix COLLATE Nocase ,
      N.Given COLLATE Nocase ,
      N.Deathyear ,
      E.DATE ,
      E.Isprimary ,
      LENGTH( N.Deathyear ) AS Bystrlen , -- Death year string length
      COUNT( 1 ) AS Deathcount            -- count up multiple Death facts
    FROM
      Nametable N ,
      Eventtable E
    WHERE
      N.Ownerid = E.Ownerid AND E.Eventtype = 2 AND E.Ownertype = 0 AND +N.IsPrimary
    GROUP BY
      1
  )
WHERE
  NOT LIKE( Deathyear , Substr( DATE , 8 - Bystrlen , Bystrlen ) )   -- the mis-match test,
  -- cannot find Death Year where it should be in the Date field
  AND DATE NOT LIKE '.' -- no date entered
  AND DATE NOT LIKE 'TUNKNOWN' -- example of suppressing a not bothersome Date value
ORDER BY
  RIN ;

Birth Year Mis-Match #date #birth

Lists individuals whose Birth Year is missing from the sidebar (and other views and reports where just the YEAR is outputted) or mismatches the value that has been stored in the Date field of the Birth fact. This is less of an issue as of RM 5.0.2.0, which incorporates an update of the BirthYear and DeathYear fields in the NameTable under the Rebuild Indexes tool in the menu File > Database Tools. However, the utility is still useful in drawing attention to where there are multiple Birth facts and where more than one is marked Primary.

New: see Rebuild Indexes and Update Birth and Death Years for a batch procedure to match up the Birth and Death years with their facts.
Download latest: BirthYearMisMatch.sqlRMtrix_tiny_check.png
Ver 4 corrects an overcount of Birth facts due to alternate names.
Ver 3 reports number of Birth facts per individual and suppresses facts with ‘UNKNOWN’ in the Date field.
Ver 2 added checking for years before 1000.

SQLiteDeveloperSQLeditor-BirthYearMisMatch.png
SQLite Developer screen shot from Ver 1.

This sql query uses some sqlite core functions to measure string length, extract a sub-string and compare to another string.

  1. In the above screenshot from the original version of the query, all the Birth Years shown are 0. The Date field could be worked on in RM4’s Edit Person dialog to change from free text (stored with a ‘T’ prefix) to a recognisable date format (stored with a ‘D’ prefix) and that should update the Birth Year.
  2. In the case of multiple Birth Facts for an individual, it is the last one saved that sets the Birth Year; any other Birth Fact not matching the Birth Year will be detected. The query could be modified to suppress reporting a mis-matched Birth Fact when another for that individual does match but it was thought to be useful to draw attention to the mis-match for review and possible correction or deletion.
  3. For RIN 681, a valid date format is stored for Jan 17 in the year 0! The mismatch that was detected compared ‘0’ to ‘0000’. The original version of the query reported a mismatch for all years <1000; later versions do not.

RMGC_Properties – Query #database

Description

Lists the properties of a RootsMagic 4 or 5 database except for dates, plus reports extra detail characterizing the database and identifying possible problems in it or in transfers to another other RootsMagic database via GEDCOM or drag’n’drop. Most SQLite managers can export the results to a .csv spreadsheet for tracking and comparison. Could be useful to compare before and after any major operations on the database.

Now has a parameter input dialog to select Summary level (matching RM’s File > Properties report) or default to the Detail level as in the sample below. Summary level works (again and only) with SQLite Developer except for some late (but not the latest) upgrades which require running the last Select after the query first runs. All others skip the parameter input and produce the Detail report. Does not yet report on the RM5 Research Manager feature.

New 2012-03-01: discovered and added class of Headless Citations for Alternate Names mispointing to a Primary Name
New 2012-02-20: corrected Orphaned Alternate Names; adapted Media Date+Description test for RM4/5 differences
Rev 2010-02-13: – Duplicate Citations
Rev 2010-07-05: – Duplicate Citations for families overstated due non-discrimination between different events – corrected.
Rev 2010-07-06: – Duplicate Citations for Individuals erroneously pointed to some family facts – corrected.
Rev 2010-07-14 – added Witness & Role Properties, using a new, unmodified blank database for detection of changes to role names and sentences.
N.B.: note the comment in the sql file about the requirement for a new, empty, unmodified database file for comparison to the file under test.
Rev 2010-08-09 – added Places and Place Details: Used, having Geo-coordinates; added FactType and SourceTemplate properties
Rev 2010-08-10 – added Version property as recorded in ConfigTable
Rev 2010-12-13 – added Multimedia duplicate filenames and Multimedia duplicate links properties

RMGC_Properties.sqlRMtrix_tiny_check.png
Also download or create a reference empty database and revise the path in the script to match its location on your system: Empty5021.rmgc

Sample Output

ValueVariableRemark
5008Versionvs Control version 5008
1157Peopleall records in PersonTable
0– Nameless Peopleno record in NameTable for that RIN
97– Unresolved Duplicate Name Pairspairs of Given and Surnames, not flagged as “Not a Problem”
16– Resolved* Duplicate Name Pairsflagged as “Not a Problem” – flags lost on transfer
2– Unresolved Duplicates with Media Linkssecondary persons’ links lost on merge
39Alternate namesall records in NameTable where IsPrimary=0
0– Orphaned Alternate names*no Primary name record found
389Familiesall records in FamilyTable
66Fact Typesno. of records from FactTypeTable
2– Custom Fact Typesno. of custom Fact Types
9– Customised Built-in Fact Typesno. of built-in Fact Types modified
35– Unused Fact Typesno. of Fact Types not used
0– Blank Fact Type NamesFactTypes must be named
0– Blank FactType SentencesFactType needing definition
2831Eventsall records of EventTable
0– Orphaned Eventsevents for which no person or family match in respective tables
135– Event WitnessesAll records in WitnessTable of persons sharing events
17— Nominal Witnessesnot Persons from database, but named in WitnessTable as sharing an event
0— Headless WitnessesPersonID (RIN) in WitnessTable missing from PersonTable
0— Witnesses to Lost EventsEventID in WitnessTable cannot be found in EventTable
0— Witnesses with blank Roleno role has been assigned from RoleTable or the RoleTable role is empty
3— Witnesses with Custom Sentencea custom sentence has been assigned, unique to this witness
27— Witnesses with Notea note has been entered for this witness to an event
81— Rolesno. of records from RoleTable
23– Custom Rolesno. of custom roles
1– Customised Built-in Rolesno. of built-in roles modified
54– Unused Rolesno. of roles not used
0– Blank Role NamesRoles needing definition
2– Blank Role SentencesRoles needing definition
775Total Placesall records in PlaceTable incl Places and Place Details (Sites)
146– System Placessystem supplied Places: LDS Temples
470– User Placesuser defined Places excl Sites
296— Used, having Geo-coordinatesnon-empty Lat or Long
27— Unused User Places*not used by EventTable, will be dropped in a transfer
159— User Place Detailsuser defined Sites
9– Used, having Place Detail Notes*Site Notes will be lost in a transfer
30– Used, having Geo-coordinatesnon-empty Lat or Long
1– Unused Place Details*Sites will be lost in a transfer
829Source Templates
  1. of records from SourceTemplateTable
416– Custom Source Templates
  1. of custom Source Templates
0– Unupdated Built-in SourceTemplates
  1. not matching reference database
393– Unused Custom SourceTemplates*lost on transfer
0– Incomplete Source Templatesmissing part of definition
139Total Sourcesall records from SourceTable
2– Unused Sources*SourceTable records unused by CitationTable
1697Total Citationsall records from CitationTable
0– Duplicate Citationsidentical in most respects, cluttering reports
0– Sourceless Citations*no SourceTable record for this CitationTable record
10– Headless Citations*CitationTable records for which no Person, Event, Family, AltName found; cleaned on transfer
12Repositoriesall records from AddressTable of type Repository
36To- do tasksall records from ResearchTable
123Multimedia itemsall records from MultimediaTable
12– lacking thumbnailprobably an imported reference to an image file that has yet to be found
0– duplicate multimedia filenamesprobably having different paths
158Multimedia linksall records from MediaLinkTable
0– with Date & Description*(RM4) if a record has both, the Description is lost in a transfer
10– with Date & Description*(RM5) if a record has both, the Description is lost in a transfer
5– duplicate multimedia linksimage appears multiple times for person, fact
20Addressesall records from AddressTable of type Address
0– blank namesName field of AddressTable record is blank
0Correspondenceall records from ResearchTable of type Correspondence
  • NOT TRANSFERABLE
via GEDCOM or Drag&Drop to another RM database

Discussions & comments from Wikispaces site


ve3meo

Tracing the “Headless Witnesses”

ve3meo
07 December 2011 17:14:19

Following Don Newcomb’s post at RootsMagic Forum, let’s look for a solution.

RMGC_Properties query reports a number of Headless citations. The core of that query gives a list of the RINs that were once witness to some event but are no longer persons in the database. Reviewing an earlier backup of the database may reveal these persons.
-- List of RINs of Headless Witnesses
SELECT PersonID FROM WitnessTable WHERE WitnessTable.PersonID > 0
EXCEPT SELECT PersonID FROM PersonTable
;

A more complex query could look up the persons in the database currently owning (the Principal role) or sharing the event.


ve3meo

Database Properties List – Query #databaseproperties

Problem

Currently under RootsMagic 4, the only way to obtain the Database Properties for a file is to go to File > Properties and use Alt+Prnt Screen (or Ctrl+Prnt Scrn) to copy the screen to the clipboard and use CTRL+V to paste the contents to a program capable of rendering the result.

Solution pt 1

In the meantime, the following code can be used to set up and populate a temporary table (outside of one’s database): (the full solution is integrated in RMtrix_tiny_check.png)

CREATE TEMP TABLE VariableTable
(
Variable TEXT,
VALUE INTEGER
);
 
INSERT INTO VariableTable
SELECT 'People', COUNT(1)
FROM PersonTable;
 
INSERT INTO VariableTable
SELECT 'Families', COUNT(1)
FROM FamilyTable;
 
INSERT INTO VariableTable
SELECT 'Events', COUNT(1)
FROM EventTable;
 
INSERT INTO VariableTable
SELECT 'Alternate names', COUNT(1)
FROM NameTable
WHERE IsPrimary = 0;
 
INSERT INTO VariableTable
SELECT 'Places', COUNT(1)
FROM PlaceTable
WHERE PlaceType = 0;
 
INSERT INTO VariableTable
SELECT 'Sources', COUNT(1)
FROM SourceTable;
 
INSERT INTO VariableTable
SELECT 'Citations', COUNT(1)
FROM CitationTable;
 
INSERT INTO VariableTable
SELECT 'Repositories', COUNT(1)
FROM AddressTable
WHERE AddressType = 1;
 
INSERT INTO VariableTable
SELECT 'To-do tasks', COUNT(1)
FROM ResearchTable
WHERE TaskType = 0;
 
INSERT INTO VariableTable
SELECT 'Multimedia items', COUNT(1)
FROM MultimediaTable;
 
INSERT INTO VariableTable
SELECT 'Multimedia links', COUNT(1)
FROM MediaLinkTable;
 
INSERT INTO VariableTable
SELECT 'Addresses', COUNT(1)
FROM AddressTable
WHERE AddressType = 0;
 
INSERT INTO VariableTable
SELECT 'Correspondence', COUNT(1)
FROM ResearchTable
WHERE TaskType = 1;

Solution pt 2

The following code can then be run to produce the results of the Database Properties List:

SELECT *
FROM VariableTable;

Finally, this code can be run to drop that temporary table (or the SQLite manager can just be closed):

DROP TABLE VariableTable;

The results of the query should be able to be copied or exported in some useful format to a file, depending on the capabilities of the SQLite manager used.

The two items not currently a part of the result set on the first pass of trying to reproduce it include Database and Created.

Discussions & comments from Wikispaces site


ve3meo

Good idea!

ve3meo
29 January 2010 14:02:56

Good idea, Romer! You could add to the page that the results may be copied or exported in some useful format, depending on the sqlite manager used, to a file.

Source Templates – Field Count #sourcetemplates

Update 2021-03-10: tested with #RM8 and minor rev for wider compatibility with SQLite managers (Pat Jones)

The complexity of a Source Template is somewhat proportional to the number of fields and their distribution between the Master Source and Source Details (citation-specific). It is probable that a source with a greater number of fields in total AND a greater number of them in Source Details will be more complex than one that has lesser numbers and will suffer more distortion and loss when exported to standard GEDCOM (see Source Templates, A Comparative Example.pdf). There are other factors that complicate the fidelity of export which could well result in some exceptions but they are harder to measure. These include:

  • fields in the RM Footnote sentence template out of order from the simple convention of Master fields first followed by Source Detail fields
  • words or strings that are within a sentence template switch that are conditional on a non-empty Source Detail field

This query does the easy task of counting the total number of fields and the number of those that are Source Detail fields.

SourceTemplates-FieldCount.PNG
Screenshot from SQLiteSpy showing results from some of the built-in Source Templates. You can sort by any column by clicking on it.

SourceTemplates-FieldCount.sql

SourceTemplates-FieldCount.sql

-- SourceTemplates-FieldCount.sql
-- 2013-04-08 Tom Holden ve3meo

-- works with RM8 PJ 2021-03-07
/*
Converts the FieldDefs blob from SourceTemplateTable to a 1-line string and
counts the number of occurrences of "<FieldName>" in it (the number of fields)
and the subset of same that are Source Detail fields "<CitationField>True".
Does so by measuring the lengths of the FieldDefsTxt strings with and without
the search string and dividing the difference by the length of the search string.
*/
DROP TABLE IF EXISTS xSrcTpltFldDefs;

CREATE TEMP TABLE IF NOT EXISTS xSrcTpltFldDefs AS
	SELECT TemplateID
		,NAME
		,REPLACE(REPLACE(CAST(FieldDefs AS TEXT), X'0A', ''), X'0D', '') AS FieldDefsTxt
	FROM SourceTemplateTable;

SELECT TemplateID
	,NAME
	,(LENGTH(FieldDefsTxt) - LENGTH(REPLACE(FieldDefsTxt, '<FieldName>', ''))) / LENGTH('<FieldName>') AS TotalFields
	,(LENGTH(FieldDefsTxt) - LENGTH(REPLACE(FieldDefsTxt, '<CitationField>True', ''))) / LENGTH('<CitationField>True') AS SrcDtlFields
	,FieldDefsTxt
FROM xSrcTpltFldDefs;

Census Needed – Named Group #namedgroup #census

Intro

Builds a group of persons whose lifetime probably spanned a user-defined Census Year and who had some event in the user-defined jurisdiction but not a Census fact for that year. The same script can be used for many named groups having different Census Years and target locations. The script includes examples of building lists of RINs that behave like RM’s manual marking and unmarking of persons for a group.

Download

2021-11-25 adapted for #RM8

CensusNeededGroup.sql (for RM4 to pre-RM8)
– initial query works on individual facts only.

CensusNeededGroup2.sql (for RM4 to pre-RM8)
rev. 2020-05-16 to correct misuse of LabelID
– this query works on all types of facts, individual, family and shared by re-using the LifeLines query – a bulldozer solution for a shovel sized pile, but, what the heck!, it was all ready to use.

Usage

  1. Backup your database before you do this every time until you are confident that it does no harm.
  2. In RM, create a named group having the words “Census needed” and “#”CensusYear and Jurisdiction (in that order and without the quotes), where CensusYear is the year of the census, Jurisdiction is the country, or province/state, or municipality.
    e.g., “*Census needed #1871 Canada”
  3. Open your RM database with SQLite Expert Personal and load CensusNeededGroup.sql or CensusNeededGroup2.sql for RM4-preRM8 or, for RM8 databases, CensusNeededGroup2-RM8.sql into the SQL editor.
  4. Execute the query (F5)
  5. You will be prompted to enter a value for @CensusYear. In this example, enter “1871” (without quotes and without the leading #).
  6. You will be prompted to enter a value for @Jurisdiction. In this example, enter “Canada”.
  7. When the query has finished, you will need to refresh the RM4 sidebar view of your group by selecting another group and then re-selecting it.

Discussion

The script looks for the group name that matches its search string. If it does not find a match, nothing happens. Otherwise, the current members list for the group is deleted and the script proceeds to build the new list. Each group has to have a unique name matching the rules above; the user of the script enters the variables year and jurisdiction for it to act on any one of the groups.

Persons having neither a Birth or Death date are ignored by the script as it cannot determine anything about their probability of being alive in the Census Year. Persons having both dates are readily evaluated against the Census Year while those having only one date are given a lifespan of 75 years – that number can be easily found in the script and changed. It could also be incorporated as a parameter specified at run time.

The @Jurisdiction parameter is used to search events with a place name containing the search string. This could be as broad as a country “Canada” as in the example; a state “, OH,” if you used abbreviations (and the leading and trailing punctuation would be needed to avoid picking up every place with “oh” in the name), or a town. If any event occurred in a matching place, that person will be selected unless they have a Census fact for that year.

Thus it is important that there be at least an estimated Birth date and/or an estimated Death date and a guess at the Place of one of these or any other individual event.

To add selected persons that are to be included or excluded from the group, look to the last two sql statements (commented out) in the CensusNeededGroup.sql script. They show how to build the lists and actually carry out the marking and unmarking of the same RINs (net change = 0).

Places – first name exploit for improved narratives #places #reports

Unlike the names of persons, RootsMagic 6 does not yet have any mechanism for automatically revolving through Place names in reports, i.e., using the full place name only if it has not been previously used for a person while taking advantage of the first or lowest level place name or the abbreviated place name for subsequent instances in the person’s narrative. It is up to the user to modify default sentences for Facts and Roles and to create custom sentences for events in order to take advantage of the :first and :short options for the Place field and thus reduce needless repetition of higher level place values such as county, state/province/shire, country. And exacerbating repetition of country is an effect of RootsMagic’s own County Checker and Gazetteer which want to populate the Place field with country.

To mitigate such repetition with a token effort, this script automatically edits the default sentence templates to use [Place:first] instead of [Place] and then goes on to generate custom sentence templates or modify existing ones for all first instances of a place in each person’s chronology to use [Place] instead of [Place:first]. That leaves all subsequent instances of the place for that person to be the default sentence having [Place:first] or to be so modified, which it also does.

For a person who was born, was schooled, worshipped, worked, married, resided, died and was buried in the same Place, his narrative will have but one instance of the full Place name (city, county, state, country) for his birth; all other instances will be just the city. More or less! A “less” is where the person appears as a child below his parent’s narratives as these phrases are not subject to any accessible template.

Places-FirstNameExploit.PNG
Screenshot from MS Word comparing the revised narrative to the original.

In the before/after example above, the Birth event was detected as the first use of Glasgow so a custom sentence for the event was generated that was the same as the original default sentence with the full Place. All the subsequent events in Glasgow, including the shared or witnessed census events, used the new default sentences having [Place:first] so “, Lanarkshire, Scotland, United Kingdom” appears deleted (red strikethrough). The one Quebec City event is a first and only so it received a custom sentence with full Place name. Then North Bay appears three times, the first with a full Place custom sentence, the rest using the first place value per the revised default sentence template.

This utility is not the be all and end all for narratives but it should be useful as a quick streamliner and a basis on which further customisation of sentence templates can deliver further improvement. It does not address all the combinations of options that can be added to [Place] because there could be many. SQLite does not have a regular expression search and replace function which would be necessary to deal with them efficiently and comprehensively.

Places-FirstNameExploit.sql

-- Places-FirstNameExploit.sql
/*
2013-03-29 Tom Holden ve3meo
rev 2013-03-30 added [Place:plain] update to [Place:plain:first] for defaults only.
    needs regexp search & replace to deal with all combinations of Place modifiers
 
Exploits the Place:first option for Place names in narratives by
setting default sentences to Place:first and customising to Place 
only for the first event for a person in any place. 
 
This makes the narrative less wordy and repetitious of the higher levels 
in a Place name. However, it has no effect on the phrases used in the 
Children list at the bottom of a person's narrative which continue 
to use the full place name.
*/
-- Set default Fact sentences to use the first Place name.
UPDATE FactTypeTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]');
UPDATE FactTypeTable
SET Sentence = REPLACE(Sentence, '[Place:plain]', '[Place:plain:first]');
 
-- set default role sentences to use the first Place name
UPDATE RoleTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]');
UPDATE RoleTable
SET Sentence = REPLACE(Sentence, '[Place:plain]', '[Place:plain:first]');
 
-- create table of first event in a place for a person including shared events
DROP TABLE
IF EXISTS xFirstPlaceEvents;
 
CREATE TEMP TABLE
IF NOT EXISTS xFirstPlaceEvents AS
    SELECT *
    FROM (
        -- INDIV events
        SELECT EventID
            ,0 AS isSharer
            ,PlaceID
            ,OwnerID
            ,SortDate
        FROM EventTable
        WHERE OwnerType = 0
            AND PlaceID > 0
 
        UNION
 
        -- Husband events
        SELECT EventID
            ,0 AS isSharer
            ,PlaceID
            ,FatherID AS OwnerID
            ,SortDate
        FROM EventTable
        INNER JOIN FamilyTable ON EventTable.OwnerID = FamilyTable.FamilyID
            AND OwnerType = 1
            AND PlaceID > 0
 
        UNION
 
        -- wife events
        SELECT EventID
            ,0 AS isSharer
            ,PlaceID
            ,MotherID AS OwnerID
            ,SortDate
        FROM EventTable
        INNER JOIN FamilyTable ON EventTable.OwnerID = FamilyTable.FamilyID
            AND OwnerType = 1
            AND PlaceID > 0
 
        UNION
 
        -- shared events
        SELECT WitnessID
            ,1 AS isSharer
            ,EventTable.PlaceID AS PlaceID
            ,WitnessTable.PersonID AS OwnerID
            ,EventTable.SortDate AS SortDate
        FROM WitnessTable NATURAL
        INNER JOIN EventTable
        WHERE EventTable.PlaceID > 0
        ORDER BY OwnerID
            ,SortDate DESC -- so next GROUP BY will pick up the smallest SortDate or first event in the group
        )
    GROUP BY OwnerID
        ,PlaceID
    ORDER BY EventID; -- so the IN() expression in the following queries will see an ordered list
 
-- set all first events for a person to use the default sentence customised with the full Place
-- except those already with custom sentences 
UPDATE EventTable
SET Sentence = (
        SELECT REPLACE(FactTypeTable.Sentence, '[Place:first]', '[Place]')
        FROM EventTable Events
        INNER JOIN FactTypeTable ON Events.EventType = FactTypeID
        WHERE EventTable.EventID = Events.EventID
        )
WHERE EventID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 0
        )
    AND EventTable.Sentence LIKE '' --change this to OR with a match to the default
    OR EventTable.Sentence LIKE (
        SELECT REPLACE(FactTypeTable.Sentence, '[Place:first]', '[Place]')
        FROM EventTable Events
        INNER JOIN FactTypeTable ON Events.EventType = FactTypeID
        WHERE EventTable.EventID = Events.EventID
        );
 
-- set all other custom sentences for first events to use the full name
UPDATE EventTable
SET Sentence = REPLACE(Sentence, '[Place:first]', '[Place]')
WHERE EventID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 0
        );
 
-- set all other custom sentences for non-first events to use the first name
UPDATE EventTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]')
WHERE EventID NOT IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 0
        );
 
--DO The Same steps for shared events
-- set all first witness for a person to use the default sentence customised with the full Place
-- except those already with custom sentences 
UPDATE WitnessTable
SET Sentence = (
        SELECT REPLACE(RoleTable.Sentence, '[Place:first]', '[Place]')
        FROM WitnessTable Witness
        INNER JOIN RoleTable ON Witness.ROLE = RoleTable.RoleID
        WHERE WitnessTable.WitnessID = Witness.WitnessID
        )
WHERE WitnessID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 1
        )
    AND WitnessTable.Sentence LIKE '' --change this to OR with a match to the default
    OR WitnessTable.Sentence LIKE (
        SELECT REPLACE(RoleTable.Sentence, '[Place:first]', '[Place]')
        FROM WitnessTable Witness
        INNER JOIN RoleTable ON Witness.ROLE = RoleTable.RoleID
        WHERE WitnessTable.WitnessID = Witness.WitnessID
        );
 
-- set all other custom sentences for first witness events to use the full name
UPDATE WitnessTable
SET Sentence = REPLACE(Sentence, '[Place:first]', '[Place]')
WHERE WitnessID IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 1
        );
 
-- set all other custom sentences for non-first witness events to use the first name
UPDATE WitnessTable
SET Sentence = REPLACE(Sentence, '[Place]', '[Place:first]')
WHERE WitnessID NOT IN (
        -- EventIDs of first events having a PlaceID for all persons
        SELECT EventID
        FROM xFirstPlaceEvents
        WHERE isSharer = 1
        );