Source Template, A Better Free Form #sourcetemplates #sources #search #replace

For a comprehensive comparison of the outputs from !MyFreeForm3, built-in and custom source templates, and Free Form see A Trio of Templates.

A Better Free Form Source Template

Give me a template that is easy to use, produces good full and short footnotes without extraneous punctuation and exports cleanly via GEDCOM to other software!
WARNING: This page has been extensively updated since 14 Oct 2011. !MyFreeForm3 supersedes !MyFreeForm and !MyFreeForm2 templates which are deprecated due to potential duplicating of the key term in the GEDCOM export along with an unwanted period.

RM4’s Source Templates are a boon and frustrating at the same time. If you can choose the right one for a given source from some 417 different templates, use it correctly, you do end up with a footnote and bibliography style that is pretty consistent with what the gurus advise it should be. That’s great for reports and websites generated from within RM4 but what gets exported to a standard GEDCOM for use by some other software is often pretty badly messed up.

The built-in Free Form template does a better job on exporting, you have total control over what gets into the TITL and PAGE tags, only limited by the order in which the tags are used to create footnotes. However, the Free Form template falls down in RM reports when a source is repeatedly cited, because there is only one Source Details field, Page, used by both the full, first footnote and by subsequent short footnotes. Thus, if lots of detail (e.g., think of a census citation) is entered in the Page field to ensure that the particular location within a source can readily be found, that same lengthy detail gets repeated in the subsequent short footnotes.

I have wrestled at length to come up with a Source Template that minimizes constraints (fewest fields=max flexibility), provides a short footnote if needed, exports nicely to standard GEDCOM, and has no extraneous punctuation. That has not been easy, given RM4’s idiosyncrasies. I believe !MyFreeForm3 is about the best that one can do. This custom template mimics the built-in Free Form template, adding one field in Source Details: Key: [ShortPage], for use in Short Footnotes instead of Details:[Page] (renamed from Page:[Page]). It also provides an additional field Punctuation:[Punc] in the Master Source with which you can override the default “, ” separator in the footnote between master source and source details.

!MyFreeForm3-Edit_Source_screenshot.png
Example of a source in !MyFreeForm3

Import this Source Template

Download the template file !MyFreeForm3.rmst to your computer. From RM4 main screen, select Lists > Source Templates to open the Source Templates window. Then select Import and browse to locate and select the .rmst file. If the new template was imported, at or near the top of the list of Source Types, you should see !MyFreeForm3. It’s now available for the creation of new sources.

Using !MyFreeForm3

Guidance on entering data

  1. For Footnote, Short Footnote and Bibliography, follow the same practices as you would with the Free Form template, but do not end any with a period.
  2. The Punctuation field only affects footnotes in RM4 generated reports, not exports. Plug in a character to see the effect in the right pane.
  3. The two Source Details fields can be both filled, both null, or either one filled. Only when both are filled will the resulting footnote and short footnote differ in any way other than what you have defined in the corresponding Master Source fields. Filling neither or one is equivalent to the Free Form template.
  4. Ending footnotes with a period is pretty standard. Due to an export issue, the template must not automatically supply it. You provide the ending period at the end of the bottom-most Source Details field you use. If both Source Details fields are empty, the footnote will not end with a period (sorry, there’s no way around it).
  5. On export, RM4 is going to stuff “; ” between non-null Details:[Page] and Key:[ShortPage] which it puts into the PAGE tag. For consistency, that’s what the full Footnote sentence template does, too.

Compose draft source with other templates

To mimic in !MyFreeForm3 the EE et al footnotes as well as one can, I try using a two-stage procedure. First, I’ll use a Source Template suited to the type of source I am citing as a guide for how I may write a !MyFreeForm source. I fill in the fields of the selected template to generate the full and short footnotes and bibliography in the side panel, and copy these results to a word processor. Then I can parse the footnotes into their Master Source values (Footnote and ShortFootnote) and into their Source Details values (Details:[Page] and Key:[ShortPage]). Having done that, then I cancel the source creation and start over with the !MyFreeForm template, copying the values into the corresponding fields. That results in footnotes that resemble, to some degree, the gurus’ footnotes, at least to the same level of detail, if not order.

Copy draft citations from Ancestry.com et al

Lots of sites provide citations of their resources and there are online Citation Machines. In effect, they have used their own templates. Breaking them down into component parts is the same as for the draft sources created using RM4 source templates, described above. It’s a whole lot easier to parse them into !MyFreeForm3 than any other RM4 template, save the Free Form.

Examples

Have a look at this page for an example of a !MyFreeForm3 template used for the 1930 US Census: Source Template, A Better Free Form – Examples
and at the discussion below for an example of a directory as a source.

Merging

Merging of Free Form sources into !MyFreeForm3 sources is lossless, datawise. All the fields of Free Form are duplicated in !MyFreeForm3; only the Punctuation:[Punc] and Key:[ShortPage] fields are unique and will remain empty after the merge. However, to take advantage of the better Short Footnote, you will have to take out the key value from the Details:[Page] field and put it into the Key:[ShortPage]field of each citation. As long as Key:[ShortPage] remains empty, the Short Footnote will come out essentiallly the same as from Free Form.

Global Conversion to !MyFreeForm3

Global conversion of Free Form sources to !MyFreeForm3 sources is easily and quickly done using SQLite. Open the database with your SQLite manager and inspect SourceTemplateTable for the value !MyFreeForm3 in the Name column. It will be at or near the bottom of the table with a TemplateID of 10000 or greater. Copy the following queries, edit them to plug in the TemplateID of !MyFreeForm3 in your database in place of YourTemplateID and run the queries to map all your FreeForm sources to the !MyFreeForm template:

-- Map Free Form sources to !MyFreeForm3 source template
UPDATE SourceTable SET TemplateID=YourTemplateID WHERE TemplateID=0
;
-- Add a [ShortPage] definition to every citation of a !MyFreeForm3 source lacking one
--  else you will see the text "[ShortPage]" outputted in RM4 reports
UPDATE CitationTable
 SET FIELDS = CAST(REPLACE(CAST(FIELDS AS TEXT),'</Fields>', '<Field><Name>ShortPage</Name><Value/></Field></Fields>') AS BLOB)
  WHERE CitationID IN
   (SELECT CitationID FROM CitationTable C JOIN SourceTable S USING(SourceID)
     WHERE TemplateID=YourTemplateID AND CAST(C.FIELDS AS TEXT) NOT LIKE '%<Name>ShortPage%'
    )
;
-- A similar update of the SourceTable is required for the [Punc] field of !MyFreeForm3
--  else you will see the text "[Punc]" outputted in RM4 reports
UPDATE SourceTable
 SET FIELDS = CAST(REPLACE(CAST(FIELDS AS TEXT),'<Field><Name>ShortFootnote', '<Field><Name>Punc</Name><Value/></Field><Field><Name>ShortFootnote') AS BLOB)
  WHERE SourceID IN
   (SELECT SourceID FROM SourceTable S
     WHERE TemplateID=YourTemplateID AND CAST(S.FIELDS AS TEXT) NOT LIKE '%<Name>Punc%'
    )
 

Follow up to enter values in the Key:[ShortPage] field of each citation that needs an improved short footnote, and take out the duplicate values from the Details:[Page] field.

How to find citations needing an improved Short Footnote

This section focuses on sources that have been globally converted to the !MyFreeForm3 templates.

Inspecting the Source List Report

For all source types and sources, one could create a RM4 Report > List > Source List including the Short Footnote and Citation Details (but not Detail Text, Comment and Media as that may clutter the report). Unfortunately, this report cannot be filtered by type of Source Template so you would find sources using the !MyFreeForm templates by searching for the term “[ShortPage]” in the report. It occurs in both the Footnote and the Short Footnote line. Doubly unfortunately, this report makes it hard to tell if a Key:[ShortPage] value has been entered, but you can use it to look for lengthy citations used more than once and thus likely to incur the use of the Short Footnote. Since the person and fact are identified in the Citation list for a Source in this report, you can then edit each citation accordingly. Best to fix one, Memorize it, Paste it into the other facts that use it, compare the original for differences in Detail Text, Comment and Media and resolve them on the copy, then delete the old. That guarantees that the key citation properties are truly identical so that Short Footnote and Ibid will come into play.

MyFreeForm3-Source_List_example_screenshot.png
“Example from Source List Report having a //ShortPage// field and therefore uses the !MyFreeForm3 template.

This screenshot from the Source List report shows that both Footnotes contain the [ShortPage] field so this is a !MyFreeForm3 source. It has two identical citations for facts about the same person so will surely use either the Short Footnote or Ibid in reports. It also has citation or source detail values for both the Details:[Page] and the Key:[ShortPage] but it is hard to tell because they are merely separated by a semi-colon. Lengthier citations with embedded semi-colons are even tougher to deconstruct into the two fields. If there is no obvious key value repeated, then either

  1. [ShortPage] is null and the short footnote may be as verbose as the full one. The absence of any semi-colon “; ” in the citation is absolute confirmation but the opposite is not necessarily true.
  2. This is a properly parsed !MyFreeForm3 citation. The absence of any semi-colon “; ” in the citation is absolute refutation but the opposite is not necessarily true.

Not overly helpful…

An SQLite query showing unparsed Source Details, longest first

Of course, SQLite could help in this process. I adapted an earlier query to produce the following report which shows the length (LenDet) of the field containing the [Page] value with a null value in [ShortPage].

Sources_by_Length_of_Detail_-_screenshot.png
Screenshot from SQLiteSpy of !MyFreeForm sources lacking a //ShortPage// value, in descending order of the length of the field containing the //Page// value.

Sorted in descending order, I work from the top down, re-running the query after each editing and replacement of duplicate citations (those with the same length most likely have identical values). Thus, the most verbose citations are tackled first. Some 156 characters out of the total shown under LenDet are used by XML tags, as can be seen at the bottom of the screen – that’s the value of the highlighted cell which contains the values of all the fields in Source Details. Download SourcesByDetailLength.sql

Discussions and comments from Wikispaces site


Geraniums

!MyFreeForm3 template

Geraniums
05 December 2011 18:01:37

If I share my database with others (some weeks another person enters data, other weeks a 2nd person enters data, etc), then we should all import your enhanced free form template (!MyFreeForm3), is that correct?

When RootsMagic has an update, does the template stay in the program? Would I ever need to re-import it?

How do I download your template? I clicked here, but then what?
!MyFreeForm3.rmst

Thanks!


ve3meo

ve3meo
05 December 2011 20:45:40

Once a custom template is imported into a database file, it stays there until deleted and cannot be deleted if it is in use by a source. If you share the database FILE, then everyone will see it. If a person having a source using a custom source template is transferred by drag and drop or by GEDCOM (including RM-specific features), the custom template spec is also transferred.

When an RM update or new release updates a database, it historically has preserved custom templates.

You should only need to re-import to a new database.

To download, right-click on the link and select ‘Save target as’, directing the save to a convenient place on your hard drive. Then follow the instructions in Help > custom source templates to import it into Lists > Source Templates.

Hope that’s clear and would appreciate your observations and experience in using the template.

Tom


Geraniums

How to use the new free form template

Geraniums
30 December 2011 19:00:05

Can I please have more examples of how to enter information in the different fields? And how the sentences read afterward.

I’m not sure what “key” means, or what type of punctuation gets entered. Where would the year of a publication be entered?

Thanks


ve3meo

ve3meo
30 December 2011 21:15:41

Using the labels beside each data entry field and enclosing them in square brackets to signify the variable data values one enters into the fields, here is what the resulting report sentences would look like:

Footnote: [footnote]<,|[punctuation]> [preamble]; [key]
Short Footnote: [shortfootnote]<,|[punctuation]> [key]
Bibliography: [bibliography].

This <,|[punctuation]> indicates that a comma will be used by default, else the character(s) you enter into the Punctuation field, such as a colon (:) or semi-colon (;); easiest to leave it empty as a comma is the most common punctuation.

I would urge you to actually try the template on a Play database so that you get the feel of it. The sentences will read the way you want them within the constraint that the Source Details always follow the Master Source data. And it’s not very different from Free Form, except for the resulting Short Footnote.

I used the term “Key” to indicate the most specific piece(s) of the Source Details that one would want in the Short Footnote and the term “Details” for the rest of the Source Details that show up with “Key” only in the long Footnote.

Year of publication would most commonly be entered in the Footnote field.

Here’s an example:
[footnote]: Lovell’s Canadian Dominion directory for 1871 (Montreal: J. Lovell, 1871); microform digital image, Canadiana.org, Internet Archive (http://www.archive.org/details/cihm_09143 : downloaded 7 April 2011)

[shortfootnote]: Lovell’s Canadian directory, 1871

[details]: entry for Sinclair Holden, Markham

[key]: page 518.

Resulting footnotes:
Footnote:
Lovell’s Canadian Dominion directory for 1871 (Montreal: J. Lovell, 1871); microform digital image, Canadiana.org, Internet Archive (http://www.archive.org/details/cihm_09143 : downloaded 7 April 2011), entry for Sinclair Holden, Markham; page 518.

Short Footnote:
Lovell’s Canadian directory, 1871, page 518.

I left [punctuation] empty so it defaulted to comma and I omitted Bibliography as it involves no other fields.

Hope that comes out on the Discussion page okay and is helpful. Glad you’re taking an interest.

Tom


RWells1938

Example

RWells1938
01 January 2012 12:47:07

Tom

Could you give a example on how you use your free from source using a 1930 census?

thanks
Roger


ve3meo

ve3meo
01 January 2012 20:56:33

Roger, I have done the example you requested as a PDF that is linked from the Page. If you look at its Table of Contents for “Examples”, you will readily find it.

Tom

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.