Standardize places by date script

#1 · 2026-10-01, 8:43 am

Quote from Jared Chenkin on 2026-10-01, 8:43 am
Hi everyone!

I guess I wanted to introduce myself (I'm Jared 👋 ) , I just joined the forum about a month back after admiring from afar. I have been building my tree fairly exclusively on Family Search for the past 8 or so years, but this year decided I wanted to have copies offline and on Ancestry that I could say is "mine" without having to get lost navigating the One Tree™ on FS. With my main use case being finding and migrating my people from FS to Ancestry, I tried both FTM and RM and founds RM to be much more usable, although it does have its limitations. After some googling, I found this site and the realization that RM is basically a frontend to an sqlite database, I was sold. As a developer by trade, this was awesome. I read through the posts here, forked Richard's github scripts and got to work.

To get to the topic, one thing that was bugging me was keeping place names not only standardized, but also correct for the time period the event fell in. FS has gotten better recently about keeping and informing standard place names and dates, even though it still allows freeform text entry. They even released a site dedicated to searching their place database. And then I went over to the "Tool request" section here and found: Change Placename if Before or After Date!

So I want to present my somewhat naïve solution to the problem. It's a couple of python objects and a wrapper script that define a grouping of places, valid place names to use based on a range of years, and search strings for how to find relevant places in RM. It will search for relevant places and repoint existing events to the place who's name is correct for the event's date. I made it interactive on the command line to ensure that unwanted places don't get swept up in the migration.

I left a lot of manual set up in place by both design and limited scope (laziness). It does rely on going into the DB and finding the PlaceID of the "correct" places from the PlaceTable as part of the configuration. ~~I also simplified the query to only search by year. I would like to be able to set time periods for places by full date but my sql-fu isn't good enough to parse the RM date string.~~ I also wanted to maybe have it search the place table rather and create the entry if missing, but I didn't feel like going through the effort there. Even better, to query the FS site directly, but with so much variation on spellings, languages, diacritics, etc, I decided that I wanted to "accept" the standard as a manual action. There were also some cases where FS didn't cover the time period and I needed to use other sources to derive what the place name and encompassing areas were called. (I also refactored some of Richard's 'common' code so this isn't a drop in to his repo, you need to clone mine wholesale).

All told, I'm pretty comfortable importing FS people and other trees and data from Ancestry into RM and then periodically running the script. Let me know if you find it useful or have feedback.

Looking forward to collaborating with everyone!

-Jared

Edit: I just refactored the whole thing because the I felt the code was somewhat poorly organized, but I also added support for handling full dates.

Hi everyone!

I guess I wanted to introduce myself (I'm Jared 👋 ) , I just joined the forum about a month back after admiring from afar. I have been building my tree fairly exclusively on Family Search for the past 8 or so years, but this year decided I wanted to have copies offline and on Ancestry that I could say is "mine" without having to get lost navigating the One Tree™ on FS. With my main use case being finding and migrating my people from FS to Ancestry, I tried both FTM and RM and founds RM to be much more usable, although it does have its limitations. After some googling, I found this site and the realization that RM is basically a frontend to an sqlite database, I was sold. As a developer by trade, this was awesome. I read through the posts here, forked Richard's github scripts and got to work.

To get to the topic, one thing that was bugging me was keeping place names not only standardized, but also correct for the time period the event fell in. FS has gotten better recently about keeping and informing standard place names and dates, even though it still allows freeform text entry. They even released a site dedicated to searching their place database. And then I went over to the "Tool request" section here and found: Change Placename if Before or After Date!

So I want to present my somewhat naïve solution to the problem. It's a couple of python objects and a wrapper script that define a grouping of places, valid place names to use based on a range of years, and search strings for how to find relevant places in RM. It will search for relevant places and repoint existing events to the place who's name is correct for the event's date. I made it interactive on the command line to ensure that unwanted places don't get swept up in the migration.

I left a lot of manual set up in place by both design and limited scope (laziness). It does rely on going into the DB and finding the PlaceID of the "correct" places from the PlaceTable as part of the configuration. ~~I also simplified the query to only search by year. I would like to be able to set time periods for places by full date but my sql-fu isn't good enough to parse the RM date string.~~ I also wanted to maybe have it search the place table rather and create the entry if missing, but I didn't feel like going through the effort there. Even better, to query the FS site directly, but with so much variation on spellings, languages, diacritics, etc, I decided that I wanted to "accept" the standard as a manual action. There were also some cases where FS didn't cover the time period and I needed to use other sources to derive what the place name and encompassing areas were called. (I also refactored some of Richard's 'common' code so this isn't a drop in to his repo, you need to clone mine wholesale).

All told, I'm pretty comfortable importing FS people and other trees and data from Ancestry into RM and then periodically running the script. Let me know if you find it useful or have feedback.

Looking forward to collaborating with everyone!

-Jared

Edit: I just refactored the whole thing because the I felt the code was somewhat poorly organized, but I also added support for handling full dates.

kevinm has reacted to this post.

#2 · 2026-10-01, 1:05 pm

Welcome to the SQLite for RM community, Jared. I've had a look at your site and a few times at Richard's and I'm going to have to dip my toes into python to really see what you two are doing! Thanks for introducing me to the FS searchable place database.

Tom

kevync has reacted to this post.

#3 · 2026-10-02, 12:49 pm

Thanks, Tom! I'm not sure if you saw this on the Gazetter thread, but I just came across the link for the developer guide to the places resource as well - https://developers.familysearch.org/main/docs/places.

I prefer to work in python largely because I am just not that experience with SQL at all beyond simple queries and clauses, but also because I find it easier (or possible?) do to higher level operations to aggregates of data, such as transforming the results of a query into a different data structure, and then using that to make database changes. A great example would be lumping sources, I couldn't see how to do that in sql alone given the amount of state needing to store and the multiple subsequent operations needed to actually do the lumping. Maybe that's just as result of my limited view of what SQL can do. And yes, I'm a lumper.

kevync has reacted to this post.

#4 · 2026-10-02, 1:51 pm

Quote from Jared Chenkin on 2026-10-01, 8:43 am

Hi everyone!

I found this site and the realization that RM is basically a frontend to an sqlite database, I was sold. As a developer by trade...

To get to the topic, one thing that was bugging me was keeping place names not only standardized, but also correct for the time period the event fell in.......

It does rely on going into the DB and finding the PlaceID of the "correct" places from the PlaceTable as part of the configuration. I also simplified the query to only search by year. . Let me know if you find it useful or have feedback.

Looking forward to collaborating with everyone!

-Jared

welcome to the club.
2027 will be year I plan to tackle places and historical names by event dates as well have them geocoded correct when possible.

Jared Chenkin has reacted to this post.

#5 · 2026-10-02, 2:07 pm

I prefer to work in python largely because I am just not that experience with SQL at all beyond simple queries and clauses, but also because I find it easier (or possible.

Python seem to be used by many people ...There is also PQ (Power Query) and a data connector via Excel or PBI.

#6 · 2026-10-02, 2:13 pm

Quote from kevync on 2026-10-02, 1:51 pm

Quote from Jared Chenkin on 2026-10-01, 8:43 am

Hi everyone!

I found this site and the realization that RM is basically a frontend to an sqlite database, I was sold. As a developer by trade...

To get to the topic, one thing that was bugging me was keeping place names not only standardized, but also correct for the time period the event fell in.......

It does rely on going into the DB and finding the PlaceID of the "correct" places from the PlaceTable as part of the configuration. I also simplified the query to only search by year. . Let me know if you find it useful or have feedback.

Looking forward to collaborating with everyone!

-Jared

welcome to the club.
2027 will be year I plan to tackle places and historical names by event dates as well have them geocoded correct when possible.

Looks like Richard is doing it too.

We really should form a club

#7 · 2026-10-02, 2:55 pm

yeah my strength is PQ and SQL & SQLITE vs python. I have used python before but usually with AI assistance

#8 · 2026-10-03, 4:49 pm

Jared,

Welcome to the site! It sounds like you will fit in here very well, both with your background and the wiliness to share.

The timing of you post is amazing as I have been working on the subject of proper place for the period. A majority of my husband's ancestors were part of the founding of the 13 Colonies. What I have found is most documentation will show (for example) a person born in 1680 at Cambridge, Middlesex, Massachusetts, United States. There was no United States prior to 1776 so the correct location would be Cambridge, Middlesex, Massachusetts Bay Colony, British Colonial America.

There are three issues:

Finding and changing everyone with the incorrect location for that period
Maintaining and using two places; prior to 1776 and after 1776
Mass updating of my tree on Ancestry.com

I use RootsMagic and tried to do a feature request to allow mass updates but that died. A majority of records on Family Search seem to have the proper location for the period even though the source shows the post 1776 location.

What I have found is using SQLite Scripts to create groups in RootsMagic for these searches; thanks Kevin. I have notes in my places that state "Prior to 1776" or "After 1776". So, item 3 is what's holding me up because I do not want to keep recreating a tree on Ancestry.

Charles

kevync has reacted to this post.

#9 · 2026-10-06, 6:55 am

There are places in United States after 1800 even after 1900 but before present day that have changed over time so it does get confusing. And sometimes it just the town our county lines that moved

#10 · 2026-10-07, 12:25 pm

Quote from Charles Banks on 2026-10-03, 4:49 pm

There are three issues:

Finding and changing everyone with the incorrect location for that period

Maintaining and using two places; prior to 1776 and after 1776

Mass updating of my tree on Ancestry.com

Thanks for your comment! My script does only attempt to solve your first two issues. From what I can tell of RM, there isn't a way to automate #3. I push updates to my Ancestry tree by hand. Recreating the tree is rough and with the changes to the Ancestry to API, every source has a unique ID so RM can't recreate on a new tree correctly. It ends up duplicating the source for every citation and it shows up as an external source, not an "Ancestry" source.

Jared

kevync has reacted to this post.