Further Results from Running Tom's RM8 to RM7 Reversion Procedure

#1 · 2023-06-02, 10:40 am

Quote from thejerrybryan on 2023-06-02, 10:40 am
I decided to make new thread, since these new test results tend to stand alone.
Basic setup for new tests of Tom’s RM8 to RM7 reversion procedure:
Use RM8 rather than RM9 for the testing to avoid any possible complications in reverting from RM9 to RM8. This limitation can surely be removed after I get the RM8 to RM7 reversion to work cleanly for my database. At that point, any loose ends in the RM9 to RM8 reversion could be cleaned up and it would be possible for me to revert from RM9 to RM7.
Use new databases P.rmgc, A.rmgc, B.rmtree, and C.rmgc. All four databases are stored in the same folder for convenience. Databases A.rmgc, B.rmtree, and C.rmgc are deleted and recreated during each iteration of the testing.
P.rmgc is a copy of my production RM7 database, created with Windows copy and paste.
A.rmgc is an extract from P.rmgc. Several different kinds of extraction are tested as described below.
B.rmtree is created by importing A.rmgc using RM8.
C.rmgc is created from B.rmtree using Tom's RM8 to RM7 reversion procedure. The goal is that C.rmgc is a fully usable RM7 database that includes all the data that was in A.rmgc with all the data being complete and unchanged.
After the reversion, use my scripts to compare tables from C.rmgc with tables from B.rmtree. It would also be possible to use the same scripts to compare tables from B.rmtree with tables from A.rmgc but doing so does not seem necessary because such a comparison was thoroughly tested during the development of the comparison scripts.
After each reversion, test the basic functioning and usability of C.rmgc using RM7 for the testing.
The five different extractions of A.rmgc from P.rmgc to be tested are as follows.
Do a File -> Copy from P.rmgc to A.rmgc from within RM7.
Drag and drop one person from P.rmgc to a new and empty A.rmgc from with RM7. Choose a person with many facts, many citations, and many media files.
Drag and drop about 200 well researched persons from P.rmgc to a new and empty A.rmgc from within RM7. These people will all have many facts, many citations, and many media files.
Drag and drop everyone in my database who is related to me and all their spouses from P.rmgc to a new and empty A.rmgc from within RM7. This represents about 30,000 people
Drag and drop everyone in my database from P.rmgc to a new and empty A.rmgc from within RM7. This represents about 47,000 people
The results from running my comparison scripts are that there is a perfect match in all five tests. The results from trying to use C.rmgc in RM7 are that the test using extraction #a fails and the other four tests succeed. Many basic functions of the reverted database C.rmgc work fine after extraction #a. But any accesses to sources or media files from within Edit Person fail. Any accesses to media files from Lists -> Multimedia list fail. The failure symptom is always an Access Violation error.
These symptoms suggest a couple of possibilities.
There is some sort of error in the RM8 to RM7 reversion procedure. But it’s hard to explain why the procedure fails in test #a and succeeds in all the other tests. It’s also hard to explain why I would be encountering such an error when other RM users have used the reversion procedure successfully.
There is some sort of malformation in my production RM7 database that does not manifest itself until after a round trip from RM7 to RM8 and then back to RM7. In this case, using drag and drop is cleaning up the malformation. The test #a that fails is the only test that does not use drag and drop. If there is a malformation in my production RM7 database, then it’s hard to picture what that malformation might be, nor how to fix it. I can't see a malformation when using RM7, nor when using my RM7 data after the data is imported into RM8 or RM9. The “malformation in my production RM7 database theory” is consistent with the observation that other RM users have used the reversion process successfully.
There are surely other possibilities, but I’m stumped.
It does occur to me that if the solution to making my RM8 database able to be reverted is to drag and drop the RM7 database before converting it to RM8, then using GEDCOM might as well be used to revert the RM8 database back to RM7. The nature of the drag and drop data losses would be about the same in either case and could mostly be mitigated. But a database level reversion from RM8 to RM7 would still be the most preferred technique if the test #a procedure could be made to work.

I decided to make new thread, since these new test results tend to stand alone.

Basic setup for new tests of Tom’s RM8 to RM7 reversion procedure:

Use RM8 rather than RM9 for the testing to avoid any possible complications in reverting from RM9 to RM8. This limitation can surely be removed after I get the RM8 to RM7 reversion to work cleanly for my database. At that point, any loose ends in the RM9 to RM8 reversion could be cleaned up and it would be possible for me to revert from RM9 to RM7.
Use new databases P.rmgc, A.rmgc, B.rmtree, and C.rmgc. All four databases are stored in the same folder for convenience. Databases A.rmgc, B.rmtree, and C.rmgc are deleted and recreated during each iteration of the testing.
- P.rmgc is a copy of my production RM7 database, created with Windows copy and paste.
- A.rmgc is an extract from P.rmgc. Several different kinds of extraction are tested as described below.
- B.rmtree is created by importing A.rmgc using RM8.
- C.rmgc is created from B.rmtree using Tom's RM8 to RM7 reversion procedure. The goal is that C.rmgc is a fully usable RM7 database that includes all the data that was in A.rmgc with all the data being complete and unchanged.
After the reversion, use my scripts to compare tables from C.rmgc with tables from B.rmtree. It would also be possible to use the same scripts to compare tables from B.rmtree with tables from A.rmgc but doing so does not seem necessary because such a comparison was thoroughly tested during the development of the comparison scripts.
After each reversion, test the basic functioning and usability of C.rmgc using RM7 for the testing.
The five different extractions of A.rmgc from P.rmgc to be tested are as follows.
1. Do a File -> Copy from P.rmgc to A.rmgc from within RM7.
2. Drag and drop one person from P.rmgc to a new and empty A.rmgc from with RM7. Choose a person with many facts, many citations, and many media files.
3. Drag and drop about 200 well researched persons from P.rmgc to a new and empty A.rmgc from within RM7. These people will all have many facts, many citations, and many media files.
4. Drag and drop everyone in my database who is related to me and all their spouses from P.rmgc to a new and empty A.rmgc from within RM7. This represents about 30,000 people
5. Drag and drop everyone in my database from P.rmgc to a new and empty A.rmgc from within RM7. This represents about 47,000 people

The results from running my comparison scripts are that there is a perfect match in all five tests. The results from trying to use C.rmgc in RM7 are that the test using extraction #a fails and the other four tests succeed. Many basic functions of the reverted database C.rmgc work fine after extraction #a. But any accesses to sources or media files from within Edit Person fail. Any accesses to media files from Lists -> Multimedia list fail. The failure symptom is always an Access Violation error.

These symptoms suggest a couple of possibilities.

There is some sort of error in the RM8 to RM7 reversion procedure. But it’s hard to explain why the procedure fails in test #a and succeeds in all the other tests. It’s also hard to explain why I would be encountering such an error when other RM users have used the reversion procedure successfully.
There is some sort of malformation in my production RM7 database that does not manifest itself until after a round trip from RM7 to RM8 and then back to RM7. In this case, using drag and drop is cleaning up the malformation. The test #a that fails is the only test that does not use drag and drop. If there is a malformation in my production RM7 database, then it’s hard to picture what that malformation might be, nor how to fix it. I can't see a malformation when using RM7, nor when using my RM7 data after the data is imported into RM8 or RM9. The “malformation in my production RM7 database theory” is consistent with the observation that other RM users have used the reversion process successfully.

There are surely other possibilities, but I’m stumped.

It does occur to me that if the solution to making my RM8 database able to be reverted is to drag and drop the RM7 database before converting it to RM8, then using GEDCOM might as well be used to revert the RM8 database back to RM7. The nature of the drag and drop data losses would be about the same in either case and could mostly be mitigated. But a database level reversion from RM8 to RM7 would still be the most preferred technique if the test #a procedure could be made to work.

#2 · 2023-06-02, 12:41 pm

I was mystified by what you mean by "test #a" as I could see no explicit definition of it. On reading and re-reading, I now realize that it is the conversion of a copy of the original production database from RM7 to RM8 and back to RM7 with no GEDCOM intervention. Any extract involving GEDCOM did not have a problem.

I think that really points to something in the ConfigTable that got modified or altered by the conversion to RM8 which is confusing to RM7. I would focus on what records in the ConfigTable are created or modified by RM8 or RM9 and delete them or, if possible, restore or repair to RM7 compatible.

#3 · 2023-06-02, 3:00 pm

I composed the message in Microsoft Word and copied and pasted it into this forum. In my Word document, my five cases were labelled #a, #b, #c, #d, and #e. It seemed very clear at that point what I was calling the cases. I didn't catch that after the paste into this forum that the five cases were relabeled as #7 through #11. Oh, the joys of copying and pasting rich text.

In any case, you correctly analyzed that the four cases where I created A.rmgc via drag and drop all worked and the one case were I created A.rmgc via a database copy failed to work. After creating database A in five different ways, the rest of my test going from database A to database B and them from database B to database C were identical.

If the problem is in ConfigurationTable, it should manifest itself in a difference in ConfigurationTable the first time I made database A and the other four times I made database A. The difference should also manifest in ConfigurationTable in database B between the first time I made database B and the other four times I made database B, and similarly for database C. I'll take a look and see what I can see. Also, I don't think I need five cases. I think I just need two cases, one with drag and drop and one without drag and drop.

#4 · 2023-06-02, 9:59 pm

I cannot yet replicate the issue you are encountering with your database peregrination, Jerry. Not that I've thoroughly tested but I took a 69,000 person RM7 database straight up to RM9 and back to RM7 and poked around in the areas that you run into the access error and they are fine. The ConfigTable of this database started with 29 records in RM7 but only 5 of which were carried over into RM9.0.4. And only these 5 can be delivered in the down conversion to RM7. Of those 5, only the DataRec of RowID 1 was changed. So it may be something subtle in here that is the trigger and it may be very dependent on how you have set views. What you might try is to replace your row 1 of your C.ConfigTable with row 1 of your A.ConfigTable. If that eliminates the access errors, at least we know where to focus the investigation and know that there needs to be some massaging of that field in the down-conversion for it to be a general solution.

#5 · 2023-06-03, 5:51 pm

The problem is still not resolved.

I have reduced my test to two cases. #1. Copy my production RM7 database to a new database called A_copy.rmgc using File=Copy. #2. Copy my production RM7 database to a new and empty database called A_dnd.rmgc using drag and drop.

Then run A_copy.rmgc and A_dnd.rmgc through the standard routine of importing them into RM8 and reverting the RM8 database to RM7 using Tom's script. The final product is two RM7 databases called C_copy.rmgc and C_dnd.rmgc.

The interim RM8 databases are called B_copy.rmtree and B_dnd.rmtree, respectively. Both of them seem to work just fine.

C_copy.rmgc continues to produce the copious Access Violation errors I have reported, and C_dnd.rmgc continues to operate perfectly.

I tried the additional step of copying ConfigTable from C_dnd.rmgc to C_copy.rmgc, and C_copy.rmgc still produces the Access Violation errors.

I need to isolate the table that is causing the problem. The only other thing I can think of to try is to copy each of the RM7 tables from C_dnd to C_copy one at a time until the Access Violation errors go away in C_copy. Or vice versa, to copy each of the RM7 tables from C_copy to C_dnd one at a time until the Access Violations appear in C_dnd.