Helion Energy expects to capture electricity and projects net energy gain this year!

Helion Energy expects to capture electricity and projects net energy gain this year!

No record copies in DB

  • By Artem V. Shamsutdinov
  • February 17th, 2024

With the current solution of copying records between Repositories there would still be multiple records in the database for the same exact record - one per repository that has a copy of that record. This is hard to maintain and comes with a performance penalty.

The solution is very straightforward - only have one record in the database, even though every repository has a copy in its Transaction log. This means that the current REPOSITORY_LID field in an AirEntity now tracks the Local Id of the Repository to which this record belongs. There is no need to track (in the database) the Repositories to which that record is copied to, since there are already relationships that depend on that record.

This approach greatly reduces the number of update operations that must be performed to just the simple update operation that is already performed on an existing repository. An additional check for local existence of the repository is needed to make sure not to run the original Creation record. Instead if the Repository being loaded isn't present locally but there are references to it the original create operation will instead be run as an Update operation.

To accomplish this, original Ids of all copied records will be stored in the COPIED_RECORD_LEDGER table. During loading of repositories this table will be checked. If any of the records in the loaded repository are present there they will be Updated instead of being Created.

Update 2/18/2024

At the time of creation of record copies the general approach is to use the "save" operation for saving copies of data (across repositories). Cross repository copies will be disallowed in the lower level "insert" and "update" operations.

A new COPY operation type will be recorded in the transaction log. It will contain the save timestamp of when the copy was made

At the time of loading copy data a check will be made against COPIED_RECORD_LEDGER to see if the record is already present in the local database. If the Repository does not exist locally a check will be made against the save timestamp to see if the newly loaded copy is more recent than the existing one. If it is then the copy record will be replaced.

When a Repository is being loaded all of its loaded records will also be checked against COPIED_RECORD_LEDGER existing records will be UPDATEd instead of being CREATEd (and the entries in COPIED_RECORD_LEDGER will be deleted).