Cross-repository '@OneToMany's

As it stands right now, repositories are linked via @ManyToOne relationships. This is easy to do since the Many side of the relationship contains the foreign key into the parent table. However, as I've been writing the first internally provided DApps I realized that there is a need for linking from the @OneToMany side as well. And the proposed solution opens up a possibility of a Graph database implementation on top of AIRport.

This can be demonstrated with linking from a Goal to Tasks. Naturally it makes sense to move the Goal and its underlying Tasks into separate Repositories. This is true because a Task is more focused and only needs to be shared with stakeholders that care about that particular task. Versus a Goal can be shared by many more people and do or do not work on tasks to accomplish that Goal. But when performing a UI navigation between a Goal and its tasks (from the Goal side of the relationship) by default there would be no way to discover the Repositories of the related tasks. This is because there is no way to store state of the @OneToMany relationship without the child records on the Many side of the relationship:

@Entity()
export class Goal extends AirEntity {

    name: string

    @OneToMany({ mappedBy: 'goal' })
    tasks: Task[]

}

@Entity()
export class Task extends AirEntity {

    @ManyToOne()
    goal: Goal

}

As you can see the @OneToMany relationship has to specify the "mappedBy" property to link to the field that it's being mapped to. This is because in the underlying table the linking identifier lives in the table with the @ManyToOne (the Tasks table in the example above).

So, when you make a Repository split between a Goal and its Tasks, by default the Goal Repository won't have any way to know about Tasks that are not physically present on the same user database.

Default solution - new Table

My default solution to this was to create a new @ManyToMany table that contains links to both the Goal and the Task:

@Entity()
export class GoalTask extends AirEntity {

    @ManyToOne()
    goal: Goal

    @ManyToOne()
    task: Task

}

And (subsequently) to change the @OneToMany relationship in the Goal to this new table:

@Entity()
export class Goal extends AirEntity {

    name: string

    @OneToMany({ mappedBy: 'goal' })
    goalTasks: GoalTask[]

}

While leaving the direct link from Task to Goal in place.

This solved the problem of the Goal Repository being able to look up Task repositories, without having Task repositories loaded in the local database. But this moved the burden of maintaining this relationship onto the App developers, which is cumbersome, error prone (developers can forget to do this) and is definitely annoying. With AIRport priding itself on being developer friendly this is not an acceptable solution in the final version of AIRport must be solved in a "transparent to developers" way before AIRport is open to third party Apps.

Copying Parent Repository records breaks Data Encapsulation.

One way to approach this issue is to copy records from the @OneToMany side of the relationship into the repository in the ManyToOne side. This is exactly what is being done to maintain query integrity and ensure that data is returned in the same way for inverse @ManyToOne links in queries. However this just doesn't work with the idea of data encapsulation. While it is OK (and obviously a good idea) for a Task to know about its parent Goal, not all Tasks should be visible from the Goal side view, at least not in some settings. The end user should have the ability to specify Tasks that are private and are not visible to all Goal participants. For example, the goal of "Get presents to this child" should have the task of "Put presents under the tree" be shared between the parents and Santa Claus. A child should have no knowledge of what it took to get those presents there, while Santa Claus should know what child is getting the present, without knowing the details of family Goals.

Note that I'm yet to find a case where a child repository needs to be insulated from a parent Repository. For example user of a Task repository should able to see what Goal it belongs to, without having to load the entire Goal Repository (or even having to have permissions to the parent Goal repository, in some security or internal strategy related cases). Same is true for a conversation that belongs to a Goal and for an Event setup for a given Task. And, in case such separation is required it can be implemented with the same many-to-many construct.

Internalize the ManyToMany

From developers point of view it can be expressed with a property on the @ManyToOne relationship:

@Entity()
export class Task extends AirEntity {

    @ManyToOne({ copyAcrossRepositories: false })
    goal: Goal

}
                        

With this the many-to-many link table becomes internal (hidden from developers) and the 'copyAcrossRepositories' flag will be 'true' by default and can be set to 'false' if needed. If it's set to false then in the absense of parent repository the queries won't return anything, while not compromising the need to maintain non-null constraints on the developer-visible (non-internal) tables. This is true since the query will always be done via the internal many-to-many record, which will be there with the child repository, even if the parent repository is not yet loaded. This also means that it's link to the One side of the relationship will not enforce referential integrity to the parent table (but again, that doesn't break integrity constraints defined by developers on non-internal tables). And once the parent repository is loaded the queries start returing parent records automatically.

Relationship maintenance

This means that for every relationship (across all tables) there will be a hidden many-to-many table. It will always be populated and maintained on @ManyToOne deletions or manual parent key swaps or key removal. I'm still not sure on simple (in memory array.slice())deletions from @OneToMany arrays causing relationship changes. All queries (both from the One side of relationships as well as the Many side) will join via the internal table and all of this will be hidden from the developers which will just see the relationship between the tables they define.

Note that any given record in the many-to-many table will have to be resent in both the parent and the child repositories and this will complicate the internal task of maintaining repoitories. But, besides that, the actual maintenance of these hidden tables is rather trivial. The one-to-many key can just be changed (in case of the change of a parent record) or the record can be removed (in case the child record is detached from the parent). The only additional task is to maintain the Original key (which will be renamed to Cross Repository key) and to change it (or null it out) as well.

Of course, the internal many-to-many table does create additional framework bloat in the case where developers want to create an Application-level ManyToMany table. But it's still worth it because it keeps things uniform and hides complexity from App developers (which also makes them happy and reduces bugs).

Future implementation of '@OneToOne's

This opens up the way to implement @OneToOne relationships, which can also span multiple Repositories. There will just have to be an additional column on the internal table that will enforce the one-to-one nature of the relationship. However, the @OneToOne relationship isn't strictly necessary and won't be in the initial release of AIRport (and thus won't be in the first public version of Turbase.org).

Graph database implementation possible

Most interestingly, moving all relationships to an internal Many-to-Many relationship enables the core feature of Graph databases - additional data on the links between tables. This can now be done by adding any number of additional columns to the internal many-to-many relationship which will act as the Graph database like link.