Introducing GraphDiff for Entity Framework Code First – Allowing automated updates of a graph of detached entities

Looking for a complete solution for automatically updating a graph of entities using the Entity Framework? Read On!

Hi, As usual I have neglected this blog as of late. It is getting harder and harder to find time to put some notes up here. But hopefully today I have something very interesting to make up for it.

So today I’m finally going to post something that we have actually been using live on our production code for quite some time, and the good news is that it is working beautifully. I’m introducing GraphDiff – an extension method allowing for the automatic update of a detached graph using Entity Framework code first. (Edit: code has been rewritten to handle multiple new features, as such no guarantee can be given on its production-ready usage, but I’m continuing to work on it to make sure it is relatively bug-free)

Working with detached graphs of entities we quite often found that it was cumbersome to use the Entity Framework to manually map all of the changes from an aggregate root to the database. By aggregate root I mean a bunch of models which are handled as one unit when updating/adding/deleting.

Below I will describe my proposed solution to this problem of automatically updating a detached graph consisting of multiple add/delete/update changes at any point in the graph. This should work for all sorts of graphs, and allows for updating entities with associated collections and single entities.

I find its always clearer with some code so lets try an example:

Say you have a Company which has many Contacts. A contact is not defined on its own and is a One-To-Many (with required parent) record of a Company. i.e. The company is the Aggregate Root. Assume you have a detached Company graph with its Contacts attached and want to reflect the state of this graph in the database.

At present using the Entity Framework you will need to perform the updates of the contacts manually, check if each contact is new and add, check if updated and edit, check if removed then delete it from the database. Once you have to do this for a few different aggregates in a large system you start to realize there must be a better, more generic way.

Well good news is that after a few refactorings I’ve found a nice solution to this problem. The proposed extension method below handles the whole diff for you in a nice convenient package.

using (var context = new TestDbContext())
{
    // Update the company and state that the company 'owns' the collection Contacts.
    context.UpdateGraph(company, map => map
        .OwnedCollection(p => p.Contacts)
    );

    context.SaveChanges();
}

Using the above code a diff will be run between the provided company graph and one retrieved from the database. The bounds of the graph are defined by the mapping which above is the company entity itself and its child Contacts. Only entities within these bounds will be included in the database diff.

The retrieval code makes use of the provided mapping configuration to get all data needed in one query at the start of the process, thus making the process quite efficient.

From this diff the algorithm will add/update/delete depending on what action needs to be performed and commit all of these changes in one batch at the end of the algorithm.

There are 2 different scenarios that this extension method must cater for. One is the situation above where you have a One-To-Many or One-To-One where the right hand side of the relationship is owned by the parent. This is defined within the mapping as an OwnedCollection/OwnedEntity.

The other scenario is that you have a Many-To-Many, or a One-To-Many and the related record exists in its own right as it not a defined part of the aggregate you are updating. That scenario is defined as an AssociatedCollection/AssociatedEntity.

The mapping configuration can handle many complex scenarios with ease. For example:

I have a company aggregate which is made of address objects and contact objects. On the contact model there is a list of accepted advertising materials (List) which are managed elsewhere in the system and not a part of the company aggregate. The list of advertisingOptions that are associated with the contact still needs to be updated however when managing the company as an aggregate.

using (var context = new TestDbContext())
{
    // Update the company and state that the company 'owns' the collection Contacts.
    context.UpdateGraph(company, map => map
        .OwnedCollection(p => p.Contacts, with => with
            .AssociatedCollection(p => p.AdvertisementOptions))
        .OwnedCollection(p => p.Addresses)
    );

    context.SaveChanges();
}

The associated collection line tells the update method that the actual entities that are AdvertisementOptions should not be marked as changed (even if the objects provided are different from the database values. We do however want to update the relation so that it reflects the fact that the contact has the provided AdvertisementOptions and this is performed during the update.

If you would like to use this code please leave all copyright marks as they are. The code has been put up on https://github.com/refactorthis/GraphDiff.

Hope this helps you as it did our team. Please note that there are a few things that can still be improved and if you have any suggestions please let me know.

Advertisements

57 thoughts on “Introducing GraphDiff for Entity Framework Code First – Allowing automated updates of a graph of detached entities

  1. Thank you Brent for this solution. I am having a problem.
    I went through your unit tests and they worked fine.
    I am using Northwind database to test your extension that I would like to use if I get it to work.
    I am trying to update the Shipper for the Order as below:
    [TestMethod]
    public void UpdateOneToOneRelationship()
    {
    Order order;
    Shipper shipper;
    using (var context = new NorthwindContext())
    {
    order = context.Orders.Include(“Shipper”).Single(o => o.OrderID == 10248);
    shipper = context.Shippers.Single(s => s.ShipperID == 1);
    }
    order.Shipper = shipper;

    using (var context = new NorthwindContext())
    {
    try
    {
    context.UpdateGraph(order, map => map
    .AssociatedEntity(o => o.Shipper));
    context.SaveChanges();
    }
    catch (DbEntityValidationException dbEx)
    {
    foreach (var validationErrors in dbEx.EntityValidationErrors)
    {
    foreach (var validationError in validationErrors.ValidationErrors)
    {
    Debug.WriteLine(“Property: {0} Error: {1}”, validationError.PropertyName, validationError.ErrorMessage);
    }
    }
    }
    }

    using (var context = new NorthwindContext())
    {
    order = context.Orders.FirstOrDefault(o => o.OrderID == 10248);
    Assert.AreEqual(order.Shipper.ShipperID, 1);
    }

    }

    And I am getting an exception
    The property ‘ShipperID’ is part of the object’s key information and cannot be modified.
    On the extension line 143:
    context.Entry(dbvalue).CurrentValues.SetValues(newvalue);
    The exception does make sense to me as the code is trying to update a key field, but I am not sure how it is working in your unit tests or projects?

  2. Thanks a lot Brent. Your fix worked! I am doing some more tests and planning to use your extension in my repositories. I will let you know if there is anything I find or I can suggest.

  3. You are a life saver. I having been struggling with this for many days. Shouldn’t all this be given as out-of-the-box functionality from EF? I mean this is pretty common problems to people dealing with Line Of Businness Apps.

    Keep up the good work.

  4. I was looking for such a method, and expected that it was builtin in EF (it seems to be the case for NHibernate)…
    But you did a very nice job !!!
    The way you are describing the relations is really a fantastic idea !
    Thanks a lot!

  5. I’m having a problem when playing with GraphDiff. I have these entities (generated using the EF5 POCO template, database-first):-

    public partial class Customer
    {
    public int Id { get; set; }
    public virtual ICollection Orders { get; set; }
    // other properties…
    }

    and

    public partial class Order
    {
    public int Id { get; set; }
    public int CustomerId { get; set; }
    public virtual Customer Customer { get; set; }
    public virtual ICollection OrderLines { get; set; }
    // other properties…
    }

    I then attempt to retrieve an existing customer and add a new order:-

    using (var context = new MyTestEntities())
    {
    customer = context.Customers.First(o => o.Id == 1);
    } // Simulate disconnect

    var newOrder = new Order { Customer = customer }; // See below
    customer.Orders.Add(newOrder);

    using (var context = new MyTestEntities())
    {
    context.UpdateGraph(customer, m1 => m1.OwnedCollection(o => o.Orders));
    context.SaveChanges();
    }

    However SaveChanges() throws the exception “Multiplicity constraint violated. The role ‘Customer’ of the relationship ‘MyTestModel.FK_Order_Customer’ has multiplicity 1 or 0..1”. It seems to be caused by me assigning the customer to the new order’s “Customer” property. If I remove this assignment then everything works. The problem is, I need to be able to pass order entities around and allow code to get at its parent Customer (via the “order.Customer” property).

    Am I doing something wrong? I admit to not really understanding the difference between “owned” and “associated”, and when to use one over the other, so is this where I’m going wrong?

  6. Hi,

    Thank you for a great library.

    But I’ve got a strange exception with one of models. Here is some details.

    The model looks like that:
    * Root entity has multiple Level1Child.
    * Each Level1Child has singular Level2Child.
    * Each Level2Child has multiple Level3Child.

    So, I’ve tried to build the UpdateGraph function:
    db.UpdateGraph(root, map => map
    .OwnedCollection(r => r.Level1Children, map2 => map2
    .OwnedEntity(l1 => l1.Level2Child, map3 => map3
    .OwnedCollection(l2 => l2.Level3Children))));

    I am getting following exception:
    Property ‘System.Collections.Generic.ICollection`1[Test.Level3Child] Level3Children’ is not defined for type ‘System.Collections.Generic.IEnumerable`1[Test.Level2Child]’

    Do you have any idea – why is it like that?

  7. The base entity gets an update even if there are no changes. Child entities get updates only if there are changes, and only to the changed fields. Can the base entity code be updated to work the same way. We are trying to use CDC for audit tracking, and a full update everytime makes the audit useless.

  8. We have a problem when attempting to save when owned collections go beyond two levels. I modified the Models a bit to perform a proof of concept before diving headfirst into using GraphDiff in production code. Here is the expression we’re having problems with…

    context.UpdateGraph(company5, map => map
    .OwnedCollection(a => a.Projects, with1 => with1
    .OwnedCollection(b => b.Managers, with2 => with2
    .OwnedCollection(c => c.Employees, with3 => with3
    .OwnedCollection(d => d.Hobbies))))
    );

    We’ve tracked the issue to the CreateIncludeExpression method and it seems things go bad when a MethodCallExpression is cast to a MemberExpression. It appears the code always expects to be dealing with Properties and dives when it has to handle a collection (the previousMember and the currentMember are both collections). We’re not experts in expressions trees and we’re attempting to make this work but if there was a patch to address this problem I know we’d benefit from it.

    Thanks for sharing your project!

  9. GraphDiff shows an error if the aggregate root is new.

    THis is exception line “return query.Single(lambda);” -> it cannot find aggregate root in database and throws an exception.

  10. Hi guys, I’m sorry for late replies.

    I don’t have time to look into these issues at present until at least August as I’m working on a large front-end project that is taking all of my time. Please feel free to debug and fix bugs it is all open sourced, if you do it would be great if you could contribute a pull request so that others can benefit and we can make GraphDiff the best solution to this existing problem in EF. I understand some of you will be relying on this code, I will try to find some time if available to look into any major issues but sorry I can’t promise anything at this time.

  11. Pingback: Updates to GraphDiff with new scenarios supported | REFACTOR (this)

  12. What should the UpdateGraph() fluent code look like if I have three entities arranged into a Many-Many relationship, and all three collections may contain new/deleted/updated entities to be processed? Say I’ve got an entity “A” which is the top of the hierarchy. This has collections of entities “B” and “C”. Lastly, both “B” and “C” have collections of entity “D”. I’m passing “A” to UpdateGraph(), and telling it that “A” own collections “B” and “C”. I’ve also got “B” owning collection “D”. Do I also need to specify that “C” owns collection “D”? When would I used “OwnedEntity”? Currently I’m having issues with this M-M scenario where context.SaveChanges() throws the exception “Conflicting changes to the role ‘xxx’ of the relationship ‘FK_xxx’ have been detected.” (in the example given, the error would be complaining about the relationship between “B” and “D”).

    • When CreateHash is called, if the database has padded the field with spaces, and the deteached object has trimmed the value, then the hash keys don’t match, and it tries to insert the same key values again. Any thoughts on how to correct this situation? Is it safe to just trim everything?

      • Um, ignore my earlier comment about the keys. I just realized someone made all the code fields char instead of varchar, and what a mess that is making of things.

  13. Opened issue #13 on GitHub with fix included. Basically, in the UpdateConfigurationVisitor, if the root.IncludeString is null then GraphDiff blows chow. Simple null check fixes it.

  14. Hi Brent

    I’m having a problem when playing with GraphDiff. I have these entities

    public class Person : IModel, IAggregateRoot
    {

    public Guid Id { get; set; }
    public string Name { get; set; }
    public DateTime? BirthDate { get; set; }
    public string NationalCode { get; set; }

    public virtual Zone BirthZone { get; set; }

    public virtual Zone ResidencyZone { get; set; }

    public virtual ICollection TravelZones { get; set; }

    }

    public class Zone : IModel,IAggregateRoot
    {
    public int Id { get; set; }
    public string Name { get; set; }
    public int? ParentId { get; set; }

    public DateTime CreateTime { get; set; }
    public DateTime UpdateTime { get; set; }

    public virtual ICollection People { get; set; }

    }
    I am trying to update the Person for the Zone as below

    public override Person UpdateEntity(Person entity)
    {

    DataContext.UpdateGraph(entity , map => map.AssociatedCollection(c => c.TravelZones).AssociatedEntity(c => c.ResidencyZone).AssociatedEntity(c => c.BirthZone));
    DataContext.SaveChanges();
    return entity;
    }

    And I am getting an exception

    An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.

    Please Help me

  15. Can you somehow conclude Owned/Associated Collections from configuration/convention – so to have only Merge method on aggregate root?

    Also, can you treat new object also (that are in graph, but need to be added to a database, even if the aggregate root is new)?

  16. Hi Brent
    Is it a very useful product! Could you check and verify a pull request on the github and include this fix into nuget repository, I really need this functionality in my project.

  17. I need help with a mapping. I have 3 entities. E1 1…* E2 * …* E3. (E3 is a 2 table join, with the second table a refence code table) I’m adding new E3s. If I do this:

    UpdateGraph(E2, map=>map.AssociatedCollection( p => p.Key))
    It works.

    If I try to update at a higher level, E1, I keep getting “System.InvalidOperationException: An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.”

    I suspect it’s my mapping, It seems like adding the same E3 key to 2 different E2 entities is seen as a key violation (by GraphDiff).

    The mapping I’m using is this:

    UpdateGraph(E1, map=>mao.OwnedCollection(p => p.key, with => with.AssociatedCollection(p => p.key)

    Thanks for the great tool!

    Sean

  18. Is there any way i can do the following in one statement:-

    foreach (collection in this.Collections)
    {
    workTypeContext.UpdateGraph(collection.vml,
    map => map.OwnedCollection(p => p.Views, with => with.OwnedCollection(v => v.ViewItems)));
    }

    workTypeContext.UpdateGraph(_locator, _Container,
    map => map.OwnedCollection(p => p.Collections));

  19. Hi, do you have a version available for Entity Framework 6?

    Also, could you include a Merge method which just takes an Entity and iterates across the entire object graph automatically? This would allow me to use a single common Save\Update method for all my entities.

  20. Does GraphDiff work for Model First projects?

    I have a graph as follows:

    Application ProcessFlow ProcessStep

    If I add a new Process Flow to the Application, it and any new Process Steps are added automatically and persist correctly.

    If I go back and modify the ProcessSteps (any CUD operation), that action does not persist.

    Can I use GraphDiff to check for the changes in the ProcessSteps to guarantee persistence?

  21. Hi Brent! Could you please add “Permissive free software licence” with the ability to modify source code(e.g. MIT) into your product on github. Thanks a lot.

  22. i think the GetFirstBaseType method in the extension class and its usage need some attention.

    if your classes inherit from interfaces, abstract classes or some other class that was not or could not defined as a set inside the DbContext, the GetFirstBaseType method returns them as the basetype causiıng the following generic method invocations fail.

    • (…just reading the code…)

      Inside RecursiveGraphUpdate method

      context.Entry(dbItem).CurrentValues.SetValues(updateItem);

      line fails if dbItem is a detached entity, ’cause CurrentValues can’t be used in this case.

  23. This is the second tool that I’ve found for client-side change tracking, the other being WCF Data Services. Looks good. But whereas WCF Data Services has “interceptors”, I don’t see an equivalent mechanism here. That’s a feature I’ll need.

  24. i Am using EF, optimistic concurrency and disconnected POCOs, and graphdiff, but when you “merge” a graph the version control property gets overwritten by the one from the database, which basically disables the concurrency check.

    Is there any way to stop grahdiff from doing this ?

  25. Pingback: reading code: GraphDiff « Home Work

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s