Looking for a complete solution for automatically updating a graph of entities using the Entity Framework? Read On!
Hi, As usual I have neglected this blog as of late. It is getting harder and harder to find time to put some notes up here. But hopefully today I have something very interesting to make up for it.
So today I’m finally going to post something that we have actually been using live on our production code for quite some time, and the good news is that it is working beautifully. I’m introducing GraphDiff – an extension method allowing for the automatic update of a detached graph using Entity Framework code first. (Edit: code has been rewritten to handle multiple new features, as such no guarantee can be given on its production-ready usage, but I’m continuing to work on it to make sure it is relatively bug-free)
Working with detached graphs of entities we quite often found that it was cumbersome to use the Entity Framework to manually map all of the changes from an aggregate root to the database. By aggregate root I mean a bunch of models which are handled as one unit when updating/adding/deleting.
Below I will describe my proposed solution to this problem of automatically updating a detached graph consisting of multiple add/delete/update changes at any point in the graph. This should work for all sorts of graphs, and allows for updating entities with associated collections and single entities.
I find its always clearer with some code so lets try an example:
Say you have a Company which has many Contacts. A contact is not defined on its own and is a One-To-Many (with required parent) record of a Company. i.e. The company is the Aggregate Root. Assume you have a detached Company graph with its Contacts attached and want to reflect the state of this graph in the database.
At present using the Entity Framework you will need to perform the updates of the contacts manually, check if each contact is new and add, check if updated and edit, check if removed then delete it from the database. Once you have to do this for a few different aggregates in a large system you start to realize there must be a better, more generic way.
Well good news is that after a few refactorings I’ve found a nice solution to this problem. The proposed extension method below handles the whole diff for you in a nice convenient package.
using (var context = new TestDbContext()) { // Update the company and state that the company 'owns' the collection Contacts. context.UpdateGraph(company, map => map .OwnedCollection(p => p.Contacts) ); context.SaveChanges(); }
Using the above code a diff will be run between the provided company graph and one retrieved from the database. The bounds of the graph are defined by the mapping which above is the company entity itself and its child Contacts. Only entities within these bounds will be included in the database diff.
The retrieval code makes use of the provided mapping configuration to get all data needed in one query at the start of the process, thus making the process quite efficient.
From this diff the algorithm will add/update/delete depending on what action needs to be performed and commit all of these changes in one batch at the end of the algorithm.
There are 2 different scenarios that this extension method must cater for. One is the situation above where you have a One-To-Many or One-To-One where the right hand side of the relationship is owned by the parent. This is defined within the mapping as an OwnedCollection/OwnedEntity.
The other scenario is that you have a Many-To-Many, or a One-To-Many and the related record exists in its own right as it not a defined part of the aggregate you are updating. That scenario is defined as an AssociatedCollection/AssociatedEntity.
The mapping configuration can handle many complex scenarios with ease. For example:
I have a company aggregate which is made of address objects and contact objects. On the contact model there is a list of accepted advertising materials (List) which are managed elsewhere in the system and not a part of the company aggregate. The list of advertisingOptions that are associated with the contact still needs to be updated however when managing the company as an aggregate.
using (var context = new TestDbContext()) { // Update the company and state that the company 'owns' the collection Contacts. context.UpdateGraph(company, map => map .OwnedCollection(p => p.Contacts, with => with .AssociatedCollection(p => p.AdvertisementOptions)) .OwnedCollection(p => p.Addresses) ); context.SaveChanges(); }
The associated collection line tells the update method that the actual entities that are AdvertisementOptions should not be marked as changed (even if the objects provided are different from the database values. We do however want to update the relation so that it reflects the fact that the contact has the provided AdvertisementOptions and this is performed during the update.
If you would like to use this code please leave all copyright marks as they are. The code has been put up on https://github.com/refactorthis/GraphDiff.
Hope this helps you as it did our team. Please note that there are a few things that can still be improved and if you have any suggestions please let me know.
Thank you Brent for this solution. I am having a problem.
I went through your unit tests and they worked fine.
I am using Northwind database to test your extension that I would like to use if I get it to work.
I am trying to update the Shipper for the Order as below:
[TestMethod]
public void UpdateOneToOneRelationship()
{
Order order;
Shipper shipper;
using (var context = new NorthwindContext())
{
order = context.Orders.Include(“Shipper”).Single(o => o.OrderID == 10248);
shipper = context.Shippers.Single(s => s.ShipperID == 1);
}
order.Shipper = shipper;
using (var context = new NorthwindContext())
{
try
{
context.UpdateGraph(order, map => map
.AssociatedEntity(o => o.Shipper));
context.SaveChanges();
}
catch (DbEntityValidationException dbEx)
{
foreach (var validationErrors in dbEx.EntityValidationErrors)
{
foreach (var validationError in validationErrors.ValidationErrors)
{
Debug.WriteLine(“Property: {0} Error: {1}”, validationError.PropertyName, validationError.ErrorMessage);
}
}
}
}
using (var context = new NorthwindContext())
{
order = context.Orders.FirstOrDefault(o => o.OrderID == 10248);
Assert.AreEqual(order.Shipper.ShipperID, 1);
}
}
And I am getting an exception
The property ‘ShipperID’ is part of the object’s key information and cannot be modified.
On the extension line 143:
context.Entry(dbvalue).CurrentValues.SetValues(newvalue);
The exception does make sense to me as the code is trying to update a key field, but I am not sure how it is working in your unit tests or projects?
Thanks for that Mustafa, It was a bug I introduced in a refactoring recently. I’ve added some tests for it and it should work fine now. Please let me know, Thanks.
I had the same problem, switch to OwnedCollection that will work.
entidades.UpdateGraph (obj, map => map
. OwnedCollection (p => p.MotivoHistoricoProcesso));
Thanks a lot Brent. Your fix worked! I am doing some more tests and planning to use your extension in my repositories. I will let you know if there is anything I find or I can suggest.
You are a life saver. I having been struggling with this for many days. Shouldn’t all this be given as out-of-the-box functionality from EF? I mean this is pretty common problems to people dealing with Line Of Businness Apps.
Keep up the good work.
I was looking for such a method, and expected that it was builtin in EF (it seems to be the case for NHibernate)…
But you did a very nice job !!!
The way you are describing the relations is really a fantastic idea !
Thanks a lot!
I too was shocked to find it wasn’t supported out of the box! Since EF is open source now I might start a discussion about possibly merging this idea into the codebase.
Thanks for the comments!
What about ef 4.1? if i change the reference in the update proj to 4.1 will it still work?
Yep should not be a problem.
nuget here https://nuget.org/packages/GraphDiff
This looks promising although I haven’t had a chance to try it yet. Will it work with EF5 DbContext?
I’m having a problem when playing with GraphDiff. I have these entities (generated using the EF5 POCO template, database-first):-
public partial class Customer
{
public int Id { get; set; }
public virtual ICollection Orders { get; set; }
// other properties…
}
and
public partial class Order
{
public int Id { get; set; }
public int CustomerId { get; set; }
public virtual Customer Customer { get; set; }
public virtual ICollection OrderLines { get; set; }
// other properties…
}
I then attempt to retrieve an existing customer and add a new order:-
using (var context = new MyTestEntities())
{
customer = context.Customers.First(o => o.Id == 1);
} // Simulate disconnect
var newOrder = new Order { Customer = customer }; // See below
customer.Orders.Add(newOrder);
using (var context = new MyTestEntities())
{
context.UpdateGraph(customer, m1 => m1.OwnedCollection(o => o.Orders));
context.SaveChanges();
}
However SaveChanges() throws the exception “Multiplicity constraint violated. The role ‘Customer’ of the relationship ‘MyTestModel.FK_Order_Customer’ has multiplicity 1 or 0..1”. It seems to be caused by me assigning the customer to the new order’s “Customer” property. If I remove this assignment then everything works. The problem is, I need to be able to pass order entities around and allow code to get at its parent Customer (via the “order.Customer” property).
Am I doing something wrong? I admit to not really understanding the difference between “owned” and “associated”, and when to use one over the other, so is this where I’m going wrong?
Hey, I just found some time to update the code and fixed the issue you are having. It will now support cyclic navigational properties.
Hi,
Thank you for a great library.
But I’ve got a strange exception with one of models. Here is some details.
The model looks like that:
* Root entity has multiple Level1Child.
* Each Level1Child has singular Level2Child.
* Each Level2Child has multiple Level3Child.
So, I’ve tried to build the UpdateGraph function:
db.UpdateGraph(root, map => map
.OwnedCollection(r => r.Level1Children, map2 => map2
.OwnedEntity(l1 => l1.Level2Child, map3 => map3
.OwnedCollection(l2 => l2.Level3Children))));
I am getting following exception:
Property ‘System.Collections.Generic.ICollection`1[Test.Level3Child] Level3Children’ is not defined for type ‘System.Collections.Generic.IEnumerable`1[Test.Level2Child]’
Do you have any idea – why is it like that?
Can you look at posting this as an issue with a a little more information on github? I’ve finally got some free time and I plan on spending some time on this library. https://github.com/refactorthis/GraphDiff/issues Thanks a lot.
Great library, saved us tons of hours. Thanks for sharing it 🙂
Thanks for the time Alex.
The base entity gets an update even if there are no changes. Child entities get updates only if there are changes, and only to the changed fields. Can the base entity code be updated to work the same way. We are trying to use CDC for audit tracking, and a full update everytime makes the audit useless.
Yes this has been fixed in the latest code. Thanks!
We have a problem when attempting to save when owned collections go beyond two levels. I modified the Models a bit to perform a proof of concept before diving headfirst into using GraphDiff in production code. Here is the expression we’re having problems with…
context.UpdateGraph(company5, map => map
.OwnedCollection(a => a.Projects, with1 => with1
.OwnedCollection(b => b.Managers, with2 => with2
.OwnedCollection(c => c.Employees, with3 => with3
.OwnedCollection(d => d.Hobbies))))
);
We’ve tracked the issue to the CreateIncludeExpression method and it seems things go bad when a MethodCallExpression is cast to a MemberExpression. It appears the code always expects to be dealing with Properties and dives when it has to handle a collection (the previousMember and the currentMember are both collections). We’re not experts in expressions trees and we’re attempting to make this work but if there was a patch to address this problem I know we’d benefit from it.
Thanks for sharing your project!
I’ve added this to issue #5 on github, I will look at this very soon.
Thanks for your work on this Brent! Much appreciated!
I’m having a serious issue while trying to expand the graph to a deeper level.
Please see details: https://github.com/refactorthis/GraphDiff/issues/5
GraphDiff shows an error if the aggregate root is new.
THis is exception line “return query.Single(lambda);” -> it cannot find aggregate root in database and throws an exception.
Hi bojanv55,
The code is meant to be used to update a graph of entities. EF already supports adding a graph of entities. At present GraphDiff is for updating only.
Hi guys, I’m sorry for late replies.
I don’t have time to look into these issues at present until at least August as I’m working on a large front-end project that is taking all of my time. Please feel free to debug and fix bugs it is all open sourced, if you do it would be great if you could contribute a pull request so that others can benefit and we can make GraphDiff the best solution to this existing problem in EF. I understand some of you will be relying on this code, I will try to find some time if available to look into any major issues but sorry I can’t promise anything at this time.
I have just used the solution for a many to many relationship.
http://stackoverflow.com/questions/17443221/mvc4-controller-with-many-to-many-relationship
Great stuff!
It would be great if you could make this a NuGet package
There is.
Search NuGet and you shall find it.
Touchè. I used the wrong search terms when I first looked.
Please close https://github.com/refactorthis/GraphDiff/issues/6 also as it’s misleading.
Love the library,
Thanks for taking the time to write it.
I’ve created a new nuget package as the original one is not owned by me and points to vip32’s fork of my code. This code is not controlled by me and is outdated. The new package is RefactorThis.GraphDiff and contains bugfixes for any closed issues on github. All updates will be added to this package, I’ll talk to vip32 to see if we can get this cleaned up.
Okay,
I thought vip32’s code was ahead and I updated it for EF6. I’ll fork yours and update that for you, can we maintain a beta version for EF6 beta?
Hey, yeah that’s a good idea, lets create a branch for EF6.
Pingback: Updates to GraphDiff with new scenarios supported | REFACTOR (this)
Great job, thanks a lot!
What should the UpdateGraph() fluent code look like if I have three entities arranged into a Many-Many relationship, and all three collections may contain new/deleted/updated entities to be processed? Say I’ve got an entity “A” which is the top of the hierarchy. This has collections of entities “B” and “C”. Lastly, both “B” and “C” have collections of entity “D”. I’m passing “A” to UpdateGraph(), and telling it that “A” own collections “B” and “C”. I’ve also got “B” owning collection “D”. Do I also need to specify that “C” owns collection “D”? When would I used “OwnedEntity”? Currently I’m having issues with this M-M scenario where context.SaveChanges() throws the exception “Conflicting changes to the role ‘xxx’ of the relationship ‘FK_xxx’ have been detected.” (in the example given, the error would be complaining about the relationship between “B” and “D”).
When CreateHash is called, if the database has padded the field with spaces, and the deteached object has trimmed the value, then the hash keys don’t match, and it tries to insert the same key values again. Any thoughts on how to correct this situation? Is it safe to just trim everything?
Um, ignore my earlier comment about the keys. I just realized someone made all the code fields char instead of varchar, and what a mess that is making of things.
Opened issue #13 on GitHub with fix included. Basically, in the UpdateConfigurationVisitor, if the root.IncludeString is null then GraphDiff blows chow. Simple null check fixes it.
Hi Brent
I’m having a problem when playing with GraphDiff. I have these entities
public class Person : IModel, IAggregateRoot
{
public Guid Id { get; set; }
public string Name { get; set; }
public DateTime? BirthDate { get; set; }
public string NationalCode { get; set; }
public virtual Zone BirthZone { get; set; }
public virtual Zone ResidencyZone { get; set; }
public virtual ICollection TravelZones { get; set; }
}
public class Zone : IModel,IAggregateRoot
{
public int Id { get; set; }
public string Name { get; set; }
public int? ParentId { get; set; }
public DateTime CreateTime { get; set; }
public DateTime UpdateTime { get; set; }
public virtual ICollection People { get; set; }
}
I am trying to update the Person for the Zone as below
public override Person UpdateEntity(Person entity)
{
DataContext.UpdateGraph(entity , map => map.AssociatedCollection(c => c.TravelZones).AssociatedEntity(c => c.ResidencyZone).AssociatedEntity(c => c.BirthZone));
DataContext.SaveChanges();
return entity;
}
And I am getting an exception
An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.
Please Help me
Can you somehow conclude Owned/Associated Collections from configuration/convention – so to have only Merge method on aggregate root?
Also, can you treat new object also (that are in graph, but need to be added to a database, even if the aggregate root is new)?
Hi Brent
Is it a very useful product! Could you check and verify a pull request on the github and include this fix into nuget repository, I really need this functionality in my project.
I need help with a mapping. I have 3 entities. E1 1…* E2 * …* E3. (E3 is a 2 table join, with the second table a refence code table) I’m adding new E3s. If I do this:
UpdateGraph(E2, map=>map.AssociatedCollection( p => p.Key))
It works.
If I try to update at a higher level, E1, I keep getting “System.InvalidOperationException: An object with the same key already exists in the ObjectStateManager. The ObjectStateManager cannot track multiple objects with the same key.”
I suspect it’s my mapping, It seems like adding the same E3 key to 2 different E2 entities is seen as a key violation (by GraphDiff).
The mapping I’m using is this:
UpdateGraph(E1, map=>mao.OwnedCollection(p => p.key, with => with.AssociatedCollection(p => p.key)
Thanks for the great tool!
Sean
Is there any way i can do the following in one statement:-
foreach (collection in this.Collections)
{
workTypeContext.UpdateGraph(collection.vml,
map => map.OwnedCollection(p => p.Views, with => with.OwnedCollection(v => v.ViewItems)));
}
workTypeContext.UpdateGraph(_locator, _Container,
map => map.OwnedCollection(p => p.Collections));
Hi, do you have a version available for Entity Framework 6?
Also, could you include a Merge method which just takes an Entity and iterates across the entire object graph automatically? This would allow me to use a single common Save\Update method for all my entities.
Sorry I just saw you have a Entity Framework 6 Branch. Awesome!!!
But a merge method which iterates across the graph would be brilliant.
Does GraphDiff work for Model First projects?
I have a graph as follows:
Application ProcessFlow ProcessStep
If I add a new Process Flow to the Application, it and any new Process Steps are added automatically and persist correctly.
If I go back and modify the ProcessSteps (any CUD operation), that action does not persist.
Can I use GraphDiff to check for the changes in the ProcessSteps to guarantee persistence?
My notation didn’t work, The relationships are
Application (mandatory) 1:M (optional) ProcessFlow
ProcessFlow (mandatory)1:M (mandatory) ProcessStep.
Hi Brent! Could you please add “Permissive free software licence” with the ability to modify source code(e.g. MIT) into your product on github. Thanks a lot.
i think the GetFirstBaseType method in the extension class and its usage need some attention.
if your classes inherit from interfaces, abstract classes or some other class that was not or could not defined as a set inside the DbContext, the GetFirstBaseType method returns them as the basetype causiıng the following generic method invocations fail.
(…just reading the code…)
Inside RecursiveGraphUpdate method
”
context.Entry(dbItem).CurrentValues.SetValues(updateItem);
”
line fails if dbItem is a detached entity, ’cause CurrentValues can’t be used in this case.
This is the second tool that I’ve found for client-side change tracking, the other being WCF Data Services. Looks good. But whereas WCF Data Services has “interceptors”, I don’t see an equivalent mechanism here. That’s a feature I’ll need.
Actually, there may still be way to use GraphDiff and accomplish what I need.
See: http://blog.oneunicorn.com/2011/04/03/rejecting-changes-to-entities-in-ef-4-1/
In fact, I think another approach is better than the one here. Instead of sending the entire modified object graph over the wire, why not instead just send the delta? You wouldn’t even need generated DTO classes in that case. If you have an opinion about this either way, please let’s discuss is at http://stackoverflow.com/questions/1344066/calculate-object-delta .
i Am using EF, optimistic concurrency and disconnected POCOs, and graphdiff, but when you “merge” a graph the version control property gets overwritten by the one from the database, which basically disables the concurrency check.
Is there any way to stop grahdiff from doing this ?
Pingback: reading code: GraphDiff « Home Work
Nuget package now depends in EF6 how to use it with EF5?
Pingback: [asp.net-mvc] EF4 POCO 개체의 변경 사항을 저장할 때 관계 업데이트 - 리뷰나라
Pingback: EF4 POCO 개체의 변경 사항을 저장할 때 관계 업데이트 - 스택플로우