Friday 31 October 2008

Using PostSharp to Allow One-Way Multiple Associations in XPO

The Express Persistent Objects (XPO) ORM from Developer Express does not support one way associations with multiplicity greater than one.  Each collection association from a type must have a corresponding reverse association from the linked type.  The reverse link can be either a unary or a collection (making either a one-many or many-many association, respectively).

I put in a request for one-way multi associations here (you may need to register to view).  The response I got back was basically that implementing one-way multi associations would "break existing functionality and this approach contradicts with the XPO concepts".  I think that this is a shame, but there you go.

So, given that I want this feature, I'm going to find the middle ground: let's have one-way associations, but use PostSharp to implement the extra code to keep XPO happy.

The Problem:
I want to be able to write this as valid XPO code:

   public class Customer : XPObject
   {
      public XPCollection Orders { get; }
   }

   public class Order : XPObject
   {
   }

 But XPO wants me to add the other end of the link to Order and to also add associated attributes:

   public class Customer : XPObject
   {
      [Association("CustomerOrders")]
      public XPCollection Orders { get; }
   }

   public class Order : XPObject
   {
      [Association("CustomerOrders")]
      public Customer Customer;
   }

So, for my one-way association there must be another property or field on the linked class as well as marking each participting property(or field with the Association attribute. Not a huge amount of extra code and not an unreasonable amount either!

(Also, it's worth noting here that I am again using PostSharp to let me use automatic properties with XPO here - there is a little more required for normal XPO code.  See my previous posts for more details on how this is done.)

A Solution:
We're going to use PostSharp to generate the reverse link for us and to add the associations that XPO requires.  Basically, I am going to write code as in the first section above and get PostSharp to enhance it at compile time so that it is actually like the second section.  Specifically, I'm going to use PostSharp to:
  • Add a public field to a class.
  • Add an attribute to this new field.
  • Add an attribute to an existing property.
Some Caveats:
If you care about the reverse link, you will want to implement it yourself as is usual in XPO.  The way we are doing it here means that the generated reverse link will not be visible at design time from inside of its own assembly (but it will be from outside the assembly).

Also, the current implementation is not compatible with the Compact Framework or Silverlight.  For this you will need to use the upcoming version 1.5 of PostSharp - more details here.

Approach
 As you will see from the PostSharp web site, PostSharp is made up of two main sections - PostSharp.Core and PostSharp.Laos.  PostSharp.Laos is a plugin on top of PostSharp.Core that simplifies a lot of the common tasks in AOP.  PostSharp.Core is the main engine that enchances your code at compile time.  It is both powerful and deep, and I am certainly not an expert in its use!  However, despite all the power and depth, it is also fairly accesssible.

The PostSharp website uses the lovely term "Pay-As-You-Use Complexity" for PostSharp - you can acheive pretty much anything you want and the common tasks are made fairly simple (by PostSharp.Laos), while other tasks may require you to gain some more knowledge of the inner workings of PostSharp (meaning PostSharp.Core). In my previous articles about PostSharp, we used PostSharp.Laos, for this task we are going to use PostSharp.Core.

In this article, I'm going to show how to add fields and attributes to classes using PostSharp.Core.  I was inspired by Ruurd Boeke's articles, which you can find here.  Ruurd uses PostSharp to enchance POCO classes for the Entity Framework use and goes a lot further with PostSharp than I do here.  I highly recommend reading Ruurd's articles for more detailed information.

Let's now have a look what we need to do to get PostSharp to do what we want.  I'm going to break this down into three sections:


Stage 1 - Attach Aspects
The first step is to create a CompoundAspect as we have done before.  The job of this attribute is going to be to add other PostSharp aspects to our classes.  The other aspects will then be picked up and used by PostSharp.Core.  The ProvideAspects method looks like this:

1 public override void ProvideAspects(
    object element, 
    LaosReflectionAspectCollection collection)
2 {
3     var targetType = (Type) element;
4      foreach (var property in targetType.UnderlyingSystemType
         .GetProperties()
         .Where(info => info.DeclaredIn(targetType) &&
                info.IsOneWayAssociation()))
5    {
6      var fieldName = GetGeneratedFieldName(property);
7      collection.AddAspect(
         property.GetCollectionTargetType(),
         new OtherEndFieldSubAspect(
           targetType.FullName,
           fieldName, 
           fieldName));
8
9      collection.AddAspect(
         targetType, 
         new AssociationAttributeSubAspect(
           property.Name, 
           fieldName));
10    }
11 }

In line 4 we use a bit of LINQ to iterate through the properties of our target type that we are interested in enchancing and then in line 6 we call a helper method to get the name of the field we want to generate.  In lines 7 and 9 we add the aspects.  In line 7 we add an aspect to the type at the other end of our collection and pass it the name of the type of the field we want to add, along with a field name and a string to be passed to the association attribute that we will add later.  Note that we are using an extension method helper (GetCollectionTargetType) to get the type at the other end, although this is nothing more than normal reflection code.  Next, in line 9, we add another aspect to the type that is declaring the collection property, passing it the name of the property that will be enchaned and a string that we will pass to the attribute that we add to the property.

So, we have used PostSharp.Laos to add aspects to our classes.  These aspects contain some information that we will use later.  In fact, as you can see below, the aspects that we have added contain little else other than the information we want to use later.

public class OtherEndFieldSubAspect : ILaosTypeLevelAspect
{
   public OtherEndFieldSubAspect(
     string fieldTypeName, 
     string fieldName, 
     string associationName)
   {
      FieldTypeName = fieldTypeName;
      FieldName = fieldName;
      AssociationName = associationName;
   }

   public void CompileTimeInitialize(Type type) {   }

   public bool CompileTimeValidate(Type type)
   {
      return true;
   }

   public void RuntimeInitialize(Type type) { }

   public int AspectPriority
   {
      get { return int.MinValue; }
   }

   public string AssociationName { get; set; }
   public string FieldName { get; set; }
   public string FieldTypeName { get; set; }
}
public class AssociationAttributeSubAspect : ILaosTypeLevelAspect
{
   public AssociationAttributeSubAspect(
     string propertyName, string associationName)
   {
      PropertyName = propertyName;
      AssociationName = associationName;
   }

   public void CompileTimeInitialize(Type type)    { }

   public bool CompileTimeValidate(Type type)
   {
      return true;
   }

   public void RuntimeInitialize(Type type) { }

   public int AspectPriority
   {
      get { return int.MinValue; }
   }

   public string AssociationName { get; set; }
   public string FieldName { get; set; }
   public string PropertyName { get; set; }
}

Note that these tasks implement ILaosTypeLevelAspect as we are applying the aspects to types.

Stage 2 - Create a Weaver Task

PostSharp's weaver is what we are going to use to create the new field and add the custom attributes.  The public parts of PostSharp are LGPL and can therefore be freely distributed under any license you choose, whereas PostSharp.Core is licensed under the copyleft GPL, and so you cannot distribute PostSharp.Core in another GPL application.  So, if we want to use this commercially, the weaver needs to be split out from the main distribution if the software is going to be used commercially.  This is not a problem as the enhancements happen at compile time and the designer of PostSharp, Gael Fratieur has made this easy for us to seperate PostSharp.Core!  The basis of this is to use PostSharp "tasks" to tell PostSharp.Core what to do at compile time.  A task has a name and points to a weaver implementation that will handle the task.  You configure the tasks in XML as below:



 
            Implementation="Weaver.WeaverFactory, Weaver">
 


The task called WeaveTwoWayAssociations is handled by the type Weaver.WeaverFactory which can be found in the assembly called Weaver.  This file and the rest of the weaver code sits in a seperate project.  Other projects find this by adding the weaver to the search path and adding an assembly level attribute to the project where we defined our AssociationAttributeSubAspect and OtherEndFieldSubAspect aspects.  We also add the weaver project's output folder to each referencing projects search path (inside project | properties).  The assembly attribute looks like this:

[assembly: PostSharp.Extensibility.ReferencingAssembliesRequirePostSharp("WeaveTwoWayAssociations", "Weaver")]
[assembly: InternalsVisibleTo("Weaver")]
Note that the task name and the assembly in which it can be found are indicated.  We also have to allow the weaver to reference our internals!  Also, note that PostSharp has another, more portable, mechanism to tell reference the weaver files, as described in Ruurd's article.  For whatever reason, I could not get this to work and so have just added the reference path.  You will need to change this path on your machine or you will get compile errors about not being able to find a plugin:



Okay, so we've now got a task setup to let PostSharp's weaver loose on our aspects.   We now just need to define what the weaver needs to do.

Stage 3 - Do the Weaving
In our configuration we said that the task called WeaveTwoWayAssociations was handled by the class Weaver.WeaverFactory.  WeaverFactory implements ILaosAspectWeaverFactory and looks like this:

public class WeaverFactory : Task, ILaosAspectWeaverFactory
{
  public LaosAspectWeaver CreateAspectWeaver(ILaosAspect aspect)
  {
     if (aspect is OtherEndFieldSubAspect)
     {
        var addField = (OtherEndFieldSubAspect)aspect;
        return new AddFieldWeaver(
          addField.FieldTypeName, 
          addField.FieldName,
          addField.AssociationName);
     }

     if (aspect is AssociationAttributeSubAspect)
     {
        var addAttribute = (AssociationAttributeSubAspect)aspect;
        return new PropertyAttributeWeaver( 
          addAttribute.PropertyName,
          addAttribute.AssociationName);
     }
     return null;
  }
}

Ttrue to its name, the factory is responsible for providing instances of types that can do some work for us.  Note that the factory is looking for the aspects that we added in stage 1.  When a OtherEndFieldSubAspect is encountered, an instance of AddFieldWeaver is created and returned to PostSharp.  When a AssociationAttributeSubAspect is encountered, an instance of PropertyAttributeWeaver is created and returned to PostSharp.

So, as indicated above, the work of adding a field is done by the AddFieldWeaver class.  You might be expecting this class to be pretty complex and have lots of IL in it.  If you are, you're in for a surprise as it's actually really clean and straight forward:

public class AddFieldWeaver : TypeLevelAspectWeaver
{
   public AddFieldWeaver(
     string fieldTypeName, string fieldName, 
     string associationName)
   {
      FieldTypeName = fieldTypeName;
      FieldName = fieldName;
      AssociationName = associationName;
   }

   private ITypeSignature fieldType;

   public override void Implement()
   {
      var newField = 
        new FieldDefDeclaration
          {
            Name = FieldName,
            Attributes = FieldAttributes.Public,
            FieldType = FieldType
          };

      ((TypeDefDeclaration) TargetType).Fields.Add(newField);

      var attribute = Utils.CreateAssociationAttribute(
        Task.Project.Module, AssociationName);
        newField.CustomAttributes.Add(attribute);
   }

   public string AssociationName { get; set; }

   public ITypeSignature FieldType
   {
      get
      {
         if (fieldType == null)
         {
            fieldType = 
              Task.Project.Module.FindType(
                FieldTypeName, 
                BindingOptions.Default);
         }
         return fieldType;
      }
   }

   public string FieldTypeName { get; set; }
   private string FieldName { get; set; }
}

The waever is an implementation of the PostSharp TypeLevelAspectWeaver and its Implement method is where the new field is created.  This method simply creates a new FieldDefDeclaration that represents the new field, sets its name, defines its visibility (by setting the Attributes property) and sets it's type.  The type is derived from a helper property called FieldType which uses PostSharp's built in Task.Project.Module to find the type of the name that was specified when we created the attribute in stage 1.
Then, the new field is simply added to the target type's fields collection.  Next, a helper method is called to create an instance of the custom attribute we want to add and is added to teh new field's custom attributes collection.  That's all there is to it!!!

PostSharp has been widely praised for having a very clean and well designed API.  After doing this I can really see why it is so well thought of - we have been sheltered from all the horrid IL generation that is usually associated with this sort of task.  Quite simply, PostSharp totally rocks and Gael Fraiteur has done an amazing job!

The Results
So, what do we have now?  If you remeber, we started with types that looked like this:

    [AllowXpoOneWayAssociations]
  public class Customer : XPObject
  {
     public XPCollection Orders { get; }
  }

  [AllowXpoOneWayAssociations]
  public class Order : XPObject
  {
  }

If we now look at the assembly in Reflector, we can see that a public field along with an custom attribute has been added to Order and an attribute has been added to Customer:





Back to XPO
So, most of the above has been about PostSharp.  Specific to XPO, the custom attribute I have added is the AssociationAttribute required by XPO to link the two sides of the association.  So, I started with a one-way association and have used PostSharp to magically turn that into a two-way one-many association.

If you want the code, you can get it from here.  Note that I've put up the XPO version and a version that doesn't require XPO.

Update: I've edited this to show that the XPCollection were the generic XPCollection 12-Nov-2008.

Monday 27 October 2008

Using C# Automatic Properties with XPO Objects - Part 2

In part 1 we looked at how XPO implements change tracking by utilising some magic from the base XPObject class.  An XPO object looked something like this:

public class Customer: XPObject
{
  private DateTime date;
  public DateTime Date
  {
    get { return date; }
    set { SetPropertyValue("Date", ref date, value); }
  }

  [Association("CustomerOrders", typeof(Order))]
  public XPCollection<Order> Order
  { 
    get { return GetCollection<Order>("Orders"); } 
  }
}

Wanting to cut down on the syntatic noise, I'd like to be able to use C# automatic properties to make my classes to look like this:

[XpoAutomaticProperties]
public class Customer: XPObject
{
  public DateTime Date { get; set; }

  [Association("CustomerOrders", typeof(Order))]
  public XPCollection<Order> Order
 
    get; private set; 
  }
}

The secret here is the XpoAutomaticProperties attribute and the work that this does.  Applying this attribute to the class makes two things happen:
  • Changing a property value will raise the changed notification events.
  • Accessing a collection will instantiate and set up the collection.
XPO objects notify observers of changesto values using a changed event.  Collections have similar events events are fired when the collection changes or when an item in the collection changes.  Note also that the GetCollection method above sets up the collection so that it is aware of being a property of a containing object.  This will also be handled by the attribute.

Along with the XpoAutomaticProperties attribute, there is one other thing that is introduced.  The SetPropertyValue call is responsible for firing the changed event.  I will introduce to allow me to call back to an object to instruct it to notify observers of changes (XPObject has exactly such a method).  The interface looks like this:

public interface IPropertyChangedNotifier
{
  void OnChanged(
    string propertyName, 
    object oldValue, 
    object newValue);
}

Okay, if you've got this far, you must be wondering how do we actually do this?  The answer is that we use the incredible PostSharp to help us add the required code we need into our objects.  PostSharp is an aspect oriented programming (AOP) library and toolset that makes it easy to add code into your objects during compilation.  As the changes occur at compile time, you avoid the runtime performance hits that would come from using reflection.

PostSharp makes the process of generating and adding IL code to your assemblies at compile time ridiculously easy to acheive for such a complex task.  Although PostSharp allows you to drop down and weave any IL code you like into your assemblies, there is a  The PostSharp website has a number of excellent examples of how to work with PostSharp, which are extremely impressive and worth checking out.  It is also very well documented.  PostSharp is an open source library that is free even for commercial use and, incredibly, is the work of one man: Gael Fraiteur.  Everything that we do here is based on the excellent examples that Gael provides with PostSharp, so check those out when you can.

From this point on, I am going to assume that you have some knowledge of AOP and PostSharp.  So, back to the XpoAutomaticProperties attribute and how this works.  The attribute is a PostSharp attribute in which you override a base method to do the work.  The method looks like:

public override void ProvideAspects(
  object targetElement, 
  LaosReflectionAspectCollection collection)

Inside this method you use reflection to interrogate the properties of the type and then add some code to that property like this:

1 if (property.CanWrite && property.CanRead && !property.IsCollection())
2 {
3   var setMethod = property.GetSetMethod(true);
4   if (!setMethod.IsStatic)
5   {
6      var aspect = new PropertyChangedNotifierSubAspect(
                    this, property.Name);
7      collection.AddAspect(setMethod, aspect);
8   }
9 }

Remember that all this code is firing at compile time so the reflection will not affect your runtime performance!  So, in line 1 we check that the property (a PropertyInfo) is read-write and is not a collection.  We then use reflection to access the property's setter in line 3 and check that we are not looking at a static property in line 4.  In line 6 we create a new PostSharp aspect - the PropertyChangedNotifierSubAspect - and add it to the collection that PostSharp supplies.  Note that the name of the property is passed in to the aspect's constructor - this is how we avoid the use of string literals for property names.  After this, PostSharp does all the work for you.  Well, PostSharp does the IL weaving for you, there's still a little more for us to do.

The PropertyChangedNotifierSubAspect decends from the built in PostSharp type OnMethodBoundaryAspect. This allows you to easily acheive a number of things when a method is called in your code. We use two of these to change the property's setter: first we store the old value when the setter is entered, then when the setter method completes we determine if the value has changed and fire the changed event accordingly.  PostSharp elegantly provides us methods that we can override to achieve all this - we never even have to leave C# or think about IL.  This is how we store the existing value when we enter the property's setter:

1 public override void OnEntry(MethodExecutionEventArgs eventArgs)
2 {
3   base.OnEntry(eventArgs);
4   var instance = (XPBaseObject) eventArgs.Instance;
5   oldValue = instance.GetMemberValue(propertyName);
6 }



We simply override the base class method and use the data supplied argument's to access the object whose property has changed.  We cast it to be an XPBaseObject and use the GetMemberValue to access and store the value of the property on entry.  Note here that the GetMemberValue is supplied by Developer Express and is performance efficient (itself using generated IL to access the property value).  If you are using XPO with your own objects that do not subclass the built in XPO types then you will have to use reflection or some other means to access the entry value.  Note also that this code is executes at runtime.

In a similar manner, we then override the OnSuccess method to execute code when the method successfully completes (i.e. exiting with no unhandled exceptions).  At this stage, we compare the current value with the existing and fire the changed event if needed.  The code looks like: 

1 public override void OnSuccess(MethodExecutionEventArgs eventArgs)
2 {
3    if (!(eventArgs.Instance is IPropertyChangedNotifier)) return;
4    var newValue = eventArgs.GetReadOnlyArgumentArray()[0];
5    if (ValuesEqual(oldValue, newValue)) return;
  
6    var instance = (IPropertyChangedNotifier) eventArgs.Instance;
7    instance.OnChanged(propertyName, oldValue, newValue);
8 }

 Again, PostSharp supplies us with event args that allow easy access to values.  We also use our IPropertyChangedNotifier interface to do the notification.  In line 4 we access the new property value from the event arguments supplied by PostSharp (again, no reflection needed!) then call a method to check for changes in line 5.  If we have a change to the property, we instruct the object to notify the world of the change in lines 6 and 7. 

Okay, so where are we now?  We've used PostSharp to create a special attribute.  We've applied the attribute to the classes in which we want to use automatic properties.  PostSharp recognises our attribute and injects code into our assemblies at compile time.  The code that is injected is defined by us in classes that we create from buit in PostSharp types.  These types are called aspects and we used the one that allows us to modify a method - specifically, the property's setter method.  To modify the method was pretty easy - we just override a couple of methods and use the information that PostSharp passes back to us to do what we want.

PostShapr also allows you to work with fields and add code to when the fields are accessed.  The way I initialise collections use this feature of PostSharp, but I'd suggest you head for the PostSharp site to learn more of what you can do with PostSharp.

Before we leave this, there's one gotcha to be aware of: the release version of PostSharp, 1.0, does not play well in partial trust scenarios.  If this is important to you then you'll be pleased to know that the next release 1.5 addresses this and version 1.5 is in CTP2 stage at time of writing.

If you want more, then you can get the code from here.

Sunday 26 October 2008

Using C# Automatic Properties with XPO Objects - Part 1

In my previous post I touched on the fact that objects in Express Persistent Objects (XPO) support change tracking. One area that this is used by their Unit of Work to track the changes made to objects. Another use is when data binding objects to a UI. So, how do you enable change tracking in your object model using XPO?

XPO supports a variety of methods for achieving this, but the easiest is to inherit your domain objects from XPObject. When you do this change tracking is enabled by calling a magic setter method in your property setters and using their built-in collection type for association properties:

public class Customer: XPObject
{
  private DateTime date;
  public DateTime Date
  {
    get { return date; }
    set
    {
      SetPropertyValue("Date", ref date, value);
    }
  }
  [Association("CustomerOrders", typeof(Order))]
  public XPCollection<Order> Orders
  {
    get { return GetCollection<Order>("Orders"); }
  }
}

Note that Association attribute on the collection property is required by XPO and indicates that the property is one end of a two-way association - either one-many or many-many. Associations are always two-way, except one-one associations which are handled differently. So, for completeness, with the example above being a one-many relationship, you will need the following to express the other end:

public class Order: XPObject
{
  private Customer customer;
  [Association("CustomerOrders")]
  public Customer Customer
  {
    get { return customer; }
    set 
    { 
      SetPropertyValue("Customer", ref customer, value); 
    }
  }  
}

It makes sense to use a collection type that supports change notification and the calls to SetPropertyValue are a fairly elegant way to notify change. Okay, this is all good, but I like to cut down the syntactic noise as far as possible. This basically means that I want to to use C# automatic properties (and I also don't want get into using design time code generation techniques to help me out). So, where do we go from here? We go to this:

[XpoAutomaticProperties]
public class Customer: XPObject
{
  public DateTime Date { get; set; }
  [Association("CustomerOrders", typeof(Order))] 
  public XPCollection<Order> Orders 
  { 
    get; private set; 
  }
}
[XpoAutomaticProperties]
public class Order: XPObject
{
  [Association("CustomerOrders")] 
  public Customer Customer 
  { 
    get; private set; 
  }
}

The classes above will give the same functionality - change tracking and collection creation - as the previous examples. Except, there is much less code to read, just automatic properties and an extra attribute on the class declaration. Take a look at part 2 to see how this can be done.

Friday 24 October 2008

ORM with Developer Express Persistent Objects (XPO)

I've been using ORMs for a long while now and love the speed and flexibility they can bring to the development process. For projects in  work we use an ORM that we wrote ourselves.  As I'm now doing some projects outside of my day job, I have been looking around at the available ORM offerings for a few months now.  I'm now pretty likely to be using Express Persistent Objects (XPO) from Developer Express - a developer tool vendor with an excellent reputation and strong background.

I've been evaluating their UI controls for a while now and I have been generally very impressed with both their products and their excellent support.  XPO comes from this background and has the main features that I want:

  • Support for the Compact Framework.
  • A LINQ provider.
  • Generation of schema from object definitions (you can also generate objects from an existing schema).
  • Wide support for a variety of databases (including my target database, VistaDB). 
  • Easy data binding of objects to controls (as you'd expect from DevExpress!).

It does have a few quirks that I'm a little unsure about at the moment.  The main ones for me now are:
  • All associations with multiplicity greater than one have to be two-way.
  • The automatic schema generation mechanism will not widen string fields in existing schemas. 
I also get the feeling from reading their forums that XPO gets a "little less love" than some of their other products.  I think that it may be seen as either largely functionally complete, or that it now plays a supporting role to their application framework, XAF.

However, these are only small niggles and the product is seems to be well constructed, uses best practices, has good support from a much respected tool vendor and is realistically priced.  If you are coming to ORM from a data-driven background, then you will probably even be comfortable with having to always having to have have two sides to an association!  It is worth noting that XPO takes care of the other end of the relationship for you - if I add an order to a customer, then the order's customer is automatically set...neat and sweet!

If you are looking for an easy to use commercial ORM that is reasonably priced, then I would recommend you take a look.   If you are not looking for a non-commercial ORM, or want to learn about the world of object-relational mapping, then head off to NHibernate.

Edit: Just thought it worth noting that the "small niggles" about XPO are in the context of my current project, which has a relatively simple object model - one for which I would happily manually write the data access code.  I would not be comfortable recommending XPO for complex object models at this stage.