Wednesday, July 23, 2014

Making stringly-typed code palatable via implicit operator

Strongly-typed code is all the rage now, but some of us still manages to stash various information to string instead of structuring the information to a class


If you can't avoid stringly-typed code, just eliminate its boilerplate codes. Use implicit operator to eliminate them


using System;

using System.Collections.Generic;


using System.Linq;


namespace Craft
{
    class MainClass
    {

        public static void Main (string[] args)
        {
            var list = new Dictionary<string,decimal> ();
            list.Add ("1_VariablePayBasis", 1337);
            list.Add ("1_DecisionAmount", 168);
            list.Add ("3_BasePay", 5201314);


            foreach (var kv in list) {

                // boilerplate codes
                string[] strings = kv.Key.Split('_');
                int pk = Convert.ToInt32(strings [0]);
                string fieldName = strings [1];


                Console.WriteLine ("{0} {1} - {2}", pk, fieldName, kv.Value);
            }
            Console.WriteLine();


            // via explicit casting, being explicit with implicit :-)
            foreach (var fv in list.Select(x => (FieldValue)x)) {
                Console.WriteLine ("{0} {1} - {2}", fv.PrimaryKey, fv.FieldName, fv.Value);
            }
            Console.WriteLine();


            // neat implicit! it still feel a bit explicit though. we can't use var here
            foreach (FieldValue fv in list) {
                Console.WriteLine ("{0} {1} - {2}", fv.PrimaryKey, fv.FieldName, fv.Value);
            }

        }
    }

    public class FieldValue
    {
        public int PrimaryKey { get; set; }
        public string FieldName { get; set; }

        public decimal Value { get; set; }



        public static implicit operator FieldValue(KeyValuePair<string,decimal> kv)
        {
            string[] strings = kv.Key.Split('_');
            int pk = Convert.ToInt32(strings [0]);
            string fieldName = strings [1];

            return new FieldValue { PrimaryKey = pk, FieldName = fieldName, Value = kv.Value };
        }
    }
   
}


Live Code https://dotnetfiddle.net/qX2UeA


Happy Coding! ツ

Sunday, July 20, 2014

protected internal

Anne Epstein has a great NHibernate article on making an entity with a composite primary key be lazy-loading-capable and cache-ready. Just forgot to hide the ORM low-level plumbing concerns (the composite primary key) from the domain model


Similar to AdventureWorks' EmployeePayHistory. Just use protected internal to hide that ORM low-level plumbing concern
public class PersonPayHistory
{
    PersonPayHistoryCompositePK _pk = new PersonPayHistoryCompositePK();
    protected internal PersonPayHistoryCompositePK PersonPayHistoryCompositePK 
    { 
        get { return _pk; }
        set { _pk = value; }
    }

    Person _person;
    public virtual Person Person 
    { 
        get { return _person; }
        set 
        {
            _person = value;
            _pk.PersonId = _person.PersonId;
        }
    }                    

    public virtual DateTime RateDate
    {
        get { return _pk.RateDate; }
        set { _pk.RateDate = value; }
    }

    public virtual decimal Rate { get; set; }
}


So we will not make the mistake of saving our entity through those composite primary key:

var ph = new PersonPayHistory
{     
    Person = session.Load<Person>(1),
    RateDate = DateTime.Today,

    Rate = 1337M
};


session.Save (ph);
session.Flush ();

Wednesday, July 16, 2014

LEFT JOIN GROUP BY HAVING on NHibernate

EmployeePayHistory payHistory = null;
   
var persons =
    session.QueryOver<Person>()
    .Left.JoinAlias(p => p.PayHistories, () => payHistory)                        
    .Where(
        Restrictions.Eq(
            Projections.Count(() => payHistory.EmployeePayHistoryCompositePK.BusinessEntityId)
        , 0
        )
    )    
    .SelectList(x => x
        .SelectGroup(p => p.BusinessEntityId)
        .SelectGroup(p => p.FirstName)
        .SelectGroup(p => p.LastName)                            
        .SelectCount(p => payHistory.EmployeePayHistoryCompositePK.BusinessEntityId)
    )
    .List<object[]>()
    .Select(cols => new
    {
        Id = (int)cols[0],
        FirstName = (string)cols[1],
        LastName = (string)cols[2],
        Count = (int)cols[3]
    });

persons.ToList();


Generated SQL:
SELECT
         this_.BusinessEntityId as y0_,
         this_.FirstName as y1_,
         this_.LastName as y2_,
         count(payhistory1_.BusinessEntityId) as y3_
     FROM
         Person.Person this_
     left outer join
         HumanResources.EmployeePayHistory payhistory1_
             on this_.BusinessEntityId=payhistory1_.BusinessEntityId
     GROUP BY
         this_.BusinessEntityId,
         this_.FirstName,
         this_.LastName
     HAVING
         count(payhistory1_.BusinessEntityId) = @p0;
     @p0 = 0 [Type: Int32 (0)]


One-stop shop code, complete working code. Just install AdventureWorks2008R2 database:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using NHibernate.Cfg;
using NHibernate.Cfg.MappingSchema;
using NHibernate.Criterion;
using NHibernate.Mapping.ByCode.Conformist;
 
using NHibernate.Linq;

 
namespace Craft
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var session = Mapper.SessionFactory.OpenSession())
            {
                
         EmployeePayHistory payHistory = null;

         var persons =
                     session.QueryOver<Person>()
                    .Left.JoinAlias(p => p.PayHistories, () => payHistory)
                    .Where(
                         Restrictions.Eq(
                             Projections.Count(() => payHistory.EmployeePayHistoryCompositePK.BusinessEntityId)
                             , 0
                         )
                     )

                    .SelectList(x => x
                        .SelectGroup(p => p.BusinessEntityId)
                        .SelectGroup(p => p.FirstName)
                        .SelectGroup(p => p.LastName)                            
                        .SelectCount(p => payHistory.EmployeePayHistoryCompositePK.BusinessEntityId)
                    )
                   .List<object[]>()
                   .Select(cols => new
                   {
                      Id = (int)cols[0],
                      FirstName = (string)cols[1],
                      LastName = (string)cols[2],
                      Count = (int)cols[3]
                   });

                persons.ToList();



                Console.ReadLine();
           } // using
        } // Main
    } //class Program
 
 
    public static class Mapper
    {
        static NHibernate.ISessionFactory _sessionFactory = Mapper.GetSessionFactory();
 
 
        public static NHibernate.ISessionFactory SessionFactory
        {
            get { return _sessionFactory; }
        }
 
        static NHibernate.ISessionFactory GetSessionFactory()
        {
            var mapper = new NHibernate.Mapping.ByCode.ModelMapper();
 
            mapper.AddMappings(
                new[] { 
                    typeof(PersonMapping) , typeof(EmployeePayHistoryMapping)
                });
 
 
            var cfg = new NHibernate.Cfg.Configuration();
 
            cfg.DataBaseIntegration(c =>
            {
                c.Driver<NHibernate.Driver.Sql2008ClientDriver>();
                c.Dialect<NHibernate.Dialect.MsSql2008Dialect>();
                c.ConnectionString = "Server=.;Database=AdventureWorks2008R2;Trusted_Connection=True;";
 
                c.LogFormattedSql = true;
                c.LogSqlInConsole = true;
            });
 
 
 
            HbmMapping domainMapping = mapper.CompileMappingForAllExplicitlyAddedEntities();
 
            cfg.AddMapping(domainMapping);
 
 
 
            var sf = cfg.BuildSessionFactory();
 
            return sf;
        }
    }
 
 
    public class PersonMapping : ClassMapping<Person>
    {
        public PersonMapping()
        {
            Table("Person.Person");
            Id(x => x.BusinessEntityId);
 
            Property(x => x.FirstName);
            Property(x => x.LastName);
 
            Bag(list => list.PayHistories, rel => rel.Key(k => k.Column("BusinessEntityId")), relType => relType.OneToMany());
 
        }
    }
 
    public class EmployeePayHistoryMapping : ClassMapping<EmployeePayHistory>
    {
        public EmployeePayHistoryMapping()
        {
            Table("HumanResources.EmployeePayHistory");
 
            ComponentAsId(x => x.EmployeePayHistoryCompositePK, x =>
            {
                x.Property(y => y.BusinessEntityId);
                x.Property(y => y.RateChangeDate);
            });
          
            ManyToOne(x => x.Person, k => 
            { 
                k.Column("BusinessEntityId"); 
                k.Insert(false); 
                k.Update(false); 
            });

            Property(x => x.Rate);
        }
    }
 
    
 
    public class Person
    {
        public virtual int BusinessEntityId { get; set; }
        public virtual string FirstName { get; set; }
        public virtual string LastName { get; set; }
 
        public virtual IList<EmployeePayHistory> PayHistories { get; set; }
    }
 
 
    public class EmployeePayHistory
    {
        EmployeePayHistoryCompositePK _pk = new EmployeePayHistoryCompositePK();
        public virtual EmployeePayHistoryCompositePK EmployeePayHistoryCompositePK 
        { 
            get { return _pk; } set { _pk = value; }
        }
 
        Person _person;
        public virtual Person Person 
        { 
            get { return _person; }
            set 
            {
                _person = value;
                _pk.BusinessEntityId = _person.BusinessEntityId;
            }
        }        
 
        public virtual decimal Rate { get; set; }
    }
 
 
    // AdventureWorks is fond of composite primary key. Composite unique is better
    public class EmployeePayHistoryCompositePK
    {
        public virtual int BusinessEntityId { get; set; }
        public virtual DateTime RateChangeDate { get; set; }
 
        public override bool Equals(object obj)
        {
            if (obj == null)
                return false;
 
            var t = obj as EmployeePayHistoryCompositePK;
            if (t == null)
                return false;
 
            if (BusinessEntityId == t.BusinessEntityId && RateChangeDate == t.RateChangeDate)
                return true;
 
            return false;
        }
 
        public override int GetHashCode()
        {
            return (BusinessEntityId + "|" + RateChangeDate).GetHashCode();
        }
    }
}


Happy Coding! ツ


Related: http://www.ienablemuch.com/2014/07/orm-expectations.html

ORM Expectations




Sometimes it's hard to give clear instruction, especially if you are not inclined to be exact with your instruction:

var persons = 
     from p in session.Query<Person>()
     select new 
     {
         p.BusinessEntityId, p.FirstName, p.LastName, Count = p.PayHistories.Count()
     };


What's the SQL generated by that Linq?

NHibernate-generated SQL:
 
select
    person0_.BusinessEntityId as col_0_0_,
    person0_.FirstName as col_1_0_,
    person0_.LastName as col_2_0_,
    (select
        cast(count(*) as INT)
    from
        HumanResources.EmployeePayHistory payhistori1_
    where
        person0_.BusinessEntityId=payhistori1_.BusinessEntityId) as col_3_0_
from
    Person.Person person0_                    
 
 
Entity Framework-generated SQL:
 
SELECT 
    [Extent1].[BusinessEntityId] AS [BusinessEntityId],
    [Extent1].[LastName] AS [LastName],
    [Extent1].[FirstName] AS [FirstName],
    (SELECT
        COUNT(1) AS [A1]
    FROM 
        [HumanResources].[EmployeePayHistory] AS [Extent2]
    WHERE 
        [Extent1].[BusinessEntityId] = [Extent2].[BusinessEntityId]) AS [C1]
FROM 
    [Person].[Person] AS [Extent1]

"That's not efficient! That's not what I wanted" you told yourself

I've often felt that much of the frustration with ORMs is about inflated expectations -- http://martinfowler.com/bliki/OrmHate.html

There are developers who insists that the above should generate a GROUP BY COUNT. Don't expect a GROUP BY will be generated on the Linq you've made above. Who told you that computers will do what you wanted or hoped them to do? You cannot wish your way around on solving a problem, especially on giving instructions to computers

With that being said, let's be exact with our instructions. Let's cut to the chase, following is how to do efficient aggregation in NHibernate and Entity Framework. Initial step, flatten the hierarchical objects via SelectMany; final step, do a group by on the flattened object. Easy-peasy!

var persons =
    from p in session.Query<Person>()
        .SelectMany(x => x.PayHistories, (person, payHistory) => new {person, payHistory})
    group p by new {p.person.BusinessEntityId, p.person.FirstName, p.person.LastName} into g
    select new {g.Key, Count = g.Count()};

SQL output:
NHibernate-generated SQL:
 
select
    person0_.BusinessEntityId as col_0_0_,
    person0_.FirstName as col_1_0_,
    person0_.LastName as col_2_0_,
    cast(count(*) as INT) as col_3_0_
from
    Person.Person person0_
inner join
    HumanResources.EmployeePayHistory payhistori1_
        on person0_.BusinessEntityId=payhistori1_.BusinessEntityId
group by
    person0_.BusinessEntityId ,
    person0_.FirstName ,
    person0_.LastName
 

Entity Framework-Generated SQL. Go home Entity Framework, you're drunk!

SELECT
    [GroupBy1].[K1] AS [BusinessEntityId],
    [GroupBy1].[K4] AS [C1],
    [GroupBy1].[K3] AS [FirstName],
    [GroupBy1].[K2] AS [LastName],
    [GroupBy1].[A1] AS [C2]
    FROM ( SELECT
        [Join1].[K1] AS [K1],
        [Join1].[K2] AS [K2],
        [Join1].[K3] AS [K3],
        [Join1].[K4] AS [K4],
        COUNT([Join1].[A1]) AS [A1]
        FROM ( SELECT
            [Extent1].[BusinessEntityId] AS [K1],
            [Extent1].[LastName] AS [K2],
            [Extent1].[FirstName] AS [K3],
            1 AS [K4],
            1 AS [A1]
            FROM  [Person].[Person] AS [Extent1]
            INNER JOIN [HumanResources].[EmployeePayHistory] AS [Extent2] ON [Extent1].[BusinessEntityId] = [Extent2].[BusinessEntityId]
        )  AS [Join1]
        GROUP BY [K1], [K2], [K3], [K4]
    )  AS [GroupBy1]

With the exact instructions, the computer will do what we told it to do, NHibernate generated what we wanted, a COUNT via efficient GROUP BY, instead of subquery in SELECT. However, even giving an exact instruction is not a guarantee that the computer will do what we told it to do, look at how stubborn Entity Framework is; though also it doesn't generate a subquery for COUNT, the approach and efficiency of Entity Framework-generated SQL is questionable and hard to reason about


A keen developer shall observe, the GROUP BY we made is not exactly the same as the subquery approach; the subquery approach could report persons without employee pay history while the INNER JOIN could not. We should do a LEFT JOIN and make a conditional count on non-null. Entity Framework solution:


var query = 
    from p in db.Set<Person>()
        .SelectMany(x => x.PayHistories.DefaultIfEmpty(), (person, payHistory) => new {person, payHistory })
    group p by new { p.person.BusinessEntityId, p.person.FirstName, p.person.LastName } into g
    select new { g.Key, Count = g.Sum(x => x.payHistory != null ? 1 : 0) };


Entity Framework-generated SQL:
SELECT
    [GroupBy1].[K1] AS [BusinessEntityId],
    [GroupBy1].[K4] AS [C1],
    [GroupBy1].[K3] AS [FirstName],
    [GroupBy1].[K2] AS [LastName],
    [GroupBy1].[A1] AS [C2]
    FROM ( SELECT
        [Join1].[K1] AS [K1],
        [Join1].[K2] AS [K2],
        [Join1].[K3] AS [K3],
        [Join1].[K4] AS [K4],
        SUM([Join1].[A1]) AS [A1]
        FROM ( SELECT
            [Extent1].[BusinessEntityId] AS [K1],
            [Extent1].[LastName] AS [K2],
            [Extent1].[FirstName] AS [K3],
            1 AS [K4],
            CASE WHEN ( NOT (([Extent2].[BusinessEntityId] IS NULL) AND ([Extent2].[RateChangeDate] IS NULL))) THEN 1 ELSE 0 END AS [A1]
            FROM  [Person].[Person] AS [Extent1]
            LEFT OUTER JOIN [HumanResources].[EmployeePayHistory] AS [Extent2] ON [Extent1].[BusinessEntityId] = [Extent2].[BusinessEntityId]
        )  AS [Join1]
        GROUP BY [K1], [K2], [K3], [K4]
    )  AS [GroupBy1]

Unfortunately, though it's a relatively easy fix on Entity Framework, we just added a DefaultIfEmpty and do a conditional count on non-null, the same won't work on NHibernate. As of the time of this writing, NHibernate Linq doesn't support DefaultIfEmpty, it can't produce a LEFT JOIN on collection nor on manual joins. NHibernate has an API for that trivial task, and its API is more complex and longer than the problem it is solving :-)


EmployeePayHistory payHistory = null;
var persons =
    session.QueryOver<Person>()
    .Left.JoinAlias(p => p.PayHistories, () => payHistory)                        
    .SelectList(x => x
        .SelectGroup(p => p.BusinessEntityId)
        .SelectGroup(p => p.FirstName)
        .SelectGroup(p => p.LastName)                            
        .SelectCount(p => payHistory.EmployeePayHistoryCompositePK.BusinessEntityId)
    )
    .List<object[]>()
    .Select(cols => new
        {
            Id = (int)cols[0],
            FirstName = (string)cols[1],
            LastName = (string)cols[2],
            Count = (int)cols[3]
        });


NHibernate-generated SQL:
SELECT
    this_.BusinessEntityId as y0_,
    this_.FirstName as y1_,
    this_.LastName as y2_,
    count(payhistory1_.BusinessEntityId) as y3_
FROM
    Person.Person this_
left outer join
    HumanResources.EmployeePayHistory payhistory1_
        on this_.BusinessEntityId=payhistory1_.BusinessEntityId
GROUP BY
    this_.BusinessEntityId,
    this_.FirstName,
    this_.LastName


With clear instructions and reasonable expectations, computers will do what you wanted it to do


Happy Coding! ツ

Wednesday, July 9, 2014

Lucky .NET developers

Java's signed bytes gets me every time I try to do image processing. Easy answer is "Don't do image processing in java", of course. -- John Carmack

Time to reflect on the fact that though statistically speaking we are not the lucky ones, we are lucky in terms of empowerment, we can make image processing application in C#, we can make games with it, we can make web applications as well as desktop applications with it, and we can even make an operating system with it


Happy Coding!

Tuesday, June 17, 2014

How to avoid the expensive ToList on generic list? Generics' OOPness redux

Have you questioned C# generics’ OOPness? Why is C# is not allowing us to return List<Autobot> to List<ITransformer> ?

This won't compile even Autobot implements ITransformer:



Can fix it with ToList, but it's an expensive operation, as it creates a copy:
public static List<ITransformer> GetTransformers(TransformerType t)
{
    if (t == TransformerType.Autobot)
        return GetAutobots().ToList<ITransformer>();
    else if (t == TransformerType.Decepticon)
        return GetDecepticons().ToList<ITransformer>();
         
    return null;
}   


To cut to the chase, return the List<ConcreteClass> to IEnumerable<Interface>:
public static IEnumerable<ITransformer> GetTransformersOop(TransformerType t)
{
    if (t == TransformerType.Autobot)
        return GetAutobots();
    else if (t == TransformerType.Decepticon)
        return GetDecepticons();
         
    return null;
}

Live code: https://dotnetfiddle.net/K3IcR5

For further information why List<ConcreteClass> is not compatible to List<Interface> even the concrete class implements the interface, read about covariance

Saturday, June 14, 2014

ORM Choice

ORM choice is a touchy subject, you can't just choose one without riling up some developers or DBAs in your team, it's a choice that must be decided carefully. When you've chosen the right ORM for the project, it is a shared joy for the team; picking the wrong one gives everyone a shared pain


As ORM has an influence how we model the domain models (facilitates shared vocabulary between the developers and the users), we should have a strategy when choosing one (or even two) for your project. We should also have a strategy when modeling our domain models. When we choose an ORM, we must embrace its principles, advantages, limitations, warts and all


And also, there are some who hates ORM, I could articulate some of the reasons why some hates ORM, but it's better to leave that to the experts. Here's the takeaway why some hates ORM, and Martin Fowler can't be wrong on positing that:

much of the frustration with ORMs is about inflated expectations


Back to the subject on choosing an ORM


You might think that this ORM drivel is about my repository framework (a repository framework to prevent the high coupling of an application to an specific ORM) I'm trying to endorse. I would not endorse it, it's a half-baked solution for what it's trying to achieve


I made a repository framework that encapsulates both Entity Framework and NHibernate in my earnest attempt to keep a project from being too coupled to an specific ORM framework. I named it ToTheEfNhX, short for: To The Entity Framework and NHibernate Elimination


However there are some principles and approaches that varies between those two ORMs that made me cease the development of that repository component project


Don't abstract out your ORM, don't abstract things out just for the reason it will be easy for you to switch to another ORM if you don't like the performance of your current ORM. Don't add another layer (e.g., repository, or if you really need to, just add very minimal or thin layer) on top of your ORM, just let the power of your ORM flows to your application



I'm not saying that it's a futile effort to abstract out ORM differences, but there are many differences in approaches between ORMs that will make one cease the effort for doing so


when in Rome, do as the Romans do


Case in point, in order to make the domain models work on both NHibernate and Entity Framework, i.e., instead of adding foreign key property to the model, just maintain an object reference, this domain model works on both NHibernate and Entity Framework:

public class Person 
{
     public virtual int PersonId { get; set; }
     public virtual string Firstname { get; set; }
     public virtual string Lastname { get; set; }

     public virtual Country DreamCountryToTravel { get; set; }
}


However, a keen Entity Framework enthusiast shall observe: "Isn't that model expensive to populate? The DreamCountryToTravel property is an independent association, it needed be assigned with an eagerly-loaded object."

person.DreamCountryToTravel = dbContext.Countries.Find(63); // .Find eagerly loads an object from the database


However, an NHibernater would beg to disagree, "that's the right way to model the business domain, it's very OOP, and you don't need to assign an eagerly-loaded object to DreamCountryToTravel. Session.Load is efficient, it lazily loads an object, it doesn't fetch anything from the database"

person.DreamCountryToTravel = session.Load<Country>(63);


An EF enthusiasts moving to NHibernate should not (and could not) force this kind of domain model to NHibernate:
public class Person 
 {
      public virtual int PersonId { get; set; }
      public virtual string Firstname { get; set; }
      public virtual string Lastname { get; set; }
 
      
      public virtual int DreamCountryToTravelId { get; set; }
      public virtual Country DreamCountryToTravel { get; set; }
 }


EF enthusiasts only can use foreign key property in NHibernate though, but it's not the OOP way to map relational to object:
public class Person 
 {
      public virtual int PersonId { get; set; }
      public virtual string Firstname { get; set; }
      public virtual string Lastname { get; set; }
 
      public virtual int DreamCountryToTravelId { get; set; }
 }


Likewise, an NHibernater moving to Entity Framework, should not force this kind of domain model to Entity Framework, as it will only incur performance problems in Entity Framework:
public class Person 
 {
      public virtual int PersonId { get; set; }
      public virtual string Firstname { get; set; }
      public virtual string Lastname { get; set; }
 
      public virtual Country DreamCountryToTravel { get; set; }
 }


The repository pattern I've made (that works on both NHibernate and Entity Framework) have the above type of a recommended domain modeling. I readied a LoadStub method for EF (4.1 then) in my utmost hope that someday Entity Framework will offer the same functionality as NHibernate's .Load method, that EF will facilitate OOP and drop the relational/persistence concern that foreign key property is. Here's the LoadStub method:

person.DreamCountryToTravel = countryRepo.LoadStub(63);


But alas, two versions (EF 4.1 was the latest version when I made ToTheEfnhX) and later, the independent association still can only receive eagerly loaded object and it is still expensive to eagerly load an object, there's still no way to just obtain an object stub/proxy with EF. There's a way though to load stub objects only, but it's for many-to-many relationship only



And the final straw that led me to cease the development of that ORM-agnostic ToTheEfnhX repository, there's a big difference in approach between the two ORMs when fetching multiple collections. It's hard to abstract them out


If you still want to create a repository pattern/component/wrappers around your ORM, just don't pull a strawman argument to justify creating one. Here's Ayende calling out one person's strawman argument against Ayende's advocacy to not use repository:


http://ayende.com/blog/153701/ask-ayende-life-without-repositories-are-they-worth-living


Don't bindly create a pattern just for the sake of creating a pattern, you might just be reinventing the wheel if you don't realized you already have a wheel. There's no shame on not creating repository pattern when your ORM already has a repository pattern bolted-in


Avoid overengineering: http://ayende.com/blog/4784/architecting-in-the-pit-of-doom-the-evils-of-the-repository-abstraction-layer



I wrote this to ease out the pain of one of the teams in our company for having to transition from one ORM to another. I hope this blog reaches the whole team and make them embrace their new ORM