LINQ to SQL not Suitable for LOB

In my previous post I mentioned that some post or blogs on the Internet are hugely misleading about the available technologies, since they tend to hide basic facts and focus on the superficial magic, which sometimes simply don’t matter when the technology is unusable.

In the company I work for, we rejected LINQ to SQL about a month ago, after trying to solve its biggest problem that of performance combined with thread safe when cashing the Data Context. So any info I have found from various sources are not available.

What is LINQ?

LINQ is basically a collection of extension methods to any Enumerable object.

What is LINQ to SQL?

LINQ to SQL is LINQ over the enumerable objects created while drag and dropping tables and procedures in a dbml file. This classes are known as Entities and the object that manages them as a DataContext.

My Sin

I really don’t like SQL. It reminds me of procedural programming which I stopped writing a decade ago. So any technology that will allow me not write SQL is more than welcome for me. So when I first started reading about LINQ to SQL, I started thinking at last an ORM from Microsoft herself.

My Sin was that in spite of the objections of our more experienced programmer, I was standing by LINQ to SQL, mainly using arguments that came from the notion that everything was great in LINQ to SQL. This notion existed because I believed the posts on the Internet.

One of the biggest objections was that of security. It was unacceptable for him, for an application to have access to the tables of a database. My lack of SQL knowledge, didn’t take that in mind, so as then, supposedly there is no security objection for the rest of the post.

First Impressions

At first I was really disappointed because there was no support for DataSet manipulation through LINQ to SQL. At first I hadn’t realized what LINQ is exactly, so I implemented a library that did this job. When building N-Tier applications, Typed DataSets are the most effective solution for the business model. I wrote about it here. Having wrote this library I was really convinced that we had a great tool for LINQ to SQL, and that it was the choice for our Data Access Layer.

Problems arising

Having spent time to build this library, it now came to check whether LINQ to SQL was the valid choice for out Application. So we created a huge table in SQL Server and started case testing and comparing with know DataSet methodology.

A Data Access Layer will be used by a Web or a Server Side of an application. In order to be thread safe, you need to create the Data Context with each call. But this is slow when data are huge, and there were some posts saying that there would be a way to cache the data context through a configuration setting. So I automatically assumed that caching and thread safe has been taken into account.

But that wasn’t the case. The sad truth is that, if you cache the Data Context you must create and maintain one for every thread or Http Context in order to make them thread safe, with a trick I read from someone else. Practically you don’t make them thread safe, but thread specific.

With caching, performance improved greatly. In some cases it was quicker than DataSet methodology. The main reason was that, while the queries run, the Data Context of each thread kept its entities in memory so no queries to server where required after a number of iterations. After this was noticed I immediately thought what about memory consumption in the server as the execution time passed. How the hell do you manage this side effect.

Conclution

LINQ to SQL might be very appealing when reading about simple objects and simple applications, but when the application gets big the coordination problems that might occur and produce data corruption, are clearly the death tomb of LINQ to SQL for LOB. And the sad part was that, I fell victim to all those glorious posts. I believe this is a risk that no company should ever take. Data integrity is something that one must never mess with. This includes the security objection mentioned above.

LINQ to SQL is a proof of concept. It is one of the things that in IT theory look great but when put in practice, it proves once again that theory sometimes doesn’t relate to practice.

Typed Dataset <–> Linq Entities

Introduction

On my previous post I discussed about how LINQ entities to not fit the world of applications that do not have a constant access to the data source. I concluded that if there was a way to connect Linq Entities and Type Dataset, then the domain of Web Applications and N-Tier Applications could be supported by the same Bussiness Object Model and a Data Access Layer over LINQ.

Assumptions – Prerequisites

Entity and Data Table Naming

Before I continue there is a basic assumption that must be kept in mind. The Business Object Model and the Typed Dataset must be constructed by their respective designer in Visual Studio, by dragging the tables into each designer. The main reason is, that the converter I have developed, assumes that the corresponding entities in LINQ and table in the Dataset have the same name.

Relations and Foreign Key Constraints

Also every relation between entities must have the same name as that between tables in the dataset. The above are automatically (great coincidence) kept, just by using the designer.

Circular Relations and all combinations have not been tested, so I do not know whether my code supports them

Database construction From LINQ

If you wish to construct the database schema from the LINQ designer then just do so, but before creating the typed dataset, the database must be created. To do this just call

LinqTestDataContext ltdc = new LinqTestDataContext(connectionString); if (!ltdc.DatabaseExists()) { ltdc.CreateDatabase(); }

where LinqTestDataContext is the DataContext the designer has created.

Column Prerequisite

Each entity must have a version property. This is because Attach(entity,true) only works if there is such a property.

The Database Schema used for testing

The LINQ schema is name LinqTest and its dataset represantion DsLinqTest.

As seen in the picture below there is a RootElement with a unique key ID, a version property TimeStamp and two string properties.

RootElement has child relation of SubRootElement entities which also have a unique key ID, a version property TimeStamp a string property and a RootID foreign key pointing to the RootElement it belongs

The corresponding Dataset will be. The relation name is the same, even thought it is not showing on the above image.

Each of the Business Object are in a separate assembly.

DataSetEntityConvertion

This is name of the assembly that does the convention between an LINQ Business Object and a Typed Dataset assuming that the above prerequisites are met.

The assembly uses heavily reflection and generics so the understanding of the above must be at least good.

Keep in mind that since the dataset is typed, every type in the dataset is specifically named so it can be used to discover the entities it relates to.

ToDataRow

Is the part where entities are used to fill the appropriate tables in the dataset.

The entry point is the Entity2DataSet class, where TEntity is the entity type and TDataSet is the dataset type. In our case RootElement and DsLinqTest respectively.

Basically the Entity2DataSet class discovers the table that corresponds to the entity, and then calls the Entity2DataRow class which in addition takes the DataTable type discovered.

There are some helping functions that through reflection fill the row, from the entity and also find the child relations of the entity if there are any. If that is true the Entity2DataSet class is called again but this time TEntity should be SubRootElement in our case.

This side of the convention is fairly easy.

ToEntity

This case deals with converting a whole dataset to its entity. The entry class is DataSet2Entity where TDataContext is the type of our DataContext and TDataSet the type of the source Dataset. In our case LinqTestBigDataContext and DsLinqTest respectively.

The first thing that DataSet2Entity does is to find the tables have no parent relations. For each of these tables DataTable2Entity is used where in addition TDataTable and TDataRow are the types of the table and its rows.

DataTable2Entity discovers the entity type that must create for each row it has and does so by using DataRow2Entity which is supplied with the knowledge of whether it is child row or not. This is crucial because if it is child row, it must be added to the related EntitySet of its parent entity instead of the entity Table in the data context.

The trick here is to know whether the original row is Added,Modified, Deleted or unchanged which is the easy part through RowState. The hard part is what to do with it.

Added

This case is easy. Just construct the entity and add it the table or the entityset and call InsertOnSubmit.

Modified or Unmodified

Here start the problems. First we must acquire the entity it self to which we will apply the values. Accordingly to if the row is a child or not, a predicate function or expression must be constructed. This part was the most difficult.

If the row is unmodified then there will be no applying of values.

Deleted

Like in Modified the entity must be retrieved from the entitytable of the datacontext in order to call DeleteOnSubmit.

Keeping track of the changes

When a row is inserted or modified, various column values need to be updated by the auto generated ones from the database. So in every entity the PropertyChanged is captured. There with the help of a dictionary the new values are applied to the original rows. This happens after the SubmitChanges of the datacontext is used.

The rest of the DataRow2Entity finds the child rows of the row for each data relation and calls another generic version of its self.

Creating Predicate Functions and Expressions

This was the hardest part, and still there are some point that I can’t understand.

When trying to acquire an entity from the table entity of the datacontext, a simple delegate function suffices. After many attempts a managed to make the creation entirely dynamic based on the primary keys of the entity.

This is done by these two functions

private System.Func<TEntity, bool> CreatePredicateFunction(TDataRow row) { return p => (IsEqual(p, row)); } private bool IsEqual(TEntity entity, TDataRow row) { for (int i = 0; i < Cache.EntityPrimaryKeys<TEntity>.Names.Count; i++) { object columnValue = null; if (row.RowState == DataRowState.Deleted) { columnValue = row[Cache.EntityPrimaryKeys<TEntity>.Names[i], DataRowVersion.Original]; } else { columnValue = row[Cache.EntityPrimaryKeys<TEntity>.Names[i]]; } if ((bool)Cache.EntityPrimaryKeys<TEntity>.EqualMethods[i].Invoke(this.entityType.GetProperty(Cache.EntityPrimaryKeys<TEntity>.Names[i]).GetValue(entity, null), new object[] { columnValue }) == false) { return false; } } return true; }

Happy as I was that I will be able to cast the above to an Expression<System.Func<TEntity, bool>> I found out that at runtime an exception is thrown telling me that IsEqual cannot be converted or something.

I assume the Expression is something far more complicated than a delegate. So in order for this to work a CreatePredicateExpression must by supplied in every DataRow of our dataset. I did like this

public static class DsLinqTestPredicators { public static Expression<System.Func<RootElement, bool>> CreatePredicateExpression(DsLinqTest.RootElementRow row) { int idValue = row.RowState == System.Data.DataRowState.Deleted ? (int)row["ID", System.Data.DataRowVersion.Original] : row.ID; return (Expression<System.Func<RootElement, bool>>)(p => p.ID.Equals(idValue)); } public static Expression<System.Func<SubRootElement, bool>> CreatePredicateExpression(DsLinqTest.SubRootElementRow row) { int idValue = row.RowState == System.Data.DataRowState.Deleted ? (int)row["ID", System.Data.DataRowVersion.Original] : row.ID; return (Expression<System.Func<SubRootElement, bool>>)(p => p.ID.Equals(idValue)); } }

Final Words for the Converter

Extension Methods are heavily used to help making the convertion as programmatically tranparent as possible.

Using the Code

Extenders

public static classDsLinqTestExtenders
{
    public static voidInsert(thisDsLinqTest extented, objectentity)
    {
        ((DataSet)extented).Insert(entity);
    }
    public static voidInsert(thisDsLinqTest extented, object[] entities)
    {
        ((DataSet)extented).Insert(entities);
    }

public static voidToEntities(thisDsLinqTest extented, DataContext dataContext)
    {
        ((DataSet)extented).ToEntities(dataContext);

}
}

Entity2Dataset

public DsLinqTest GetDsFromID(int id) { LinqTestDataContext ltdc = new LinqTestDataContext(connectionString); RootElement re = ltdc.RootElements.Single(p => p.ID.Equals(id)); DsLinqTest ds = new DsLinqTest(); ds.Insert(re); ds.AcceptChanges(); return ds;}

DataSet2Entity

public void SaveGeneralDs(DsLinqTest dsLinqTest) { LinqTestDataContext ltdc = new LinqTestDataContext(connectionString); dsLinqTest.ToEntities(ltdc); ltdc.SubmitChanges(); }

Source Code for the above assemblies