Project MSE-iT Upgrade Framework starts

As mentioned in previous posts, i want to have a declarative way to tag upgrades in code. After some research i came to conclusion that this can be separated in a own library or framework. It should bring a generalized form to tag upgrades to a object model. Any object persistence implementation can use this framework to generate upgrades and downgrades of persistent data.

To put it simply: The framework expresses an upgrade and an downgrade path for a type:
Upgrade("TypeName") -> List
Downgrade("TypeName") -> List

The backup and sync provider can use this list to perform the real actions on the datastore. Of course this methods don't exist because real world upgrade scenarios are more complex. I will publish the real API on first release.

The project will be hosted on CodePlex, open source, GPL licensed and is currently in setup state. Hiere is the Project: MSE-iT Upgrade Framework on CodePlex But for the moment, it is not published. I will post when i check in the initial development.

DDoS Attac on InterNetX nameservers

Our servers and this blog was down because of a DDoS attack on our domain registrar InterNetX. That shows again the vulnerability of Internet based applications and the importance of a distributed infrastructure. I'll continue working on synchronization issues, to protect systems against this.

jQuery backgroundCanvas update

Do you know jQuery? It is a famous JavaScript framework for DOM manipulation.

It was turn of the year 2007/2008, i asked about design possibilities for our new homepage. Because our new software has a "Office 2007 Ribbon", i want to have round corners, gradients and a fluent layout. Staff members told me, this is impossible because you cannot have scaling background gradient images in CSS. And if we will clutter the html markup with layouting DIVs, we will be ranked badly by Google. I thought: Search engine optimization is important, this argument strikes. But i don't want to give up, therefore i looked in my spare time after other solutions. I was new in web developement and everybody told me, that i wouldn't find something. Ok, the web pros know what they talk about? Or not:

I found an interesting HTML element, which brings the solution: The canvas. Inserting a canvas object in the web page and drawing on it gives all the possibilities i want to have. I've developed a jQuery plugin for this in my spare time and it is great. Look at my private homepage:
jQuery backgroundCanvas plugin

I've just moved the source to Goggle Code and updated it. It is licensed under LGPL and easy to use.

Upgrade path declaration for automatic upgrade and downgrade of persistent data

In persistence frameworks peer synchronization and restore of backups requires knowledge about older versions. To follow the SCSE pattern from x-tensive upgrade paths should be declared in the object model.

If the domain-model changes, it is in general necessary to upgrade the clients too. In windows installer terms, these kind of upgrades are called "major upgrade", because the change of the model will almost always break the interoperability to related assemblies.

Important: The term "major upgrade" has nothing to do with the version numbering schema of assemblies which has a major version and minor version. But a major upgrade should have at least a minor version number upgrade.

An upgrade path is a linear path along major upgrades (releases):

V1.1.x -> V1.2.x -> V2.0.x -> V2.1.x -> V2.2.x

There are at least six primitive upgrade operations:

• persistent class added
• persistent class renamed
• persistent class removed
• persistent field added
• persistent field renamed
• persistent field removed

To declare this, three attributes are announced:

• RelAdded for adding a class or a field
  
• RelRenamed for renaming a class or a field
• RelDeleted for deleting a class or a field

This is an example of a class with release attributes:

[assembly: RelGroup("MseIt.Persistency.UnitTests")]
[assembly: RelDeleted("MseIt.Persistency.UnusedClass", Release.B_0_2)]

namespace MseIt.Persistency
{
    static public class Release
    {
        public const int B_0_1 = 1;
        public const int B_0_2 = 2;
    }

   [Sync]
   [RelAdded(Release.B_0_1)]
   [RelRenamed("MseIt.Persistency.UpdatedClass",Release.B_0_2)]
   [RelDeleted("Payload",Release.B_0_2)]
   class UpdatedEntity : SyncEntity
   {
       [Field]
       [RelAdded(Release.B_0_1)]
       public string Payload1 { get; set;  }

       [Field]
       [RelAdded(Release.B_0_2)]
       public string Payload2 { get; set; }
   }
}

A upgrade path is extractable from this declarations. So a automatic upgrade can be performed on restoring old data or synchronizing with peers having previous versions.

Additionaly it is a nice documentation which modifications are done on a specific release. External tools could check this update information for consistency.

The need for a unique Syncronsation, Update and Backup in Persistence Frameworks

Long time ago I've head the idea of combining the synchronization issues, backups and the update process together. Our application for travel agency accounting is based on a SQL Server and installed in hundreds of travel agencies in Germany. About one third uses SQL replications to synchronize data with their branch offices or for telework.

Now we need solutions for the following problems:

• The sql servers connected via replications must be updated together
• Software upgrades requires downtime of the application
  
• Restoring an old backup of the data requires an upgrade of the SQL database
  
• Crash of the consolidated database requires a complete regeneration of the replication from one of the remote databases
• Crash of the remote database requires generating a new database from the consolidated database
• And so on...

With the new system we want:

• Upgrade our product with no downtime
• Upgrade branch offices and teleworkers step by step with no downtime
• Restore backups of old versions with automatic upgrade
• Restore a backup in a branch office and synchronize it to the head office
• synchronize between branch offices if the head office becomes unavailable (peer sync)

This could only be reached, if we put the upgrade and synchronization layer in the business logic. This is the goal for the synchronization, backup and upgrade framework on top of the famous DataObject.NET 4.0 from x-tensive.

Switch to the persistency framework Data Objects.NET 4.0

Long time ago I've decided to use DataObjects.NET as ORM framework. The content management of our company homepage www.mse-it.de relies soley on it. It also uses some JQuery plugins for layouting. You can find them on my private homepage under JQuery Projects.

A view months ago they announced the new 4.0 version. It is licenced under GPL and comes along with some propertiary licence models. It is a awesome framework, with goundbreaking changes to any ORM i know.

But i miss some important things:

• Syncronsiation or replication like SQL - but on the level of business objects
• Update and upgrade support
• Backup and resorte support

So i've decided to use syncronsation and distribution knowledge from our software for travel agencies (comes with replication, backup update and so on) and build a framework for it that is able to:

• Syncronize
• Backup
• Deploy

In the moment i am working on syncronisation and update issues. Because x-tensive follows the "sorce code is source of everything" (SCSE) pattern, i will do too.

Here is a sample class form my NUnit tests:

[Sync]
class TestEntity : SyncEntity
{
   [Field]
   [RevAdd(Rev.B_0_1)]
   public string Payload { get; set;  }

   [Field]
   [RevAdd(Rev.B_0_1)]
   [RevRenamed("FieldNoSync",Rev.B_0_1)]
   [SyncVersionNoUpdate]
   public string FieldWithSyncVersionNoUpdate { get; set; }

   public string NoFieldProperty
   {
       get { return "";  }
       set {}
   }
}

The RevXXX (Revision) attributes will tell the framework when a property is introduced, deleted or the name changed. So the backup and synchronization layer can automatically do the necessary transformations between the versions.