[Windows Azure] 將 Table 的 Entity 結構由 ITableEntity 中解放吧

Windows Azure Platform 的 Table Storage 是一個結構化的資料儲存地,一般來說 (連我的書也是這麼寫),在使用 Table 之前,我們需要對 Table 中的資料列做型別宣告,也就是要建立一個 Table Entity 的類別,然後用 DataServiceContext.AddObject() (Storage Client 1.0) 或是 TableOperation.Insert (Storage Client 2.0) 來存取它,但這對於很多 NoSQL 的應用很難適應,因為 NoSQL 是 Free-Schema,但 Table 的 Entity 限制反而形成了 Schema,對 NoSQL 應用有相當的副作用...

Windows Azure Platform 的 Table Storage 是一個結構化的資料儲存地,一般來說 (連我的書也是這麼寫),在使用 Table 之前,我們需要對 Table 中的資料列做型別宣告,也就是要建立一個 Table Entity 的類別,然後用 DataServiceContext.AddObject() (Storage Client 1.0) 或是 TableOperation.Insert (Storage Client 2.0) 來存取它,但這對於很多 NoSQL 的應用很難適應,因為 NoSQL 是 Free-Schema,但 Table 的 Entity 限制反而形成了 Schema,對 NoSQL 應用有相當的副作用。

使用 Entity 做結構描述其實不是不好,只是一來喪失了 NoSQL 最重要的特性,會讓應用程式在寫起來倍感棘手,其實 ITableEntity (2.0) /
TableServiceEntity (1.0) 並不是要強制符合某些資料結構,它只是想要強制每一個存進去的資料列都具有 PartitionKey, RowKey 和 Timestamp 等必要的屬性而己,因此或許我們可以用一些更彈性的作法來取代 ITableEntity/TableServiceEntity 的功能,但一樣擁有這些基礎類別或介面的能力。

在尋尋覓覓之際,筆者發現了這篇文章,作者介紹了使用 Dynamic Type Factory 建造動態組件型別的方法,令筆者靈機一動,結合之前發的文章,我們是否可以直接將匿名物件 (anonymous type object) 透過這個手法來自動產生一個動態型別,以處理這個棘手的問題呢?事實證明是肯定的。

所以,我們將 Developenator 那篇文章的概念以及範例實作借過來 (主要部份是 CreateType() 的實作),結合匿名物件以及 Reflection/Reflection.Emit 的作法,實作出一個可容納匿名物件的工具類別如下所示:

private static ITableEntity CreateDynamicEntity(object AnonymousObject)
{
    if (AnonymousObject == null)
        return null;

    // check property map.
    PropertyInfo[] properties = AnonymousObject.GetType().GetProperties();
    IEnumerable<PropertyInfo> typeProperties = null;
    Type objectType = null;
    bool typeFound = false;

    foreach (KeyValuePair<string, Type> type in EntityTypeDictionary)
    {
        typeProperties = type.Value.GetProperties().Where(c => c.DeclaringType == type.Value);
        int propertyMatchedCount = 0;

        foreach (PropertyInfo property in properties)
        {
            if (property.DeclaringType == AnonymousObject.GetType())
            {
                var q = from p in typeProperties
                        where p.Name == property.Name && p.PropertyType == property.PropertyType
                        select p;

                if (q.Count() == 1)
                    propertyMatchedCount++;
            }
        }

        if (propertyMatchedCount == properties.Where(c => c.DeclaringType == AnonymousObject.GetType()).Count())
        {
            typeFound = true;
            objectType = type.Value;
        }

        if (typeFound)
            break;
    }

    if (!typeFound)
    {
        objectType = CreateType(AnonymousObject);
        typeProperties = objectType.GetProperties().Where(c => c.DeclaringType == objectType);
    }

    object o = Activator.CreateInstance(objectType);

    // apply value.
    foreach (PropertyInfo property in properties)
    {
        if (property.DeclaringType == AnonymousObject.GetType())
        {
            var q = from p in typeProperties
                    where p.Name == property.Name && p.PropertyType == property.PropertyType
                    select p;

            q.First().SetValue(o, property.GetValue(AnonymousObject, null));
        }
    }

    ITableEntity entity = o as ITableEntity;

    if (properties.Where(c => c.Name == "PartitionKey").Count() > 0)
        entity.PartitionKey = properties.Where(c => c.Name == "PartitionKey").First().GetValue(AnonymousObject, null).ToString();
    if (properties.Where(c => c.Name == "RowKey").Count() > 0)
        entity.RowKey = properties.Where(c => c.Name == "RowKey").First().GetValue(AnonymousObject, null).ToString();

    return (ITableEntity)o;
}

private static Type CreateType(object AnonymousObject)
{
    Random rnd = new Random();
    int typeRandomNumber = 0;

    do
    {
        typeRandomNumber = rnd.Next(1000000, 9999999);

    } while (EntityTypeDictionary.ContainsKey("T" + typeRandomNumber.ToString()));


    // create a dynamic assembly
    AssemblyName assemblyName = new AssemblyName();
    assemblyName.Name = "TA" + typeRandomNumber.ToString();
    AssemblyBuilder assemblyBuilder = Thread.GetDomain().DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run);

    // Create a module
    ModuleBuilder module = assemblyBuilder.DefineDynamicModule(string.Format("T{0}Mod", typeRandomNumber));

    // create a new type builder (with or without use of a base type)
    TypeBuilder typeBuilder = module.DefineType(
        string.Format("T{0}", typeRandomNumber),
        TypeAttributes.Public | TypeAttributes.Class, typeof(TableEntity));

    var properties = AnonymousObject.GetType().GetProperties().Where(
        c => c.DeclaringType == AnonymousObject.GetType());

    // Loop over the fields that will be used as the properties in our new type
    foreach (var f in properties)
    {
        Type propertyType = f.PropertyType;
        string propertyName = f.Name;

        // Generate the field that will be manipulated with the property's
        // get and set methods
        FieldBuilder field = typeBuilder.DefineField(
            "_" + propertyName, propertyType, FieldAttributes.Private);
        // Generate the public property
        PropertyBuilder property = typeBuilder.DefineProperty(
            propertyName, System.Reflection.PropertyAttributes.None,
            propertyType, new Type[] { propertyType });

        // Define the attributes for the getter and setter of the property
        MethodAttributes propertyAttributes =
            MethodAttributes.Public | MethodAttributes.HideBySig;

        // Declare the accessor (get) method for the field we made previously.
        MethodBuilder getMethod = typeBuilder.DefineMethod(
            "get_value", propertyAttributes, propertyType, Type.EmptyTypes);

        // Write a method in IL to read our field and return it
        ILGenerator accessorILGenerator = getMethod.GetILGenerator();
        accessorILGenerator.Emit(OpCodes.Ldarg_0);
        accessorILGenerator.Emit(OpCodes.Ldfld, field);
        accessorILGenerator.Emit(OpCodes.Ret);

        // Declare the mutator (set) method for the field we made previously.
        MethodBuilder setMethod = typeBuilder.DefineMethod(
            "set_value", propertyAttributes, null, new Type[] { propertyType });

        // Write a method in IL to update our field with the new value
        ILGenerator mutatorILGenerator = setMethod.GetILGenerator();
        mutatorILGenerator.Emit(OpCodes.Ldarg_0);
        mutatorILGenerator.Emit(OpCodes.Ldarg_1);
        mutatorILGenerator.Emit(OpCodes.Stfld, field);
        mutatorILGenerator.Emit(OpCodes.Ret);

        // Now we tie our fancy IL to the actual property in our assembly,
        // and the deed is done!
        property.SetGetMethod(getMethod);
        property.SetSetMethod(setMethod);
    }

    // Create and store the type for future use (while memory lasts)
    Type resultingType = typeBuilder.CreateType();

    EntityTypeDictionary.Add(string.Format("T{0}", typeRandomNumber), resultingType);

    // And finally, return our brand new custom type
    return resultingType;
}

這段程式碼可以允許我們傳入匿名型別,然後透過 Emit 的實作在執行期間建立一個全新的動態型別 (dynamic type),這個型別是繼承自 TableEntity (已實作 ITableEntity) 類別,可相容於 TableOperation 的要求,也就是說,我們可以用下列的程式碼來設定資料列:

string pk = Guid.NewGuid().ToString();

TableOperation op1 = TableOperationUtil.Insert(new
    {
        p1 = "1",
        p2 = "2",
        PartitionKey = pk,
        RowKey = "4"
    });
TableOperation op2 = TableOperationUtil.Insert(new
    {
        p1 = "1",
        p2 = "2",
        PartitionKey = pk,
        RowKey = "5"
    });
TableOperation op3 = TableOperationUtil.Insert(new
    {
        p5 = "1",
        p6 = "2",
        PartitionKey = pk,
        RowKey = "6"
    });

TableBatchOperation ops = new TableBatchOperation();
ops.Add(op1);
ops.Add(op2);
ops.Add(op3);

table.ExecuteBatch(ops);

如何?不用再宣告一個 TableEntity 的型別,就可以直接存取 Table,這樣才像是一個 NoSQL 的應用程式應有的樣子,而不是為了 Table 還要綁一堆強型別物件。

這個工具只有對 Insert, Update 和 Delete 使用,至於查詢的部份因為 Storage Client 2.0 本身有設計了一個 DynamicTableEntity,可支援 Free Schema 的 NoSQL 需求,詳情可參考:http://blogs.msdn.com/b/windowsazurestorage/archive/2012/11/06/windows-azure-storage-client-library-2-0-tables-deep-dive.aspx

Full Source Code: https://gist.github.com/regionbbs/25d609730d3d4ab3a1d8

PS1: 雖然本文中成功的讓匿名物件能順利使用,但不代表可以省略 PartitionKey/RowKey 和 Timestamp,至少 PartitionKey 和 RowKey 仍然要按規矩設定。
PS2: 本工具類別是針對 Storage Client 2.0 設計,不相容於 1.0。

Reference: http://developenator.blogspot.tw/2011/09/dynamic-type-factory-for-azure-table_27.html