Day 2 - 09:00 : A first look at Oslo, Dublin and WF 4.0

David Chappell presented this session and, after yesterdays two post-keynote and frankly less engaging presentations, explained the upcoming new features of oslo, dublin and workflow foundation 4.0 very clearly.

Applications can be distilled down to 3 major abstractions - they consist of workflow, services and models. The three new features supporting these abstractions are;

Dublin - extensions to windows server for hosting services.

Oslo - modelling technology to describe applications and other models.

WF 4.0 - technology for coordinating work within software.

What is workflow?
In simple terms, workflow is a bunch of activities executed as a process.

For what do we need workflow?
Scalable applications that must pause and start (long running applications) and applications that must co-ordinate parallel work can all benefit from using WF without having to build the pause/restart/persist/un-persist mechanisms etc from scratch..

What's new in WF 4.0?
WF 4 brings improvements to performance, new visual designers and more activities. It also introduces a new workflow type beyond the sequential and state machine workflows called flowchart. This new type is more powerful than the basic sequential workflow whilst is easier to work with than the even more powerful state machine. It provides the missing middle ground.

Earlier versions of WF have been difficult to work with as the facilities for hosting workflow isn't provided out of the box. Microsoft are addressing this issue in WF 4.0 through Dublin.

What does Dublin provide?

• A scalable host for WCF and WF services and applications
• Built in persistence for service state
• Management tools
• Auto start and heartbeat services - failed services restart etc.
• Message forwarding - content based routing
• Tracking and logging what's going on within your services
• Dublin will be "Free" (included in windows, but initially offered as a separate download)

What about Oslo?
Oslo completes the picture by offering the ability to model against different schemas. But what is a model?

A model is an abstract description of something - for example, a city map is an abstract description of the city. Models omit detail - the city map doesn't include terrain or information about all of the buildings, what colour they are and so on - it's an abstraction of the city for a given purpose - in this case navigation.

Many different things can be described with models - WF workflows, services, applications and business processes for example. Models are able to be purely descriptive and informative or may even be executable  eg: in the case of a WF workflow. Models can also be linked to other models.

Oslo itself is a general purpose modelling platform with 3 key components;

• A repository - storage for both schema (kinds of models) and instances of schemas (individual models)
• "M" - a modelling language used to define schemas. This divides into two other components MSchema and MGrammar discussed later.
• "Quadrant" - a visual modelling tool used against

The repository is where all the information about schemas and models are stored. Initially it's envisaged that Oslo will ship with schemas for processes, applications, workflow, activities, services and environments, but using "M", you will be able to define any schema you can imagine.

It's worth noting that the repository itself is just a simple SQL database - you can query it directly to interrogate models or schemas, and you don't even have to use "M" to define your schemas. If you can understand the database structure for a schema, you can create your own tools.

"M", the modelling language is used to define schemas. It splits into two areas: "MSchema", which is a C# type language used to define the structure of models and "MGrammar" which is used to define textual domain specific languages (DSLs).

MSchema defines structure of the model, the relationships between the structural elements and ultimately is used to generate T-SQL to define storage of model instances.

MGrammar is used to define the syntax for new DSLs and provides tools for creating parsers of these DSLs. Examples of DSLs include SQL, regular expressions and also MSchema - Interestingly, and perhaps obviously, MSchema is a DSL created with MGrammar.

The quadrant toolset is  a graphical tool that consumes schemas to provide a modelling surface with appropriate views. Schemas can be defined to have different viewers and control what tools appear in the designer for manipulation on these design surfaces.

Day 1 - 17:45 : When you have too much data, good enough is good enough (ARC303)

Presented by Pat Helland, this session was a high level talk that resided largely in the theoretical space and offered no real answers but rather problems were posed and left as exercises for the attendee to think about.

The session aimed to challenge how we think about data and how rigid and prescriptive we are about our interfaces to the data and the usages of that data.

This culminated in a look at how organisations often find themselves compromising data quality the larger that data gets. Amazon was used as a case study, where specifically their merchant API contracts aren't overly prescriptive about what data they expect in an effort to encourage merchant adoption over data quality. Instead, they have processes which attempt to reconcile data together, but ultimately they sacrifice data quality for the sake of simplicity for the merchants.

For example, take shoes - they have no unique code - there's no ISBN or similar unique identification systems, yet if you were able to buy shoes on Amazon, pair of shoes X from manufacturer Y, sold by merchant A would appear as the same product on the site sold by merchant B, with the same unique Amazon product code. Merchant A may have sent distinctly less data than Merchant B, yet the Amazon service is able to apply logic to work out that the products are the same thing and present them as such.

Merchant A and B both might send the colour and manufacturer name for instance, whilst merchant B might provide a host of additional information on top of this that can flesh out the product data. The idea being the colour and manufacturer name might be the prescriptive contract whilst the additional information from Merchant B is completely optional, not strictly defined and would be used to flesh out extra information on the product data (for both merchants) if it was available (from either). He even went so far as to suggest that contracts offer key/value pairs to allow any data to be passed optionally and used. (This gives me chills of the bad variety, I've got to say).

Of course this is where the quality issues appear - it's not always 100% possible to match the two together so sometimes the same product will be brought into the catalogue as separate items. For Amazon, this is deemed acceptable rather than forcing a regimented API that merchants must adhere to.

The ideas presented revolved around how classic RDBMS systems offer crisp answers over relatively small amounts of data, but new systems have huge amounts of data, high rates of change and large volumes of queries. As systems grow, data quality and it's meaning becomes more fuzzy - any schema, if it's even present, may vary across data and the origin of the data may be stale and we must be able to work with this data within given tolerances of staleness.

For example, if we have an ordering API that allows our customers to place orders for products with us and that API exposes also a list of prices that updates at midnight each evening. If someone submitted an order at 11:59pm to your services that was processed at 12:01am, is your system going to reject the order because the pricing is stale? No - it should allow either the stale or the current pricing to be used for a period of time before enforcing such rules.

This was discussed further as the concept of inside data and outside data. Inside data being the transactional systems we're all used to - you start a process, you freeze the database in time using a transaction and then commit when you're done. This is the historic model of databases, but today we have services to contend with that are outside of the transaction and so aren't within the same space/time as the database transaction. We have to deal with this in our systems in future.

To cut the remainder of the story short, he theorised that for many businesses, just like Amazon, they are happy with "good enough" if it gives benefit elsewhere.

Day 1 - 16:00 : The future of composite applications and SOA - everything they told you and why it isn't true (SOA202)

The presentation opened with a video about the history of models. You can view this light hearted video here: www.modelsremixed.com

Presented by Mark Berman and Steven Martin, this session covered (in quite simple terms) the future of SOA and some misnoma's about SOA in general. To begin with they addressed the following mistakes;

SOA is a product?
Service oriented architecture is not a what, it's a how. You don't go and buy Microsoft SOA or IBM SOA, you build applications using the principles of SOA. It's another way to build applications or even another way to re-purpose them. In fact most recent high ROI implementations of SOA haven't been green field developments, but instead have been a wrapper over existing software, exposed as SOA.

SOA aligns business to IT
No, this is what people do. No technology can align business to IT, only people can achieve this.

SOA governance fixes everything
This is wrong, SOA governance alone is not enough - governance needs to span all of IT, not just SOA.

SOA stops at the firewall
Notice in many organisations today how the administrator of the firewall holds immense power! To truly benefit from SOA we need to be thinking of SOA as starting at the firewall.

The presenters followed this up with a claim that essentially Microsoft pioneered SOA. This is claimed as Microsoft, along with other constituents, pioneered web services, a fundamental part of SOA itself.

One of the issues Microsoft are looking to address is how, it's perceived at least, only blue chip / fortune 500 / FTSE companies are able afford to implement SOA successfully. They wish to make it available for the smaller organisation too.

Dublin is the codename for a set of extensions to the windows server platform (IIS7 specifically) that are the next evolution of the WAS/IIS platform that will be used to host, run and manage windows communication foundation and windows workflow foundation applications.

Released after visual studio 2010, dublin will provide facilities to manage, throttle and inspect individual services within a deployed application. Combined with OSLO, the graphical modelling toolset and DSL environment, building, deploying and managing services is purported to become easier than it is at present.

Whilst this session was titled the future of composite applications and SOA, there wasn't enough depth to take away anything about the future of SOA beyond the rough idea of what dublin will be.

Day 1 : The Keynote

After introductory comments and reinforcement of the Azure message, Jason Zander, general manager of visual studio team developer division of Microsoft took to the stage to present the future of visual studio in the guise of visual studio 2010. It got off to a flying start as VS crashed within a few minutes of the demo - but lets face it this IS pre-alpha software, so is easily forgiven.

Amid many new enhancements in vs.net 2010, there were key improvements discussed in the following areas;

• Understanding the code
• Building web applications
• Creating office business applications
• Using the power of C++

There was quite a lot of depth to some of these changes, but to summarise some of what I felt were the more important ones;

A new "architecture explorer", is being introduced to help you visualise the structure and dependencies within your solution. It visually displays the relationships between assemblies within a project and provides drill down into the assemblies to see relationships between individual classes and namespaces and the connections between them are weighted according to the level of dependency.

In addition to visualising the dependencies between areas of your solutions, individual portions of code can be extrapolated into UML 2.1.1 sequence diagrams - which is a useful addition, and I'm hoping this eludes to visual studio 2010 ultimately supporting UML modelling out of the box.

Testing was a major focus for the next release, with 2010 introducing the testing activity centre, code-named Comono(?). This is a test environment for managing and running manual scripted tests. This is a major boon for the end-to-end experience and integrates into new debugging facilities in visual studio and TFS. Some of the highlights for this new facility;

• Testers are presented with scripted steps to follow - test steps are marked as passing or failing.
• During testing, the system is able to take video of what the tester is doing as part of their session and also record replay information about the state of the application that can be replayed in the debugger within visual studio.
• When failures occur, a bug can be entered directly into TFS, attaching any video or replay session.
• In TFS, viewing any raised bugs from this process shows a list of steps the tester took, and each step provides a hyperlink to the timecode within the video so you can see exactly what the tester was doing.
• Historical debugging of the testers snapshot allows debugging of the testers session.
• The aim and objective of this suite is to eliminate the "unable to reproduce" responses developers often run into and to bring manual testing into the managed process.

Alongside this new manual test studio, is the test lab management facilities. This allows virtual test environments (eg; different servers for various tiers) to be provisioned and used within the test. Again this is integrated back to TFS and visual studio, allowing virtual machines to be restored into the state they were at the point a bug was raised.

Within visual studio, the editor itself has had a complete WPF overhaul. As a result of this, the editor can take advantage of lots of WPF goodness like the ability to contain and display any form of information, from the source code we know and love, to diagrams, graphics and other glyphs that might help us to understand or navigate the code more efficiently. The editor uses the extensibility framework (MEF) allowing for multiple add-ins for the editor and add-ins on add-ins on add-ins. X-copy deployment of extensions will also be supported as a by-product.

The editor provides for intellisense support for jQuery and also offers a bunch of new refactoring capabilities - which I can't help but think treads on the toes of Resharper, which has got to annoy JetBrains somewhat.

A new configuration transformer toolset is provided, which allows you to define transformation rules that will be applied when an application (web in this case) is deployed to various environments. Eg: Ability to change connection strings when preparing for production deployment versus debug deployment etc. One touch publishing of websites is also a new facility, where your entire web application is built and zipped into a single file ready for deployment with msdeploy.

In the sharepoint space, demonstrations were given on the new sharepoint server explorer extensions and the WSP importer, which looks like it should speed up sharepoint development no end, especially tied to the new packaging explorer which allows you to package and deploy sharepoint applications easily.

Finally, Jason demonstrated several new features of C++ including running parallel for loops and such like for performance.

Datasets - what's the problem?

I was talking with a friend of mine last night, an experienced developer who's leading a team of guys building a large commercial enterprise application. We were just chewing the fat over data access and how we approach things differently - I'm a fan of the domain model and use OR/M for persistence - usually NHibernate, but my buddy was advocating the use of the Dataset.

Many of us are told that the Dataset is just evil incarnate, but that's not actually true - the Dataset does serve a purpose, just not one in an enterprise application with complex domain logic.

The issue with a dataset is that it's not representational of data, it merely contains it. Beyond the activity of querying data and performing straight forward operations, it requires additional components. Representing complex logic and interrelated business entities is harder than it needs to be.

A domain driven approach on the other hand, with entities representing data, provides more flexibility to describe your logic - for instance, a customer object will contain properties to represent itself, along with business logic for operations and rules that govern a customer object and it's relationships to other objects.

Does that mean that the DataSet is intrinsically evil? No!

Sure dataset's have their problems - amongst others, they enforce a database centric view of data and when they serialise they also serialise a description of the schema of the data resulting in a bloated payload (this can be disabled). Despite these problems however, for trivial systems, they can serve a purpose.

It's a matter of choice. Even for trivial applications I would advocate a domain approach, but that's my preference. For anything non-trivial however and for any enterprise system, I'd avoid the dataset.

Microsoft kill off LINQ to SQL

http://blogs.msdn.com/adonet/archive/2008/10/29/update-on-linq-to-sql-and-linq-to-entities-roadmap.aspx

Microsoft are dropping support for LINQ to SQL as of .NET 4.0. This is a kick in the face for anyone who has invested time in what is a reasonable, if very basic, ORM solution - get porting folks!

I'm just glad I chose to continue using NHibernate.

ASP.NET error: could not load file or assembly App_Web*

Reminder for me more than anything....

Useful information about this annoying error:
http://yetanotherdeveloper.com/post/2008/08/10/Could-not-load-file-or-assembly-App_Web.aspx

Summary of options:

• Turn off batch compilation in web.config (<compilation batch="false"..)
  
• Clear down the contents of Temporary ASP.NET Files (in windows\Microsoft.NET\Framework\vX.xx\)
  

Silverlight and Cross domain services

Silverlight 2 allows you to invoke a web service on a different domain to where your XAP came from. In order to do this however, the owner of the service needs to enable it to allow cross domain calls.

This entails creating an xml file in the root of the service web server called clientaccesspolicy.xml. A skeleton of this file, allowing any caller from any domain, is shown below;

<?xml version="1.0" encoding="utf-8" ?>
<access-policy>
  <cross-domain-access>
    <policy>
      <allow-from http-request-headers="*">
        <domain uri="*" />
      </allow-from>
      <grant-to>
        <resource include-subpaths="true" path="/" />
      </grant-to>
    </policy>
  </cross-domain-access>
</access-policy>

Exposing new properties for control templates using attached properties

There are many instances where you want to write a groovy looking control template to apply to your WPF controls only to find that the control you want to template falls short in the properties it provides - eg: Try specifying the over state gradient brush for a button using only it's exposed properties. In these situations you need some custom properties on the control to be able to tell your template what to do.

In this situation there are several options open to you - you could hijack a property - bad idea when it comes to maintainability. Alternatively you could inherit the control and introduce the new properties - again, not the most ideal situation under WPF's composition model.

By far the most flexible and easiest way to achieve this behaviour is with attached dependency properties.

In the following example, we want to extend all of our buttons to provide not just a single line of text as you'd find on normal buttons, but we also want to have customisable sub-title line. To achieve this we start with the custom dependency property;

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

public class ButtonPropertyExtender : DependencyObject {   public static readonly DependencyProperty SubTextProperty =     DependencyProperty.RegisterAttached("SubText",       typeof(String), typeof(ButtonPropertyExtender),       new FrameworkPropertyMetadata(null,         FrameworkPropertyMetadataOptions.AffectsMeasure |         FrameworkPropertyMetadataOptions.AffectsArrange));   public static void SetSubText(UIElement element, object o)   {     element.SetValue(SubTextProperty, o);   }   public static string GetSubText(UIElement element)   {     return (string)element.GetValue(SubTextProperty);   } }

Quite simply this registers a custom attached property with WPF and allows it to be attached to any element. The next thing we do is use the new property on our button thus;

1	<Button xc:ButtonPropertyExtender.SubText="Subtitle text"> Click Me! </Button>

And finally, to use our new property in the control template, we bind to it - notice however how we cannot use the {TemplateBinding} shortcut - instead we must use the full binding expression to get to the new custom property.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

<ControlTemplate TargetType="Button">   <Border Background="{TemplateBinding Background}"       BorderBrush="Black"       BorderThickness="2"       CornerRadius="10" >     <StackPanel HorizontalAlignment="Center">       <ContentPresenter Content="{TemplateBinding Content}"/>       <TextBlock         Text="{Binding RelativeSource={RelativeSource TemplatedParent},          Path=(xc:ButtonPropertyExtender.SubText)}"/>     </StackPanel>   </Border> </ControlTemplate>

Database Schema and Source Control - the tools

A couple of weeks ago I said I would release some example source code for managing database change and getting schemas into source control painlessly. Attached is the full source to the tool used in this approach. It's by no means production quality, I literally knocked this up in a couple of hours this evening, but it works and should provide a basis for improvement.

The tool allows you to follow the approach discussed in my earlier post and have your scripts applied to your database as needed. The tool will also generate database creation and update scripts instead of applying them to your database directly - which is handy for packaging releases.

You'll find 3 projects in the solution - an example database project to run the tools against, a command line implementation of the schema tool and the schema engine itself.

To use the command line tool, use the following arguments;

Deepcode.SchemaTool.Cmd.exe - followed by; (items in red are mandatory);

-h	Display usage information
-l [location]	The location of your database project
-m [dbscript]	Specify db to run in database update mode - scripts will be applied to the database specified. Specify script to generate scripts to create the entire database and to patch it.

When running in database update mode (-m db), use these additional arguments;

-s [server]	The SQL Server to target (default to (local))
-d [database]	The database to target
-id [databaseid]	The id of the database in the DatabaseVersions tracking table.
-c	If present, instructs the engine to create the database if it doesn't exist.
-cd	If present, instructs the engine to create the database. If it exists already, it will first be dropped.
-u [username]	The SQL username to login to the server as - if not present the system will use windows authentication as the current windows identity.
-p [password]	The password to connect to SQL server, when -u is being specified

When running in script generation mode (-m script), use these arguments;

-o [location]

The location to store the generated SQL scripts.

To use this as part of your development process, place a copy of the exe into your trunk/library directory (or whatever strategy you use for SCC of dependencies), and create two batch files - one to update the database and one to generate the scripts. Then, whenever you make a change, just run the update script. When you release a version, run the generation script.

I didn't have time to do it tonight, but it should be trivial to wrap the same functionality as the command line host into a custom MsBuild task and automate everything. If anyone does this, please post a link to your wrapper source.

And finally, again, remember this isn't production quality, I offer no warranty etc...use at your own risk.

Click here for the source code