SOA
Over on the Adding Simplicity blog I came across a very interesting presentation on "How eBay Scales" (along with a ton of other posts that I have queued up for later reading). While very few people have to worry about scaling to the level that the likes of Amazon and eBay do, its still very interesting to hear about the things they do to help them scale. The section I found most fascinating was "Scaling the Data Tier". It turns out that due to the high volume of SQL requests made against their database they do whatever they can to limit the amount of work done by the database. This includes partitioning the databases horizontally by "primary access path", as well as removing all business logic from the database. This means no stored procedures and only very simple triggers for populating default values. Additionally they move CPU intensive work to the application layer instead of the relying on the database. This means doing referential integrity checks, joins, and sorting all within the application layer rather than the database. One final interesting point is that they don't allow any client side database transactions. Oh I almost forgot, they also use an internally developed "pure Java" O/R Mapping solution.
Anyway, if you're interesting in hearing about how eBay scales their site checkout the presentation and subscribe to the Adding Simplicity blog.
Technorati tags:
ebay,
architecture
Tim Ewald’s latest post points out the major problem we all face when designing services…we can’t have everything we want!?!?! Tim presents the following three desires:
- A flexible system that can evolve
- A system that's easy to build without knowing anything about the XML layer that we're all relying on
- Code that is easy to write and hard to break
He goes on to say that we have to pick our preferred approach and understand the implications of that chosen approach. Unfortunetly I think Tim is right on the mark. Although it would be great if we could create a system based on services that gave us all three it just isn’t realistic. What do we do?
I think the answer depends largely on what type of system you’re developing and how you’re using services within that system. If you’re creating a set of services that’s sole purpose is to be consumed by others then you definitely want to make sure that you do everything in your power to ensure that those services can be versioned and stay usable at all times, no matter how many versions you deploy. In this type of scenario you want to always keep your messages clearly separated from any internal representations or structures.
What happens if you’re building a set of services whose primary purpose is to provide one of your own applications with the data and functionality it requires? Does the same level of separation need to be applied for services that serve very different purposes? What if the services that you’re providing are focused on CRUD like behaviour necessary for some of your other services? Is it realistic to think that your domain and contracts will be able to evolve completely independently? If I add a new field to my backend system which then ends up on my domain object won’t it also need to end up on my contract if you have a service that is used to create new records in your backend system? In this type of scenario would it be so terrible to have support within your architecture for your domain objects to be turned into contracts without requiring them to be two completely separate things?
While general guidance on how services should be developed is extremely important I think we often make the guidance too black and white. We need to look at the things that are important to us and understand what implications that has on how we develop our services. It would be wonderful if we could have it all but that simply isn’t realistic. Every design decision has a set of associated design tradeoffs that have to be understood.
Craig, Tim, and Aaron have been having a nice discussion about WSDL first services development.
Enjoy!
In my previous two posts I talked about developing services contract first and how it leads to having the messages sent and received by our service defined completely separate from our domain layer. In my “Designing services contract first” post I talked about having a translator class that has the responsibility of converting our contracts to our domain objects and our domain objects to our contracts. While that class is pretty simple I wonder if the translation logic could live inside our contracts like so:
namespace MyCompany.Contracts {
[DataContract]
public class CustomerContract {
public CustomerContract() {}
public CustomerContract(DomainLayer.Customer customer) {
this.Name = customer.Name;
}
[DataMember(Name=”Name”, Order=1, IsRequired=true)]
private string name;
public string Name {
get { return name;}
set { name = value;}
}
public DomainLayer.Customer ToDomain(Contracts.CustomerContract customerContract) {
DomainLayer.Customer customer = new DomainLayer.Customer();
customer.Name = customerContract.Name;
return customer;
}
}
}
So rather then having a separate translator class we have our contract class support being constructed via our associated domain class as well as being converted to our domain class via a ToDomain() method. The obvious downside is that the contract and domain objects are now coupled together. The advantage is that everything related to your contracts (including translation) is in one place rather then being spread out across 2 (or more classes). What do you like better, a translator class, or a slightly smarter contract? If the behaviour on our contracts is hidden behind our service boundary do we really need to have our contracts be strictly “data classes”?
We’ve recently been having a lot of discussions about how we develop our services. We’re using WCF (Indigo) to develop our services so we get to let it handle a lot of the plumbing for our services. We develop our services by creating an interface that defines our service contract and then create a concrete implementation of our service by creating a class that implements our service interface. All well and good. When we get into the implementation of our services we start to feel all the uber-SOA dudes of the world give us bad looks. We’re currently doing the following things that most SOA fanatics would beat us over the head for:
1) We’re not applying the rule that I wrote about way back when. Web services should have a single parameter. This means that instead of something like this:
public class CustomerService {
public Customer FindCustomer(int customerId) { /// }
}
we should be wrapping our customerId up in a DataContract so that our services can be versioned as it evolves like so
[DataContract]
public class FindCustomerRequest {
[DataMember(Name = “CustomerId”, Order=1, IsRequired=True)]
private int customerId;
public int CustomerId {
get { return customerId;}
}
}
public class CustomerService {
public Customer FindCustomer(FindCustomerRequest request) { /// }
}
2) We’re passing our domain objects back and forth in our services. So our domain objects are being decorated with [DataContact] and [DataMember] attributes. Since 99% of the time our services need to return a message that looks exactly like our domain objects we’ve been doing just that and returning them as is across our service boundry. In order to separate our services implementation from our domain layer we would ideally have a separate DataContract defining our messages that can be transformed into our domain objects. So instead of…
[DataContract]
public class Customer {
[DataMember(Name=”Name”, Order=1, IsRequired=true)]
private string name;
///yada yada yada…
public SaveObjectResponse Save() { //…}
}
We should have our domain object with no DataContract/ DataMember, a separate DataContract used only by our service, and a translator class that converts between the two….
namespace MyCompany.DomainLayer {
public class Customer {
private string name;
public string Name {
get { return name;}
set { name = value;}
}
}
}
namespace MyCompany.Contracts {
[DataContract]
public class CustomerContract {
[DataMember(Name=”Name”, Order=1, IsRequired=true)]
private string name;
public string Name {
get { return name;}
set { name = value;}
}
}
}
namespace MyCompany.ServiceTranslators {
public class CustomerTranslator {
public DomainLayer.Customer CustomerFromContract(Contracts.CustomerContract customerContract) {
DomainLayer.Customer customer = new DomainLayer.Customer();
customer.Name = customerContract.Name;
return customer;
}
public Contracts.Customer ContractFromCustomer(DomainLayer.Customer customer) {
Contracts.Customer customerContract = new Contracts.Customer();
customerContract.Name = customer.Name;
return customerContract;
}
}
}
While the sample code for the sample above isn’t too bad, imagine how repetitive and boring this could get 
None-the-less it does provide a clean separation between your service and domain layers which helps ensure that changes in our services don’t directly affect your domain layer as well as vice versa. With tools like the Web Services Software Factory the grunt work required for implementation your services using the above method can be reduced to a couple runs of their wizards. But as you know I don’t like wizards. Could there be a better way?
Jeffrey Palermo has a nice summary writeup of Ron Jacobs thoughts on SOA. On a related note I saw a number of interesting podcasts on Ron Jacobs podcast page relating to service orientation (SO).
-
| SO Technology Model Applied | Enough beating around the bush - now we get to the meat of how this thing gets implemented. | [mp3] |
| The SO Technology Model | Ron and Beat discuss how the Technology model aligns with the SO model | [mp3]
| Service Orientation Models | Ron and Beat chat about three models for architecting solutions with the principles of service orientation. | [mp3
]
As I’ve mentioned several times recently I’m thinking a lot about different architectural issues relating to the development of a set of services for “our” product. One of the things that I’ve been realizing is that we have very little real world guidance for how to build services. We talk a lot about the high level concepts such as sharing schema not type, following the four tenets of service orientation, and etc. but we talk very little about the real stuff that we need to think about when designing services.
A data driven application that is using a domain model is pretty well covered. We have things like Domain Driven Design that help us layer our application into entities, factories, and repositories. We have all the information we could ever want about DataSet’s. Yet we have very little about real world development of services. Perhaps it’s out there and I just haven’t come across it yet. Perhaps it’s talked about just not in the circles that I frequent.
As I learn and experiment with various real world services issues I’m going to try and record my findings, thoughts, and experiences here. Below are some things that I’ll likely be developing some thoughts around:
– How does O/R Mapping fit in with services?
– How does one leverage their domain model when developing services?
– How do we best provide customized views of our data to our services while leveraging an O/R Mapper behind the scenes?
– How does having to support multiple databases impact the way our domain is developed? (Ok, this one isn’t very service focused)
– And much much more….
TheZel asked for some material on Indigo. There’s probably a bunch more out there but below is the material that I’ve come across so far….
WinFX SDK: What is Indigo
Programming Indigo: The Programming Model
Programming Indigo: Contracts
Introduction to Indigo (video)
I started a new gig recently which I’m really enjoying. The job offers a lot of changes which I’m a fan of. These include the following:
1) Moving from a consulting shop to a software shop.
2) Moving from the web world to the windows world.
3) Moving from one place trying agile to another place trying to do agile (hrm, I guess that’s kinda the same).
4) Moving from a place that did minor “services” work to a team that is planning on making a lot of our application available via services.
#4 is the one that spurred me to pick up David Pallman’s “Programming Indigo” book. Over the past couple weeks I’ve been reading through the book learning about the various “things” that make up Indigo. This involves how do define contacts using [DataContract] and [DataMember] attributes, how to define our service contacts using [ServiceContact] and [OperationContact], as well as how to define the behavior of our services and methods using the [ServiceBehavior] and [OperationBehavior] attributes.
After reading Programming Indigo I have to say I’m impressed with the programming model. Things seem very easy to setup and configure in Indigo. I’ve started to work on a prototype app that will be using Indigo so I’ll be curious to see if things start to get more confusing or convoluted when I start doing “real services work”.
If your interested in services, and particularly Microsoft’s vision for services in the future I’d definitely recommend picking up Programming Indigo. The book doesn’t go into a lot of detail but it does provide all the information necessary for getting up to speed on the primary building blocks of Indigo. I would have liked more discussion on the various options for security and examples for when you’d use each, as well as case studies that dove deeper into the internals of Indigo, but, overall it was a book I’d read again.
I’ve recently been dropping in on various people in the industry to get their thoughts on the best way to build distributed systems (SOA if you will) using today’s technology. I threw in the wrinkle that performance was very high on the importance scale.
When we start talking about performance people usually begin to shy away from web services, as I did, however some recent investigation from Kirk Allen Evan’s shows that the performance difference between web services and Enterprise Services (or .NET Remoting) becomes somewhat irrelevant as soon as you start doing a decent amount of processing within your service. The milliseconds you save on transport pale in comparison to the amount of time that the actual logic takes to run within your service. Benjamin Mitchell followed up on Kirk Allen Evan’s post with a post of his own entitled “The fastest transport isn’t always the best choice.” In his post he states:
Worrying about the speed of the transport without thinking about the amount of work done in each call is a little bit like worrying how fast a car can rev if you have it in neutral.
As I was looking over the validation controls in Genghis I kept thinking that the ValidationSummary control that we have in ASP.NET was missing. Michael Weinhardt has a series of articles on MSDN from last year that overviews how to create a ValidationSummary for WinForms. I’m wondering why this hasn’t been incorporated into Genghis? Anyway the articles are here:
Extending Windows Forms with a Custom Validation Component
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnforms/html/winforms03162004.asp
Extending Windows Forms with a Custom Validation Component, Part II
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnforms/html/winforms04202004.asp
Extending Windows Forms with a Custom Validation Component, Part III
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnforms/html/winforms05182004.asp
I’ve been enjoying the content in “The Architecture Journal”. This month we have a couple more good topics which are related to some things I’m currently working on.
Roman Kiss has an interesting
article on WS-Transfer and Indigo over on Code Project. I’ve just started to take a close look at Indigo and so far I’m really liking what I’ve seen. Hopefully I’ll have more reasons to learn and post on Indigo in the coming months.
Since the URL is so easy to remember I thought I’d drop myself a link to the code samples for Programming Indigo. On top of providing me with a reminder to their location it provides everyone else with a link to some Indigo code that they can checkout!
http://www.microsoft.com/mspress/companion/0%2D7356%2D2151%2D9/
One of the items that is currently on my plate from an architectural standpoint is trying to decide how to architect our application around services yet still gain the advantages of an O/R Mapper to ensure we aren’t hand coding a ton of dynamic SQL (we need to support 3+ databases). One of the difficulties in this is that if we’re designing our services correctly we’ll be doing so by modelling them in XSD or possible with the help of Indigo attributes. It’s not likely that these messages will map directly to the “domain objects” that need to be persisted to the database. Or is it? The limited number of services and operations that we’ve “mapped out” thus far accept an object or a set of objects that match up pretty closely with our “domain”. I anticipate additional services coming along that don’t follow this mold but I think there is the potential that a decent number of our services will expose operations that accept/return messages that are composed of domain objects. This means we should be able to pull out a domain object and pass it along to our O/R Mapper for persistence untouched. If we use an O/R Mapper that supports POCO objects then we shouldn’t have a lot to worry about with this approach as our POCO objects should be relatively easy to include in our messages.
I’d be interested in hearing the opinions of those on the Indigo team relating to this. How do they anticipate the objects defining the contracts for their services fitting into the “persistence story” of applications?