December 2007 - Posts

• Adding a node to an existing cluster

  In this post, I will add another node to a existing cluster that was created in the post Creating the first node in a cluster.   In my example and post series I am creating an Active / Active SQL Server cluster.  This step is adding a second node to the cluster. To add an existing node to a cluster, log onto the cluster from either the new node or the existing node, you do not have to be logged onto the node directly.  Once logged on verify the following Windows Server 2003 Enterprise Public and Heartbeat networks active and can communicate with other servers Machine part of the domain. You have the cluster service account password Now add the node to the cluster. Click Start menu. Click Administrative Tools. Click and launch the Cluster Administrator. The cluster administrator should show Click the File menu, New menu, Node menu.   Alternatively, you can add the node by right clicking the name of the cluster and choose Add, Node. In the Welcome to Add Nodes Wizard, Click Next>. Type the second node computer name in the Computer name textbox.  i.e. SQL2.  Then Click the Add.. button.   The Analysing Configuration step analysis the new node to ensure that it can successfully be added to the cluster. Should any errors appear, fix them and do not continue. Click Next>>. Type in the Password for the cluster service account. Click Next>>. Review the settings for the new cluster, click Next>>.  The cluster node will be created and a summary page created, Review any problems and fix them afterwards. Click Next>>. Click Finish.   In the cluster administrator you should see both nodes. Posted Dec 23 2007, 12:56 PM by Brett with no comments Filed under: Cluster

• Creating the first node in a cluster

  Creating a cluster is a simple task when the hardware and operating system have been set up correctly.  For the purposes of this blog I will presume you have the following network configuration: DC   Operating System Windows Server 2003 R2 - Standard Role(s) Active Directory Exchange Server 2003 SP 2 WINS DNS IP 192.168.1.1 / 24 (Public) SQL1   Operating System Windows Server 2003 R2 - Enterprise Role(s) Cluster Node IP 192.168.1.1 / 24 (Public) SQL2   Operating System Windows Server 2003 R2 - Enterprise Role(s) Cluster Node IP 192.168.1.1 / 24 (Public) The cluster hardware In the test lab I have created, I have used VMware Workstation.  Although officially VMware does not support clusters, you can easily build a cluster.  I have posted a blog on how to Create a Clustered Virtual Machine.  The configuration for a simple cluster will require a shared SCSI drive for the quorum disk and then a second SCSI drive for the data.  In a future post I will use this drive for the actual SQL database files (i.e. mdf, ndf and ldf files).  In this scenario, I will build and Active\Passive cluster.  In many production environments, the quorum and resource drives are stored on a SAN. It is also important to note that the servers need two network cards (yeah, yeah you can share traffic) as this is the right way of doing things.  The first network is a heartbeat network which the clusters use to monitor each other.  The second network is the normal network the servers will accept client connections from. Once the physical nodes (SQL1, and SQL2) are clustered virtual machines will be exposed onto the network Preparation Steps Before starting the building of the cluster, you will need to do the following steps: Log onto the Domain Controller and create an account that the cluster service account will run under. Account: Service_Cluster1 Password: Pa$$w0rd User must change password at next logon: Unchecked User Cannot Change Password: Checked Password Never Expires: Checked Ensure the Quorum disk is formatted and assigned the same drive letter on both nodes, Q: (Quorum) Ensure the Resource disk is formatted and assigned the same drive letters on both nodes, R: (Resource) Check that the Nodes can ping the Domain Controller. Ensure that the Nodes are part of the domain. Ensure that the nodes can communicate with each other on the public network and from other public computes, use ping. Ensure that the nodes can communicate with each other on the heartbeat network, again use ping. Disable, WINS on the heartbeat network adapters. Creating the first node and the cluster The following steps will guide you through creating the cluster on the first node. Logon to the first node in the cluster with administrative rights, i.e. log onto SQL1. Click the Start Menu. Click Administrative Tools. Click Cluster Administrator. The cluster administrator will appear and the Open Connection to Cluster dialog box will display. Choose Create new cluster from the Action drop down list.     Note: If the Open Connection to Cluster dialog does not display, choose File menu, Open Connection... Click OK. The New Server Cluster Wizard will appear. Click Next >. The Cluster Name and Domain wizard step will appear. Select the Domain from the domain drop down. Type in the NetBIOS name for the cluster name, in my example type Cluster1. Click Next>>. The Select Computer step will appear. By default, the current machine will appear in the computer name.  Change this if you must. Click Next>>. The Analysing wizard step will show and will start analysing your server.  This may take a minute or two. Verify that there are no errors and that the Tasks complete bar shows in green.  The above dialog indicates a successful analysis.  Should you have any errors, you will need to fix them and restart the process.  DO NOT CONTINUE IF THERE ARE ERRORS ... FIX THEM! Click Next>>. The IP address wizard step will appear. Type the IP address that the cluster should be exposed onto the network as.  This will be used for administrative tools.  In my example, I used 192.168.1.10. Click Next>>. The cluster service account step will appear Type the user name of the service account the cluster will run under as created in the preparation steps.  Type Service_Cluster1 in the User name box. Type the password of the service account. Ensure the domain is correct Click Next>>. The Proposed Cluster Configuration step will appear Click the Quorum... button. In the Cluster Configuration Quorum dialog box, choose the correct drive Disk Q: as the quorum disk.  Yours may be different. NB.  Make sure you choose the correct disk! Click OK. Click Next>>. The cluster gets built and may take a few minutes. Ensure that there are no errors. Click Next>>. Click Finish. The first node in the cluster should be created. What next In my next post I will show you how to add additional nodes to a cluster.  Following that I will show you how to create resource groups and how to fail over a resource group or node. Posted Dec 18 2007, 10:49 PM by Brett with 2 comment(s) Filed under: Virtual PC, Cluster

• Create a Clustered Virtual Machine

  After a company has procured a Clustered Server Farm, it quickly gets provisioned into the production environment.  Many of us do not have the opportunity to sit and play with clustering and sharpen our skills ... enter the world of Virtual Machines.  By creating a virtual networkCreating a virtual machine environment helps you simulate, practice and build a clustered environment. Unfortunately you cannot build a clustered set of Virtual Machines on Microsoft Virtual PC, you will either need VM Ware or Microsoft Virtual Server.  This is fortunately in your MSDN or TechNet subscriptions.  Virtual Server runs within IIS and needs to be installed, note that there are some challenges to get it working on Vista, however there are many blogs on this. A basic cluster In order to create a cluster you will need the basic hardware configuration External NetworkThe cluster needs to have network adapters exposed to the server farm network and thus is the access point for your clients machines. Heartbeat NetworkThe cluster will also need another network adapter that allows the servers in the cluster to communicate with each other.  This heartbeat network should be dedicated for cluster communication only and should not be part of any application traffic. OS Hard Drive. Each node in the the cluster should have access to its own local hard drives for operating system and applications installation. Quorum DriveThe quorum disk should be a shared SCSI drive that all the nodes in the cluster share.  This is where Virtual Server is different from VPC as it allows you to create shared SCSI drives.  This drive does not have to be big and a 200Mb install is more than sufficient. Resource DriveIt is not recommended that you store your shared data on the same drive as the quorum, a resource disk is created for all shared data and is also a shared SCSI drive. Domain You will need to add your server nodes to a domain. The basic configuration looks a follows: Summary of Tasks You are going to need to set up this network for a successful cluster test Create a Heartbeat Network Create a Base Virtual Machine Add Hard drive for OS Add SCSI Adapter Add Quorum disk to SCSI adapter Add Resource disk to SCSI adapter Add Second network adapter bound to Heartbeat network Install O/S Install Utilities Create clone for Node1 of cluster Create clone for Node2 of cluster Create Cluster Install cluster software Software you will need Microsoft Virtual Server 2004 R2 Microsoft Windows Server 2003 R2 Enterprise (ISO) SysPrep New SID Any other software you find necessary for your virtual machines. Below is a detailed set of instructions of how to create virtual servers for a clustered solution.   Using Virtual Server Install and configure Virtual Server according to the products setup instructions Open the virtual server by navigating to http://localhost/VirtualServer/VSWebApp.exe?view=1 This will display the Virtual Servers Main page, shown below: Create a base virtual machine Open the Virtual Server Administration Web Site Under Virtual Machines, click Create. Set the Virtual Machine Name to Base-Win2003R2Ent Set Memory to 512 Mb Create Virtual hard disk that is big enough to hold your OS and software, I used a 16Gb IDE.  I would also suggest a dynamically expanding virtual hard disk as it will only allocate space when needed. Set Virtual network adapter to the Internal Network. Click Create   Install base operating system In order to work with clustering you will need to install the Enterprise Edition of Windows Server.  Before booting, Set your virtual server's CD to the Operating System .iso or physical CD Boot your Virtual Machine Install your Operating System Activate your Operating System Apply patches to your System Install utilities and other common software Run SysPrep   Create common resources The next step in the process is to create the common quorum and resource disks.  Once these have been created the two nodes in the cluster need to be created. Create a Heartbeat Network Open the Virtual Server Administration Web Site  Under Virtual Networks, click Create Set the Virtual network name to Heartbeat Network. Set the Network adapter on physical computer to None (Guests Only). Click OK Create Quorum Disk It is important to note that the shared SCSI drives must be Fixed Size and NOT Dynamically Expanding. In “BASE-W2003R2Ent” Configuration, under Virtual Disks, click Create, then Create Fixed Size Virtual Hard Drive.   Type in the path to the Virtual hard disk file name, C:\VPC\SharedDisks\Quorum.vhd. Set the size of the Quorum to 200Mb Click Create. Create Resource disk It is important to note that the shared SCSI drives must be Fixed Size and NOT Dynamically Expanding. In “BASE-W2003R2Ent” Configuration, under Virtual Disks, click Create, then Create Fixed Size Virtual Hard Drive.   Type in the path to the Virtual hard disk file name, C:\VPC\SharedDisks\Resource.vhd. Set the size of the Resource to 8Gb (or any size you feel that you may need for data). Click Create.  You may need to wait a few minutes for the drive to create.   Create a Clone Open the Virtual Server Administration Web Site Under Virtual Machines, click Create.   Set the Virtual Machine Name to Node1 (or any other machine name) Set Memory to 512 Mb Set the Virtual hard disk to Attach a virtual hard disk later (None). Set Virtual network adapter to the Internal Network. Click Create Add SCSII Adapter In “NodeX” Configuration, Click SCSI Adapters Click Add SCSI Adapter>>.   Select Share SCSI bus for clustering. Click Add SCSI Adapter>>. Select Share SCSI bus for clustering. Select 6 from the SCSI Adapter ID. Click OK. Create a differencing Disk for the operating System Open the Virtual Server Administration Web Site Under Virtual Disks, click Create, Choose Differencing Virtual Hard Disk. In the location drop down, choose the folder for the machine you have just created, i.e. Node1 Append and type in the name of the new vhd file to create.  Typically this is the name of the virtual machine.vhd In the known virtual hard disks, choose the base vhd, i.e C:\VPC\Base-Win2003R2\Base-Win2003R2.vhd Add operating system disk to node In “NodeX” Configuration, Click Hard Disks.    Set the attachment to Primary chanel (0). Set Known virtual hard disks to C:\VPC\Node1\Node1.vhd or the name of the differencing disk you have just created. Resume with next step, or click OK. Add Quorum disk to node In “NodeX” Configuration, Click Hard Disks.   Set the attachment to SCSI 0 ID 0. Set Known virtual hard disks to None. Fully qualified path to file to C:\VPC\SharedDisks\Quorum.vhd Resume with next Step, or Click Ok Add Resource disk to node In “NodeX” Configuration, Click Hard Disks.    Set the attachment to SCSI 0 ID 0. Set Known virtual hard disks to None. Fully qualified path to file to C:\VPC\SharedDisks\Resource.vhd Click Ok Add Heartbeat network adapter In “NodeX” Configuration, Click Network Adapters Click Add Network Adapter>>. Choose Heartbeat Network from the Connect to drop down list. Click OK. Configure Virtual Machine The final step is to boot your virtual machine and do some final configurations. Boot your VPC Rename your VPC.  If you did not use SYSPREP on the base image, I would suggest your run the SysInternals Tool NewSID to rename the machine and create a new SID.  This should require a reboot Configure the IP address for the internal network Configure the IP address for the heartbeat network Add the Machine to your test domain.  If you do not have a domain, create one. On the first node, format and allocate drive letters for the quorum and resource disks.  On the second node, you will only need to assign drive letters. Shutdown the machine. Create other virtual machines Duplicate the steps above to create the second node Create yourself a test client to test connectivity and failover (or simply use the DC) Test Boot all your virtual PC's Test that each machine can ping other machines on the public network Test that each cluster node can ping each on the heartbeat network Test that both nodes can can see the Quorum (Q) and Resource (R) disks. Test that both machines are on the domain. Shutdown the machines Save and backup your Machines. You are now ready to build the cluster, I will post this soon. Posted Dec 18 2007, 07:33 PM by Brett with 2 comment(s) Filed under: Virtual PC

• Instructor Led Training will never die

  As a certified trainer with some 21 years training experience, I may be a bit biased towards Instructor Led Training (ILT) versus Computer Based Training (CBT) and Online Distance Learning (ODL).  This is my take on ILT. I see many companies pushing and promoting CBT and Online Distance Learning over the Internet.  This has been going on for years and these companies have been competing against Classroom training.  Yes, they have made money but they cannot seem to go beyond a point and really compete.  Why is this? I will attempt to explain why it does not work by asking you to place yourself the following scenario.  A manager arranges a CBT or Distance Learning for you to complete, the topic is IT product related.   What are your initial feelings to this?  A typical initial reaction is "Cool, Nice!" and "About time!" and sadly this is where the enthusiasm stops.  When push comes to actually doing it the first 30 minutes are nice and then it starts getting a bit long winded.  Over the next module or two the initial enthusiasm may dwindle to,  a "Oh, I forgot my appointment at the dentist for root canal surgery without anaesthetic.  I have to go!".  And that is where the lessons typically stop. The reasons for CBT not working are Most people are expected to fit it into their schedule.  This is exceptionally hard where deadlines and business issues are looming.  To find a hour or two here and there is nearly impossible.  Ones motivation for doing the sessions is low, especially when you know you have other more important things to do. Asking people to do it in personal time does not work, simply because "My time is my families time and this is work" and "Work must be done in work time".  It benefits both parties at the end of the day. People find CBT sessions not personal and directed to them. One cannot ask questions there and then and get answers immediately. You cannot ask the simulation to "please explain that differently" or "I don't understand". Interruptions give stop-start learning. CBT cannot adapt to your learning style as it is presented in a single style. Instructor lead training works because. We are human and social by nature.  We like talking to other people even if highly introverted we like listening to other people and seeing there expressions.  By looking at an instructor, you can sense whether he/she is on the ball or out of their depth.  As humans, we have feelings, moods and emotions.  This totally affects the way we are receptive to information.  A classroom provides a learning environment, where emotionally one has bought into the concept "I am in class, I must learn" where as the emotional buy-in at ones desk is "I am at my desk, I must work (not learn)". In a classroom, a bit of peer pressure is wonderful as everyone has the idea to learn and normally want to learn. One of the trainers jobs is to keep motivation going and keeping the learners in a mode to learn.  It is hard to just get up and say, I have had enough in a training room whereas a CBT application can simply be closed. Training off premises even extends the emotional buy in to "I am on course (therefore I must learn)" In a classroom environment, you pick up experiences from other people.  Hear their questions and assess your own knowledge.  This also indicates whether one is on top of things or highlight that one is out of depth.  It thus gives a mechanism of benchmarking. Most people will answer one of the following; " I don't have the time.",  whereas time will be made in a dedicated training. When doing a CBT, it is easy to be distracted especially on hard topic.  In a classroom environment, the trainers job is to help get through these hard topics.  Motivation, persistence is key. When one is offsite, colleagues cannot interrupt with "urgent problems that need attending to".  CBT normally challenges one with stop-start learning. Trainers normally bring a bit of entertainment and character into the lesson and adjust according to the room.  In CBT sessions, cracks jokes are normally weak and once they start to irritate you, the pain of the sessions increase.  Trainers are skilled at adapting to the learners, adjusting there style to the groups needs.  I look at myself, some courses are by the book and very formal where others are full of jokes and laughs.  The end results are the same, depending on how people like receiving information. On an emotional level, it is easier to relate to a person standing in front of you than to a screen recording.  Again, we are social beings. In a classroom environment, the trainer not only talks to the room but also to you directly. Trainers also learn to read body language, as a trainer I can see when people understand things and the penny drops.  I can also see when there is confusion. As a trainer, in many cases I do not have to give a "Test" to determine whether a person understands a topic.  A few subtle but strategic questions will yield the same information. These subtle human emotions, play a big part to our motivation and willingness to learn.  Without them it is very hard to achieve effective learning. Conceptual topics are easier to present to groups as buy in concepts can be conveyed easier. From my training experience, no two courses are the same each course has its own pace with its own challenge areas.  Some topics are easily understood by one group yet a major stumbling block for another group.  As trainers we adjust.  Synergy CBT and distance learning is not a total waste of time, when used in the correct situations it can yield great results.  Short 10-15 minute sessions that deal with a single topic carry powerful messages. Topical subjects with "How To" information are very effective. Extra information to supplement instructor lead training works well. Short additions and value adds very useful Demonstrations and practical of hard concepts reinforce training. CBT distance learning used to reinforce instructor lead training delivers a highly effective solution. I am leaving the post here for now and am challenging you for comment, feedback and debate.  Please comment below: Posted Dec 18 2007, 06:32 PM by Brett with 1 comment(s) Filed under: Opinionated Posts

• Using the sites profiles, avatars, passwords and Private Messages

  I am so excited about how this community has grown in the past few months, pretty awesome stuff.  Thank you for supporting us and getting involved.  One of the cool features about community server is the ability for you to update your profile, load an avatar, send Private Messages (PM) and see contact detail.  Many members are and will create logons that are aliases, this is cool, but please could you put your name in your profile so we know who we are talking to (not mandatory but nice if you could). Updating your profile It is pretty easy!  First you need to register with the site and then log in.  After you logged in you will see the Edit Profile hyperlink in the top right of your screen. Click it an the profile will display,  there are many options you can choose and some privacy  settings.  PLEASE set your mail address as this will enable you to receive private messages.   Navigate though the tabs and once done, click the Save button.  (NOTE that changes may not immediately display on the web site due to page caching, it may take 10 minutes to refresh) Avatars If you update your avatar, it must be a maximum of 80 X 80 px.  If you don't have a mug shot of yourself and want something different, just hit as search engine and search Avatar.  There are many sites that have it,  take a look at http://www.allavatars.com/ as they have a nice collection. Viewing a members details In the forums, in blog posts and comments simply click any members name and their profile page will appear.  Test it to have a look at what it looks like. Sending a member a Private Message This is also easy,  simply view the profile of the person you wish to send a private message to.  In the right hand navigation bar click the "Send ????? a private message".  If anyone sends you a private mail, it will automatically be set sent to the email address in your profile.  Changing your password If you need to change your password, edit your profile and click on the Password and Statistics tab, click the Change Password button. Posted Dec 18 2007, 05:32 PM by Brett with no comments

• SharePoint and SQL Express, SQL Standard and SQL Embedded

  In a discussion in class this week, I had a chat with a few of students regarding the editions of SQL Server that are installed.  This prompted me to write this blog as this comes up quite often.  The information on the SharePoint site is limited and therefore one has to dig a bit for it.  Microsoft Windows SharePoint Services 3.0 can be installed onto a single server farm, when this is done the SQL Server Embedded Edition SSEE is installed in an instance name of MICROSOFT##SSEE.  This is not SQL Server Express and it does NOT have the 4Gb limit of SQL Server.  It however has remote connectivity limitations. WSS can also be installed into a Farm, and in this configuration any edition of SQL server can be used, including SQL Express.  Obviously if you choose SQL Express you are limited to 4Gb.  I do not see any reason why you would install SQL Express over SQL Server Embedded Edition with a standalone WSS install. Microsoft Office SharePoint Server is however a bit different,  if you install MOSS into a standalone server install, SQL Express is installed and you are limited to 4Gb.  MOSS can also be installed into a farm using any other versions of SQL Server (except SSEE).   SQL Enterprise, Standard, Workgroup SQL Express SQL Embedded Edition (SSEE) Limit None* 4Gb None*         Standalone       ·         WSS     Y ·         MOSS   Y           Farm       ·         WSS Y Y   ·         MOSS Y Y   * Limits of SQL and disk apply Note.  WSS can be installed on a standalone machine that uses SSEE and does not have any limitation to database size.  However MOSS in a standalone configuration is installed onto SQL Express with a limit. Posted Dec 08 2007, 06:06 PM by Brett with 1 comment(s) Filed under: SharePoint

• What is a Service-Oriented-Architecture (SOA)?

  A Service Orientated Architecture (SOA) is a software design pattern for creating are reusing common business process.  A business process that performs a common task, thus providing functionality is packaged as a service.  Line-of-Business (LOB) applications can at any time make use of the service thus not having to duplicate the code.   Common business tasks that are system related are broken down or decomposed into small and distinct units of work, called a service.  The service is then published over the network and can be accessed by other line-of-business applications.  As an organisation matures into a service oriented architecture, the reusability of services becomes more and more prevalent and integrated. Business problem solved by SOA When looking at different operational processes you will typically notice many business processes involve different people with different roles.  The process may span several systems as well requiring interaction with these systems and by different users.  Take for example a simple placement of an order from a customer.  The customer needs to be validated whether they exist and the account is still active.  The customers credit needs to be checked.  The stock on hand needs to be verified and an order placed.  Once the order has been placed, the items need to be picked from the warehouse and distributed to the client.  Sales balances, general ledger values and stock values need to be adjusted.  This example typically involves many systems. With modern business, supply chain management is becoming a greater importance where system-to-system communication is no longer internal but extends to customers, vendors and trading partners.  Electronic data interchange is becoming more and more common. Over time many initiatives have started to facilitate communicate, an example is Electronic Data Interchange (EDI) which is a well structured rigid mechanism.  Although it is powerful, it does not scale with the high demands of today's systems.  Another common problem is the reuse of components, a large enterprise may have a lengthy process to create a new customer with many sub processes and validations.  Due to the size of the organisation and the many services and products it delivers this process might be duplicated in many different systems thus duplicating code.  For example, a customer may be suspended due to non payment of service, legal issues, electively not using company any more or other reasons.  By exposing a "Suspend Customer" service any applications such as the Credit system, the legal system or CRM system will be able to call the Suspend customer service. Synchronisation of data can also be solved in a service oriented architecture as each system may expose a create customer interface.  At a high level a single service is created to "create new customer" and after a customer has been created, it calls other systems services to "Add new customer" or even "Change customer details". Another challenge for system architects is to loosely couple applications, thus should a single application change or be replaced it does not require changes in other applications.  Many designers find it hard not to decouple applications and thus become reliant on other systems and should changes be made, all applications need to undergo modification. What is SOA? Service orientated architecture looks at common business processes that are repetitively used or accessed from multiple sources and exposes this system functionality as a service.  For example you could create a "Place order" service that allows multiple systems to call it an start the new order process.  When the service is executed, an order document or information is passed to the service and the process starts.  This "place order" service could be started from a Internet facing portal, a Business-to-business interface, a Order Application, a CRM system or even a sales system; no matter where the order came from it is reusable.  The complex logic of the service is embedded into the service and should changes be made, it only needs to be made in one place. The service is thus not dependant on any other system and can be called from anywhere and at any time (obviously in a security context).  When working with bigger business processes, it is very common that several services are called and thus stringed together to form a chain of activities.  By defining a service as a small, single unit of work will allow this to happen. The architecture is not tied to a specific technology or communication framework, but a variety of technologies can be used.  These include SOAP, RPC, DCOM, CORBA, Remoting, Web Services and WCF.  The key goal is that the service can accept requests and possible produce responses, how it gets the information and in what format the information can be negotiated.  The common trend is to use web services with XML data structures, however it can be done through csv files in a file location.  Ideally the service should be available 24/7/365 but not mandatory and could even move to a batch solution. SOA therefor is considered a programming style of providing a function that can be called by any client at any time.  It is not built into a single application as a class library or DLL but presented as a single, standalone service.  A machine can obviously host many services. Business requirements for SOA To implement a SOA in your organization, you will need to meet the following requirements. Systems need to be viewed differently whereby business functionality is exposed as a central service.  Front-end and other applications may then call these services to perform tasks. You need to want to build a federation of resource tasks, that can be reused by many different systems. Interoperability between systems needs to a priority in your organisation Loose coupling of applications a core driver The bottom line to SOA, is a mindset change to how your applications work. Exposing applications for SOA An application needs to be able to publicly expose functionality that other systems can interact with and use, the functionality could be from querying and getting data to inserting, updating and deleting several processes.  In my illustration below we have a "Customer system" that stores information about our customer and this system should be the central repository for all systems. As a SOA designer, one may choose functionality that can be used by other applications and a set of methods exposed; let us say for example we choose to expose them as a web service.  In the design it is decided that other applications can Create new customers Suspend customers Query customer information Update credit balances based on purchases and payments A web service can be created with methods to expose the functionality.  Note that the technology used to expose the information can be anything and in a complex environment could by many different technologies used together.   Once the functionality has been exposed, client applications can call these methods within their processes.  For example, the diagram below indicates an order capturing system.  Once a user captures the order and submits it for processing a process starts, this process contains the logic to call and consume multiple web services, it can for example check if the client exists and not suspended as well as whether the client has sufficient credit.  The application runs through the process of call and thus consuming many web services. Although this illustration is simple, the same web service can be consumed by many applications and reused.  Overtime the service may still expose the same functionality but actually update a different customer system. Posted Dec 03 2007, 03:18 PM by Brett with no comments Filed under: Architecture

• Service Oriented Architecture and Business Process

  In this post, I will discuss the relevance of business processes in the light of a service orientated architecture.  The importance of understanding processes and how they affect the organisation and our daily lives will impact the success of the solution.  A while back I posted an article on Splitting business and development analysis and discussed the reasons for project failure; one of the many reasons is the lack of business process. Process in business If you have attended one of my sessions, I am sure you have heard me say "Business, life and everything is a process with a clear objectives" Take a while to think about this statement as it is most probably one of the most concepts as a programmer to understand.  For the purpose of this post I will discuss business only and not personal and life objectives. The primary objective of all commercial business ventures is to make profit for the shareholders, a simple but harsh fact. How do you do that?  The answer is to follow a process of finding customers, deliver goods or service and get paid.  Although simply defined, the actual implementation of the process is somewhat harder. Digging into this simple objective and processes you should observer that as people we work in generalisations and abstract ideas and concepts; our minds are capable of comprehending this based on our life experiences and education. As the number of people collaborating and working together to achieve this objective increases, so does the number of interpretations resulting in confusion, frustration.  The process then becomes obscured and often the objective lost thus resulting in failure and as research shows 84% of software products will fall into this category. Defining processes is not an easy task as there are typically many individuals or party involved, each with a set of required objectives that often conflict with the primary objective.  In organisations, there are many legislative and compliance requirements that processes need to adopt and these often oppose the the primary objective of the process.  Immature individuals will tend to "selectively ignore" the requirements and move forward as their objective is different.  The processes normally become hard to define as there are so many permutations, hidden objectives and human emotions and egos involved that  A process without and objective is pointless; what is the point of doing something to get no result.  Many processes in organisations have been put into place to sort either enhance communication, sort out some problem, intervene in some political agenda.  These processes normally patch a symptom and not the underlying cause of the problem, thus resulting in frustration. Strategic Processes Executive management steer the business with vision and insight, their objective is not to work on the finite details of the process but rely on middle-management to define these processes.  Their mandate is clear and business direction and conceptual high level processes defined.  Their task is to review processes in line with business strategy, determining whether it is viable or not.  When strategic processes are defined they typically define the rough and thus the macro processes.  After some quick analysis, it becomes clear that these processes are flawed with many holes and many details missing.  As these processes are refined with logistical, tactical and governance activities the clearer the vision and strategy.  Any issues raised during the detailed definition of the process need to be checked against the strategy and any conflicts with strategy should be addressed with executive management. Legislative processes Every country has company, legal, taxation, employment acts and others to adhere to.  Processes need to be adapted to these rules and regulations to avoid breaking laws.  The greater majority of individuals respect the need for laws, however often miss the details of the laws and unknowingly break them.  Processes need to be verified that all legislative requirements are met. Many industries are also governed by industry regulators that impose certain processes and requirements that too need to be adopted.  Again many individuals fail to full understand the implications of their decisions with respect to the laws. Governance processes Often communication within the organisation is not up to scratch and corporate governance comes into play.  In many organisations the coroprate governace also includes legislative and industry regulations.  However in this definition, there is a difference between laws and internal needs.  Auditing, security, safety and other controls are in place to prevent against disaster.  These governance processes are normally sweeping and affect all business areas and detailed processes need to adhere to them. Operational processes Each and every department has a set mandate to deliver service, supporting service or product to the organisation and typically have many processes defined.  These processes are normally complex with many different scenarios and different outcomes based on different factors.  When drilling down deeper into the process, one quickly realizes that processes normally involve many different people in different roles and across different business units.  The process typically contain manual and system tasks. User interface processes When a system is developed, there is a navigation structure a user must learn in order to work with the system.  The system also moves from one screen to another based on clicks and menu navigation.  This navigation forms part of a user process. System processes When a user initiates a task within an application, the application follows a sequence of tasks,  e.g. Connect to database, validate data, insert record, insert audit trail and display success notification.  These tasks are system processes and activities that occur.  The user interface processes typically launch the system processes based on user input or events.  Also, the system process typically form part of a larger operational process which combines manual work by staff as well as data capture. System-to-System In larger organisations systems are distributed by functionality and often rely on common information, such as customer.  Systems need to communicate with other system in order to keep up to date as well as to automate some processes.  This communication reduces manual error and enhances communication. Importance of defining processes As one defines the process and drills deeper and deeper into the process, more and more steps are indicated, this process decomposition is important to go through as this defines what the business is doing.  Without fully understanding the process and what the business does, where information needs to go, who owns the information and what are expected timings it is impossible to to define a system. Should you not understand your business processes, you will not be able to successfully write any application that  benefits the business. Why processes are important to SOA In a service orientated architecture, we need to understand the business and what the business is doing.  In defining all the business processes, you will start noticing some common tasks that are performed and especially system tasks.  When you start noticing a common task being performed by multiple different systems, this system process could be an ideal candidate for a service. In a SOA, a service provides an interface to perform a common task, i.e. it exposes a service for other applications to use.   Ideally this service should be used by multiple different applications. Example. A business my have many different applications that create a customer, you could easily create a customer service that not only creates customers but can perform other tasks to customers as well.  Any application that needs to create a customer simply uses the create customer service.  The logic within the create customer service thus does not need to be duplicated in all applications.  However the application must be aware of the service and how to call it. Understanding the business processes will help you determine what can be  automated, what is duplicate and what can be made into a service.  In my next post, I will discuss SOA and what it does. Posted Dec 03 2007, 11:48 AM by Brett with no comments Filed under: Architecture, Analysis

• Myths about Service-Orientated-Architecture

  "Service-Oriented-Architecture" is one of those terms where most folk have and idea to what the objective is, however completely miss the point to what it is.  Many vendors, marketers, IT pros and managers have latched onto term without fully understanding what SOA is all about.  When learning about this architecture, it becomes confusing as there are many different interpretations.  In this post, I will discuss the myths about SOA and will follow up with a post about what SOA actually is. Myths about SOA SOA is an abstract concept Many people get the fact that SOA is used to represent distributed processing, however they lack to define the actual implementation of the solution.  The term is marketed and well understood, but the implementation and solution is ambiguous;  this results in a lot of developer frustration as the developer does not know where to start, how to implement it or even what to do. SOA is compromised of Web services A big marketing drive by many vendors was that Web Service functionality in their products, claming it to be the "Enterprise Application Integration" solution to all problems, thus by using the product SOA is enabled.  SOA was promised asking companies to invest in web services and should they do this their problems will miraculously disappear.  Many vendors wrongly tried to re-brand their products to be SOA enabled. A web service application is Service orientated. You may as well say things like "if it has an engine, it is a car".  There is a big difference between distributed \ central processing and a service-orientated application. SOA is distributed computing Whilst functionality and tasks are accessed on remote machines using different connectivity mechanisms. It does not necessarily mean that the product delivers a universal service that can be consumed by any other application.  Many solutions created distributed processes that were not enterprise wide services. SOA simplifies distributed computing Applying SOA in a real world is not a simple task, it takes a great deal of business understanding with the right choice of enabling technologies.  SOA is a principle with a lot of work, when matured in an organization it is greatly beneficial. WS-* enabled applications is service orientated Because you have to connect to it remotely, does not mean it is a SOA. SOA is easy This attitude typically will yield a 100% chance failure in a product.  Although the concept is easy, there are many factors that need to be considered to make the solution work within the enterprise.  The business processes need to be fully understood and the information \ data flow clearly defined.  The choice of enabling software is important  as is the implementation of this software.  A few "simple" misunderstandings will cause failure.  SOA needs to be carefully planned and thought through and is not recommended to be done by a new programmer with some battle wounds. SOA will make everything interoperable The ultimate goal is to move everything into a service orientated environment, however existing applications that have not been written to expose services will never change.  Investment, time and change will be needed on these systems to adopt them to deliver services, by no means can it change without writing code. SOA makes no impositions A service-orientated-architecture will require infrastructure as well as changes to applications that require SOA can easily be implemented An organization with weak business processes will find it a challenge to implement SOA.  Information, ownership of information and distribution of information needs to be clearly understood. What to expect The real world of SOA needs to be understood and in the next couple of posts I will take you through SOA.  I will touch on distributed processing and Service orientation and will discuss how you could use this within your organisation as well as what you need to do to enable it. SOA is going to require you to make some serious mindset changes, your organisations business logic will need to be reviewed and in many cases bedded down.  This typically will require you and your team to resolve processes from the  top-down and implement changes.  This most probably will be a slow and lengthy process.  The good news it does not have to be a big-bang and can slowly be introduced through the organisation.  It is going to take work and you will need some guidance. Posted Dec 02 2007, 10:36 AM by Brett with no comments Filed under: Architecture

• BizTalk Server Users Group

  On 5th December, the first BizTalk users group will be kicking off at Microsoft SA in Bryanston.  The reason for this is to have talks and discuss topics that will help you implement BizTalk more effectively.  Sharing experiences and knowledge is an ideal way to grow the expertise of all people involved. I will be at the BizTalk launch and will be talking about Service-Orientated-Architecture and hope to see you there.   To register for the event click here. If you do miss the event, I will be blog about SOA and put the slides up for download on the day. Posted Dec 02 2007, 09:45 AM by Brett with no comments Filed under: BizTalk, News