November 2007 - Posts
-
Understanding the structure of SharePoint is vital to any successful installation and future administration to the product. The future success, scalability and reliability of the product depends on your initial understandings of how the product is placed together. As a developer, it is important for you to understand how the actual structure of SharePoint web site as this will change your development approach, thus leading to a successful project. SharePoint (both WSS and MOSS) is naturally a hierarchical structure, this structure starts with the servers in a farm and moves through to a web site and a page within the site. The diagram below basically depicts the basic structure of a SharePoint site. 
Farm The first step is to install either WSS or MOSS onto your server, this simply creates the 12 hive with all the necessary components to run SharePoint. The installation does not actually create any web site. Once installed, you need to either create a new web farm for your server or join an existing web farm. In this process of creating a new farm, a new configuration database is created with the administration central web site; this allows the SharePoint administrator to actually create web sites. 
All servers in a Farm share the same configuration database. Web Application A web application are web sites that run on Internet Information Server (IIS). These web applications are created and managed by SharePoint and are extended onto every server in the farm. When installing a new web application, SharePoint will automatically create a root site collection for the site. Managed Paths A managed path is simply a dynamically configured URL to a site collection, e.g. http://portal.internal/ refers the root site collection and http://portal.internal/sites/sales refers to the sales site collection. These managed paths are dynamically managed by SharePoint and configured when creating the site collection. A web site collection may also have child sites which are also managed as child sites under the site collections managed path. For example, a document library for sales proposals is created under the sales site collection may have a URL of http://portal.internal/sites/sales/proposals. By default there are two managed paths in every web application created, the first points to the Root web site collection Site Collection A site collection is just that, a collection of different web sites that all fall under the same administrative context. After a web application has been provisioned, a root site collection is created. Additional sites can be created under the site collection and these site have the same security context. Additional site collections can be provisioned within a web application for different security contexts and rights, e.g. you may wish to create a site collection for every department within your company where each site collection needs to be managed differently and by different people. The users within the site collection are determined by the web application and thus all sites and site collections have the same users. Administrative permissions are given to different users at a site collection level thus allowing the business unit to manage their own site. When provisioning a site collection, a content database needs to be chosen. A site collection can have its own content database or can be added to an existing content database used by another site collection. There are many reasons for creating different site collections; namely back and restore sites, bringing data closer to users in physically distributed locations Site A site is a collection of of lists, libraries and web parts as well as the web pages that are used to manage them. After a use has provisioned a site collection a top level site (often called root) is created. The top level site is the same as any other site in SharePoint, however when editing the site there are additional options to manage the site collection, galleries. Sites can be provisioned under other sites thus building a hierarchy of sites with different content, careful planning should be done before building up your site hierarchy. The terminology of a site is often confusing and is sometimes referred to as a web or workspace. There are some confusing uses within the development API which developers should get used to; A SPSite is a Site Collection and a SPWeb is a site (legacy). Normally a child site is created for a meeting, short term project that a small set of users need to use. These sites are often of a collaborative nature and these sites are normally referred to a workspaces. Workspaces are often seen from office products such as outlook and word, these typically point to a site. A user can provision a site anywhere within a site collection where he/she has rights. A site typically provides a web page with web parts to provide some business functionality. When the user provisions a site, he/she chooses the type of site to provision and there are many different templates to choose from. There are additional installed with Microsoft Office SharePoint Services; additionally you can install others features and templates. Portal A portal is a set of SharePoint Site collection are spread across an organisation through geographic locations and functional areas. The portal is viewed as a single access point to content within the organisation
|
-
Since launching grounding.co.za, I have used the Free Edition of Community Server by Telligent. Their payoff line is "The world's leading platform for creating vibrant online communities" is 100% spot on, let me tell you why. The product runs on top of ASP.NET and .NET 2.0 using SQL Server or Express as the database. To me this was great as I have a good knowledge on the platform, thus making it easy to support. Hosting the solution was easy as there are hosting providers that allow you to host ASP.NET + SQL sites and it is a simple copy of files, an edit to a web.config. The initial installation was really simple as the setup application is very good. Creating blogs, forums and community members could not be easier. The day-to-day administration of the product is also easy, including site, membership, moderation and reports. I would also like to extend my thanks to Telligent, as they have really help us get things up and running and there sponsorship is really noticed and appreciated. The truly build communities in more ways than just product. What drew me to the product was the number of blog sites, communities out on the web both private and commercial that use the product. My thinking was to rather use than build, and to find a product that is used. Looking at their client profile of who actually use the product, I thought it was worth trying out. I am really glad that I have made it. Should you be considering a community or know someone that is establishing a community, or even someone that is dissatisfied with the product. Take a look at Community Server. I am sure you won't regret it.
|
-
When starting to brand a SharePoint site, there are 29 Placeholders to get to know in the default.master. This can be a daunting task especially if you are new to SharePoint, which is normally one of the first tasks assigned as the corporate image is vital (I won't dig at management about getting functionality first ... oops I did). In this post I have a set of placeholders drawn on top of a the default master page which hopefully will give you an idea of what is in each place holder. The purple placeholders are the main place holders on a page, whilst the orange placeholders are embedded default content of a parent placeholder. Over time, I hope to post about the default content of many of these placeholders. 
| | Placeholder | Description |  | PlaceHolderGlobalNavigation | The global navigation |  | PlaceHolderGlobalNavigationSiteMap | The global navigation breadcrumb. This is a nested placeholder. |  | PlaceHolderSiteName | Site name |  | PlaceHolderSearchArea | Search box area |  | PlaceHolderTopNavBar | Top navigation area |  | PlaceHolderHorizontalNav | Top navigation menu for the page. This control is nested inside the PlaceHolderTopNavBar |  | WSSDesignConsole | The page editing controls when the page is in Edit Page mode (in the browser, click Site Actions, and then click Edit Page) |  | SPNavigation | Empty by default in Windows SharePoint Services 3.0. Can be used for additional page editing controls such as in a publishing portal. |  | PlaceHolderPageImage | Page icon in the upper-left area of the page |  | PlaceHolderTitleLeftBorder | Left border of the title area |  | PlaceHolderTitleBreadcrumb | Main content breadcrumb area |  | PlaceHolderPageTitleInTitleArea | Page title shown immediately below the breadcrumbs |  | PlaceHolderMiniConsole | A place to show page-level commands, for example, WIKI commands such as Edit Page, History, and Incoming Links |  | PlaceHolderTitleRightMargin | Right margin of the title area |  | PlaceHolderTitleAreaSeparator | Shows shadows for the title area |  | PlaceHolderLeftNavBarDataSource | Data source for the left navigation menu |  | PlaceHolderCalendarNavigator | Shows a date-picker for navigating in a calendar when a calendar is visible on the page |  | PlaceHolderLeftNavBarTop | Top of the left navigation area |  | PlaceHolderLeftNavBar | Left navigation area |  | PlaceHolderNavSpacer | The width of the left navigation area |  | PlaceHolderLeftNavBarBorder | Border element on the left navigation bar |  | PlaceHolderBodyLeftBorder | Border element for the main page body |  | PlaceHolderPageDescription | Description of the page contents |  | ContentPlaceHolder | Main content of the page |  | PlaceHolderBodyRightMargin | Right margin of the main page body |  | PlaceHolderFormDigest | The "form digest" security control |  | PlaceHolderUtilityContent | Extra content at the bottom of the page |  | PlaceHolderBodyAreaClass | Additional body styles in the page header |  | PlaceHolderTitleAreaClass | Additional styles in the page header | One able to identify the placeholders, the next step is to understand the default content that is rendered inside of each placeholder, followed by each application and site page and the placeholders they use.
|
-
Microsoft Office SharePoint Designer 2007 is a product used to creating and customizing SharePoint Web sites. The product also allows you to build standard sequential workflow within SharePoint. SharePoint Designer allows you to build interactive SharePoint sites without writing any code. Designers can use it to designing custom SharePoint sites, brand the site. SharePoint developers and designers can create custom content and applications. SharePoint Designer also provides several reports indicating navigation and performance. Managing of permissions can also be achieved by using the code. No-code Using SharePoint Designer, you can build sophisticated and complex solutions without having to write a single line of code. You can create Data Views and data forms to display vital company information. This powerful functionality simplifies creating ASP.NET pages intended to run in SharePoint. Workflow solutions can be developed using a wizard interface. Data rich pages Using SharePoint Designer you are able to connect to many different data sources and display them on pages within your site. This information can be filtered and sorted and can be obtained from different sources. The appearance of the information can be easily formatted into different layouts. Below is a list of some data sources - .NET SQL Client
- .NET Oracle Client
- OLEDB Source
- ODBC Source
- XML Data Source
- Server-side Script Source
- XML Web Service
- Linked data source
- SharePoint lists
- SharePoint libraries
- RSS feeds
Design Professional Web Design tools are within SharePoint Designer allowing you to create visually rich web sites. SharePoint Designer has many tools such as CSS Manager and Apply Styles allowing you to create, manage and apply styles. Inheritance of styles can easily be viewed and edited. Whilst designing your SharePoint Site, content areas such as navigational bars and links can be viewed with actual content from SharePoint, giving you a true representation of your site. Master Pages Master pages allow you to create a consistent look-and-feel to your site, thus branding your site. Using SharePoint Designer Master Pages and Nested Master Pages can easily be edited and managed. Content areas inherited and replaced with more intuitive views and placements. The SharePoint Designer user interface is far more intuitive and realistic looking that that of Visual Studio. This enables you to visualize what you are creating. Workflow Using SharePoint designer you can customize and build simple sequential workflows from scratch. Custom Pages You can create ASP.NET pages and insert custom ASP.NET controls by using the same powerful on-control activity menus and control property grid found in development tools such as Microsoft Visual Studio 2005. Access Control Managers and Information workers can easily control access to the site through SharePoint Designer. You are able to create user groups and assign them permissions, give access to specific regions on a page. Limit access to who can edit pages, master pages and style sheets. Reporting There are many reports allowing you to track changes to your site, such as hyperlinks, track customised pages, CSS usage, navigation structure and more.
|
-
Microsoft announced the November 2007 CTP of SQL 2008, you can download it from here.
|
-
In a previous post, I chatted about the goals of ECM and the business problems it aims to resolve. This post describes how ECM is architecturally viewed from a MOSS perspective. The diagram below highlights the components within the ECM architecture. 
User Interface
 The user interface is an important component in any system as this is the buy-in to the system. Users need to feel comfortable and confident with the interface otherwise the resistance to the product will increase. The interface should also provide a productive means of working, where things are easy to get hold of and without problems. MOSS is a powerful and leverages of several other products. Microsoft Office The average business user working on a PC is already comfortable with tools such as Word, Excel, PowerPoint and Outlook and spend most of their PC work day in these products. The wide spread adoption of these products through organisations helps companies introduce SharePoint to assist with ECM. The content generated from this is a perfect candidate for an ECM solution. Microsoft Office and SharePoint are made to work with each other. Content can created, edited in these products and centrally stored in a SharePoint document library. The organisation my create a simple document library with folders or the better practice of many different document libraries in different sites brining the content closer to a user and in a place where the user will normally work. When a document is created, it can be directly stored on SharePoint by connecting to the site. Mail is integrated into SharePoint where SharePoint can send mails, notifying users of tasks, Users can create and read mails. SharePoint can even receive mails with attachments and store them into a document library. Web Browser SharePoint is a portal in a box and is hosted on a web server. Users can use a browser to navigate through the site to gain access to information from various other sources within the portal. Users can search for and retrieve content from the library and edit the document. With web access, users can remotely connect to the site and get the information they need. 3rd Party Other products such a SQL Server Reporting Services and Dynamics CRM can be integrated with SharePoint, giving a user a single and central access point to all data. SharePoint is extensible and developers can create custom content components that run within SharePoint, giving a true portal look and feel. Additional front-end products can be created to store and retrieve information from SharePoint. ECM Components
 SharePoint addresses a majority of ECM needs with different components and features. These features are accessed from the front-end through SharePoint sites. Each site created in SharePoint will use a combination of unified services to deliver the required solution. Records Management Many companies have ignored records management in their solutions, thus not adhering to regulations and other compliance rules. These problems can now be addressed using Record management functionality. The product assists with keeping records up-to-date thus preventing outdated records. Auditing can be implemented thus knowing who is using the information. Records are locked thus preventing change and protecting the information. Powerful search functionality assists in finding the correct data. Retention policies can be put into place thus managing the library. All of these features assist protecting a company with legal and compliance adherence. Some key deliverables - A repository for structured and unstructured content
- Record schedules through retention policies
- Audinging
- Secure repository
- Changes to content including metadata cannot be made
- Searchable
- Access control
Web Content Management Web presence within a company helps businesses communicate effectively and information disseminated to all parties throgh a managed environment can be achieved. Not only can the content be delivered to staff, also customers, partners, suppliers thus facilitating transparent communication. Content of the site is put into the hands of users allowing them to provision new content an new sites, thus reducing the burden on IT. This provisioning obviously runs under security and can be fully process controlled. Web content management allows an organization to easily manage multiple Web sites with different content, in different languages and in geographically dispersed and managed environments Key points: - Users can publish their own content
- Editors are designed for business users
- Workflow can manage publishing of content, e.g. approval
- Critical content can be published immediately
- Consistent corporate branding
- Content deployment and management is simple
- Can target intranet, extranet and internet sites
- Multi-lingual site
- Mobile devices
Business Process Management A workflow solution typically follows a single process stage within SharePoint and facilitates the movement of data through various channels. This could be from a simple approve / decline to a more complex workflow requiring input from different individuals. SharePoint uses the Windows Workflow Foundation to deliver User Interface Workflow and application workflow solutions. Workflow thus manages a process from a SharePoint view, however in an enterprise the process may span multiple other systems and communication between systems becomes vital. SharePoint has been developed to be extended, products like BizTalk server can easily integrate with SharePoint using the SharePoint Adapter in BizTalk to manage the enterprise process. Custom components can also be created to integrate the solution. Document Management Departments within organisations produce large volumes of unstructured content such as Microsoft Office documents. This poses a huge storage problem and management nightmare as users store this content in different places resulting in duplicates, different versions and unknown content. Generally this content lies around for years and no-one actually knows what it is. Through SharePoint, an organizations content can be effectively managed. Through different unified services, document management goals enforce policies through the entire documents lifecycle. Documents are stored centrally, with optional version control and check in/out functionality. Workflow can be associated with the document to mange approval processes through the company. Metadata can be placed on the document such as account numbers, bar codes helping documents to be discovered an managed. Documents can be secured and controlled through security and can be sent integrated with Record Management. Information rights can also be enforced on documents thus preventing documents be removed from the organization, features such as printing can be prohibited and view restrictions made to a specific user account. Key points: - Centrally store content
- Classify content
- Index and search information
- PowerPoint Slide libraries enable slide reuse
- Protect sensitive information
- Integrated rights management
- Excel services for protecting spreadsheets
- Streamline document collaboration
- Document workflow templates
Forms Management Business have forms that are vital to many processes. Using electronic forms, they can be filled out in a web browser thus allowing it to be available to many users. These forms do not only have to be internally facing and could face internet and intranet sites as well. Once the forms have been captured with the required information, they can be submitted for processing. A workflow can be associated with a form to start a process, even external systems such as BizTalk and SQL could process the data. Forms can also be used within Outlook 2007, then sent as e-mail messages. Using Microsoft Office InfoPath, forms are easily created with a consistent appearance and behavior across all applications. The forms can be from simble capture only forms to once with data connections and complex pre validations. Once forms have been created they can be deployed to a forms library in SharePoint and in MOSS they can be rendered onto the web. Key points - Helps automate manual processes
- Use Windows workflow foundation
- Monitor in-progress and completed workflows
- Rapidly design forms
- Centrally manage forms
- Use office InfoPath 2007 for easy form creation
- Track status of forms
- Extend information gathering across boundaries
- Design forms once, run on rich client, browser, mobile devices
- Access LOB system data thru the business data catalog
- Data entry validation rules
Unified Services
 The SharePoint offering delivers a set of components that are available though all components, these include workflow, metadata, policies, library services, security, collaboration, information rights management, Microsoft Excel® Services and search. These rich features not only provide consistency through the product, each component can be reused and customised for company specific needs. No matter if you are using Records Management, Document management or content management, the services are used. Key features that are shared amongst the ECM components are - Workflow
- Metadata
- Policies
- Library services
- Search
- Security
- Information Rights Management (IRM)
- Collaboration
Unified Storage
 The heart and core of MOSS is a SQL server database, all content, metadata is stored in various different databases. The server farm is managed through a configuration database, which manages the services and roles of each server offers. Provisioned features and templates are through this database as well. Content is stored in different content databases, from a single to multi-partitioned tables. The SQL backend can be fully scaled from a single SSEE database to a multi-clustered, geographic dispersed solution. Your budget is your only limit!
|
-
I often get the question, why must I put in WSS or SharePoint into my company? SharePoint is a versatile application that facilitates Collaboration in any organization using Office and web technologies. There are many features to the product, one of the biggest features is to expose and manage information in a controlled manner; this is Enterprise Content Management. This post discusses ECM and what it entails. Every company has a large volume of information assets sitting in documents, and this valuable content needs to be well managed. Enterprise Content Management (ECM) is a strategy undertaken to use information technology to manage this content. This management entails looking at the creation, storage and retention of the documents, the security and version control the documents. Accessibility to the documents and abilities of users to find information also needs to be addressed. Content can be electronic documents, mails, scanned documents and even audio recordings. ECM is not a light subject and a huge area of concern. Business problems faced Non existing ECM and very poorly planned ECM solutions can have a seriously negative impact to any company and some common problems are - Users create many document and store them in different locations, such as local drives, network drives, home folders.
- Historical changes to the document are not stored and new and old versions my get invalidly used.
- Attachments are sent via e-mail leading to multiple copies being stored by multiple users.
- Documents are stored all over where nobody actually knows what the documents are, and everyone is too afraid to remove them in case it is important.
- Finding documents are hard.
- Paper based documents are not stored and managed and correlated to relevant projects, customers or tasks.
- Approval processes may be ignored, resulting in negative impacts to people. e.g. Legal department may not review a document before being publicly released.
- Information is well documented, but not well shared.
- Security and access control to information weak and people see documents that they should not really see.
- Copyright protection may be violated as documents are freely available.
- Trade Secret documents could easily be copied to disk \ flash and taken home or given to a "friend".
- Legislative requirements may be ignored and 5 year retention of information may be breached.
These and many more problems have impacts on the business, from staff productivity, to legal and reputation damage of the company and finally the weakening of communication and collaboration. Areas that ECM address with technology Technology delivering Enterprise Collaboration Management can have a significant positive impact on a company when introduced properly to the organisation. Through ECM areas of document, records, web content, forms and business processes can be managed. Collaboration, sharing and accessibility of information will assist people to work on, communicate on and finally act on information. 
Document Management Document management allows you to centrally store, organise and protect your information. Check in/out policies can be implemented to prevent multiple people changing the same document at the same time, thus overwriting other peoples work. Version control of documents can be introduced to track changes and statuses of documents. Templates can be forced within document libraries. Documents can be indexed and then searched using powerful technologies that search engines provide. Additional meta-data \ information about the document can be stored and thus linking the document to a project \ customer \ supplier. Records Management Records Management identifies, classifies, archives, preserves and possibly destroys records. Typically a countries company acts and legal system enforce record management and retention. Any information from paper based, tangible objects and electronic documents need to be kept and archived. In record keeping, the information needs to be captured, possibly approved and classified and then retained for a period of time. The discipline requires the early identification of new information and classification. Long term storage plans need to be thought through and retention policies acted on. Web Content Management With the great demand customers / suppliers and partners have on a company, places a huge demand on the companies web presence and systems. Information needs to be captured and managed through process of validations such as legal, language localisation, design and branding. In larger companies publishing of information may need to go through a process and through different environments, e.g. development, testing and production. Launch of content and revocation of information may need to coincide with marketing campaigns, launches and product life. Legalities such as copy protection need to be in place and technologies enforced. Security may need to address whom can access what. Last, but not least the navigation, searching and indexing needs to be considered. Business Process Management Business Process Manages the companies process across different boundaries whit manual steps, system-to-system integration, business-to-business communications and then the processes within an single system. Moving information around from person-to-person or entity-to-entity typically follows a process, these processes may include approval from one or more people. Acceptable timings and service level agreements may need to be implemented and actually managed and monitored. Reminders to deadlines may need to be sent and more importantly people need to be notified that they are required to act on information. The components of ECM A typical ECM solution requires one to look at different aspects of ECM. 
Capture This involves the generation, preparation, creation and data capturing of information. This could be from a simple query on a database to a highly complex process require both manual and technology steps. The sources of information - Relational databases
- XML
- Microsoft Office Documents
- Emails
- Multimedia such as video, audio, telephony
- EDI or other SOA integrations
- Scanned documents
- Faxes
- OCR
- Barcode forms
- Electronic forms
- System front-ends
- Aggregation services
Manage The information needs to be categoriesed, retention policies put into place and managed. Authorisation of user access needs to be created and then managed. In larger companies, bandwidth between different regional locations needs to be managed. Store Database access, administration, backup \ restore and performance tuning needs to be implemented. Disaster recovery solutions put into place for mission critical information. Online, near-line and off-line storage solutions potentially need to be implemented. Indexing of information does take up huge volumes of space and need to be planned. Capacity planning of storage. Preserve Long term archiving solutions for expired records need to be put into place and managed. Deliver Delivery and availability of services need to be looked at. The integration to other systems such as mail need to be carefully planed and managed. Bandwidth needs to be considered based on volume of traffic on the network. Physical location and network access points considered, this may entail replication solutions put in to bring data closer to users in countries where bandwidth is an issue. Locate Indexing and searching need to be implemented to give users a better experience to the system. Web site navigational structure and data organisation and categorisation have to be carefully planned by trying to make it intuitive for users to find things.
|
-
In Windows SharePoint Services 3.0 and Microsoft Office SharePoint Server the actual installation location is: %CommonProgramFiles\Microsoft Shared\web server extensions\12 Where %CommonProgramFiles% = C:\Program Files\Common Files 
In WSS 3.0, The 12 hive is directory structure where the product is installed into. This folder structure is where all the relevant web pages, features, Site definitions and web services are installed to. Of note is the TEMPLATE directory, this directory contains all the necessary web page files that have been provisioned by default. WARNING!
You should not add to, modify or delete files in the 12 Hive. To add custom functionality to the 12 Hive (including branding) should be done through SharePoint Designer or provisioning using .wsp files. Tips - Create a shortcut to this folder structure on your desktop as it is an important folder structure for you to use.
- Add a search path by editing your environment variables to:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN
|
-
This weekend I was doing some work on my pet little portal project and needed to create a logout link where the user can logout or sign-out. The membership interface did not have any logout method and abandoning the session did not help either. Although the LoginStatus control did log the user out, I needed to do this programmatically and not through the UI. I searched the web with many different keywords hoping to find an answer but with no avail. I knew it could be done as other products have the feature so I jumped into MSDN to find the answer, after a few hours of my life I found one line of code that does it: FormsAuthentication.SignOut(); The System.Web.Security.FormsAuthentication class can be really useful when customising and controlling authentication.
|
-
At Tech-Ed in Barcelona, the Vice President of the Developer Division at Microsoft, S. (Soma) Somasegar announced that Visual Studio 2008 will ship at the end of November 2007. Catch the press release here. I have been saying that I will move this onto my production box as soon as it released, I cannot wait.
|
-
The different products on the shelf today with even more emerging that use workflow in one way or another. Information workers, developers and IT Pros are being bombarded with workflow technology and it is becoming difficult to choose which one to use and where. The answer is unfortunately not that simple as you firstly need to understand what you are trying to do before matching product. In my blog article High level overview to what workflows are, I attempt to explain the different types of workflows and I suggest you read that article before continuing as I assume you understand workflows. System-to-system workflow In our business environment we have many different systems, systems we have bought off the shelf, customised and enhanced; there are also all the systems developed by different business units and different development teams. It is highly unlikely that you will have a single system that does everything. When looking at integrating systems there are a few things that are important. Firstly can you get or send the data to or from the system using technology? For example, is there an API, web service, import/export functionality or application interface adapter or direct database statements? If there is not way to communicate with the application, and you don't know how to perform miracles you will not communicate with it. Using BizTalk we can facilitate system-to-system communications. BizTalk has great functionality to get and send data from systems, the list below is not comprehensive - Get or send data in XML, Flat-file (e.g. CSV) or EDI.
- Transport it through FTP, HTTP, SMTP, POP3, MSMQ .... and a whole lot more.
- Specific application called adapters to SAP, SIEBEL, SWIFT and many more.
- Adapters to databases such as SQL server
- Adapter for Windows Communication Framework and Web Services.
- Adapters to SharePoint.
The data can either be pushed or pulled. Through pipelines the data can be encrypted\decrypted, compressed\decompressed and mime encoded to make sure it is in the correct format. In many cases the formats of the sending and receiving system are different and conversion of data is required. BizTalk's strong mapping capabilities allows you convert the data, perform aggregations , string manipulation, date and time manipulation. Data movement in not simply an Extract-Transform-Load, it often requires the data to flow to different systems based on a workflow (called an orchestration in BizTalk). The orchestrations can process data quickly and yield responses quickly, alternatively they could run for days, months or years. An example of a long running orchestration is a annual subscription, near the end of the subscription you could resume the orchestration to try to renew the subscription and if not renewed start terminating the subscription, notifying all other systems that the subscription is canceled. Decisions can be made and data routed accordingly. Using the Business Rule Engine in BizTalk, the rules can be created that business users can change the rules through a rule editor without having to compile or change the data. Using rules data can be redirected differently, calculated differently and processed differently. The power of BizTalk is that it allows you to loosely couple your systems by using a powerful workflow + ETL mechanism. Thus supporting tools around the product are great. SharePoint workflow As a portal solution, a workflow can be initiated either manually by the user or automatically when a list changes. The workflow allows you to add approval processes to document libraries and other lists, create tasks for users to do work. The workflow is aimed to increase collaboration within company, if the out-of-the-box workflows are not sufficient, you can develop your own using the SDK and Windows Workflow Foundation. It is important to note the SharePoint functionality is around processing of SharePoint data, typically lists (libraries and lists) and to facilitate communication, i.e. it is specific to the SharePoint functionality, e.g. When something happens, process the data and notify these people via lists or mail Windows Workflow Foundation WWF in the .NET 3.0 framework allows you to write custom workflows for your application. To develop WWF, take a visit to Microsoft and download the WWF SDK for Visual Studio. There are basically two categories of workflow you will be looking at when writing your application. The first is the Human Workflow Interfaces which manages the display of forms or pages, i.e. when the app starts it will open up page a, when you click button it will display form b or c. The HIW is aimed at navigation through the system and prevents you having to hard code navigation into your form. For the hard core techies, form data and state (cancel or ok) is sent to the workflow and it decides on what next to display. This allows you to change the applications navigation based on role. The second is process automation, these workflows will take data and move them through the process, making decisions and performing tasks. Instead of you writing code to call methods sequentially, you transfer this code into a workflow. A single application may have a number of processes. Through Human Interface Workflow and Process Workflows you can chain execution, e.g. HIW shows a form, user fills it out clicks OK, HIW then starts Process Workflow and on completion Shows next form. You may decide to launch a process to respond to dialog in the background and immediately navigate the user to another form. WWF is typically aimed at a single application. It is interesting to not that custom SharePoint workflow use WWF but have custom activities to interact with SharePoint objects, such as lists and tasks. SharePoint Workflow typically implement an business process.
|
-
Do you get the feeling of "Workflow ... workflow ... workflow .... aaargh .... wtf"? Whilst chatting to many students, I find that there is a lot of ambiguity around workflows and what they are and where to use them. I will be writing a few blogs on the topic, this topic what a workflow is. As the technologies being used are gearing to workflows, I will discuss the different technologies and where to use what in another post. What is a workflow? A workflow is simply a reliable and repeatable set of activities that drive a process. In our businesses we create business silos or departments to deal with different processes, e.g. sales is responsible for performing activities to get new business, provisioning is responsible for delivery of goods, hr is responsible for managing staff and IT for performing tasks to provide supporting technologies. Within these departments there are many processes each aimed to perform a different task. We may even divide these processes into geographic locations for branches. Business is all about process! A workflow is a business process that has a set of characteristics: - It has a set of events that will cause the workflow to start.
- It may need certain information to start such as a document artifact.
- A workflow has an owner.
- A workflow has a business benefit.
- A may workflow delivers an outcome.
- A workflow may have several activities.
- A workflow is deterministic through a set of decision structures
- A workflow activity may be automated by a system or manually done by a person, thus having someone\something responsible for executing the process.
- A workflow is traceable and can be audited.
- A workflow may start another process and even wait for an outcome.
Why is workflow hard? An organization has thousands of processes and normally these processes are understood in the minds of individuals but not well documented, thus leaving room for ambiguity. Many people use subjective reasoning and experience to make decision which leads to inconsistencies as the process is not well defined in their minds. Unfortunately many decisions made and actions taken result in consequences, thus causality. Thus one event (cause) results in a related activity (effect). In business many effects are counterproductive as a result of bad decisions in previous processes. System development and defining a workflow is hard as computer logic expects explicit decision structures and processes, there can not be any ambiguity or any step left out. As a technical resource developing a workflow, the business process needs to be implemented exactly and it is our task is to extract the process from the business user. However, most business users have never given great thought to the process as the live on a wing and a prayer dealing with causality on a daily basis. This often yields to flawed processes which are technically hard coded, resulting in a flawed system. A flawed business process will result in a system workflow that automates the process, thus a botch-up results in an automated botch-up. An application is workflow No matter how you look at it we ultimately write applications to facilitate or automate some business process. The activities the user performs on the system, is part of some business process. The application itself is a set of methods that create objects and execute other methods based on logic. The statements within each method also forms part of a process of tasks. Is this not a workflow. Windows events may occur, triggering off processes, such as clicks and timers. This leads us to a complex design issue as we are actually dealing with many different types of workflows. Types of workflows Business workflow Our organizations are full workflows that you, me and our coworkers carry out each and every day. All of these activities require us to do something in a set manner with deterministic outcomes. These tasks may require us to discuss a topic with others, perform manual work, validate and check, make a decision based on criteria or even capture information into a system. Human Interface Workflow A single system attempts to guide the user through different screens with deterministic outcomes. If you click on a button a certain dialog will appear, and if you click on another button something else will appear. As we navigate through the system, information is passed from screen to screen thus attempting to guide you through a process. The challenge is that applications facilitate many processes activities, and the user needs to learn what to click in order to resume the process. The user either needs to capture some information to start off a process within the application, alternatively the user needs to query or read some information to resume a manual process. The human interface workflow thus is used to assist the user navigating through your application and retaining state of what the user is doing. You may want to think of your application as a giant wizard interface with screens and reports that you allow the user to navigate through by clicking or typing things. In writing a human workflow, technical tasks such as retrieving or storing data, checking spelling is done through System workflow's. There is a coupling between the Human Interface and the System Workflow, i.e. events in the HIW trigger off system workflows. System Workflow Systems either monitor user events (clicks) or system events (timers, data arrival, i/O) and based on the event a sequence of code activities (methods) execute. These activities normally perform both technical overhead tasks such as opening connections to databases, beginning and committing transactions, to computations, decision trees and even storing and retrieving data. As I am writing for technical people, each and every method you write should do a single task (activity), there may also be a need for you to write a method that chains the calling of other methods. Your system workflow is the nuts and bolts that make the application do what it is supposed to do and this is normally good old fashioned programming. System-to-System workflow In our organization we cannot expect to have one system as we have divisions such as departments with different processes, these differences require different systems that allow a user to do his/her job. Many times a single person may interact with different systems as they are performing different processes. This is normal in business. However, when looking at the business holistically, there are many commonalities that are shared across systems, take for example the customer. You need a system to capture and identify the customer, however the customer is used by sales, by accounting, by credit control, by provisioning and many other departments. It is impossible virtually impossible to have one system that runs the entire business, as we may custom develop systems in different time frames, we may also purchase off-the shelf systems. There is no single system that can run the entire business, from web presence and portal, document manage, CRM, Financial, HR and Payroll, Security and authentication, printing, Communication, Data base, Business intelligence and forgive me for not going on for pages and pages. Typically systems were done at a departmental and role level and required manual communication when discussing with other departments, e.g. Sales receiving an order and forwarding it to billing, credit and provisioning. With technology today we want to interchange data between our systems and automatically trigger off workflow on those systems so that we stop human miscommunication. This is what system to system workflow are all about. This can be tricky as security, information format and interchange methods come to play, i.e. What information, how does it look, how do we protect it, how do we send it and and in what format do we send it. Business-to-business workflow Business-to-business workflows are ultimately System-to-system workflows but they now are going to vendors, partners and customers which are different legal entities. There are challenges of using standards to totally decouple technology, not everyone is using MS products and all of their products. There are legal issues to worry about and things such as non-repudiation. Business-to-business workflow are normally more legally intense and I feel that they need to be managed differently.
|
-
Microsoft BizTalk 2006 R2 was recently launched and there are a host of improvements that make life easier. Take a look at these features Windows Communication Foundation (WCF) Adapters With the WS-* endpoint standards, BizTalk now has several adapters allowing communication with WCF applications. The following adapters are available - WCF-WSHttp
- WCF-BasicHttp
- WCF-NetTcp
- WCF-NetMsmq
- WCF-NetNamedPipe
- WCF-Custom
- WCF-CustomIsolated
You are also able to publish an orchestration as a WCF service which is pretty neat. You are able to set the WCF binding, behavior, and behavior extensions for the receive location. Microsoft WCF Line-of-Business (LOB) Adapter SDK A new software development kit has been released allowing you to make your own business adapters. These LOB adapters are based on WCF thus allowing WCF consumers to search for and get the contract. The nice part is these adapters can be reused in other products such as Microsoft Office SharePoint Server, SQL Server Integration Services and if you can code in .NET, you can also use it. BizTalk Adapter Pack This has not been released yet and planned for 2008 Q1\Q2. The big idea here is to enable adapters like Sieble, SAP and Oracle to be exposed as WCF Line-of-Business Adapters, thus making them available to any .NET application. BAM Interceptors for WF and WCF The Business Activity Monitoring (BAM) Interceptors now use Windows Workflow Foundation (WF), Windows Communication Framework (WCF) and other runtimes. This opens up a window of opportunities for your applications. EDI Ok, I am not a fan of EDI but it is still used extensively for BTB communications. EDI was supported in previous versions of BizTalk, from a quick look see it looks like MS has improved the interface and ironed out some weaknesses. You can download the white paper from Microsoft's web site. AS2 EDI over the Internet (EDIINT) is an alternative to EDI as it allows communication using the Internet for data exchange. To get this working you will need to use Applicability Statement 2. BizTalk Server 2006 R2 uses AS2-defined methods to send, receive, encrypt, decrypt, sign, and verify messages between partners using HTTP over the Internet. BizTalk Server helps ensure the security of messages through the use of encryption keys, digital signatures, certificates, and non-repudiation. AS2 is not restricted to sending EDI data and you can send XML for example. Microsoft BizTalk RFID The RFID technology has radically improved over the past year and a bit and now a really feasible technology to track your goods and assets and get down to proper supply chain management. The benefits are huge if you are moving stock or assets around and you should really look in this to minimise shrinkage and manage stock movement. BizTalk 2006 R2 now can interface with many readers and expose the unique id on the RFID tag. You can write applications to process the RFID Tag and even pass it through an orchestration in BizTalk. The integration of RFID into your enterprise and other products such as SQL, .NET and Dynamics is awesome. Key Benefits - Uniform management and configuration of heterogeneous RFID devices, via the Device Service Provider Interface
- Operational simplicity
- Innovative process architecture that decouples design time and deployment time activities
- Reliable and fast event processing engine that supports using business rules or custom .NET components
- Synchronous and asynchronous communication with RFID devices
- Extensible development platform for creating custom event handlers
- Part of the Windows Server Systemâ„¢
BAM Add-in for Excel BAM Add-in for Excel (XLA) has been updated to support Microsoft Office Excel 2007. BAM XLA continues to support for Office Excel 2003. Enterprise Single Sign-On 4.0 BizTalk can now use SSO 4 and the set of tools are greatly improved from the previous version. Support for Windows SharePoint Services 3.0 and Microsoft Office SharePoint Server (MOSS) Finally no more butchering your WSS 3.0 and MOSS installation to get the web service installed and the BizTalk SharePoint Adapter working. It is out of the box now. :)
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Deploying a custom SSIS task
Deploying the SSIS unzip task is not a normal deployment and requires a few steps.
- Deploy the assembly to the Global assembly cache.
- Deploy the SharpZipLib to the Global assembly cache.
- Copy the assembly to the tasks directory within SQL Server C:\Program Files\Microsoft SQL Server\90\DTS\Tasks\
I have automated this in the build process by adding the following to the Post-Build events within the project
- Edit the properties of the project.
- Select the build events tab.
- Copy the following script into the Post-build event command line:
cd $(ProjectDir)
@SET TASKDIR="C:\Program Files\Microsoft SQL Server\90\DTS\Tasks\"
@SET GACUTIL="c:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\gacutil.exe"
Echo Installing dll in GAC
%GACUTIL% -if "$(OutDir)$(TargetFileName)"
%GACUTIL% -i "bin\ICSharpCode.SharpZipLib.dll"
Echo Copying files to Tasks
copy "$(OutDir)$(TargetFileName)" %TASKDIR%
- Build the project.
Registering the control on the toolbar
- Create a new Integration Services Project.
- Open a package for editing and display the toolbox.
- Right click on the toolbox, and select Choose Items...
- Select the SSIS Control Flow Items tab
- Find the UnzipTask and check the checkbox.
- Click Ok
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Adding an icon to a custom SSIS task
In order to add a icon to the task as to display it in the designer, you first need to create an icon resource. Then you need to update the custom SSIS task to use the icon.
This is done by following these steps
- Add an icon file to your project
- Add a 32x32 icon to the icon file
- Add a 16x16 icon to the icon file.
- Click the Unzip.ico icon file in the Solution explorer and change the Build Action property to Embedded Resource.
- Finally edit the Unzip task classes attribute to indicate the name of the icon. Then name of the icon should be <dll name>.<icon filename>
[DtsTask(DisplayName = "UnzipTask",
Description = "A custom Unzip task for demonstration purposes.",
TaskType = "BrettTasks",
IconResource = "Brett.Demo.SSIS.Tasks.Unzip.Unzip.ico",
UITypeName = "Brett.Demo.SSIS.Tasks.Unzip.UnzipTaskUI,Brett.Demo.SSIS.Tasks.Unzip,Version=1.0.0.0,Culture=Neutral,PublicKeyToken=f73e70619ef50b13"
)]
public class Unzip ...
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Adding a User interface to edit properties for the custom task
This post describes how to add a custom user interface to the task, thus allowing a user within the designer to right click a task and click edit. This however is a bit tricky as the custom dialog needs to update the actual properties of the class of the task. This is done by:
- To achieve this a class needs to be created that implements the IDtsTaskUI interface. There are methods in the class to create your custom form and initialize it with your properties using the TaskHost.
- Then the form needs to be created and using the task host it should set your controls to the value of the properties when loaded. On exiting the form, the properties need to be updated with the control values.
- Finally, the attributes of the task needs to be edited to indicate the IDtsTask implementation that will be used to create the dialog.
These steps are discussed below.
Adding a IDtsTaskUI interface
The IDtsTaskUI interface is used to define a collection of methods and properties called by the SSIS Designer to initialize and display the user interface associated with a task.
The interface
Delete
The designer calls this method when a task is deleted from the package.
Initialize
The client designer application (Visual Studio) calls this interface method when the task is added to the package. This method is also called when the package is opened up in the designer. It is important to note that TaskHost object is passed to the interface, this needs to be stored as it is used to establish communication between the designer and the custom dialog.
GetView
The client designer will call this method when it wants to create an instance of the form. The method should create a new object instance of the dialog. When creating the dialog, you will notice that a custom constructor is created on the form in order to pass the TaskHost object to the form, thus the form can communicate with the designer.
New
The designer calls this method when a new task is added to the package.
The code below implements the UnzipTaskUI which is responsible for creating the code. using System;
using Microsoft.SqlServer.Dts.Runtime;
using Microsoft.SqlServer.Dts.Runtime.Design;
namespace Brett.Demo.SSIS.Tasks.Unzip
{
public class UnzipTaskUI : IDtsTaskUI
{
#region // Fields
private TaskHost _taskHost;
#endregion
#region IDtsTaskUI Members
public void Delete(System.Windows.Forms.IWin32Window parentWindow)
{
// Do nothing for this project
}
public System.Windows.Forms.ContainerControl GetView()
{
return new UnzipUIForm(_taskHost);
}
public void Initialize(TaskHost taskHost, IServiceProvider serviceProvider)
{
_taskHost = taskHost;
}
public void New(System.Windows.Forms.IWin32Window parentWindow)
{
// Do nothing for this project
}
#endregion
}
}
The User interface
The user interface is just a normal System.Windows.Forms.Form
The form constructor
The form constructor is changed to take a TaskHost object. When executed the constructor will call InitializeComponent() to create the child controls and then
it will use the taskHost to get the properties. The code below demonstrates this.
You will notice the taskHost has a collection of Properties with an Indexer taking in a string with the Property name. Reflection will be used to get the property value and return the relevant data type. public UnzipUIForm(TaskHost taskHost)
{
InitializeComponent();
_taskHost = taskHost;
if (_taskHost != null)
{
if (this._taskHost.Properties["ZipFilename"] != null)
this.txtZipFilename.Text =
this._taskHost.Properties["ZipFilename"].GetValue(_taskHost).ToString().Trim();
if (this._taskHost.Properties["TargetDirectory"] != null)
this.txtTargetDirectory.Text =
this._taskHost.Properties["TargetDirectory"].GetValue(_taskHost).ToString().Trim();
if (this._taskHost.Properties["OverwriteExisting"] != null)
this.chkOverwriteExisting.Checked =
(bool)this._taskHost.Properties["OverwriteExisting"].GetValue(_taskHost);
if (this._taskHost.Properties["ZipMode"] != null)
this.optZipMode_UnZip.Checked =
((ZipMode)this._taskHost.Properties["ZipMode"].GetValue(_taskHost) == ZipMode.Unzip);
}
}
Synchronising the properties between the dialog and the custom task
When the dialog box is closed by clicking the Ok button, the taskHost object will be used and the properties will be set to the dialog variables. private void cmdOk_Click(object sender, EventArgs e)
{
this._taskHost.Properties["ZipFilename"].SetValue(this._taskHost, this.txtZipFilename.Text);
this._taskHost.Properties["TargetDirectory"].SetValue(
this._taskHost, this.txtTargetDirectory.Text);
this._taskHost.Properties["OverwriteExisting"].SetValue(
this._taskHost, this.chkOverwriteExisting.Checked);
this._taskHost.Properties["ZipMode"].SetValue(this._taskHost, ZipMode.Unzip);
this.DialogResult = DialogResult.OK;
this.Close();
}
Updating the Unzip class attributes to use the user interface
The last step is to update the Unzip classes DtsTask attribute to indicate the user interface. A new named parameter is added called UITypeName. This parameter needs a four part assembly name referring to the class, that is "<Class name>,<Dll name>,Version=<Dll version>,Culture=<Dll Culture>,PublicKeyToken=<PublicKeyToken>"
In order to accomplish this you will need to assign a strong name to the dll, compile it and use the sn.exe -T option to get the Token name. This is documented in the blog post Determine the Public Key Token for assembly. [DtsTask(DisplayName = "UnzipTask", Description = "A custom Unzip task for demonstration purposes.",
TaskType = "BrettTasks",
UITypeName = "Brett.Demo.SSIS.Tasks.Unzip.UnzipTaskUI,Brett.Demo.SSIS.Tasks.Unzip,
Version=1.0.0.0,Culture=Neutral,PublicKeyToken=f73e70619ef50b13"
)]
public class Unzip ...
NOTE that the string has been split over two lines for readability.
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Adding saving and loading of properties to the SSIS package
Integration Services packages are saved as XML documents and contain information that describes their objects, properties, and settings. When the run-time engine saves a package, the tasks contained by the package are automatically saved, but with only a limited amount of information. This information includes task creation properties and properties of the TaskHost object that wrap the task.
When writing a custom task, you must implement the IDTSComponentPersist interface to allow your custom task to save custom properties and settings in the package XML
The IDTSComponentPersist interface
The code below basically describes the interface // Summary:
// Allows properties and settings for components and tasks to be saved and loaded
// from the package XML.
public interface IDTSComponentPersist
{
// Summary:
// Loads component information from XML.
//
// Parameters:
// node:
// The node that contains the information to be loaded.
//
// infoEvents:
// An object that implements the Microsoft.SqlServer.Dts.Runtime.IDTSInfoEvents
// interface for raising events (errors, warnings, and so on) during persistence.
void LoadFromXML(XmlElement node, IDTSInfoEvents infoEvents);
//
// Summary:
// Saves a component to XML. Tasks and containers implement this method.
//
// Parameters:
// doc:
// The XML document to which to save the information.
//
// infoEvents:
// An object that implements the Microsoft.SqlServer.Dts.Runtime.IDTSInfoEvents
// interface for raising events (errors, warnings, and so on) during persistence.
void SaveToXML(XmlDocument doc, IDTSInfoEvents infoEvents);
}
Adding persistence to the unzip class
The code below ensures that a UnzipTask node exists and then attempts to read the properties into the fields: void IDTSComponentPersist.LoadFromXML(System.Xml.XmlElement node, IDTSInfoEvents infoEvents)
{
// This might occur if the task's XML has been modified outside of the Business Intelligence
// Or SQL Server Workbenches.
if (node.Name != "UnzipTask")
{
throw new Exception(string.Format("Unexpected task element when loading task - {0}.", "UnzipTask"));
}
else
{
// let error bubble up
// populate the private property variables with values from the DTS node.
this._zipMode = node.Attributes.GetNamedItem("ZipMode").Value.ToLower() == "zip" ? ZipMode.Zip : ZipMode.Unzip;
this._zipFilename = node.Attributes.GetNamedItem("ZipFilename").Value;
this._targetDirectory = node.Attributes.GetNamedItem("TargetDirectory").Value;
this._overwriteExisting = Convert.ToBoolean(node.Attributes.GetNamedItem("OverwriteExisting").Value);
}
}
The code below ensures that a UnzipTask node exists and then attempts to read the properties into the fields: void IDTSComponentPersist.SaveToXML(System.Xml.XmlDocument doc, IDTSInfoEvents infoEvents)
{
//create node in the package xml document
XmlElement taskElement = doc.CreateElement(string.Empty, "UnzipTask", string.Empty);
// create attributes in the node that represent the custom properties and add each to the element
// ProgramId property
XmlAttribute zipModeAttribute = doc.CreateAttribute(string.Empty, "ZipMode", string.Empty);
zipModeAttribute.Value = this._zipMode.ToString();
taskElement.Attributes.Append(zipModeAttribute);
// Package Name property
XmlAttribute zipFilenameAttribute = doc.CreateAttribute(string.Empty, "ZipFilename", string.Empty);
zipFilenameAttribute.Value = this._zipFilename;
taskElement.Attributes.Append(zipFilenameAttribute);
// Package Description property
XmlAttribute targetDirectoryAttribute = doc.CreateAttribute(string.Empty, "TargetDirectory", string.Empty);
targetDirectoryAttribute.Value = this._targetDirectory;
taskElement.Attributes.Append(targetDirectoryAttribute);
// Enabled Switch property
XmlAttribute overwriteExistingAttribute = doc.CreateAttribute(string.Empty, "OverwriteExisting", string.Empty);
overwriteExistingAttribute.Value = this._overwriteExisting.ToString();
taskElement.Attributes.Append(overwriteExistingAttribute);
//add the new element to the package document
doc.AppendChild(taskElement);
}
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Implement the zip logic
The method below implements the logic to zip a file /// <summary>
/// This method is called to start zipping based on the internal fields.
/// </summary>
/// <remarks>
/// The StartZipping method will ensure the directory to zip exists. It also
/// validates whether the zip file exists and should be overwritten.
/// The method then creates a zip file and starts adding the files in the
/// _target directory to the zip file by calling the method AddFolderToZip.
/// </remarks>
private void StartZipping()
{
// Perform some simple parameter checking. More could be done
// like checking the target file name is ok, disk space, and lots
// of other things, but for a demo this covers some obvious traps.
if (!Directory.Exists(_targetDirectory))
{
throw new Exception(string.Format("Cannot find directory '{0}'", _targetDirectory));
}
//Check if the zip file already exists
if (File.Exists(_zipFilename))
{
// If the overwriteExisting flag is not set then we cannot replace the file
// threfore throw an exception
if (!_overwriteExisting)
{
throw new Exception(string.Format("The zip file {0} already exists and cannot be overwritten!", _zipFilename));
}
//Delete the file
File.Delete(_zipFilename);
}
try
{
// 'using' statements guarantee the stream is closed properly which is a big source
// of problems otherwise. Its exception safe as well which is great.
using (ZipOutputStream s = new ZipOutputStream(File.Create(_zipFilename)))
{
s.SetLevel(9); // 0 - store only to 9 - means best compression
AddFolderToZip(_targetDirectory,s);
// Finish/Close aren't needed strictly as the using statement does this automatically
// Finish is important to ensure trailing information for a Zip file is appended. Without this
// the created file would be invalid.
s.Finish();
// Close is important to wrap things up and unlock the file.
s.Close();
}
}
catch (Exception ex)
{
Console.WriteLine("Exception during processing {0}", ex);
// No need to re-throw the exception as for our purposes its handled.
}
}
/// <summary>
/// This method adds a folder, child folders and files to an already created zip
/// file.
/// </summary>
/// <remarks>
/// The method will recursively iterate through all child folders of add all the files
/// to the zip file. It is important to not that this code is not production optimized
/// for large volumes of files.
/// </remarks>
/// <param name="directory">The folder to iterate through and add to the zip file</param>
/// <param name="s">A zip stream (aka open zip file) to add the folder contents to.</param>
private void AddFolderToZip(string directory, ZipOutputStream s)
{
byte[] buffer = new byte[4096];
// Recursively add child folders to the zip file
string[] directories = Directory.GetDirectories(directory);
foreach (string childDirectory in directories)
{
AddFolderToZip(childDirectory, s);
}
// Depending on the directory this could be very large and would require more attention
// in a commercial package.
string[] filenames = Directory.GetFiles(directory);
foreach (string file in filenames)
{
// Using GetFileName makes the result compatible with XP
// as the resulting path is not absolute.
ZipEntry entry = new ZipEntry(Path.GetFileName(file));
// Setup the entry data as required.
// Crc and size are handled by the library for seeable streams
// so no need to do them here.
// Could also use the last write time or similar for the file.
entry.DateTime = DateTime.Now;
s.PutNextEntry(entry);
using (FileStream fs = File.OpenRead(file))
{
// Using a fixed size buffer here makes no noticeable difference for output
// but keeps a lid on memory usage.
int sourceBytes;
do
{
sourceBytes = fs.Read(buffer, 0, buffer.Length);
s.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}
}
}
Implementing the Unzip logic
The logic below implements unzipping. /// <summary>
/// This method is called to unzip a file to the target folder.
/// </summary>
private void StartUnzipping()
{
using (ZipInputStream zip = new ZipInputStream(File.OpenRead(_zipFilename)))
{
ZipEntry theEntry;
while ((theEntry = zip.GetNextEntry()) != null)
{
string fileName = Path.Combine(_targetDirectory, theEntry.Name);
string directoryName = Path.GetDirectoryName(fileName);
// Skip the file if it exists and task set NOT to overwrite
if (!_overwriteExisting && File.Exists(fileName))
{
continue;
}
// create directory
if (directoryName.Length > 0 && !Directory.Exists(directoryName))
{
Directory.CreateDirectory(directoryName);
}
if (fileName != String.Empty)
{
using (FileStream streamWriter = File.Create(fileName))
{
int size = 2048;
byte[] data = new byte[2048];
while (true)
{
size = zip.Read(data, 0, data.Length);
if (size > 0)
{
streamWriter.Write(data, 0, size);
}
else
{
break;
}
}
}
}
}
}
}
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Overwrite the base classes Execute method
The SSIS runtime will call the Execute method to start processing the task, and you will need to override the base classes Execute method in order to implement your own logic. In this example, the execute method will either zip or unzip a file based on the ZipMode.
Syntax public virtual DTSExecResult Execute (
Connections connections,
VariableDispenser variableDispenser,
IDTSComponentEvents componentEvents,
IDTSLogging log,
Object transaction
)
Parameters
| connections |
A collection of connections used by the task. |
| variableDispenser |
A VariableDispenser object for locking variables. |
|
componentEvents |
An object that implements the IDTSComponentEvents interface. |
| log |
An object that implements the IDTSLogging interface. |
| transaction |
The transaction object that the container is a part of. This value can be null. |
Returns
Once the task is run, you need to return with one of the following DTSExecResult enumerations
|
Canceled |
The task was cancelled. |
|
Completion |
The task ran to completion. |
|
Failure |
The task failed. |
|
Success |
The task ran successfully. |
The code below implement the Execute logic: /// <summary>
/// The main use of the Execute method is for it to be inherited and overridden when you
/// create a custom task. i.e.
/// </summary>
///
/// <param name="connections">A collection of connections used by the task.</param>
/// <param name="variableDispenser">A VariableDispenser object for locking variables.
/// </param>
/// <param name="componentEvents">An object that implements the IDTSComponentEvents
/// interface.</param>
/// <param name="log">An object that implements the IDTSLogging interface.</param>
/// <param name="transaction">The transaction object that the container is a part of.
/// This value can be null.</param>
///
/// <returns>A value from the DTSExecResult enumeration. </returns>
public override DTSExecResult
Execute(Connections connections,
VariableDispenser variableDispenser,
IDTSComponentEvents componentEvents,
IDTSLogging log,
object transaction)
{
try
{
if (_zipMode == ZipMode.Zip)
{
StartZipping();
}
else
{
StartUnzipping();
}
}
catch
{
return DTSExecResult.Failure;
}
return DTSExecResult.Success;
}
Override the base classes Validate Method
The Validate method reviews properties and settings for inaccuracies or incorrect settings. The method does not touch data, or connect to data sources to validate connections. However, it ensures that required fields are populated and contain appropriate values. The fields that are validated differ depending on what object is being validated.
The primary use of Validate is when writing a custom task. The Validate method is called by the SSIS Designer when a task is dropped onto the design surface and again, potentially multiple times, when properties are being set. However, in code, the Validate method on individual objects is not commonly used, as it is recommended that you call the Validate method on the Package when you need to validate objects. However, the method is available on individual objects should you find a unique circumstance where it is needed.
The Validate method is overridden in custom tasks, either for validation of the object when used in the SSIS Designer, or when called by code. For more information on writing the Validate method for a custom task, see Implementing the Validate Method in Custom Integration Services Tasks.
Syntax public virtual DTSExecResult Validate (
Connections connections,
VariableDispenser variableDispenser,
IDTSComponentEvents componentEvents,
IDTSLogging log
)
Parameters
| connections |
A collection of connections used by the task. |
| variableDispenser |
A VariableDispenser object for locking variables. |
|
componentEvents |
An object that implements the IDTSComponentEvents interface. |
| log |
An object that implements the IDTSLogging interface. |
Returns
Once the task is run, you need to return with one of the following DTSExecResult enumerations
|
Canceled |
The task was cancelled. |
|
Completion |
The task ran to completion. |
|
Failure |
The task failed. |
|
Success |
The task ran successfully. |
The code below implement the Execute logic: /// <summary>
/// The Validate method is called by the SSIS runtime to ensure that all properties
/// are valid
/// </summary>
///
/// <param name="connections">A collection of connections used by the task.</param>
/// <param name="variableDispenser">A VariableDispenser object for locking variables.
/// </param>
/// <param name="componentEvents">An object that implements the IDTSComponentEvents
/// interface.</param>
/// <param name="log">An object that implements the IDTSLogging interface.</param>
///
/// <returns>A value from the DTSExecResult enumeration. </returns>
public override DTSExecResult Validate(
Connections connections,
VariableDispenser variableDispenser,
IDTSComponentEvents componentEvents,
IDTSLogging log)
{
//TODO add property validation
return base.Validate(connections, variableDispenser, componentEvents, log);
}
|
-
This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS).
Source code
Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty.
Creating a Visual Studio project for the custom SSIS control flow
- Open up visual studio and create a new Class Library project called "Unzip".
- Edit the properties of the project and set the following
- On the Application tab, set the Assembly name to Brett.Demo.SSIS.Tasks.Unzip.
- Set the default namespace to Brett.Demo.SSIS.Tasks.Unzip.
- Edit the Assembly Information as with your details.
Add references to the Microsoft SSIS assemblies
- Add a reference to the Microsoft.SqlServer.Dts.Design and Microsoft.SqlServer.ManagedDTS
- Add a reference to the .NET Zip Library, by choosing the dll.
If you do not see the assemblies, you do not have SQL installed with SSIS services.
Creating a class for the control flow
- In your project add a new class to the solution, called Unzip
- Add the following using statements to your code
using System;
using System.IO;
using System.Xml;
using System.Runtime.InteropServices;
// Unzip namespace
using ICSharpCode.SharpZipLib.Zip;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
// SSIS namespace
using Microsoft.SqlServer.Dts.Runtime;
- Add an enumeration for the ZipMode allowing Zip and Unzip
public enum ZipMode { Unzip, Zip }
- Add a public class Unzip that inherits from Task and IDTSComponentPersist
public class Unzip : Task, IDTSComponentPersist
{
}
- Add attibutes to the Unzip class as to allow the class to appear in the toolbox.
[DtsTask(DisplayName = "UnzipTask",
Description = "A custom Unzip task for demonstration purposes.",
TaskType = "BrettTasks",
)]
public class Unzip ...
- Add fields and properties for the ZipMode, ZipFileName, TargetDirectory and a OverwriteExisting flag.
#region // Fields
private ZipMode _zipMode;
private string _zipFilename;
private string _targetDirectory;
private bool _overwriteExisting;
#endregion
#region // Properties
/// <summary>
/// The ZipMode property allows you to set the task to either
/// Zip a target folder to a zip file or to Unzip a zip file to
/// a target folder.
/// </summary>
public ZipMode ZipMode
{
get { return _zipMode; }
set { _zipMode = value; }
}
/// <summary>
/// The full pathname to the zip file.
/// </summary>
public string ZipFilename
{
get { return _zipFilename; }
set { _zipFilename = value; }
}
/// <summary>
/// The folder name that the files in the zip file will be unziped to
/// or the folder to zip.
/// </summary>
public string TargetDirectory
{
get { return _targetDirectory; }
set { _targetDirectory = value; }
}
/// <summary>
/// This flag is used to either overwrite a zip file when zipping or
/// the over write files in the target directory when unzipping. If the
/// flag is set to true, the file will be replaced.
/// </summary>
public bool OverwriteExisting
{
get { return _overwriteExisting; }
set { _overwriteExisting = value; }
}
#endregion
- Add a default constructor that does nothing
#region // Ctor
public Unzip()
{
}
#endregion
|
-
Microsoft SQL Server Integration Services (SSIS) provides powerful Extract-Transform-Load (ETL) and in 99% of the time sufficient for most companies. There are however a few situations where out-of-the-box tasks do not provide the functionality needed. There are two ways you can get this functionality, firstly you can visit you favorite search engine and look for a vendor that has already made it; there are many awesome tasks out there. The second option is to write one yourself. I would strongly suggest you try purchase one before you start developing as it will be cheaper. Should all you options be exhausted, you will need to develop one. This series of posts will guide you through the process of developing a unzip control flow. I hear you say, run a command line process task to do it and I would agree as it does not involve development, this however is a demonstration. SSIS is very extensible as it sits on top of the .NET 2.0 Framework and it provides a set of classes and interfaces that you can use do develop your own components, these include - Control flow tasks
- Data flow tasks
- Event handlers
- Data sources and destinations
- Enumerators for foreach control flow
Series As this is a rather large topic, the blog will be posted as a series of posts. This series of posts will teach you how to develop a control flow for SQL Server Integration Services (SSIS). Source code Download the sample code from here. Please not that you are free to use this code however you wish, but do not come knocking at my door if it messes up your environment. Its as-is and without guarantee or warranty. System Requirements to develop SSIS tasks Project scope The purpose of the project is to create a simple zip and unzip control flow task that will run within SSIS. The objective is to teach you the basics of creating a control flow task. Requirements The zip\unzip control flow task should be able to: - Create a zip file and add the contents of a directory including files and sub-directories to the zip file.
- Optionally allow the new zip file to be replaced should it already exist.
- Unzip a zip file including the sub-directories and all files to a target directory structure.
- Optionally allow the target directory files to be replaced should they already exist.
- Return a Success to the SSIS runtime should the zip or unzip work.
- Return a Failure to the SSIS runtime should any error occur.
- Provide an editable dialog box allowing users to configure the control flow.
- Persist properties to the package once the control flow has been added.
- Raise events when the control flow zips or unzips a file
Design The class diagram below describes the classes created for the Unzip control flow 
Properties and fields ZipMode
This property indicates whether the task should create a new zip file or unzip an existing zip file. ZipFilename
The full pathname to the zip file to either create or unzip. TargetDirectory
The full pathname to the directory to unzip files in the zip file to, alternatively the pathname to the folder to scan and add the contents to a new zip file. OverwriteExisting
Indicates whether and existing zip file should be replaced while creating zip. Alternatively it indicates whether the task should overwrite existing files when unzipping. Methods Execute (Task override)
This method either calls StartZipping or StartUnzipping based on the ZipMode. The method also returns wheter the package ran successfully or not. Validate (Task override)
The Validate method reviews properties and settings for inaccuracies or incorrect settings LoadFromXML (IDTSComponentPersist interface)
This recieves and XML element and sets the fields to the attributes of the element. SaveToXML (IDTSComponentPersist interface)
The method creates an XML element and creates attributes for all the fields. StartUnzipping
This method opens up the zip file specified by the ZipFilename property and extracts all content to the TargetDirectory folder. StartZipping
This method creates a zip file using the ZipFIlename property. It then calls AddFolderToZip passing in the TargetDirectory. This directory will be scanned and all files and sub-directories will be added to the zip file. AddFolderToZip
This method is called from the StartZipping method and will search for child directory, the method will recursively call itself passing in the child directory. The AddFolderToZip will then search for all files in the directory and add them to the zip file.
|
-
Strong names are needed should you want to add your assembly to the Global Assembly cache, or use tight version control when loading assemblies. This article shows you how to add a strong name to your project, and then walks you through how to find the Public Key Token. In brief you run sn.exe -T <assemblyname> (note the capital T) - First you need to created a strong name for the assembly, by editing the Project Properties, i.e. Right click on the project in the solution explorer and then click properties.
- In the Choose a strong name key file drop down, choose <New...>
- Fill out the dialog for the strong key. Give your key a name and optionally protect it with a password.
 - Build your project.
- Choose the Visual Studio 2005 Command Prompt from your Start menu
 - Change directory to the bin\Debug project folder using the CD command
- Type sn -T Brett.Demo.SSIS.Tasks.Unzip.dll and specify the name of the assembly.
 - Make a note of the Public key token.
|
More Posts
|
|
|