Ware House Solutions

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Thursday, 26 July 2012

Building an Auditing Process for SSIS - Part 2

Posted on 11:42 by Unknown
In this post I'l continue where I left off last time with the configuration of an SSIS package which allows the population of an audit schema. 

We use this to support a DBA Dashboard showing the progress of ETL operations and as a method of documenting the package mechanism. I've found administering complex dynamic SSIS packages which use parameters and expressions can be time consuming to debug even for relatively simple issues. In my organisation we now use this audit schema to allow the rapid diagnosis of issues and poor performance.

In a future post I'll blog our DBA dashboard which links into this schema and provides us a snapshot of all SSIS processes in much greater detail than can be derived from SQL Agent errors.

Configuration


Each Package requires a connection manager. Keep this distinct from any other connection managers as a dedicated metadata connection. Configure the connection manager with the sufficient security to write to the audit schema in your chosen Utility database.

Control Flow Elements

The package logic should be wholly contained within a Sequence Container with a SQL task as a pre-requisite step which initiates the BatchLog and another SQL task as a final step which closes that Batch.

Initiate Audit - SQL Task General Pane











The SQL statement reads as follows: EXEC Audit.LogPackageStart @BatchLogID = ?,@PackageName = ?,@ExecutionInstanceID = ?,@MachineName = ?,@UserName = ?,@StartDatetime = ?,@PackageVersionGUID = ?,@VersionMajor = ?,@VersionMinor = ?,@VersionBuild = ?,@VersionComment = ?,@PackageGUID  ,@CreationDate = ?,@CreatedBy = ?

Initiate Audit - SQL Task Parameter Mapping Pane









Initiate Audit - SQL Task Result Set Pane










Staying with the Control Flow pane we move onto the configuration of the Audit Termination.

Terminate Audit - SQL Task General Pane



The SQL Statement reads as follows: EXEC Audit.LogPackageEnd @PackageLogID = ?, @BatchLogID = ?,@EndBatchAudit = ?

Terminate Audit - SQL Task Parameter Mapping Pane



As this post is getting long I'll break here as the configuration of the Control Flow pane is now complete. Next time we'll move onto configuration of the Event Handlers.


Email ThisBlogThis!Share to XShare to Facebook
Posted in Auditing, SSIS | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • SQL Bad Practice - Hardcoding
    Legacy systems present design challenges different to problems faced during greenfield development. One mistake I've seen time and again...
  • Data Warehousing Resources List
    Listed here are the best resources I've found for learning about data warehousing, useful techniques and methodologies. In no particular...
  • Big Data in the Real World - Part 2 of 2
    Part 1 of this series described the first 2 stages in the processing of clickstream data to allow sales conversion analysis. This post desc...
  • A 7-Zip Archiving Task for my SSIS Package Template
    I use an SSIS package template to speed up project delivery. 7-zip is my archiving tool of choice because it provides encryption and passwor...
  • Book Review - The Checklist Manifesto
    Periodically you read books which change your habits and make you more effective. The Checklist Manifesto is one such book. Authors who can ...
  • Pragmatic Principals - Agile
    The Data Warehouse exists to provide Business Intelligence. Its users will not always start with the end in mind. They will browse, discover...
  • Big Data in the Real World - Part 1 of 2
    There is so much talk of Big Data nowadays, mostly from sales people but also from technologists. Karen Lopez aka DataChick took issue at ...
  • Refactoring Legacy Databases
    Know your enemy - It might not always seem so, but legacy code was written with a process in mind. Determining what the code was originally ...
  • Building an Auditing Process for SSIS - Part 1
    An audit schema published in the excellent book SSIS Problem– Design – Solution allows the efficient capture of SSIS progress and errors in...
  • Pragmatic Principals - Focus
    The DW is a business system not a techie playground. Use Business Analyst's to focus all development on revenue generating (or saving) w...

Categories

  • 7-Zip
  • Architecture
  • Auditing
  • Bad Practice
  • Big Data
  • Book Review
  • Collations
  • Data Cleansing
  • Data Profiling
  • Data Warehouse
  • DBA
  • Design
  • Disaster Recovery
  • Error Handling
  • ETL
  • OLTP
  • Package Template
  • Principals
  • Professional Skills
  • Project Management
  • Quote
  • Refactoring
  • Replication
  • Resources
  • sFTP
  • SQL Agent
  • SSIS
  • T-SQL
  • WinSCP

Blog Archive

  • ►  2013 (8)
    • ►  September (1)
    • ►  July (1)
    • ►  May (2)
    • ►  March (2)
    • ►  February (1)
    • ►  January (1)
  • ▼  2012 (15)
    • ►  December (1)
    • ►  September (1)
    • ►  August (1)
    • ▼  July (3)
      • Building an Auditing Process for SSIS - Part 2
      • Building an Auditing Process for SSIS - Part 1
      • ETL Cleansing : Remove Whitespace
    • ►  June (2)
    • ►  April (1)
    • ►  March (2)
    • ►  February (3)
    • ►  January (1)
Powered by Blogger.

About Me

Unknown
View my complete profile