Ware House Solutions

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Saturday, 22 September 2012

A 7-Zip Archiving Task for my SSIS Package Template

Posted on 14:21 by Unknown
I use an SSIS package template to speed up project delivery. 7-zip is my archiving tool of choice because it provides encryption and password protection in addition to good compression. It has all the scripting options you'd expect in the command line version which makes it easy to use with SSIS. Documentation for command line scripting can be found here.
I call 7-ZIP from an Execute Package Task (EPT). This requires the 7-Zip executable to be present on the server where the SSIS package executes. The standard 7-Zip executable (7z.exe) doesn't work as a command line tool. Instead you need to use the 7za.exe. Both can be downloaded here.

In my SSIS Template I have a preconfigured EPT task with expressions constructed to build the complex strings required by the command line tools from SSIS variables. These variables can be set for simple recurring packages or with minor amendments populated from configuration values.

Configuring the Task

I always name each task using a standard acronym for the task followed by a brief description. This makes debugging much easier both during development using the package explorer pane and once the package is live as all the packages created using my template log to an audit schema. This audit can be invaluable when searching for a failure within a complex process. In addition, the log allows reporting on the duration of each step which aids performance tuning.

The screenshot below shows the general pane for the Execute Process Task used in this example.


Next follows the all important Process pane. This is where configurations are set. Or is it? With complex strings required as command line parameters it is better to configure the task dynamically using expressions and variables.


In this version on my package template I set only two values, the Arguments and Working Directory. I probably should have set the Executable location via a variable as well but I guess that'll wait for the next version.

The 3 string variables used are;
  • [User::ZipFileName]
  • [User::ZipFileLocation]
  • [User::ZipPassword]


The most complex expression is that which sets the commend line arguments, shown here;
"a -tzip " + @[User::ZipFileName] + "_" + (DT_STR,4,1252)YEAR(GETDATE()) + RIGHT("0" + (DT_STR,2,1252)MONTH(GETDATE()),2) + RIGHT("0" + (DT_STR,2,1252)DAY(GETDATE()),2) + ".zip *.csv -P" +  @[User::ZipPassword]
And this results in the following string; 
a -tzip DataExport_20130822.zip *.csv -PPa55word!

In Summary

This is a useful addition to my SSIS package template which I use regularly. I don't believe in reinventing the wheel, instead I prefer simple modular solutions I can combine to make more complex packages. 

The time taken to establish a template is well worth the investment. I'm often very busy in my role and I prefer to focus my limited time and brain cycles on higher level problems than configuring an SSIS task. I recommend building your own template or using mine which I'll be sharing shortly.

Check out my WinSCP configure task also used as part of my SSIS package template.
Read More
Posted in 7-Zip, Package Template, SSIS | No comments
Newer Posts Older Posts Home
Subscribe to: Posts (Atom)

Popular Posts

  • SQL Bad Practice - Hardcoding
    Legacy systems present design challenges different to problems faced during greenfield development. One mistake I've seen time and again...
  • Data Warehousing Resources List
    Listed here are the best resources I've found for learning about data warehousing, useful techniques and methodologies. In no particular...
  • Big Data in the Real World - Part 2 of 2
    Part 1 of this series described the first 2 stages in the processing of clickstream data to allow sales conversion analysis. This post desc...
  • A 7-Zip Archiving Task for my SSIS Package Template
    I use an SSIS package template to speed up project delivery. 7-zip is my archiving tool of choice because it provides encryption and passwor...
  • Book Review - The Checklist Manifesto
    Periodically you read books which change your habits and make you more effective. The Checklist Manifesto is one such book. Authors who can ...
  • Pragmatic Principals - Agile
    The Data Warehouse exists to provide Business Intelligence. Its users will not always start with the end in mind. They will browse, discover...
  • Big Data in the Real World - Part 1 of 2
    There is so much talk of Big Data nowadays, mostly from sales people but also from technologists. Karen Lopez aka DataChick took issue at ...
  • Refactoring Legacy Databases
    Know your enemy - It might not always seem so, but legacy code was written with a process in mind. Determining what the code was originally ...
  • Building an Auditing Process for SSIS - Part 1
    An audit schema published in the excellent book SSIS Problem– Design – Solution allows the efficient capture of SSIS progress and errors in...
  • Pragmatic Principals - Focus
    The DW is a business system not a techie playground. Use Business Analyst's to focus all development on revenue generating (or saving) w...

Categories

  • 7-Zip
  • Architecture
  • Auditing
  • Bad Practice
  • Big Data
  • Book Review
  • Collations
  • Data Cleansing
  • Data Profiling
  • Data Warehouse
  • DBA
  • Design
  • Disaster Recovery
  • Error Handling
  • ETL
  • OLTP
  • Package Template
  • Principals
  • Professional Skills
  • Project Management
  • Quote
  • Refactoring
  • Replication
  • Resources
  • sFTP
  • SQL Agent
  • SSIS
  • T-SQL
  • WinSCP

Blog Archive

  • ►  2013 (8)
    • ►  September (1)
    • ►  July (1)
    • ►  May (2)
    • ►  March (2)
    • ►  February (1)
    • ►  January (1)
  • ▼  2012 (15)
    • ►  December (1)
    • ▼  September (1)
      • A 7-Zip Archiving Task for my SSIS Package Template
    • ►  August (1)
    • ►  July (3)
    • ►  June (2)
    • ►  April (1)
    • ►  March (2)
    • ►  February (3)
    • ►  January (1)
Powered by Blogger.

About Me

Unknown
View my complete profile