SSIS ETL Framework v1 goes Beta!
A quick update on the status of the BI Monkey SSIS ETL Framework (on Codeplex @ http://ssisetlframework.codeplex.com/)
The Framework v1 has gone into Beta – i’ve completed all the testing, and now just need to tidy up the reports and add some extra logging capability – but it is now fully functional. I’ve left it in Beta as I want to get some feedback on it before I move it live, plus fix those small details I mentioned.
So now I will press on with updating the documentation (yes, really!) and start laying the foundations for the more Enterprise level v2 framework.
I look forward to your feedback – please take advantage of Codeplex’s issue logging functionality to help me manage bugs and improvements.
AUSSUG Followup
After the great session at AUSSUG on Tuesday, I thought i’d share the presentation materials, so here’s my powerpoint slides. I enjoyed the shorter, more interactive format.
I’ve also been busy updating the documentation for the BI Monkey SSIS ETL Framework on the codeplex site, and have uploaded a fresh alpha release – which is significantly more tested and reliable – it’s already had quite a few downloads so i’m looking forward to some feedback soon.
The BI Monkey SSIS ETL Framework
I have, after at least 3 incarnations of building SSIS based ETL control frameworks, decided to do a fourth and (possibly) final one – for all to share. The alpha release of this is available now on CodePlex @ http://ssisetlframework.codeplex.com. I’ve put an aplha up, rather than a better tested Beta because the youngest BI Monkey in my household appears to be unhappy being a baby right now and is taking me away from this side project.
It’s fairly standard stuff from my point of view – it’s metadata driven framework that consists of a single control package and a template execution package. I’ve aimed to include the following features, all of which can be turned on or off in the metadata without altering any code.
- Recoverability
- Extraction constraints
- Execution order
- Dependencies
- Failure handling
The SSIS packages are pretty much a simple interface for where the meat of the decision making is occurring – in a bundle of stored procedures where it’s a lot easier to write and maintain complex decisions about processing than in SSIS.
I’m adhering to my own self-set principles on usability as well, namely to hit these targets:
- High visibility of operation via reports
- Fully commented code
- No acronynms in table and field names
- Any codes used exposed in a metadata table
I will be doing a demo of it at the AUSSUG session in Sydney this coming Tuesday as part of the SSIS round table which is taking place:
James Beresford: ETL Frameworks
Design and Implementation considerations when building ETL Control Frameworks, Including the debut demo of the BI Monkey ETL Framework
Kevin Wong: Practical ADO. NET
Three practical showcases using ADO.NET in SSIS including re-usable in memory resultsets
Glyn Llewellyn: Putting the T back into ETL
Looking at alternative ways in SSIS to perform data transformations and correction and discussing the advantages and disadvantages of each method
(Full details at the AUSSUG website)
I hope to see you there!