Flat File Sources and the Decimal Data Type

Question: What Data Type should you use for importing a column in Flat File containing Decimal data?

Answer: numeric [DT_NUMERIC]

You cannot use the decimal [DT_DECIMAL] type, because in the Advanced Editor of the Flat File connection, the decimal type for some reason only allows you to set the Scale (the number of digits after the decimal point) – the Precision is greyed out (precision is the total number of digits). The numeric data type allows the setting of both values.

Fortunately the SSIS numeric type maps to SQL Server decimal columns without complaint, so you don’t have to add a Data Conversion to change numeric to decimal before using it. I have raised a bug on Connect – please vote it up if you consider this worth fixing.

Read More

The Data Conversion Transformation

b
Fig 1: The Data Conversion Transformation

In this post I will be covering the Data Conversion Transformation. The sample package can be foundĀ here for 2005 and guidelines on use are here.

What does the Data Conversion do?

The transformation takes input columns and creates a copy of that column with a new data type. This is a vital component in SSIS because SSIS is very fussy about Data Types – you cannot force an integer typed column into a string destination. It should go – in theory – but this strictness prevents sloppy errors and more importantly allows SSIS to be truly platform independent by ensuring its internal data types can be reliably matched to the target.

Configuring the component is pretty simple – you pick your input column, set the name of the output and pick the target data type. Depending on the target data type you may have other configuration options such as precision, scale, length or code page. The snapshot below is taken from the sample package.

b
Fig 2: Configuring the Data Conversion Transformation

An important thing to be aware of is how the Data Conversion transform handles missing data. A blank field will return an error when you try to convert it to pretty much anything but a string – if you have missing values that you want to sail through this component, you must convert them to NULLs first, as a NULL can be converted (to the appropriate NULL) but a blank string is technically a value, and one that cannot be validly converted. It would be good if there was an option to treat blanks as nulls, but for now there isn’t, so be careful. There is a demonstration of this in the sample package in Data Flow 2.

When would you convert Data to a new Type?

The need for this kind of change is manifold – two of my most common uses would be converting data from text files to the correct types – from strings into dates, numbers, currency etc. – and when moving data between relational systems. While an ADO .NET source may give you data as one general type – say an int, you may want to put it in a different type field for an SQL Server database, such as a tinyint.

MSDN Documentation for the Data Conversion Transformation can be found here for 2008 and here for 2005.

Read More

SSIS Data Types

This is a useful guide on MSDN for mapping SSIS Data Types to that of various databases.

http://msdn.microsoft.com/en-us/library/ms141036.aspx

The top part is an explanation of each SSIS data type. Further down is a mapping of various databases (notably SQL Server, Oracle and DB2) data types to SSIS types.

Brian Knight has a nice grid mapping SSIS Types to SQL Server which is a little easier to read for those working in a pure SQL Server environment.

Read More