Wednesday, March 7, 2012

Integration Services: ?Table refresh (UPDATE/INSERT)“

Hello

I have a question about the new Integration Services of the MS SQL Server 2005.

Situation:
- SQL Server 2005 (standard edition)

- 2 tables with identical structure (same attributes)

- the table ?TestSource“ will be constantly extend (new records & updates).

- the table ?TestDestination“ will just be refreshed by SSIS (Data Warehouse table)


I would like to create a Integration Service, witch refreshes the table ?TestDestination“ with the data from table ?TestSource“.

Existing records (ID already exists) should be updated (UPDATE), not existing records should be created (INSERT).

I would like to use the IS Data Flow Task, because in future i won’t just copy the data. I also will use Toolbox items like ?Data Conversion“, ?Derived Column“ and so on.

Alike I won’t use an easy SQL-Query, because it would be complicated to make changes and to Log the transactions.

Just clear and refill the whole table is not possible because of performance and availability requests (large data).


Question:
How can I implement this workflow as Data Flow in a Integration Service?
Witch components from the Toolbox do I need?

Greetings

Not sure if you can use this within SSIS, but the tablediff.exe that comes with sql server 2005 is pretty slick. I wrote a c# wrapper for it and I have a config file that holds all the table names that I want to sync as well as the source and destination servers. It then generates the sql change file as well as gives output of the number of rows that are out of sync. It works really well for me especially using the C# wrapper that I wrote.

Just a thought....

No comments:

Post a Comment