Change Data Capture Tables



A Change Data Capture (do not confuse it with Change Tracking) is a solution for datawarehousing applications when you want to perform incremental data loads on DW databases.
This info is stored in special system tables created stored in each database where this option is set.

This option is set at database level and you must use these stored procedures to set this property on the affected database:


EXEC sys.sp_cdc_enable_db --Enable CDC
EXEC sys.sp_cdc_disable_db --Disable CDC

Then you enable it for an specific table using the following stored procedure, using the standard schema.table convention:


EXEC sys.sp_cdc_enable_table
       @source_schema = '<schema_name>'
       ,@source_name = '<table_name>'
       ,@role_name = '<role>'

For simple applications, these options are fairly enough. Once enabled you can use SSDT to handle the data to suit your needs, using the CDC Source and CDC Split Transformation inside a Data Flow task:

"Sample" usage


CDC Source validates that the table you select is CDC enabled, while CDC Splitter splits each operation based on the __$operation Column on the table.


References:

https://docs.microsoft.com/en-us/sql/relational-databases/system-tables/change-data-capture-tables-transact-sql

https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/enable-and-disable-change-data-capture-sql-server

https://docs.microsoft.com/en-us/sql/integration-services/data-flow/cdc-flow-components

Comments

Post a Comment

Popular posts from this blog

Install Python on Windows

Image
Python is the most famous development language today. This is because is open-source, scalable and robust. It can run in almost any device with a processor (or microprocessor, we will see that in coming posts) and is very simple to use and learn, even if you don't have any programming backgroud or experience. Start using it is very simple and you can learn the basics in the official site .   Installing Python For windows systems, you can download the latest version here . Once downloaded, open the executable file as administrator. Then, for the uses we will give on the articles on this site, please install it this way: Make sure you have selected the add Pyton to PATH option - This will allow us to run the Python engine from multiple sources (As is intended for use on multiple applications) .   Make sure you install Python for all users - This will allow usto run the script commands .   Select PIP feature - This is used to install adittional libraries and functionalities, one of
Read more

Checking Azure SQL DB service tier via T-SQL

Image
If you have to manage or work with an Azure SQL database and want to know what service tier the database is currently operating, but you don't have access to the Azure subscription or CLI. Or you want to know the status of the service tier after a scale up or scale down, directly from the database; you can do it via T-SQL Just query the sys.database_service_objectives DMO to obtain this information, this will give information about the service tier, and also will tell you if the database is part of an elastic pool or not. Basic usage (in the context of the database you need the information): SELECT * FROM sys.database_service_objectives; This will return the following information for the current database: I am running on a General Purpose tier with 1 VCore What if you want the information for all the databases created? Just change the context to the master database and execute the following query: SELECT D.name AS DB , D.create_date , SO.edition , SO.service_o
Read more

Quick tip: Zoom in Azure Data Studio

Image
If you finally have given a try t o Azure Data Studio , and if you use it on a regular basis, maybe you want to customize it to suit your needs. Among the huge customization options it has, you can control the text size in form of zoom. To do change it, just use the following keyboard combinations: (Ctrl + = ) For Zoom in. (Ctrl + - ) For Zoom out. (Ctrl + 0) For Zoom reset.  You can see it in action: Also, as with everything on this tool, you can access this functionality from command palette (Ctrl + Shift + P) , and type "zoom" (you can fine-control the editor zoom from here): If you haven't tried it yet, you can download Azure Data Studio here .
Read more