Integrated Security

Friday, March 30, 2012

Internal Activation Best Practices?

I am looking for an example of a SP that shows the best practices for internal activation? In BOL this topic describes the typical patter for reading messages from a queue. What is the typical pattern for reading messages from a queue using an internally activated SP? Do we still need to loop (considering the message arrival actually fired the sp)?

Any advice provided would be helpful.

Thanks!

Yes, you should still loop. If you don't loop, you risk getting at maximum only one activated procedure, instead of the value set for max_queue_readers. If a procedure is launched and it exits imedeatly (expecting to be launched again), the activation algorithm timers will be reset by this event, so the activation will never reach the point when it has to launch a new instance of the procedure.

The Activation algorithm is described here: http://msdn2.microsoft.com/en-us/library/ms171601.aspx

Another important reason why you should loop is the need to batch commit. If your procedure RECEIVEs one message, process it and commits, you won't be able to scale. Even the best tuned system will only be able to process only tens of messages per second, maybe a best a few hundred. You have to batch several tens, even hundreds, of messages in one transaction, to be able to process thousands of messages per second. The trick here is to avoid holding up a transaction in wait for a message. That is, if you have a batch pending, do a plain RECEIVE, not a WAITFOR(RECEIVE). If the RECEIVE returns empty, commit the pending batch and only then do the WAITFOR(RECEIVE).

Do linger few seconds in your loop before exiting. That is, have a WAITFOR (RECEIVE), TIMEOUT 1000 (or 2000) to drive the loop. This way, the activated procedure will be already started if a new message arrives quickly after you just drained the queue. Don't linger to much, as you're keeping a server thread occupied.

Don't fake an empty queue. That is, don't have a RECEIVE ... WHERE that will return empty rowset even when there are messages in the queue. This will fool the activation algorithm to think that you drained the queue, and will prevent new instances of the procedure to be launched.

Activated procedures have a different execution context than user connection launched procedures. They are under an EXECUTE AS USER context, thus under all the restrictions described here: http://msdn2.microsoft.com/en-us/library/ms188304.aspx. So try to design the procedure avoiding the need to access resources outside the database. If you must, then you'll have to either turn on the trustworthy bit on the database, or folllow these complex steps: http://blogs.msdn.com/remusrusanu/archive/2006/03/07/545508.aspx

Do make sure your procedure handles the system message types:
http://schemas.microsoft.com/SQL/ServiceBroker/Error
http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog
http://schemas.microsoft.com/SQL/ServiceBroker/DialogTimer

Don't rollback intentionally in your procedure. It will trigger the poison message detection and disable your queue.

Do experiment with various ways of writing the RECEIVE loop, e.g.:
- TOP (1) into T-SQL variables
- INTO @.tableVariable and open cursor over @.tableVariable
- CLR procedure and RECEIVE a whole resultset into a SqlDataReader
See which one gives the bets performance for your environment. Some are better when you're likely to have always one message returned, some are better when you're likely to have many messages returned by RECEIVE. Consider that RECEIVE returnes only messages on one conversation_group.

HTH,
~ Remus

|||

I has been brought to my attention that one of my comments is wrong:

If a procedure is launched and it exits imedeatly (expecting to be launched again), the activation algorithm timers will be reset by this event, so the activation will never reach the point when it has to launch a new instance of the procedure.

This is incorrect, the exiting the procedure does not reset the timer and the activation algorithm works as expected in these conditions: the max_queue_readers is reached.

HTH,
~Remus

|||Great info Remus!

Internal Activation - calls stored procs in other DBs

Hi all

I am using internal activation on a queue to process the messages, should an error be encountered I call stored procedure A in the same database to log the error. Part of the processing in stored procedure A is a call to stored procedure B in another database (on the same server), however I have not been able to get this call to B to work. Currently I get the error "The server principal XXXXXX is not able to access the database YYYYYYY under the current security context".

I have tried various combinations (too many to remember) of database owners, roles and permissions as well as EXECUTE AS on both A and B and the Queue but none seem to work. Can anyone give me simple example of a setup which would allow this cross database call to work?

Thanks

Ian

You are hitting the 'Extending database impersonation under EXECUTE AS context' issue. I have a series of posts in my blog tackling this problem:

http://blogs.msdn.com/remusrusanu/archive/2006/03/07/545508.aspx
http://blogs.msdn.com/remusrusanu/archive/2006/03/01/541882.aspx
http://blogs.msdn.com/remusrusanu/archive/2006/01/12/512085.aspx

The first link is posted today and is an actual example on how to call a procedure in another database under activation.

|||

Thanks for the information - it's just what I was looking for.

Ian

|||

One more question....

Is it essential that the owner of the other DB is the same as the owner of the activated stored proc?

I don't seem to be able to get this to work if they are different.

Thanks

Ian

|||You can use any user with receive permission on the queue.|||I think you must grant AUTHENTICATE permission on the 'other' DB to the user from the EXECUTE AS clause of the CREATE/ALTER procedure. If the EXECUTE AS is OWNER, then to the owner of the activated procedure)

Intermittently Slow query - Left Join

Here's a little background on the query. I have a list of documents by an id number in one table and the description of the sheets in another table. It's a one to many relationship, so for each description, there may be multiple entries in the documents table that it applies to. For example:

Descriptions table:

ID | Title
Doc1 | Document 1
Doc2 | Document 2

Documents table:

ID | Parent
Doc1 | 10400
Doc2 | 10400
Doc1 | 20189
Doc3 | 20189

View:

ID | Parent | Description
Doc1 | 10400 | Document 1
Doc2 | 10400 | Document 2
Doc1 | 20189 | Document 1
Doc3 | 20189 | (null)

So the query I am using uses a left join to combine the data from the one table into the other. There might not be an entry for the description, so for some Document entries, the description field may be blank. For some reason, certain queries take about 2 minutes longer than others who retreive 5 times the information.

In SQL Manager, is says "Executing Query. Waiting for response from data source." After about 20 seconds it says "Retrieving Data..." then about a minute later, it finally comes up with the data. I can select another parent that has a lot more items and it comes up in about 3 seconds max.

It's running on SQL Server 2005 with 2GB of RAM.

Any suggestions on tracking down the reason for the slowness would be great.

Thanks in advance!!!

-DanPost the query you are executing.|||SELECT
dbo.Table1.ItemType,
dbo.Table1.ItemLabel,
dbo.Table1.ItemParent,
dbo.Table1.ItemID,
dbo.Table1.ProjectID,
dbo.Table1.ItemDate,
dbo.Table1.Active,
dbo.Table1.ItemBaseLabel,
dbo.Table2.ItemDescription
FROM dbo.Table1 LEFT OUTER JOIN dbo.Table2
ON dbo.Table1.ProjectID = dbo.Table2.ProjectID
AND dbo.Table1.ItemBaseLabel = dbo.Table2.ItemBaseLabel

Here you go!

-Dan|||where are the indices placed? have you looked at the execution plan?|||That's weird... I just checked the indices on the base table and there are three. One is By ItemID. Another is by ItemLabel and another is by ItemLabel, ItemParent, Item ID, and another field. Could they be throwing off the way the data is being stored?

There are no indices on the view and I cannot add one. It comes up with an error saying that it can't add an index because it's not schema based. Please, bear with me - I'm new to 2005. This was originally stored on MSSQL 7.0 and it worked fine there.

Thanks!

-Dan|||typically you want indices on search conditions, primary keys (there is usually a clustered one there) and the foriegn keys but you should examine the execution plan first to make sure this will help. if they are already there, make sure they are not fragmented, the statistics are up to date, and then if this is a sp, recompile it. I am not working with 2K5 just yet.|||I checked the indices and everything seems to be in order. I also tried it through a stored procedure and it still takes way too long.

Any other ideas?

-Dan|||for the third time... have you looked at the execution plan?|||Sorry. The execution plan said that a table scan on the dbo.Table2 was 100% cost. So I looked at the fields it was referencing and added a column to the Table2 index and then re-ran the execution plan and it was more divided among the tasks, now with 2 index scans.

Went back to the query and now it's blazing fast.

Apparently my listening skills are about as good as my databasing skills...

Thanks Thrasymachus. :)

-Dan

intermittently my SSIS Packages which are run through the JOB are failing

Hi SSIS Guru's,

I am facing a very strange problem.

We have 2 Physical Servers (A and B) on which we have installed the SQL Server, one primary (A) and other as secondary (B). And there is a cluster (C) available to acces the running server. I have created some SSIS packages which we installed on the Server A (Primary), and created the job on the cluster server which initiates the SSIS packages, whcih are installed in the File System.

The problem i am facing is the some thing related to Connection time out. And interestingly i am not getting this error Always. Approxiamtely For Every 5 Times once it;s Failing. I am copying the errors Which i encountered in the different runs.

The thing i am confused is why i am not geting the error all the time? And Why am i getting this error all the time in a different data flow task. My SSIS Package structure is I have created one master package and 6 Child packages. I am getting the connection string for the Data base from the Configuration file which is defined in the XML File.

The connection string that i am using is

Data Source=<<server name>>;User ID=DOMAIN\user;Initial Catalog=DatabaseName;Provider=SQLNCLI.1;Integrated Security=SSPI;

*************************************************************************************************************************************

RUN 1 - Error
Executed as user: AMR\sys_calyp. ...sion 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights

reserved. Started: 2:00:07 PM Error: 2007-09-15 14:02:35.92 Code: 0xC0202009 Source: ssis_emp Connection manager

"DBCONNECTION" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An

OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Unable to complete

Source: infr_char Get the Records from emp 1 [72] Description: SSIS Error Code

DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager

"DBCONNECTION" failed with error code 0xC0202009. There may be error messages posted before this with more information on

why the AcquireConnection method... The package execution fa... The step failed.

*************************************************************************************************************************************

RUN 2 - Error
Message
Executed as user: AMR\sys_calyp. ...sion 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights

reserved. Started: 9:15:01 AM Error: 2007-09-15 09:17:01.64 Code: 0xC0202009 Source: ssis_emp Connection manager

"DBCONNECTION" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An

OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Unable to complete

Source: Data Flow Task Get the Records from emp [473] Description: SSIS Error Code

DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager

"DBCONNECTION" failed with error code 0xC0202009. There may be error messages posted before this with more information on

why the AcquireConnection me... The package execution fa... The step failed.

*************************************************************************************************************************************

Run -3 Error
Message
Executed as user: AMR\sys_calyp. ...sion 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights

reserved. Started: 11:30:01 PM Error: 2007-09-14 23:32:21.28 Code: 0xC0202009 Source: ssis_dept Connection

manager "DBCONNECTION" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code:

0x80004005. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Unable

to complete login process due to delay in opening server connection". End Error Error: 2007-09-14 23:32:21.28 Code:

0xC020801C Source: Data Flow Task Get the Records from dept [632] Description: SSIS Error Code

DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager

"DBCONNECTION" failed with error code 0xC0202009. There may be error messages posted before this with more information on

why the AcquireConnection method ... The package execution fa... The step failed.

*************************************************************************************************************************************

Run - 4 Error

Message
Executed as user: AMR\sys_calyp. ...sion 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights

reserved. Started: 11:00:02 PM Error: 2007-09-14 23:02:21.46 Code: 0xC0202009 Source: ssis_emp Connection

manager "DBCONNECTION" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code:

0x80004005. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Unable

to complete login process due to delay in opening server connection". End Error Error: 2007-09-14 23:02:21.46 Code:

0xC020801C Source: infr_itm_char_val Get the Records from emp_master [1] Description: SSIS Error Code

DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager

"DBCONNECTION" failed with error code 0xC0202009. There may be error messages posted before this with more information on

why the AcquireCon... The package execution fa... The step failed.

*************************************************************************************************************************************

Run -5 Error

Message
Executed as user: AMR\sys_calyp. ...Execute Package Utility Version 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp

ssis_salgrade Connection manager "DBCONNECTION" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred.

Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005

Description: "Unable to complete login process due to delay in opening server connection". End Error Error: 2007-09-14

21:12:23.25 Code: 0xC020801C Source: Data Flow Task - ssis_salgrade get salgrade [3227] Description: SSIS Error

Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager

"DBCONNECTION" failed with error code 0xC0202009. There may be error messages posted before this with more information on

why the AcquireConnection method c. The step failed.

*************************************************************************************************************************************

I thought i found the solution my self... I changed the delay validation to all the packages to true and the problem dissappeared Smile