Tag Archives: #SQLServer

All about Microsoft SQL Server

SQL Server – Profiler – Part 6 – Correlating the Profiler Trace with Windows Performance Log Data


Based on popular demand from my colleagues and you, the kind reader, I am currently writing a series of posts on the SQL Server Profiler. Five parts have been published till date, and for your kind reference, here are the links to them:

  1. What is a SQL Trace? Permissions, space requirements and launching the Profiler
  2. Profiler Templates, Template Types & creating custom templates
  3. Trace execution options – Save trace to file or a table, Auto-scroll and keyboard shortcuts
  4. Review T-SQL code to identify objects no longer supported by Microsoft – Deprecation Event
  5. Replaying a trace – from a file or a table – using breakpoints during trace replay

Today, in the last part of the series, we will be looking at another important application of the SQL Server Profiler – Correlating the Profiler Trace with Windows Profiler Log Data. This is helpful when you, as an IT or database administrator suspect that SQL Server is either being held up by or is the cause of an I/O, memory or processor contention.

Windows Reliability & Performance Monitor Log Data

Windows Reliability and Performance Monitor is a Microsoft Management Console (MMC) snap-in that combines the functionality of previous stand-alone tools including Performance Logs and Alerts, Server Performance Advisor, and System Monitor. The tool of interest to us is the Performance Logs and Alerts, erstwhile known as “Perfmon”.

Because this is a post on the SQL Server Profiler, demonstrating the capture of data using the Windows Reliability and Performance Monitor is out of scope. You may refer the References section for more details on the Windows Reliability and Performance Monitor.

For this test, I have created a Performance trace of the following counters (because the goal of this exercise was just to have a demonstration, very basic counters have been chosen. Under “live” circumstances, the counter range would be different):

  • Total Disk – Reads/sec
  • Total Disk – Writes/sec
  • Total Disk – Current Queue Length
  • SQL Server Buffer Manager – Page reads/sec
  • SQL Server Buffer Manager – Page writes/sec

SQL Server Profiler Trace Data

Using the custom trace developed in earlier editions of this series, capture a SQL Server Profiler Log for the following script. Notice that the script has been designed to have a distinct set of read & write activities:

/**********************************************
           !!!! WARNING !!!!
THIS SCRIPT IS PROVIDED AS-IS AND WITHOUT
WARRANTY. THE AUTHOR AND BEYONDRELATIONAL.COM
ARE NOT RESPONSIBLE FOR ANY DAMANGE CAUSED BY
THE USE OF THIS SCRIPT.
**********************************************/
USE tempdb
GO
CREATE TABLE DefaultTableTest
( DataId INT IDENTITY(1,1),
  RandomId INT DEFAULT (RAND()*1000),
  DataName VARCHAR(10) DEFAULT 'DEFAULT'
)
GO

/******************************************
   Insert some test data into the table
- This should generate lots of writes
******************************************/
USE tempdb
GO
INSERT INTO DefaultTableTest DEFAULT VALUES
GO 20000

/******************************************
   Fetch the test data
- This should generate lots of reads!
******************************************/
USE tempdb
GO
SELECT * FROM DefaultTableTest
GO
SELECT * FROM AdventureWorks2008R2.HumanResources.Employee
GO
--Wait for some time
WAITFOR DELAY'00:00:20'
--Regenerate some activity
USE tempdb
GO
SELECT * FROM DefaultTableTest
GO
SELECT * FROM AdventureWorks2008R2.HumanResources.Employee
GO

It is important to remember that the time ranges for the Windows Performance Monitor Log and SQL Server Profiler Trace log must overlap for the Profiler to be able to correlate them.

Correlating the Profiler Trace with Windows Performance Log Data

Once the data has been collected, now is the time to correlate the Profiler trace with the Windows Performance Log data. Below are the steps that you can use to do so:

In the SQL Server Profiler, open the saved Profiler trace file image
On the SQL Server Profiler File menu, click Import Performance Data image
In the Open dialog box, select a file that contains a performance log.

In the Performance Counters Limit dialog box, select the check boxes that correspond to the System Monitor objects and counters that you want to display alongside the trace. Click OK.

image

Notice that the performance monitor graph came out to be exactly as expected

  • Lot of Page writes in the beginning
  • Very small period of disk reads and buffer page reads towards the end
  • Note that disk writes continue in an even fashion throughout – indicating disk writing by the SQL Profiler and the Performance monitor
image

Select an event in the trace events window, or navigate through several adjacent rows in the trace events window by using the arrow keys.

The vertical red bar in the System Monitor data window indicates the performance log data that is correlated with the selected trace event.

image
Click a point of interest in the System Monitor graph.

The corresponding trace row that is nearest in time is selected.

Once the graph receives the mouse click, you can also use the right & left arrow keys to navigate

image
To zoom in on a time range, press and drag the mouse pointer (as if you are selecting a section of an image) in the System Monitor graph image

NOTES:

  1. The time ranges of the Profiler & Windows Performance Log data must overlap
  2. For accurate correlation with System Monitor data, the trace must contain both StartTime and EndTime data columns
  3. Correlation of the Profiler & Windows Performance Log data is not possible for a running trace that is still active & collecting data

References:

I hope you liked reading all 6 parts of my series on the SQL Server Profiler. Do leave your feedback – I really appreciate and thank-you for taking the time out of your busy day to read my posts.

Until we meet next time,

Be courteous. Drive responsibly.

Subscribe to my posts via E-mail: Subscribe here | Read my posts via your favourite RSS reader: Click Here!

SQL Server – Profiler – Part 5 – Replaying a trace – from a file or a table – using breakpoints during trace replay


Based on popular demand from my colleagues and you, the kind reader, I am currently writing a series of posts on the SQL Server Profiler. Four parts have been published till date, and for your kind reference, here are the links to them:

  1. What is a SQL Trace? Permissions, space requirements and launching the Profiler
  2. Profiler Templates, Template Types & creating custom templates
  3. Trace execution options – Save trace to file or a table, Auto-scroll and keyboard shortcuts
  4. Review T-SQL code to identify objects no longer supported by Microsoft – Deprecation Event

Today, we will be looking at another important application of the SQL Server Profiler – Replay. From Books-On-Line:

“Replay is the ability to save a trace and replay it later. This functionality lets you reproduce activity captured in a trace. When you create or edit a trace, you can save the trace to replay it later.

SQL Server Profiler features a multithreaded playback engine that can simulate user connections and SQL Server Authentication. Replay is useful to troubleshoot an application or process problem. When you identify the problem and implement corrections, run the trace that found the potential problem against the corrected application or process. Then, replay the original trace and compare results.”

To demonstrate replay, let’s the following piece of code, which is rigged to fail. (There is no particular reason why I am using a code that is rigged to fail). I used this piece of code in my posts – Exception handling in T-SQL/TRY…CATCH – Underappreciated features of Microsoft SQL Server and Sunset for RAISERROR and sunrise for THROW – SQL 11 (“Denali”).

IF EXISTS (SELECT * FROM sys.objects WHERE name = 'InnerProc' AND type = 'P')
    DROP PROCEDURE InnerProc
GO

CREATE PROCEDURE InnerProc
AS
BEGIN
    BEGIN TRANSACTION ExceptionHandling
       BEGIN TRY
          PRINT 'In the TRY block of the Inner Procedure...'
          SELECT 1/1

          RAISERROR('An error occured in the Inner procedure.',17,1)  --Line #10 considering CREATE PROC...as Line #1

          COMMIT TRANSACTION ExceptionHandling
       END TRY
       BEGIN CATCH
          SELECT ERROR_NUMBER() AS ErrorNumber
                ,ERROR_SEVERITY() AS ErrorSeverity
                ,ERROR_STATE() AS ErrorState
                ,ERROR_PROCEDURE() AS ErrorProcedure
                ,ERROR_LINE() AS ErrorLine
                ,ERROR_MESSAGE() AS ErrorMessage;

          IF @@TRANCOUNT > 0
             ROLLBACK TRANSACTION ExceptionHandling

          PRINT 'Throwing error from the CATCH block of the INNER Procedure...';   
          --Preceding statement MUST be a semi-colon ';'
          THROW
       END CATCH
END
GO

IF EXISTS (SELECT * FROM sys.objects WHERE name = 'OuterProc' AND type = 'P')
    DROP PROCEDURE OuterProc
GO
CREATE PROCEDURE OuterProc
AS
BEGIN
    BEGIN TRY
        PRINT 'In the TRY block of the Outer Procedure...'
        EXEC InnerProc
    END TRY
    BEGIN CATCH
        PRINT 'In the CATCH block of the Outer Procedure...';
        --Preceding statement MUST be a semi-colon ';'
        THROW
    END CATCH
END
GO

--Executing the outer procedure
EXEC OuterProc

Capturing the Replay trace

The SQL Server Profiler ships with a default “TSQL_Replay” trace. For this demo, we will be using this trace type.

image

As demonstrated in the previous posts of the series, capture the trace for the T-SQL script under review.

Replaying the trace

Open the Trace file saved during the trace collection image
Notice the changes to the Profiler toolbar image
In case one needs to stop execution at a certain step/statement, select the statement and hit F9 to set a breakpoint image
Hit F5 or the yellow arrow key to start the Replay.

Apply the required Replay options.

 image image
Once the options are set, click on OK. Notice that the trace replays till the breakpoint is hit.

Alternatively, one can also use the F10 key to step-through the replay trace one step at a time or the (Ctrl + F10) combination to run to a selected cursor position.

image

Notice that during the replay options configuration, you can choose whether or not to have the SQL Server blocked process monitor running. This is especially useful if you are trying to replay a trace that you suspect is involved in heavy locking/blocking on your server.

Also notice the status bar, which now shows the number of open connections and the percentage of trace that has finished the replay.

In my next post…

The next post will be the final one of the series, in which I will discuss how to correlate performance counters with the Profiler data.

If you have suggestions on any applications of the Profiler that you want me to cover, drop me a note, and I will try to cover them in future posts.

References

Until we meet next time,

Be courteous. Drive responsibly.

Subscribe to my posts via E-mail: Subscribe here | Read my posts via your favourite RSS reader: Click Here!

SQL Server – Profiler – Part 4 – Review T-SQL code to identify objects no longer supported by Microsoft – Deprecation Event


Based on popular demand from my colleagues and you, the kind reader, I am currently writing a series of posts on the SQL Server Profiler. Three parts have been published till date, and for your kind reference, here are the links to them:

  1. What is a SQL Trace? Permissions, space requirements and launching the Profiler
  2. Profiler Templates, Template Types & creating custom templates
  3. Trace execution options – Save trace to file or a table, Auto-scroll and keyboard shortcuts

Today, we will be looking at one of the events that I came across when we were certifying our product against Microsoft SQL Server 2008 when it was originally released. You may want to do the same when certifying your application against SQL 2012 (code named: “Denali”), currently in CTP03. We will see how the Profiler can help us in identifying T-SQL code that would no longer be supported going forward.

Deprecation Announcement Event Class

This Event class can be used to capture events that occur when a feature scheduled to be deprecated in some future release of Microsoft SQL Server is encountered. These features will still be available in the next major release of SQL Server, but may not be available in future releases.

If such code is encountered, it is recommended to consider replacement with suggested replacements.

Deprecation Final Support Event Class

This Event class can be used to capture events that occur when a feature scheduled to be deprecated in the next release of Microsoft SQL Server is encountered. If any such code is encountered, it is definitely time to refactor the code to use the recommended replacements.

An example

Let us intentionally create a scenario that would trigger one of the Deprecation Event classes.

Create a New Profiler Trace image
Select the “Deprecation” events from the Event Selection screen image
Execute the following T-SQL code against the instance of Microsoft SQL Server being monitored
/*
** Per Books On Line (http://msdn.microsoft.com/en-us/library/ms143729(SQL.110).aspx),
** SET ROWCOUNT for INSERT, UPDATE and DELETE will no longer be supported in the next
** Microsoft SQL Server release.
** Therefore, we expect this to be captured in the Deprecation Final Support Event Class
*/

USE AdventureWorks2008R2
GO

BEGIN TRANSACTION BRDemo
    SET ROWCOUNT 10
    UPDATE HumanResources.Employee SET VacationHours += 1
ROLLBACK TRANSACTION BRDemo
Notice the data captured by the Profiler image

In my next post…

Essentially, two major applications of the SQL Server Profiler remain:

  1. Correlating Performance data
  2. Replaying a trace

We will be discussing these in the next 2 posts, which will be the final ones in the series. If you have suggestions on any applications of the Profiler that you want me to cover, drop me a note, and I will try to cover them in future posts.

References:

Until we meet next time,

Be courteous. Drive responsibly.

Subscribe to my posts via E-mail: Subscribe here | Read my posts via your favourite RSS reader: Click Here!

SQL Server – Profiler – Part 3 – Trace execution options – Save trace to file or a table, Auto-scroll and keyboard shortcuts


Based on popular demand from my colleagues and you, the kind reader, I am currently writing a series of posts on the SQL Server Profiler. The first part was an introduction attempting to answer the question – What is a SQL Trace? Permissions, space requirements and launching the Profiler. The second part dealt with Profiler Templates, Template Types & creating custom templates. Today, we will attempt to answer the following questions:

  1. How to “execute” a basic Profiler trace?
  2. Which are some of the usability features that may be of use during trace execution?
  3. Can the Profiler be manipulated via the keyboard?

Creating a basic Profiler trace

To execute a Profiler trace, the first step is, obviously to launch the Profiler. This can be done by following the steps in my first post – What is a SQL Trace? Permissions, space requirements and launching the Profiler.

Next, go to File –> New Trace to launch the “Trace Properties” window once successful authentication takes place.

image

General Properties

Trace Information (# 1 and 2)

You can provide a name to the Profiler trace for easier future reference. You can also choose the template type to suit the exercise. For this test though, let’s choose the custom template created in my previous post – Profiler Templates, Template Types & creating custom templates.

Note that depending upon the connection created during authentication, the system shows the Trace Provider name, Provider Type and Version.

Saving the trace output (#3)

Normally, a profiler trace is shown to the user on the grid. However, depending upon the number of events and data columns being captured, there is a performance overhead associated to this method. It is much better instead to dump the trace data to a trace file.

When saving to a file, the user has the option to set a maximum file size to ensure that the file does not grow out of control. When “Enable file rollover” is checked, if the max. file size is reached, the existing file is closed and a new trace file is created. If the “Enable file rollover” is unchecked, the profiler stops capturing events when the max. file size is achieved.

As an alternate to saving trace data to a file, you may also want to dump the data to a table on any Microsoft SQL Server instance. This can be achieved by checking the “Save to Table” checkbox. Also, just as file-sizes can be limited, the number of rows in the trace table can be limited by specifying the max. row count in the “Set maximum rows (in thousands):”.

(Tip: It is advisable to store the trace data to another instance of Microsoft SQL Server to avoid interference with the activity being monitored.)

The “Server processes trace data” must be used with caution. This causes the service running the trace to process trace data instead of the client application. The drawback is that server performance may be affected.

Time-bound tracing (#4)

To have the SQL Server Profiler automatically stop tracing after at certain time, check the “Enable trace stop time” and specify the required stop date and time.

Event Selection

image

This page allows you to do the following:

  1. Review the Events & data columns to capture
  2. Add or Remove the Events to capture
  3. Add or Remove data columns to capture
  4. Apply one or more filters to one or more data columns
  5. Specify data columns to Group the results on and also specify the order in which data columns should be shown to the user
  6. Brief description of the event class selected (changes when event class selection changes)
  7. Brief description of the data column (changes as the user hovers over the data column)

A step-by-step guide on customizing the template using the controls specified by #2, 3, 4 and 5 is available in my previous post – Profiler Templates, Template Types & creating custom templates.

Executing a trace

Once the Profiler trace is defined, one just needs to click on the “Run” button in the “Template Properties” window. The SQL Server Profiler will now start showing events to the user depending upon the chosen template.

image

  1. Basic controls – start, pause & stop
  2. Launch SQL Server Management Studio
  3. Launch the Performance Monitor
  4. Clear the trace data grid
  5. Review the trace properties (The “Trace Properties” window opens, but in read-only mode if the trace is running or is paused)
  6. Find a string within any of the data columns

image

Auto-scroll – a nice usability feature

Most of the usability features (launching of SSMS, PerfMon and the flexibility of the “Find” window) are quite straight-forward. However, there is one feature that I like the most. Look at the toolbar on the profiler, and note the icon with the grid and a little, blue down arrow:

image

This icon is the “Auto-scroll” button. When selected, the grid scrolls to the most recently collected data row as more and more data starts collecting on the grid. When not selected, the grid needs to be scrolled manually.

Typically, if I am trying to capture and analyze a trace for a particular database and host and want to see things as they happen, I prefer the enable “Auto-scroll”. If I am saving data to a file or a table, I would most probably be looking at the data later on and would therefore have the “Auto-scroll” turned off.

Can the Profiler be Manipulated via the Keyboard?

Of course, it can! Here’s where you can find a list of all keyboard shortcuts – http://msdn.microsoft.com/en-us/library/ms174164.aspx

In my next post…

SQL 2012 is just around the corner. Most of the community members are probably evaluating SQL 2012. For those who are not yet ready, but will ultimately move to SQL 2012 must most definitely be moving to SQL Server 2008 R2. We will see how to use the Profiler to identify which objects are in most dire need of replacement for certification against a release of SQL Server.

We would be approaching the end of the series and would therefore start looking at other scenarios where the Profiler can be used.

References

Until we meet next time,

Be courteous. Drive responsibly.

Subscribe to my posts via E-mail: Subscribe here | Read my posts via your favourite RSS reader: Click Here!

SQL Server – Profiler – Part 2 – Profiler Templates, Template Types and Creating Customized Templates


Based on popular demand from my colleagues and you, the kind reader, I am currently writing a series of posts on the SQL Server Profiler. You can read the first part on What is a SQL Trace? Permissions, space requirements and launching the Profiler. Today, we will attempt to answer the following questions:

  1. What is a Profiler Template?
  2. What are the various types of Templates available within the SQL Server profiler?
  3. How to create customized templates?

Profiler Templates

We talked yesterday about SQL Traces and it’s components. SQL Server Profiler ships with templates that define the event classes and data columns to include in traces. It is important to understand that when we execute the SQL Server Profiler, we are capturing a SQL Trace and the template gives body to the trace, i.e. the template is used to represent the events and data columns captured in the trace, but the template is itself never “executed”.

Template Types

SQL Server Profiler offers predefined trace templates that allow you to easily configure the event classes that you will most likely need for specific traces. To view the list of available templates, click on the “New Trace” button on the toolbar, and then expand the “Use the template” drop-down.

image

NOTE: If you are using a version of the SQL Server Profiler that’s different than the SQL Server instance, then the template drop-down may be blank, and you may be required to use a customized template.

Below is a brief description of the various template types available in the SQL Server Profiler. To know more about the various Event Classes associated to each, please refer the Books On Line documentation referenced in the “References” section of this post. The ones I have used most in my career till date are: TSQL_Duration, TSQL_Locks and TSQL_SPs.

Template Name Description
Blank This is a blank template that can be used during template customization
SP_Counts Captures the event raised whenever a stored procedure starts execution
Standard This is the default trace.
This is the most generic and commonly used trace that captures:
1. Connections (connect and disconnects)
2. Completion of Remote Procedure Calls
3. Start & completion of T-SQL statements & batches
TSQL

Captures all client-submitted T-SQL statements and the time issued.

Use to debug client applications.

TSQL_Duration

Captures all client-submitted T-SQL statements, their execution time (in milliseconds), and groups them by duration.

Use to identify slow queries.

TSQL_Grouped

Captures all client-submitted T-SQL statements and the time they were issued. Groups information by user or client that submitted the statement.

Use to investigate queries from a particular client or user.

TSQL_Locks Captures the client-submitted T-SQL statements and associated exceptional lock information.

Use to troubleshoot deadlocks, lock time-out, and lock escalation events.

TSQL_Replay

Captures detailed information about Transact-SQL statements that is required if the trace will be replayed.

Use to perform iterative tuning, such as benchmark testing.

TSQL_SPs Although not the default, this is perhaps the most useful trace for most systems (provided it is not to be replayed).

Captures detailed information about all executing stored procedures.

Use to analyze the component steps of stored procedures. Add the SP:Recompile event if you suspect that procedures are being recompiled.

Tuning

Captures information about stored procedures and Transact-SQL batch execution.

Use to produce trace output that Database Engine Tuning Advisor can use as a workload to tune databases.

Customizing a Profiler trace

To have a best of all worlds, one might want to create a custom Profiler template. To create a custom template, one can follow the following steps:

Go to
File
–> Templates
–>
New Template
image
Select “Server Type” image
Supply a name for the template, and also choose whether the template should be based on an existing template or not image
Go to “Events” tab and add/remove events captured using the “Show All Events” checkbox.

NOTE that I have removed a few default events and have added 2 new ones.

image
Using the “Show All Columns”, I added one additional data column capture image
Go to “Column Filters” to apply filters on the data columns 

NOTE: If multiple criteria are defined, the “AND” operator is used

image
Go to “Organize columns” to define the sequence in which columns should appear and also to group by any particular column image
Once all adjustments are done, click on “Save” to save the template image
The trace will then appear in the “Use the template” drop-down when starting a new trace for the given server type image

Setting the Default Trace

Let’s assume that we need to set the trace we just created as the default trace. Here are the very simple sequence of steps to follow:

Navigate to File –> Templates –> Edit Template image
Select the Server Type and Template Name to edit image
Check the “Use as a default template for selected server type”.

NOTE: You can also use this interface to delete a trace template.

image

Storage & Exporting/Importing templates

Customization of templates is machine and user specific. That means that the template must be stored on the disk as a physical file. A simple file-system search reveals the location of the template file:

For SQL 2012 (code named: “Denali”), the template is available at: C:Users<<<user name>>>AppDataRoamingMicrosoftSQL Profiler11.0TemplatesMicrosoft SQL Server110

So, here’s a tip – the next time you backup your system, also backup your templates so that you don’t have to set them up again! You can Export or Import your existing templates by going to File->Templates->Export Template/Import Template:

image

In my next post…

In the next post, I will be writing about the various options available to execute the trace. Finally, towards the end of the week we would look at other uses of Profiler (specifically, trace replay).

References

Please find below some of the reference Books On Line documentation that I found when researching for this post:

Until we meet next time,

Be courteous. Drive responsibly.

Subscribe to my posts via E-mail: Subscribe here | Read my posts via your favourite RSS reader: Click Here!