Today I wanted to share with you that some DBAs seem to overlook the transaction log when thinking about optimizing database performance.  I myself was guilty of not considering optimizing the transaction logs for my production databases.  I am going to blog about Virtual Log Files (VLFs) and why it is important to consider this option when talking about overall database health and performance.  Here is a little background first… I first learned about VLFs when I attended the last Dev Connections Conference in Las Vegas for SQL Server 2008.  At his conference I learned about VLFs from one of the prominent SQL Server experts Kimberly Tripp.  Here is a more detailed breakdown of Virtual Log Files:

For Performance Impact please refer to Linchi Shea’s post on SQLBlog (link must be cut and pasted into your browser):

http://sqlblog.com/blogs/linchi_shea/archive/2009/02/09/performance-impact-a-large-number-of-virtual-log-files-part-i.aspx

For more information, please refer to Kimberly’s posts on her website (also, these must be cut and pasted into your browser as the direct links will not work):

http://www.sqlskills.com/BLOGS/KIMBERLY/post/8-Steps-to-better-Transaction-Log-throughput.aspx

http://www.sqlskills.com/BLOGS/KIMBERLY/post/Transaction-Log-VLFs-too-many-or-too-few.aspx

I hope I have given all of you something new to ponder as you are getting ready to optimize performance on your production databases.

Now back in the days of SQL Server 2000 and earlier you needed to write a T-SQL script that allowed access to the system tables, grab the SIDS from syslogins (sysxlogins) and update the SIDs in the user database.  Now lets fast forward to SQL Server 2005… Microsoft recogonized the need to sync database users to logins and vice versa for different servers.  So in SQL Server 2005 came the stored procedure sp_change_users_login.  To find out more information on this stored procedure you can visit this site:   http://technet.microsoft.com/en-us/library/ms174378.aspx .

Here is a T-SQL script that you run in a query window to allow you to sync database users to SQL Server logins.  Now one thing to note here… make sure you run the script in the context of the database that you want to sync the users for.  Here is the script:

SET NOCOUNT ON
SET QUOTED_IDENTIFIER OFF
GO

USE <database name>
GO

DECLARE @sql varchar(100);

DECLARE curSQL CURSOR FOR
SELECT ‘EXEC sp_change_users_login ”UPDATE_ONE”, ”’ + name + ”’, ”’ + name + ””
FROM sysusers
WHERE issqluser = 1
AND suser_sname(sid) IS NULL
AND name NOT IN (‘dbo’, ‘guest’)

OPEN curSQL

FETCH curSQL INTO @sql

WHILE @@FETCH_STATUS = 0 BEGIN
EXEC (@sql)
PRINT @sql
FETCH curSQL INTO @sql
END

CLOSE curSQL
DEALLOCATE curSQL
GO

I hope this helps! Let us know if you have another way to sync database users to SQL Server logins.

DTSRun OnStart:  DTSStep_DTSDataPumpTask_6   DTSRun OnProgress:
DTSStep_DTSDataPumpTask_6; 1000 Rows have been transformed or copied.;
PercentComplete = 0; ProgressCount = 1000   DTSRun OnProgress:
DTSStep_DTSDataPumpTask_6; 2000 Rows have been transformed or copied.;
PercentComplete = 0; ProgressCount = 2000   DTSRun OnProgress:
DTSStep_DTSDataPumpTask_6; 3000 Rows have been transformed or copied.;
PercentComplete = 0; ProgressCount = 3000   DTSRun OnProgress: 
DTSStep_DTSDataPumpTask_6; 4000 Rows have been transformed or copied.;
PercentComplete = 0; ProgressCount = 4000   DTSRun OnProgress: 
DTSStep_DTSDataPumpTask_6; 5000 Rows have been transformed or copied.;
PercentComplete = 0; ProgressCount = 5000   DTSRun OnProgress: 
DTSStep_DTSDataP…  Process Exit Code 1.  The step failed.

Now looking at this error message does not really provide all the detail as to why and what failed.  So I did some searching through BOL, MSDB and Google and I finally found what I was looking for.  IF you have logging enabled on the DTS package (I do for all of mine) and have the option to write to SQL Server instead of a log file then you can query for the exact error message produced.  Here is what you need to do:

In SQL Server 2000 open up a Query Analyzer window and select ‘MSDB’ from the drop down or type in ‘USE MSDB’ and execute that and you will now be placed into the context of MSDB.  Next, look at the job that failed and notice the name of the of the DTS package being called.  Take the name and then type and execute the following T-SQL statement:

    select *
    from sysdtspackagelog
    where name = ‘SendCustomerData’  — this is the name of your DTS package
    order by starttime desc

In the result set look at the first record and look for the column that reads ‘lineagefull’ and copy that value in the first record.  The value will look something like this:  E1E90023-93EB-4355-B995-43A762AD43C1

After grabbing that value, execute the following T-SQL statements and use that value in the WHERE clause.

     select *
     from sysdtssteplog
     where lineagefull = ‘E1E90023-93EB-4355-B995-43A762AD43C1′ 

The table sysdtssteplog displays all of the tasks in the DTS package so 1 task equal 1 record in the table.  Scroll over to your left in the result set pane and look for a column called: 

errordescription

Here you will be able to view the actual error that was generated by one or more tasks in your DTS package!

I hope you found this helpful!

The other day I was posting in the SQL Server news groups and I came across a person asking about which performance counters should be collected to obtain a performance load \ baseline in their production SQL Server environment.  It occurred to me there is a lot of information out there about performance counters and what shoud be collected to determine a true load \ baseline for a production environment.  Now granted everyone’s production SQL Server environment is different however, the following performance counters can be used to give you a wide or general overview of the activity going on in your environment.

In the past I have worked with a company called Scalability Experts on a very large SQL Server consolidation project.  Scalability Experts ran the same counters to get a general overview\load\baseline on all production SQL Servers.  These counters can be ran on SQL Server 2000, 2005 and 2008 in either a stand-alone or cluster configuration.  Here are the performance counters to run within Performance Monitor:

Memory - Page Reads/sec
Memory - Page Writes/sec
Memory - Pages Input/sec
Memory - Pages Output/sec
Memory - Pages/sec
Network Interface(*) - Bytes Total/sec
Network Interface(*) - Output Queue Length
Network Interface(*) - Packets/sec
PhysicalDisk(*) - % Disk Time
PhysicalDisk(*) - Avg. Disk Bytes/Read
PhysicalDisk(*) - Avg. Disk Bytes/Write
PhysicalDisk(*) - Avg. Disk Queue Length
PhysicalDisk(*) - Avg. Disk Read Queue Length
PhysicalDisk(*) - Avg. Disk sec/Read
PhysicalDisk(*) - Avg. Disk sec/Write
PhysicalDisk(*) - Avg. Disk Write Queue Length
PhysicalDisk(*) - Disk Read Bytes/sec
PhysicalDisk(*) - Disk Reads/sec
PhysicalDisk(*) - Disk Transfers/sec
PhysicalDisk(*) - Disk Write Bytes/sec
PhysicalDisk(*) - Disk Writes/sec
Process(sqlservr) - % Privileged Time
Process(sqlservr) - % Processor Time
Process(sqlservr) - % User Time
Process(sqlservr) - IO Data Operations/sec
Process(sqlservr) - Page Faults/sec
Process(sqlservr) - Private Bytes
Process(sqlservr) - Thread Count
Process(sqlservr) - Virtual Bytes
Process(sqlservr) - Working Set
Processor(*) - % Interrupt Time
Processor(*) - % Privileged Time
Processor(*) - % Processor Time
Processor(*) - % User Time
SQLServer:Buffer Manager - *
SQLServer:Cache Manager(*) - *
SQLServer:Databases(*) - Data File(s) Size (KB)
SQLServer:Databases(*) - Log File(s) Size (KB)
SQLServer:Databases(*) - Transactions/sec
SQLServer:General Statistics - *
SQLServer:Locks(*) - *
SQLServer:Memory Manager - *
SQLServer:SQL Statistics - Batch Requests/sec
SQLServer:SQL Statistics - SQL Compilations/sec
SQLServer:SQL Statistics - SQL Re-Compilations/sec
System - Context Switches/sec
System - Processor Queue Length

My next post will be demostrating how to add these counters into Performance Monitor and setting the specifics (duration, frequency, etc) on the the log.  If you have any other performance counters that you use please post them here!

So what I did was… I opened up the trace file in SQL Profiler and I was able to view the events and columns for the trace. Then I compared the events and columns to what I found in the trace file to what I found on the Internet. The comparison turned out to be an exact match for events as well as columns. So I started asking around and nobody seemed to know how it got there on the server and if it was being used. Of course when it comes to something like this nobody wants to fess up.

Here is the idea of a “black box” trace. A black box trace records the last 5MB of activity on the SQL Server when it becomes problematic. The trace captures the queries and\or errors right before the server becomes unrepsonsive or crashes. This information is useful for sending to Microsoft to help diagnose the problem. The black box trace records the following:
Stored Procedure Execution (RPC:Starting)
T-SQL Batch Execution (SQL:BatchStarting)
Errors and Warnings (Attention and Exception)

You can go to google and query “sql server black box trace” and you will get a lot more information. From what I have read and I can tell on my own server the black box trace does not cause any performance issues. If you have experience using the black box trace feature please post and let us know!

I thought today’s posting to my blog should cover how to get the results of DBCC CHECKDB into a table.  Unfortunately, Microsoft does not allow you easily per se to insert the results into a table so that you might be able to generate administrative reports off of.  Well in my case that is exactly why I developed this process… to generate administrative reports off of.

Now what I did as you will see in the T-SQL scripts below, I created a table based upon the result set of the DBCC CHECKDB execution.  Once I had the column names and data types I went ahead and created the table.  The next step was to write a stored procedure that would iterate through all databases on the server, run the DBCC CHECKDB statement and record the output into the table I just created.  So after much testing and a little taste of success I give you the scripts below.  I call the stored procedure in a SQL Server Agent job and schedule the job to run in the early hours of the morning and ’off peak’ hours.  So in the morning I can review the results of DBCC CHECKDB from the previous night.  Now you can modify this process to just append the data or just truncate it like I did.  I hope you find this code helpful!… Let us know!…

Here is the table script (this script is for the SQL Server 2005 CHECKDB output)

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_PADDING ON
GO
CREATE TABLE [dbo].[CheckDBResult](
 [ServerName] [varchar](100) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
 [Error] [int] NULL,
 [Level] [int] NULL,
 [State] [int] NULL,
 [MessageText] [varchar](7000) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
 [RepairLevel] [int] NULL,
 [Status] [int] NULL,
 [DbId] [int] NULL,
 [Id] [int] NULL,
 [IndId] [int] NULL,
 [PartitionId] [int] NULL,
 [AllocUnitId] [int] NULL,
 [File] [int] NULL,
 [Page] [int] NULL,
 [Slot] [int] NULL,
 [RefFile] [int] NULL,
 [RefPage] [int] NULL,
 [RefSlot] [int] NULL,
 [Allocation] [int] NULL,
 [insert_date] [datetime] NOT NULL CONSTRAINT [DF_CheckDBResult_insert_date]  DEFAULT (getdate())
) ON [PRIMARY]

GO
SET ANSI_PADDING OFF

Here is the CREATE PROCEDURE script that utilizes the CHECKDB 2005 output and here is the T-SQL syntax to execute the procedure: EXEC mDBMaintenance ‘Lumiere’ <- server name to run the script against

EXEC Admin_GetCheckDBResults ’server name’

set ANSI_NULLS ON
set QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[Admin_GetCheckDBResults]
 @ServerName VARCHAR(100)
 
AS
DECLARE @Database SYSNAME
 
– Remove Previous Weeks Results
TRUNCATE TABLE DBA_Admin.dbo.CheckDBResult
 
DECLARE cDatabases CURSOR FOR
SELECT [name] from master..sysdatabases AS sdb
WHERE sdb.[name] NOT IN (‘tempdb’)
OPEN cDatabases
FETCH FROM cdatabases INTO @Database
WHILE ( @@FETCH_STATUS = 0)
BEGIN
 SET NOCOUNT ON
 
 INSERT INTO DBA_Admin.dbo.CheckDBResult(
  Error,
  [Level],
  [State],
  MessageText,
  RepairLevel,
  [Status],
  [DbId],
  Id,
  IndId,
  PartitionId,              — specific to SQL Server 2005, remove for SQL Server 2000 results
  AllocUnitId,             — specific to SQL Server 2005, remove for SQL Server 2000 results
  [File],
  Page,
  Slot,
  RefFile,
  RefPage,
  RefSlot,
  Allocation
  )
 EXEC (‘DBCC CHECKDB(”’ + @Database + ”’) WITH TABLERESULTS’)
 
FETCH FROM cdatabases INTO @Database
END
CLOSE cDatabases
DEALLOCATE cDatabases
 
– Remove all details except CHECKDB information
DELETE
FROM DBA_Admin.dbo.CheckDBResult
WHERE MessageText NOT LIKE ‘CHECKDB%’
 
– Update server Name
UPDATE DBA_Admin.dbo.CheckDBResult
SET ServerName = @ServerName
 
– Select Statment To Return Rows
SELECT ServerName,
 MessageText,
 Error,
 [Level],
 [State],
 RepairLevel,
 [Status],
 [DbId],
 Id,
 IndId,
 PartitionId,          – specific to SQL Server 2005, remove for SQL Server 2000 results
 AllocUnitId,         – specific to SQL Server 2005, remove for SQL Server 2000 results
 [File],
 Page,
 Slot,
 RefFile,
 RefPage,
 RefSlot,
 Allocation,
 Insert_Date
FROM DBA_Admin.dbo.CheckDBResult
GO

Now as you can see I only want to see the last line in the DBCC CHECKDB command… however, you can modify it keep all the detail from the execution.  I hope this code helps and let us know how you are using it!

select db_name(dbid) as [database name],
hostname,
program_name,
nt_username,
loginame,
net_address,
last_batch
from SQL1.master.dbo.sysprocesses

The problem here is… the function db_name() was pulling the “local” database name for the remote dbid on the SQL Server 2000 server. I was unable to find documentation on this “issue” on BOL and the Internet. I found a workaround for this. Here is the work around:

select d.[name] as [database name],
p.hostname,
p.program_name,
p.nt_username,
p.loginame,
 p.net_address,
p.last_batch
from SQL1.master.dbo.sysprocesses as p
inner join SQL1.master.dbo.sysdatabases as d
on p.dbid = d.dbid

This solution pulls the database name for the remote server and not the local database name on the SQL Server 2005 server. If you have experienced this issue or if this a known “bug” or not a “bug” please post and let us know!

DMVs return server metric information that can be used to monitor the health of an instance, diagnose problems, and performance tune. One of the things I was looking for was a way to identify slow performing queries outside of using SQL Profiler. I came across the following script on SQL Server Performance website. This script returns a result set that contains queries that are performing slowly.

SELECT creation_time,
last_execution_time
,total_physical_reads
,total_logical_reads
,total_logical_writes
, execution_count
, total_worker_time
, total_elapsed_time
, total_elapsed_time / execution_count avg_elapsed_time
,SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,
((CASE statement_end_offset
WHEN -1 THEN DATALENGTH(st.text)
ELSE qs.statement_end_offset END
- qs.statement_start_offset)/2) + 1) AS statement_text
FROM sys.dm_exec_query_stats AS qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
ORDER BY total_elapsed_time / execution_count DESC;

The sys.dm_exec_query_stats view returns aggregate performance statistics for cached query plans. To find out more about this view click here. In my upcoming posts I will be covering the different DMVs in SQL Server 2005 in great detail. If you have other scripts that utilize this view please post them here!

Today is going to be a short posting on how to calculate whether or not the hour in either the function getdate() or a datetime field in a table is either even or odd. I am using this to determine which tables (evenhour or oddhour) are recording page impressions. Here is the T-SQL script:  select datepart(hour,getdate())%2 as evenodd

The % is know in SQL Server as “modulo” and what this does is it provides the remainder of one number divided by another.  When using % with the datapart and getdate functions it returns either zero if the hour of datepart is even or will return one if the hour of datepart is odd.

This logic comes in very handy when you need to know which hour table (even or odd) you need to process hits, impressions, clicks, etc.  Tomorrow I will show you how to implement this in a SSIS package to calculate the even or odd hour and to move the data to their respected tables based upon the result of %.  So keep checking back tomorrow for this solution!