Tag Archives: Guidance

Articles intended to provide guidance to the readers based on my past experiences.

#0357 – SQL Server – Best Practices – Avoid Implicit conversions as they may cause unexpected issues (Part 01)


Often, I see database developers relying on implicit conversions more than they should. Implicit conversions can use unexpected problems if not used judiciously.

In this post, I will demonstrate how implicit conversions can cause issues when trying to insert data in a table using the VALUES clause. Assume the following sample object into which we want to insert the data.

USE tempdb;
GO
--Demo objects
BEGIN
    IF OBJECT_ID('dbo.BewareOfImplicitConversion','U') IS NOT NULL
        DROP TABLE dbo.BewareOfImplicitConversion;

    CREATE TABLE dbo.BewareOfImplicitConversion
        ( RecordId                   INT         NOT NULL IDENTITY(100,10),
          RowDescription             VARCHAR(20) NOT NULL,
          IntMasqueradingAsCharacter VARCHAR(20)     NULL,
          IntValue                   INT             NULL
        );
END
GO

The column – dbo.BewareOfImplicitConversion.IntMasqueradingAsCharacter has been specifically named because we will be trying to insert numerical data into this column (which is supposed to hold character data) – it is perhaps one of the most common scenarios as far as integrating systems are concerned, however, we will be adding a twist – we will be inserting numeric data as-is, i.e. as integer values.

INSERT INTO dbo.BewareOfImplicitConversion (RowDescription, IntMasqueradingAsCharacter, IntValue)
VALUES ('One',   NULL, 1),
       ('Two',   NULL, 4),
       ('Three',    3, 6),
       ('Four',     4, 8),
       ('Five',   'V', 10);
GO

What we also did was to try and insert, in the same batch, one row of data that actually has character data in the column of interest. When executed, we end up in the following error:

Msg 245, Level 16, State 1, Line 3
Conversion failed when converting the varchar value ‘V’ to data type int.

However, when executed as shown below, the statements work just fine and insert all the 5 records into the table (dbo.BewareOfImplicitConversion).

INSERT INTO dbo.BewareOfImplicitConversion (RowDescription, IntMasqueradingAsCharacter, IntValue)
VALUES ('One',   NULL, 1),
       ('Two',   NULL, 4),
       ('Three',    3, 6),
       ('Four',     4, 8);

INSERT INTO dbo.BewareOfImplicitConversion (RowDescription, IntMasqueradingAsCharacter, IntValue)
VALUES ('Five',   'V', 10);
GO

SELECT * FROM dbo.BewareOfImplicitConversion;

Result:

image

Root Cause

The root cause here is very simple – before inserting data into the object, SQL Server has to first parse the queries populating the source data. During this process, the rules of data-type precedence come into the picture.

Data type precedence states that whenever an operator combines data from two expressions of different data types, the data-type with the lower precedence (in this case, it is the character data type – VARCHAR) is converted implicitly to a data-type with higher precedence (in this case – INT). When an implicit conversion fails, SQL Server will return an error.

In our example, the implicit conversion worked for the first 4 records being inserted because they did not require any implicit conversion, Implicit conversion is implemented for the data in the 5th record. Because they are part of the same statement, SQL Server will try to convert the string data implicitly to an integer value – causing the failure.

Solution/Best Practice:

Always use proper data-type casting and reference styles as shown in the query below and it is guaranteed to work.

INSERT INTO dbo.BewareOfImplicitConversion (RowDescription, IntMasqueradingAsCharacter, IntValue)
VALUES ('One',   NULL, 1),
       ('Two',   NULL, 4),
       ('Three',  '3', 6),
       ('Four',   '4', 8),
       ('Five',   'V', 10);
GO

Further Reading:

Until we meet next time,
Be courteous. Drive responsibly.

#0356 – SQL Server – Search with the LIKE operator and the underscore (_) wildcard character


String searches in most applications are based on the “contains” or “begins-with” paradigms – which are implemented by the use of the LIKE logical operator. The LIKE operator allows for the use of wildcard characters (underscore being one of them), which is the part that most people unknowingly overlook when they have been working with T-SQL for quite some time.

One such incident happened the other day at work when I was asked by a colleague to help him out with a query that appeared correct, but failed to give the expected results. For the sake of brevity, I have condensed the issue into a test scenario below:

DECLARE @testTbl TABLE (Value VARCHAR(50));

INSERT INTO @testTbl (Value)
VALUES ('SQL 2012'),
('SQL_2014');

SELECT * FROM @testTbl WHERE Value LIKE 'SQL_20%';

image

As can be seen in the screenshot above, the results are not what one would expect them to be at the first glance. One would have expected it to return only the string “SQL_2014” and yet the T-SQL query is returning both the records.

Normally, data coming in via flat-files from raw-data collection systems would require such kinds of string searches and manipulations. A well designed system would have these static values as a look-up so that all references look at the same version of the string value

Solution(s)

There are not one, but two possible solutions to choose from to work around this issue.

Using square brackets ([]) around the wildcard character

Because we want to consider the underscore as a character and not as a wildcard, the easiest thing to do would be to surround it with square brackets, similar to what is shown in the query below.

SELECT * FROM @testTbl WHERE Value LIKE 'SQL[_]20%';

image

Using the ESCAPE keyword

The other option, which I have discussed in one of my earlier posts, is to use the ESCAPE keyword. The ESCAPE keyword works on the basis of a user specified escape sequence. When encountered, the query engine will simply ignore the wildcard character and treat it as a normal character.

SELECT * FROM @testTbl WHERE Value LIKE 'SQL*_20%' ESCAPE '*';

image

Conclusion

The moral here is that one must always remember the basics – this thumb rule not only applies to software, but in everything that we go about doing in our day to day lives.

The other key learning that I encourage everyone to take from this is that Occam’s Razor is real – from the multiple explanations that a problem may have, the explanation with the least number of assumptions should be selected as the answer. In this case, the simplest theory was –  human error, which was indeed the case.

References

Until we meet next time,
Be courteous. Drive responsibly.

#0353 – SQL Server – Locate an object in a SQL Server Instance (using documented features)


A while back, I wrote a post on locating an object in a SQL Server instance (post #351). One of the items I received as part of the feedback was to implement the logic using documented (and therefore supported) features.

As you may know, I had used the sp_MSforeachDB system stored procedure in my earlier post. This is an undocumented procedure and therefore the recommendation is to assume that it is unsupported and not use it in production environments. In today’s post, I present before you a script that achieves the same objective (i.e. locate an object in a SQL Server instance), but using fully documented (and therefore fully supported) features.

Improvements over the script using sp_MSforeachDB:

  • Uses fully documented features
  • Does a comparison on both – the schema name and the object name
  • Search is only limited to user databases by default (system databases can be easily reintroduced by commenting out appropriate filter conditions in the query below)

The entire script is provided below for your reference:

USE master;
GO

SET NOCOUNT ON;

--Declarations
BEGIN
    --User to fill in
    DECLARE @schemaToSearch NVARCHAR(255) 
                            = N'dbo';
    DECLARE @objectToSearch NVARCHAR(255) 
                            = N'uspGetBillOfMaterials';

    --Internal variables
    DECLARE @databaseInformation TABLE 
                (DatabaseId INT NOT NULL,
                 DatabaseName NVARCHAR(255),
                 IsProcessed BIT NOT NULL
                );
    DECLARE @databaseId INT = 0;
    DECLARE @databaseName NVARCHAR(255) = N'';
    DECLARE @sqlStmnt NVARCHAR(1000) = N'';
END

--Fetch all user databases into temporary table variable
--for convenience
BEGIN
    INSERT INTO @databaseInformation (DatabaseId, 
                                      DatabaseName, 
                                      IsProcessed)
    SELECT sdb.database_id AS DatabaseId,
           sdb.name AS DatabaseName,
           0 AS IsProcessed
    FROM sys.databases AS sdb 
    WHERE CAST((CASE WHEN SUBSTRING(sdb.name,0,14) 
                          IN ('master',
                              'msdb',
                              'model',
                              'tempdb',
                              'ReportServer$') 
                          THEN 1 
                     ELSE sdb.is_distributor 
                END) AS BIT) = 0;
END

--Begin processing one database after another
WHILE EXISTS (SELECT dbi.DatabaseId 
              FROM @databaseInformation AS dbi 
              WHERE dbi.IsProcessed = 0
             )
BEGIN
    SELECT TOP 1
           @databaseId = dbi.DatabaseId,
           @databaseName = dbi.DatabaseName
    FROM @databaseInformation AS dbi
    WHERE dbi.IsProcessed = 0;

    --Build the statement to fetch the objectId of an object 
    --and then check for it's existence
    SET @sqlStmnt = N'USE ' + QUOTENAME(@databaseName) + N';
    DECLARE @objectId INT;
    SET @objectId = OBJECT_ID(''' 
                + QUOTENAME(@schemaToSearch) 
                + N'.' 
                + QUOTENAME(@objectToSearch) 
                + N''')
    IF EXISTS (SELECT *
               FROM sys.objects AS so
               WHERE so.is_ms_shipped = 0
               AND so.object_id = @objectId
             )
    BEGIN
        SELECT DB_NAME() AS DatabaseName;
    END;'

    --Execute the query
    --PRINT @sqlStmnt;
    EXEC sp_executesql @sqlStmnt = @sqlStmnt;

    --Mark the database as done
    UPDATE dbi
    SET dbi.IsProcessed = 1
    FROM @databaseInformation AS dbi
    WHERE dbi.DatabaseId = @databaseId
      AND dbi.DatabaseName = @databaseName; 

END
GO

Until we meet next time,
Be courteous. Drive responsibly.

#0352 – SQL Server – Working with Temporary Tables (L100) – Scope


In this post, I provide a Level 100 primer on the scope related considerations for local temporary tables.

While I have written about temporary tables a lot in the past (and the same has also been done by other writers), there are some questions that I keep encountering from the team or in various forums on the Internet which is why I am writing this post.

Temporary tables use the tempdb and come in two variants:

  1. Local Temporary Tables
    • As the name implies, they are local to the connection that created them
    • Therefore, if multiple connections request creation of a temporary table with the same name, each connection gets it’s own unique copy of the temporary table
    • The local temporary tables are implicitly deleted when the connection is closed
    • To create a local temporary table, prefix the desired table name with a single hash (#) mark, e.g. A local temporary table for employees might be named #Employee or #HumanResources
  2. Global Temporary Tables
    • These are global to the entire SQL Server instance and are therefore shared by all connections to the server
    • They are implicitly deleted when the last connection referencing them is closed
    • To create a global temporary table, prefix the desired table name with a double hash (##) mark, e.g. A global temporary table for employees might be named ##Employee or ##HumanResources

A demo on local temporary tables

The most important point that is missed out by most novice developers is that temporary tables are session specific. To briefly demonstrate this, allow me to follow the following steps:

  1. Open SQL Server Management Studio and launch two (2) query editor windows
    • In each window, connect to the same database and same SQL Server instance
  2. Run the following in window #1. This snippet creates a local temporary table and populates some records into it
--Window 01
USE AdventureWorks2012;
GO

--Create the local temporary table if it does not exist
IF OBJECT_ID('tempdb..#BasicEmployeeList','U') IS NULL
BEGIN
CREATE TABLE #BasicEmployeeList 
    ( BusinessEntityID INT          NOT NULL,
      JobTitle         NVARCHAR(50) NOT NULL,
      Title            NVARCHAR(8)      NULL,
      FirstName        NVARCHAR(50) NOT NULL,
      MiddleName       NVARCHAR(50)     NULL,
      LastName         NVARCHAR(50) NOT NULL
    );
END;
GO

INSERT INTO #BasicEmployeeList
    (BusinessEntityID,
     JobTitle,
     Title,
     FirstName,
     MiddleName,
     LastName
    )
SELECT p.BusinessEntityID,
       e.JobTitle,
       p.Title,
       p.FirstName,
       p.MiddleName,
       p.LastName
FROM Person.Person AS p
INNER JOIN HumanResources.Employee AS e
ON p.BusinessEntityID = e.BusinessEntityID;
GO
  1. Without closing window #1, Switch over to window #2
  2. Attempt to run the statement shown below
    • The system will encounter an error
USE AdventureWorks2012;
GO
SELECT bel.BusinessEntityID,
       bel.JobTitle,
       bel.Title,
       bel.FirstName,
       bel.MiddleName,
       bel.LastName
FROM #BasicEmployeeList AS bel;
GO

/* ERROR
Msg 208, Level 16, State 0, Line 1
Invalid object name '#BasicEmployeeList'.
*/
  1. The error is received because the temporary table was created as a local temporary table and is therefore inaccessible to other connections

Important points to note

  • Because temporary tables are persisted in the tempdb, the tempdb can see significantly large I/O operations if your application makes use of a lot of temporary storage
  • SQL Server gives Local Temporary tables an auto-generated name which is transparent to the caller to allow multiple connections to the use local temporary tables with the same name [Link]
  • Local temporary tables can be shared to referencing stored procedures as long as they use the same connection
    • This is documented in my post [Link]
  • Creating named constraints on temporary tables prevent them from being used concurrently
  • Temporary tables do not support:
    • Foreign key constraints [Links]
    • Triggers

Until we meet next time,

Be courteous. Drive responsibly.

#0351 – SQL Server – Locate an object in a SQL Server instance


Interacting with people on a SQL Server forum is always something that I look forward to. It gives me an insight into the challenges that are being faced out in the field – some of which I may have already encountered & addressed myself in the past, but thought it to be something trivial. A recent question was before me which prompted this post.

A friend of mine was involved in a knowledge transfer of a legacy product from one team to another. Documentation was insufficient and most of the product knowledge was in the brains of the people who had originally developed the system. One of the points covered in the transfer that all databases of that product would have a particular database object (a table or a stored procedure). If that object did not exist, the database was not a valid database for that product. The question therefore was:

“Given a SQL Server instance, is it possible to check for existence of a particular database object across all databases in the instance?”

The simple answer is yes.

However, the answer involves the use of a undocumented (and therefore unsupported) system stored procedure – sp_MSforeachdb.

USE master ;
GO
EXEC sp_MSforeachdb 
       N'USE ?;
         IF EXISTS (SELECT *
                    FROM sys.objects AS so
                    WHERE so.is_ms_shipped = 0
                      AND so.name = ''uspGetBillOfMaterials''
                   )
            SELECT ''?'' AS DatabaseName;' ;
GO

/*RESULTS

DatabaseName
------------------
AdventureWorks2012

*/

Disclaimer: Because of the use of unsupported system stored procedure, it is recommended that the script provided below is limited to use in the development and quality assurance environments only.

Further Reading

Until we meet next time,

Be courteous. Drive responsibly.