Philip Hendry's Blog

September 29, 2009

Generating a list of numbers in T-SQL

Filed under: SQL Server, T-SQL, Tip — philiphendry @ 8:33 am

I wanted to create a list of numbers so I could cross join them to another set of data to generate some test data quickly. There are ways to generate numbers using a CTE (Common Table Expression) or if you’re in a real hurry you can make us of a fairly undocumented table called master..spt_values. Seeing as this table isn’t well documented don’t go using it for production code since it might change but for quick one-off tasks it seems fine.

The following SQL creates a list of numbers from 0 to 2047 :

select [number]
from master..spt_values
where type = 'P'

September 21, 2009

Sproc Compile Locks

Filed under: MSDN Article Summary, SQL Server, Tip — philiphendry @ 7:28 pm

I came across an article on Microsofts support site which has raised a point which I’d never considered before. It basically says that if the user that executes a stored procedure is not the owner of the procedure then locks are acquired during the process to find a cached plan that may lead to blocking. It also states this could be avoided by fully qualifying the sproc name (e.g. dbo.mysproc) when calling it to avoid the extra lookups. This has been a problem for me in the past since the projects I’ve worked on had been setup with an account running as dbowner, however, if I was setting up a database from scratch I’d set up a specific user(s) for accessing the database and give it just the permissions it requires (sproc execute permissions for example.) I think, therefore, this would fall into these problem. However, I would also make use of the schema prefix (e.g. Sales.Order table) rather than leaving it as dbo and therefore avoid this problem altogether.

September 15, 2009

Importing from Excel into SQL Server

Filed under: SQL Server, T-SQL — philiphendry @ 10:54 am

I was going through a mountain of pain trying to get an import working from an Excel spreadsheet working into SQL Server 2005 but I finally managed it and here’s how…

Here’s the SQL I’m using (slightly abridged!) :

set transaction isolation level read committed

declare @sql nvarchar(1024)
set @sql = 'select * into ##Temp from opendatasource(''Microsoft.Jet.OLEDB.4.0'', ''Extended Properties=Excel 8.0;Data Source=' + @ExcelFileToImport + ''')...' + @ExcelWorkSheet
exec sp_executesql @sql

Points to note :

  • I’m using dynamic sql because the opendatasource() doesn’t like being passed variables… so be sure to validate the contents of @ExcelFileToImport and @ExcelWorkSheet.
  • Because I was forced to use dynamic sql I had to use a global temporary table. I wanted to load the data into a temporary table so that the columns were generated on-the-fly and I could check the correct columns were supplied by examining tempdb.sys.columns. It therefore had to be a global temporary because a standard one would have been thrown away after sp_executesql() had finished! For my situation the global temporary table is ok… but you’ll have to consider your own situation and whether concurrency allows.
  • I kept getting the error below on the subsequent re-tries of uploading but worked around this by adding ‘set transaction isolation level read committed’ in the sproc.

    The requested operation could not be performed because OLE DB provider "Microsoft.Jet.OLEDB.4.0" for linked server "(null)" does not support the required transaction interface.

I was having trouble executing within a transaction but with the changes made above the sql could be called with the following :

using (TransactionScope scope = new TransactionScope())
{
    UploadExcelIntoDatabase(importTicket, importFilename);
    scope.Complete();
}

September 11, 2009

Conditional Aggregate Totals in SQL

Filed under: SQL Server, T-SQL, Tip — philiphendry @ 8:03 am

I’ve been working on a SQL Server Reporting Services report which had to display several counts on each row. Each row represented a project and each project could have several tasks that were either ordinary tasks or milestones and could also be escalated. The report need to show something like this :

Project Name Milestone Counts Task Count Escalated Task Count
1st Week 2nd Week 3rd Week >=4th Week Total 1st Week 2nd Week 3rd Week >=4th Week Total

As you can see a task initially has to break down into two major colours of milestone and ordinary task counts but then is broken down into counts in the 1st, 2nd, 3rd and greater than 4th week totals including the total itself.

Initially this seemed tricky when thinking in terms of using count() but there’s an incredibly simple technique that solves this but which isn’t immediately obvious. The Sum() aggregate function can contain an expression and in this example that expression could compare the task date against late week dates and return either a 1 or 0 :

select ProjectName,
   sum(case when IsMilestone = 1
      and TaskDate < @LateDate
      and Taskdate >@OneWeekLateDate
      then 1 else 0 end) as Week1LateMilestoneCount
from Project
group by ProjectName

I’ve only included one example there but I think it’s enough to get the idea.

September 2, 2009

Selecting a result set from a stored procedure in SQL Server

Filed under: SQL Server, T-SQL, Tip — philiphendry @ 10:55 am

Selecting data from a stored procedure in SQL Server is already a documented feature and here’s an example:

insert into #systables exec sp_executesql N'select * from Northwind.sys.tables'

…problem is this example doesn’t run without first creating the temp table and therefore knowing all the column definitions. When I’m running quick queries this isn’t exactly convenient. I’ve seen blogs posts using a linked server but there’s another way :

select * into #systables
from openrowset(
   'sqlncli',
   'server=.;trusted_connection=Yes',
   'sp_executesql N''select * from Northwind.sys.tables'''
)

I wouldn’t necessarily use this as a day-to-day process on a production environment but for administration or scripting installations I think it fits the bill.

July 31, 2009

String Concatenation in T-SQL and Importing from Excel

Filed under: SQL Server, T-SQL — philiphendry @ 10:34 am

I keep forgetting how simple this little trick is so it’s time to blog it.

I wanted to write some T-SQL that would confirm that the columns just created by an import from an Excel spreadsheet into a temporary table contained the expected columns irrespective of order or other columns that exist in the file. I also wanted to be able to inform the user of the problem instead of just fail with a horrible error. I chose a fairly simple method, I would concatenate the columns from the temporary table (#tempImport in this case) and compare that with an expected string. If they don’t match I can report back to the user that the expected columns do not exist (in this implementation I’m not interested reporting which columns are missing.) The implementation I chose was this :

select * into #tempImport
from openrowset('Microsoft.Jet.OLEDB.4.0',
   'Excel 8.0;Database=e:\temp\aSpreadsheet.xls', Sheet1$)

declare @columns varchar(512)
set @columns = ''

select @columns = @columns + ',' + [name]
from tempdb.sys.columns
where [object_id] = object_id('tempdb..#tempImport')
    and [name] in ('Store','Store Name','PSA1','PSA2','Mtrg')
order by [name]

if (@columns <> ',Mtrg,PSA1,PSA2,Store,Store Name')
    print 'The expected columns in the spreadsheet do not exist'

This is a brute-force type of implementation but I like it because it’s very simple and keeps everything in one place. There will be better ways to implement this but for the moment this fits the bill.

Update:

http://support.microsoft.com/kb/326548 is a useful guide to using the ISAM provider.

September 5, 2008

Generating random numbers in a SQL Update

Filed under: SQL Server, T-SQL — philiphendry @ 8:11 am

I wanted to fill a column in a table with random numbers so off the top of my head wrote the following sql :

update code set serialNumber = floor(rand() * 30000000000)

Unfortunately this set the serialNumber to exactly the same value since RAND() isn’t being re-executed for each row. I was in a bit of a hurry so instead found the following on google :

update code set serialNumber = abs(cast(cast(newid() as binary(4)) as int)) + 30000000000

Newid() is executed for each row and in this example it’s being used to created (aka ‘bodge’) an int. An interesting solution and did the trick for me.

March 14, 2008

SQL Server Replication : Lesson 1

Filed under: Lesson, Replication, SQL Server — philiphendry @ 5:13 pm

I’ve decided to chronicle things I learn about replication (above and beyond the normal things to learn in the documentation) after a very hard lesson yesterday during an upgrade which lasted a lot longer than necessary.

So, what’s the first lesson I’ve learnt…

  • Just because you don’t have any subscribers doesn’t mean to say that the publisher won’t end up doing a lot of work.
  • Snapshot generation can be locked by the log reader agent.

So how did I come across this fact? Well, we had planned a production server upgrade which involved installing a new server with all the publications against an existing database of about 12Gb. However, before installing the subscribers, it was planned to run some updates against the database in order to prevent a lot of data from being replicated and therefore extending the upgrade time.

However, the lesson learned was that logread.exe (the log reader agent) scans through all the transactions found in the transaction log and inserts into the distribution database whether there are any subscribers or not. I’ve assumed that logread.exe only includes data from the transaction log for replication articles and ignores anything else.

These current facts lead to the problems experienced in the implementation. We had run a couple of very substantial queries which had filled the transaction log to almost 8Gb therefore giving a lot of work for the log reader agent to perform – although we had assumed there wouldn’t be any work because there were no subscribers!

After the queries had finished we began creating our subscribers – however, we found that the snapshot agents would not complete. A quick look with sp_who2 identified the problem – they were being blocked by the log reader agent – something we definitely didn’t expect.

We decided that to expedite the implementation we would stop the log reader agent then run the snapshots. Big mistake! The log reader stopped but issued a rollback. What we didn’t realise was that the transaction must have been huge and we waited almost 45 minutes for it to stop! However, once it had finished we could trigger the snapshots and continue as expected.

So next time, I’ll not underestimate the effects of any query I run on a server that has replication installed!!

March 5, 2008

Exclusive Queries in SQL Server

Filed under: SQL Server — philiphendry @ 4:58 pm

We had a major problem last week when someone ran a query manually that was only ever designed to be run once under controlled conditions. To put this right we decided to add some protection to the stored procedure that would prevent it from running twice in a row however, because we don’t employ a transaction around the whole stored procedure (it does too much for that) we couldn’t use the simplest option sp_getapplock which requires a transaction.

Although a little nasty, a simple and effective solution was used. A temporary table was created by the stored procedure is that temporary table didn’t already exist in tempdb.sys.objects. When the stored procedure ends the temporary table is automatically removed (although if the stored procedure raises an exception there’s a bug in SQL Server 2005, fixed in SP2, that leaves the temporary table there!!)

Blog at WordPress.com.