Tuesday, February 23, 2010

Understanding the GROUPING SETS

SQL Server 2008 introduced GROUPING SETS as an extension to the GROUP BY, facilitating us to combine more than one group into a single result set. We can think it is as, combining multiple grouped aggregations into a single result set by using UNION ALL.

This is almost similar to CUBE and ROLLUP operators but easy to implement and offers best performance. Not only that, it is ANSI SQL 2006 compliant too.

Let’s understand this. The below code creates a SalesTransactions table and inserts 6 records. Then query to see the Sales Amount by (1.) Year, (2.) Customer, (3.) Year, Customer and Product. Result is shown too.

   1: USE tempdb
   2: GO
   3:  
   4: CREATE TABLE SalesTransactions
   5: (    [Year] smallint
   6:     , Customer varchar(30)
   7:     , Product varchar(30)
   8:     , Amount money
   9: )
  10: GO
  11: INSERT INTO SalesTransactions VALUES
  12:     (2001, 'Hendergart', 'Mountain-100 Black, 42', 100)
  13:     ,(2001, 'Hendergart', 'AWC Logo Cap, 42', 200)
  14:     ,(2001, 'Hendergart', 'Long-Sleeve Logo Jersey, M', 600)
  15:     ,(2001, 'Collins', 'Long-Sleeve Logo Jersey, M', 400)
  16:     ,(2002, 'Collins', 'Mountain-100 Black, 42', 300)
  17:     ,(2002, 'Collins', 'Mountain-100 Black, 42', 100)
  18:  
  19: SELECT [Year], SUM(Amount) Amount
  20: FROM SalesTransactions
  21: GROUP BY [Year]
  22:  
  23: SELECT Customer, SUM(Amount) Amount
  24: FROM SalesTransactions
  25: GROUP BY Customer
  26:  
  27: SELECT [Year], Customer, Product, SUM(Amount) Amount
  28: FROM SalesTransactions
  29: GROUP BY [Year], Customer, Product

2
If all grouped aggregations need to be compiled into a single result set, we can use UNION ALL as below.

   1: SELECT [Year], NULL AS Customer, NULL AS Product, SUM(Amount) AS Amount
   2: FROM SalesTransactions
   3: GROUP BY [Year]
   4: UNION ALL
   5: SELECT NULL AS [Year], Customer, NULL AS Product, SUM(Amount) AS Amount
   6: FROM SalesTransactions
   7: GROUP BY Customer
   8: UNION ALL
   9: SELECT [Year], Customer, Product, SUM(Amount) AS Amount
  10: FROM SalesTransactions
  11: GROUP BY [Year], Customer, Product
union

Okay, that’s the old technique, see how easily we can generate the same result set by using GROUPING SET. All you have to do is, add required grouping in GROUPING SET.

   1: SELECT [Year], Customer, Product, SUM(Amount) AS Amount
   2: FROM SalesTransactions
   3: GROUP BY GROUPING SETS ([YEAR], (Customer), ([YEAR], Customer, Product))
grouping

I see a small issue with this result set. Too much of NULLs. Do you see it too? These NULLs are not real NULLs, how can we identify whether the NULLs in the result set are real or not? There are two ways, you can use either GROUPING function that accepts one parameter as the column name or GROUPING_ID that accepts multiple columns as a parameter. First function returns 0 or 1 that indicates whether it is a real NULL or result of the GROUPING respectively. The second function returns a bitmask that shows whether passed columns contains NULLs or not.

   1: SELECT [Year], Customer, Product, SUM(Amount) AS Amount
   2:     , GROUPING_ID([Year], Customer, Product) AS Bitmask
   3:     , GROUPING([Year]) AS YearGrouping
   4:     , GROUPING(Customer) AS CustomerGrouping
   5:     , GROUPING(Product) AS ProductGrouping
   6: FROM SalesTransactions
   7: GROUP BY GROUPING SETS ([YEAR], (Customer), ([YEAR], Customer, Product))

Monday, February 22, 2010

Understanding the TOP and TABLESAMPLE operators

Result set can be filtered in many ways. Most common ways is, using the 3rd primary property of the SQL SELECT statement which is the WHERE condition which filters specific set of rows from the result set.

This post is not about WHERE condition, it is all about TOP and TABLESAMPLE.

The TOP operator gives you top n number of records as you need. The “top n” can be changed by changing the order of the result set by using ORDER BY clause.

The TABLESAMPLE operator allows you to randomly pick data from the table. You can instruct to SQL Server to return specific number of records or percent of rows.

Where we can use this? I think that this is commonly used for getting the average from a large result set. If you have a table with millions of records and average has to be calculated on one of the columns, it is worthwhile to use TABLESAMPLE (10 PERCENT) rather than using all records.

   1: -- use all rows in the table for calculating
   2: SELECT AVG(Freight)
   3: FROM Sales.SalesOrderHeader
   4: -- use approximately 10% of rows
   5: SELECT AVG(Freight)
   6: FROM Sales.SalesOrderHeader
   7: TABLESAMPLE (10 PERCENT)

REPEATABLE OPTION
If you execute the above two codes again and again, the average returns from first statement is always same but second statement. This is because of the way it picks records. If you need the same average for all the execution of second statement, REPEATABLE option should be used. The REPEATABLE option has to be used with a repeat_seed, and as long as the repeat_seed is same and no records have been change, same average is returned.

   1: SELECT AVG(Freight)
   2: FROM Sales.SalesOrderHeader
   3: TABLESAMPLE (10 PERCENT) REPEATABLE (1)

What is SYSTEM option?
The SYSTEM option is optional, but it is used by default though you do not use it in your query.

   1: SELECT AVG(Freight)
   2: FROM Sales.SalesOrderHeader
   3: TABLESAMPLE SYSTEM (10 PERCENT)

The SYSTEM option returns approximate percentage of rows and generates a random value for each data page. SQL Server decides which data pages to be included for the sample based on the random value generated and the percentage specified in the query. If a page is decided to to be included, all rows in the page is included for sampling, else the page is excluded. If you use set the TABLESAMPLE with number of rows, instead of a percentage, number of rows will be converted to a percent and process same way.

Monday, February 15, 2010

Are you calculating Average correctly?

Almost all engineers have used the AVG aggregate function, but have you ever checked whether the result is correct? This was discussed while I was conducting my new SQL class, here is an example for explaining it, where you might make a mistake;

   1: CREATE TABLE dbo.EmployeeCommision
   2: (
   3:     EmployeeId int PRIMARY KEY,
   4:     CommisionGiven money NULL
   5: );
   6:  
   7: -- Inserting five employees' commison records
   8: -- In real implementation, two tables 
   9: -- will maintain for holding employees records
  10: -- and commison records
  11: INSERT INTO dbo.EmployeeCommison 
  12:     (EmployeeId, CommisionGiven)
  13: VALUES
  14:     (1, 50), (2, 50), (3, NULL), (4, 50), (5,50)
  15:   
  16: -- Calculating the average commision given 
  17: -- to an employee, You may use LEFT OUTER JOIN 
  18: -- if two tables are maintained
  19: SELECT AVG(CommisonGiven) 
  20: FROM dbo.EmployeeCommision
  21: -- Result of above is 50, which is wrong in 
  22: -- this case to correct the issue, use 
  23: -- ISNULL function with AVG function
  24:  
  25: SELECT AVG(ISNULL(CommisonGiven, 0)) 
  26: FROM dbo.EmployeeCommision
  27: -- Now the result is 40, which is correct.

The reason for this is because, all aggregate functions ignore NULL values. Only exception is COUNT(*) that counts NULL values.

Wednesday, February 10, 2010

Developers Training Kit for SQL Server 2008 R2

Microsoft has published "SQL Server 2008 R2 - Training Kit for Developers" that contains presentations, demos, videos and hands-on lab related to new features of SQL Server 2008 R2. It is free, all can download. Here is the link: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=fffaad6a-0153-4d41-b289-a3ed1d637c0d This requires new AdventureWorks database, it is available here: http://msftdbprodsamples.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=24854

Tuesday, February 9, 2010

Hands-On Labs: Authoring Reports with Reporting Services 2008 R2

I will be doing a Hands-On labs session tomorrow (10th Wednesday) at TechEd Sri Lanka 2010, on Authoring Reports with Reporting Services 2008 R2. This session includes some of the new features of R2 like Maps, Sparklines, DataBars, Shared DataSets, Shared Report Items, and more if time is permitted. If you interest, make sure you are at the lab after lunch, it has been scheduled from 1.15pm to 3.15pm.

Monday, February 8, 2010

TechEd Sri Lanka 2010

It is happening..... It started yesterday and continues till Wednesday. It seems that a lot of good sessions have been scheduled, will blog about sessions later. If you guys come, visit our MVP stall, we have arranged a kind of raffle... you can try out an easy puzzle and grab a portable hard-disk :). If you guys interest in Business Intelligence, visit my company stall, IronOne stall and see what we have as Business Intelligence products. for more info about TechEd, visit http://www.teched.lk.

Monday, January 11, 2010

First Presentation in 2010: Reporting Services 2008 R2 – New Features

2010, My first public presentation for the year, going to start with exciting features of Reporting Services 2008 R2.

I will be doing a session on Wednesday (tomorrow) at Microsoft and will be discussing many new features such as Map, Sparklines, Indicators, Report Parts, and much more.

As usual, we will be doing two sessions, second one will be delivered by Gogula on CLR Integration. Please visit http://sqlserveruniverse.com/content/ssslugmain.aspx for info, and join with us in the evening.

images are taken from http://blogs.msdn.com/blogfiles/seanboon/WindowsLiveWriter/HowToBuildSparklineReportsinSQLServerRep_13C99/SSRS%20Sparkline_thumb.jpg and http://blogs.msdn.com/blogfiles/robertbruckner/WindowsLiveWriter/SQLServer2008R2TechEd2009_6DC/SalesStrategy2009_thumb.png

Monday, January 4, 2010

Awarded Microsoft MVP again for 5th time

MVPLogo I am fortunate enough to continue my IT career with MVP title again!

Microsoft has awarded the title me for the 5th time, thanks for Microsoft, Microsoft Sri Lanka, Wellignton and Lilian for recommending to get this title, letting me to continue my community support.

Thursday, December 31, 2009

Happy New Year!

Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010! Wish you all a very Happy and Prosperous New Year 2010!

Tuesday, December 8, 2009

Fixed: SharePoint 2010 Installation Error: Microsoft Geneva Framework Runtime: download error

No more errors with SharePoint 2010 prerequisites installation, and good thing is, no manual installation of any components.

If you refer my previous post that speaks about issues with SharePoint 2010 prerequisites installation and some fixes I did (I found from various post), you can see that it was unable to fix and install SharePoint by installing any. I started again, here is the screenshot with new installation....

Untitled

All I did was:

  • Installed W2008 R2 64-bit
  • Updated Windows (with automatic updates)
  • Installed SQL Server 2008 R2 Nov CTU

NO additional downloads and installations, if you face for same, try again and see.

Tuesday, December 1, 2009

SharePoint 2010 Installation Error: Microsoft Geneva Framework Runtime: donwload error

I have been trying to install SharePoint 2010 Beta on Windows 2008 R2 (64-bit) for a day but not successful yet.... Initially, I got four issues: Microsoft "Geneva" Framework Runtime: download error Microsoft Chart Controls for Microsoft .NET Framework 3.5: Installation skipped Microsoft Filter Pack 2.0: Installation skipped Microsoft SQL Server 2008 Analysis Services ADOMD.NET: Installation skipped In order to solve these issues, I downloaded all four and manually installed: Microsoft "Geneva" Framework Runtime Microsoft Chart Controls for Microsoft .NET Framework 3.5 Microsoft Filter Pack 2.0 Microsoft SQL Server 2008 Analysis Services ADOMD.NET I installed a hotfix for Windows (KB976462) too. While searching, two more hotfixes were found to be installed, KB971831 and KB968930, but OS did not allow me to run them. Finally I ended up with two issues, one with Geneva Framework Runtime and Filter Pack 2.0. Anyone can help me on this?

Sunday, November 29, 2009

Deep-Dive sessions at Peradeniya University

This was my third visit to Peradaniya University, we (Wela, Prabath, Susantha, Tharindu, and me ) did set of sessions on .NET, C#, and SQL Server, and everything went very well, a lot of questions..... Here are some photos: Wela: Showing Silverlight
Dinesh: Delivering SQL Server presentation
Prabath: Delivering presentation on C#
Susantha: Showing Windows Operating Systems
Tharindu: Explaining the importance of communities-Student Champ
Part of the audience

Wednesday, November 25, 2009

A lesson from my son - History of Computers

Thought to make a post on this, may help you to brush up your computer history knowledge too :) • The name of the first computer is ENIAC (Electronic Numerical Integrator and Calculator) • Abacus is the first known calculating device invented in China. • Charles Babbage is known as the Father of computers, who invented the first mechanical computer which was called the Analytical Engine.

Be in the know

I was not in the know, if you too, read this Merill's post, it is about the history of Microsoft Office Web Apps.

Tuesday, November 24, 2009

Connect to 64-bit Oracle 10g from Reporting Services 2008

Recently, I had to create couple of Reporting Services 2008 reports by using Oracle 10g as the data source. I had to use one machine as the developer machine and the server. Server was installed with 64-bit Windows 2008 and SQL Server 2008 64-bit. Oracle was installed in another machine and it was 64-bit too. Here are some of the problems, issues I faced, It may help you too. Connectivity problem with BIDS First problem I faced was connectivity with Oracle in BIDS. I tried to use Oracle provider that comes with SQL Server installation but it did not work. So the solution was, install Oracle full client (Administrator). What should be installed? 32-bit or 64-bit? In order to make the connection via BIDS, we need to install Oracle 32-bit client though the server is 64-bit. The main reason for this is, BIDS is a 32-bit application. Once 32-bit Oracle client installed, BIDS was able to connect to Oracle. Connectivity problem with Report Manager This was the second problem. Once the reports are publish, I got the same error when reports are viewed with Report Manager. This is because it requires Oracle 64-bit client. Once Oracle 64-bit client installed, it started working. OLEDB Provider for Oracle or Microsoft Oracle Provider? When I google, I saw some posts related this provider, it seems that Microsoft Oracle provider had not worked for many but it worked for me. I was able to make connection by using both providers. Passing Parameters to Oracle As usual, we can use "?" for represent parameters in the query if the provider is OLE DB (eg. WHERE column1 = ?). But remember, you need to use ":" for parameters if the provider is Microsoft Oracle provider (eg. WHERE column1 = :Parameter1) IN Clause with WHERE "IN" is not supported by OLE DB provider. If you have a requirment that needs to use "IN" with "WHERE", use Microsoft Oracle provider instead.

Monday, November 16, 2009

IIS 7 Stopped. PerformancePoint Error ???

It hit me again, this is the second time I experienced this problem. My VPC with Windows 2008, SQL Server 2008 and MOSS 2007 was working fine until I installed and configured the PerformancePoint. My bad, I did not take a backup of my VPC. Once the PerformancePoint configured (with SP3), I noticed that the Default Application Pool was getting stopped even after restarting and resetting. The entire IIS 7 stopped working. The error was 503, Service is unavailable. I applied all the solutions what I found with my searches, but as most have done, I have to start from the beginning. No solutions, but reinstall whole thing.......

PerformancePoint Server 2007 Service Pack 3

If you are unaware of SP3, here is the link, x86 and x64.

Wednesday, November 4, 2009

PerformancePoint error: Part III - Unable to connect to Server - SSRS 2008

It hit again, this time it is with Reporting Services 2008. PerformancePoint allows to connect with Reporting Server and lets to browse reports in the server, but the problem comes when we try to connect with the report. When the report is selected from the Browser, it throws an error saying "Unable to connect to server". After few minutes, we realized that the problem comes only with reports that have parameters, other reports can be connected. This is a bug and the hotfix is available here. For your reference, I had blogged two more issues I faced before, you can find them here (Part II)and here (Part I).

Monday, November 2, 2009

DML, DDL, DCL, TCL, DQL

When we discuss, when I teach, the mostly discussed SQL Server related languages are DML and DDL. But do you know that there are few more categories, they are DCL, TCL and DQL. Here is a brief note of it; DML - Data Manipulation Language - statements that perform changes to the database. Eg. INSERT, UPDATE, DELETE DDL - Data Definition Language - statements that modify objects in the database. Eg. CREATE TABLE, CREATE VIEW, DROP PROCEDURE DCL - Data Control Language - statements that controls the rights to objects. Eg. GRANT, DENY, REVOKE TCL - Transactional Control Language - statements that controls transactions in the database. Eg. COMMIT, ROLLBACK, SAVE POINT DQL - Data Query Language - statements that query the database. Eg. SELECT

Wednesday, October 28, 2009

Native OLE DB\SQL Server Native Client 10.0 does not list out the available databases

This held our work for few hours. When we tried to make a connection to SQL Server 2008 server with “Native OLE DB\SQL Server Native Client 10.0” provider from a developer’s machine through SSIS, it did not list out available databases, and did not allow to connect to a database even the name of the database is just typed in the “Select or enter a database name” input box. We doubt that the problem was related to some network problem or to some installed components but, since it allowed to connect with “Native OLE DB\Microsoft OLE DB Provider for SQL Server”, thought that it is related to “SQL Server Native Client 10.0”. We spent hours to find the reason but could not find any issue related to SQL Server components. Though the connection can be established with “Microsoft OLE DB Provider for SQL Server”, since it does not allow to run some of the new TSQLs and specially it does not recognize “Date” data type properly, we had to dig deeper…. Finally, my colleague Buddhika found the issue. Windows Firewall Settings was blocking the Native Client 10.0 requests. Once it is turned off, connection could be established. Again, turning off the settings is not a good practice, we tried to find a way to make a connection while Firewall Settings is on, and we found. Simply, the TCP port 1433 added as an exception for the Firewall Settings. It worked. So, in shorter form, if you cannot make a connection to SQL Server 2008 with Native OLE DB\SQL Server Native Client 10.0” provider, make sure that the SQL Server listening port is added as an exception with Windows Firewall Settings. I am sure that we will be getting the same with Analysis Services too, and adding TCP port 2383 as an exception might solve the issue.

Tuesday, October 27, 2009

Slowly Changing Dimension: Char for Type II Attributes

I used to use Char data type for character data types of variable length columns, if the maximum number of characters that will set for the attribute is less than 10. This is what most DBAs/DBEs do. Not only the maximum length, there are few more considerations for using Char instead of Varchar for character data types of variable length. One would be, when the length of all data values is approximately same. I made a mistake; this cannot be applied in everywhere, especially in Relational DW with Type II attributes. For example, if I have to make a dimension table with Marital Status that contains either Single or Married, I can make the column as either varchar(8) or char(8), so I made it as char(8) because I prefer to make my design according to experts suggestions. Unfortunately this is a Type II attribute which is designed to maintain history, hence when the record contains “Single”, even though the same value comes with a new data set, it considers as a changed (because of the length) and inserts a new record, making the old record as a historical record and it continues with next data set... It unnecessary adds new records to the table making the old ones as historical records. So, I made the all the Type II columns as varchar columns that were set as char to solve the issue, what do you think, any suggestions, any thoughts on this?

Thursday, October 22, 2009

Another SQL Server new course.

Last week, two batches, one for 2780B (SQL 2005) and another for 6231A (SQL 2008) were successfully completed and will be starting my next class, most probably within next two weeks, for 2778: Writing Queries with Microsoft SQL Server 2008, at NetAssist. If you interest to join my class, you are welcome. Although I started my conducting classes on.NET, last two years I have been doing classes, workshops only for SQL Server and Business Intelligence. Just thought to focus on two more subjects. Working closely with one of the reputed institutes regarding these two subjects, hope will be able to start them by January, 2010.

USER_NAME(), SUSER_NAME(), ORIGINAL_LOGIN()

Sometimes, we switch the execution context to different account when required. One of requirements when connected in such a manner may be, finding out the original account. This can be retrieved from ORIGINAL_LOGIN function. Not only this, the other functions such as USER_NAME, SUSER_NAME are useful too, if you need to info return from them. Thought to put down a small code, just to show the different between these functions; Login as "sa" and execute... -- Create a login and a user USE master GO CREATE LOGIN TestLogin WITH PASSWORD = '123', CHECK_POLICY = OFF GO USE AdventureWorks GO CREATE USER TestUser FROM LOGIN TestLogin -- Test the functions -- Returns dbo SELECT USER_NAME() -- Returns sa SELECT SUSER_NAME() -- Switch the execution context EXECUTE AS LOGIN = 'TestLogin' -- Returns TestUser SELECT USER_NAME() -- Returns TestLogin SELECT SUSER_NAME() -- Returns sa SELECT ORIGINAL_LOGIN() REVERT; USER_NAME: Returns the current user in the current context. If the user_id is submitted, returns the name of the given id. SUSER_NAME: Returns the current login in the current context. If server_user_id is submitted, returns the name of given id. ORIGINAL_LOGIN: Returns the original login in the session in which there are many implicit or explicit context switches.