Showing posts with label SQL Tips. Show all posts
Showing posts with label SQL Tips. Show all posts

Tuesday, March 19, 2024

Updating Table Containing Xml Column via LinkedServer

If you are trying to update a table containing XML column via Linked Server in SQL Server, and you are not able to, you are not alone.

There are 2 main issues in this scenario:
1. Accessing a table containing XML column via Linked Server
2. Updating the XML value

When trying to select the XML column of a table via linked server:

SELECT col1, col2, XmlColumn
FROM [LinkedServer].[DatabaseName].dbo.TableName

Error encountered:

Xml data type is not supported in distributed queries. Remote object '[LinkedServer].[DatabaseName].dbo.TableName' has xml column(s).

There are a few workarounds to resolve this (we will focus on using view in this post):

1. Use OPENQUERY

2. Create a view that selects a converted non-XML value from the XML column in the destination DB.

--In destination DB
CREATE VIEW vw_NonXmlColumn
AS
SELECT col1, col2,
    XmlColumn = CONVERT(NVARCHAR(MAX), XmlColumn)
FROM TableName WITH(NOLOCK)

Then, on the source DB where you create the Linked Server, you can access the view instead:

--In source DB
select col1, col2, XmlColumn
from [LinkedServer].[DatabaseName].dbo.vw_NonXmlColumn

This resolves our first issue: accessing a table containing XML column via Linked Server.

Share:

Wednesday, May 12, 2021

Creating Dynamic Pivot Table using SQL

 In this post, I want to share SQL tips on how to create dynamic pivot table in SQL.

The columns in the pivot table is not fixed in this case. Hence, there is a need to use dynamic SQL to achieve this.

In this example, we will be forming a pivot table containing spending by outlet from spending data.

SQL Query:

--prepare data source
create table #TmpSpending
(
	MemberID nvarchar(100),
	Outlet nvarchar(100),
	TransactDate date,
	Spending numeric(18,4)
)

insert into #TmpSpending values ('M0001', 'Outlet A', '2021/01/05', 100)
insert into #TmpSpending values ('M0001', 'Outlet B', '2021/01/15', 200)
insert into #TmpSpending values ('M0002', 'Outlet A', '2021/02/09', 300)
insert into #TmpSpending values ('M0003', 'Outlet B', '2021/01/10', 400)
insert into #TmpSpending values ('M0003', 'Outlet B', '2021/02/15', 500)

select * from #tmpspending

--temporary table for output
create table #tmpOutput (
	Outlet nvarchar(100)
)

--initial insert into #tmpOutput
insert into #tmpOutput
select distinct outlet
from #TmpSpending
order by outlet

--store the list of [month year] first
select distinct [Year] = year(transactdate), 
	[Month] = month(transactdate),
	MonthYear = left(datename(month, transactdate), 3) + ' ' + convert(varchar, year(transactdate))
into #tmpDates
from #tmpspending

--construct the table with dynamic columns using cursor
declare @year int, @month int
declare @monthyear nvarchar(8)
declare @sql nvarchar(max) = '' --dynamic sql for columns addition

declare cur1 cursor for
select [year], [month], MonthYear
from #tmpDates
order by [year], [month]

open cur1

fetch next from cur1
into @year, @month, @monthyear

while @@FETCH_STATUS = 0
begin
	
	--add column to #tmpOutput
	set @sql = 'alter table #tmpOutput add [' + @monthyear + '] numeric(18,4)'
	exec (@sql)

	--update #tmpOutput
	set @sql = 'update o '
	set @sql += 'set [' + @monthyear + '] = s.spending '
	set @sql += 'from #tmpOutput o '
	set @sql += 'inner join (	select outlet, spending = sum(spending) '
	set @sql += '				from #tmpspending '
	set @sql += '				where year(transactdate) = ' + convert(varchar, @year) + ' '
	set @sql += '				and month(transactdate) = ' + convert(varchar, @month) + ' '
	set @sql += '				group by outlet '
	set @sql += ') s on s.outlet = o.outlet '
	exec(@sql)

	--fetch next record
	fetch next from cur1
	into @year, @month, @monthyear
end

close cur1
deallocate cur1

select * from #tmpOutput

drop table #tmpDates
drop table #TmpSpending
drop table #tmpOutput


First, we prepare the data source which looks like this.



Afterwards, we use cursor (or you can use while loop) to create additional columns on the table on the fly, to form up the pivot table.
This is the end result of the pivot table.


Happy querying! :)
Share:

Wednesday, November 4, 2020

Split Comma-Delimited Column into Multiple Rows

We had post about combining multiple rows into a column in the past. Sometimes, on the contrary, we would want to split character-delimited column into multiple rows. In this post, we will see try to split comma-delimited column into multiple rows. Sample Data:
End result after split into rows:
Query:

-- Create temporary table to store UserGroup
create table #UserGroup (GroupID int, UserGroup nvarchar(1000), Users nvarchar(1000))
insert into #UserGroup values (1, 'Group 1', 'Himura,Selvi,Superman')
INSERT INTO #UserGroup VALUES ( 2, 'Group 2', 'Luck,Smarty')
INSERT INTO #UserGroup VALUES ( 3, 'Group 3', 'Lucky')

--Display the sample data
select * from #UserGroup

--Split comma-delimited column into multiple rows
;WITH tmp(GroupID, UserGroup, Username, Users) AS
(
	SELECT GroupID, 
		UserGroup,
		convert(nvarchar(1000), LEFT(Users, CHARINDEX(',', Users + ',') - 1)),
		convert(nvarchar(1000), STUFF(Users, 1, CHARINDEX(',', Users + ','), ''))
	FROM #UserGroup
	where Users is not null
	UNION all

	SELECT GroupID, 
		UserGroup,
		convert(nvarchar(1000), LEFT(Users, CHARINDEX(',', Users + ',') - 1)),
		convert(nvarchar(1000), STUFF(Users, 1, CHARINDEX(',', Users + ','), ''))
	FROM tmp
	WHERE
		Users > ''
)
select GroupID, UserGroup, Username
from tmp
order by GroupID, UserGroup
Share:

Thursday, September 19, 2019

Using SQL Cursor for Looping

If you ever heard of SQL cursor, this is probably not a new thing to you. If you never heard or use cursor in SQL before, we usually use cursor to replace the ordinary looping in SQL. The performance is generally better compared to using WHILE or ordinary looping method, depending on how it is implemented. Reason being, cursor is stored in memory.

I have a good example on usage of cursor in SQL.
In CRM & loyalty, we are looking at membership and loyalty points. Customers earn points when they transact or purchase something.
In this scenario, there is a points cap for members transactions, in which customers can only earn x number of points from their purchase.

We want to retrospectively calculate the correct points earned given customer's transactions and the points they earned.

Example of the records:

Points is capped at member-level. This means, different customer / member can have different points cap, denoted in the last column in the example above.
Since we are looking at member-level, we will look at the re-arranged version of the records.

This is the end result we are looking at. Observe the last field in the table. The content in the Correct_PointsEarned is always within the points cap for each member, in the order of the transaction sequence. Once the points cap has been met, the Correct_PointsEarned will be 0 for that member.

You should familiarize yourself with how to create a cursor first.
This is a template of how a cursor looks like:
--START CURSOR
declare cur1 cursor for
select *
from 
order by 

open cur1

fetch next from cur1
into @zzz, @sss, ......

while @@FETCH_STATUS = 0
BEGIN
 --do the necessary processing here
 
 
 --fetch next record
 fetch next from cur1
 into @zzz, @sss, ......
END

close cur1
deallocate cur1
--END CURSOR


This is the script to create temp table to populate the data:
create table #TmpPointsCap (
 TransactAutoID bigint identity(1,1),
 MemberID nvarchar(100),
 TransactDate date,
 TransactPoints numeric(18,4),
 PointsCap numeric(18,4),
 Correct_PointsEarned numeric(18,4) default(0.00),
 AddedOn datetime default(getdate())
)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0001', '2019/01/01', 100, 700)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0002', '2019/01/02', 200, 500)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0004', '2019/01/02', 400, 300)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0001', '2019/01/03', 500, 700)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0003', '2019/01/03', 700, 1000)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0001', '2019/01/05', 200, 700)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0002', '2019/01/06', 100, 500)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0003', '2019/01/06', 300, 1000)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0002', '2019/01/06', 400, 500)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0002', '2019/01/08', 200, 500)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0003', '2019/01/08', 100, 1000)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0004', '2019/01/09', 100, 300)

insert into #TmpPointsCap (MemberID, TransactDate, TransactPoints, PointsCap)
values ('M0005', '2019/01/10', 200, 200)

--Original table content
select * from #Tmppointscap

--The data sorted in the order we want
select TransactAutoID, MemberID, TransactDate, TransactPoints, PointsCap, Correct_PointsEarned = 0
from #TmpPointsCap
order by MemberID, TransactDate, AddedOn

And this is how we use cursor to achieve the correct point earned in the last column:
--Temporary variables to store the table fields
declare @MemberID nvarchar(100), @TransactAutoID bigint, @TransactPoints numeric(18,4), @PointsCap numeric(18,4)
declare @PrevMemberID nvarchar(100) = ''
declare @RunningPoints numeric(18,4) = 0


--START CURSOR
declare cur1 cursor for
select MemberID, TransactAutoID, TransactPoints, PointsCap
from #TmpPointsCap
order by MemberID, TransactDate, AddedOn

open cur1

fetch next from cur1
into @MemberID, @TransactAutoID, @TransactPoints, @PointsCap

while @@FETCH_STATUS = 0
BEGIN
 --do the necessary processing here
 if @MemberID <> @PrevMemberID
 begin
  --set initial running points as the 1st TransactPoints
  set @RunningPoints = 0
 end
 
 if @RunningPoints < @PointsCap and (@RunningPoints + @TransactPoints) > @PointsCap
 begin
  --if after adding the TransactPoints, it is bigger than the pointscap
  --only earn at most, as much as the pointscap
  update #TmpPointsCap
  set Correct_PointsEarned = @PointsCap - @RunningPoints
  where TransactAutoID = @TransactAutoID

  --Update RunningPoints = PointsCap
  set @RunningPoints += @TransactPoints
 end
 else if (@RunningPoints + @TransactPoints) = @PointsCap
 begin
  --earning all the transactpoints just good
  update #TmpPointsCap
  set Correct_PointsEarned = @TransactPoints
  where TransactAutoID = @TransactAutoID
  
  set @RunningPoints += @TransactPoints
 end
 else if @RunningPoints < @PointsCap
 begin
  --update the running points
  set @RunningPoints += @TransactPoints
  
  --update the correct points earned in the table
  update #TmpPointsCap
  set Correct_PointsEarned = TransactPoints
  where TransactAutoID = @TransactAutoID
 end
 else
 begin
  --not earning any points anymore
  update #TmpPointsCap
  set Correct_PointsEarned = 0
  where TransactAutoID = @TransactAutoID
 end
 
 set @PrevMemberID = @MemberID

 --fetch next record
 fetch next from cur1
 into @MemberID, @TransactAutoID, @TransactPoints, @PointsCap
END

close cur1
deallocate cur1
--END CURSOR

--Check the temp table again after updated
select TransactAutoID, MemberID, TransactDate, TransactPoints, PointsCap, Correct_PointsEarned
from #TmpPointsCap
order by MemberID, TransactDate, AddedOn

Good Luck, and Happy Query-ing!
Share:

Monday, June 20, 2016

Find Maximum Value from Multiple Columns

A colleague asked about how to get maximum value from multiple columns. After searching it online, I found a pretty good way to get it, and thought it would be good to share here.

For example, I have a table with MemberID and 3 date fields. I want to get the last date from the 3 date fields for each MemberID.


The expected end result look like:

Creating the sample data:

create table #member ( MemberID nvarchar(100), modifiedon datetime, deletedon datetime, addedon datetime )
insert into #member values ('M1', GETDATE(), GETDATE(), GETDATE())
insert into #member values ('M2', GETDATE()-1, GETDATE()-2, GETDATE()-3)
insert into #member values ('M3', GETDATE()-3, GETDATE()-2, GETDATE()-1)


Query to get the max for multiple columns:

SELECT 
 MemberID, 
 LastUpdateDate =
 ( SELECT MAX(LastUpdateDate)
  FROM (VALUES (modifiedon),(deletedon),(addedon)) AS UpdateDate(LastUpdateDate)
 )
FROM #member

Share:

Friday, January 22, 2016

Calculating Customer's Latency

There were times when I was asked to retrieve the latency of the customers in my work. At first, I was confused about how to retrieve this. After some thought, I find a solution, which is actually pretty simple, and the SQL usage is widely available. Unfortunately, when I tried searching it online, the SQL technique is seldom associated with latency problem in retail industry.

This is an illustration of what we want to achieve.
Given members and their transaction date. We want to find the average latency of members. To get the average latency, we would need to get the difference in days, between member's 1st and 2nd visit, 2nd and 3rd visit, 3rd and 4th visit, etc. The average of these differences will be the average latency of the members.


To do this on SQL, we can rank the member's transaction based on the TransactDate, and then self-join to the same set of records, but shift the ranking by one order. This way, we will get the difference of days between a transaction and the subsequent transaction.

Here are the SQL queries to prepare the data:

--Create temp table
CREATE TABLE #Transact
(
 MemberID NVARCHAR(100),
 RecieptNo NVARCHAR(100),
 TransactDate DATE
)

--Insert dummy data
INSERT INTO #Transact VALUES ('M001', 'R001', '2015/11/20')
INSERT INTO #Transact VALUES ('M001', 'R002', '2015/12/30')
INSERT INTO #Transact VALUES ('M001', 'R003', '2016/01/05')

INSERT INTO #Transact VALUES ('M002', 'R004', '2015/09/12')
INSERT INTO #Transact VALUES ('M002', 'R005', '2015/09/15')
INSERT INTO #Transact VALUES ('M002', 'R006', '2015/09/25')
INSERT INTO #Transact VALUES ('M002', 'R007', '2015/10/30')

Here are the SQL queries to get the Latency per member:

--Store the ranked transaction into temp table. Alternately, use common table expression
SELECT MemberID, TransactDate, RankNo = ROW_NUMBER() OVER(PARTITION BY MemberID ORDER BY TransactDate)
INTO #1
FROM #Transact

--Get the average latency per member
SELECT MemberID, Latency = AVG(CONVERT(NUMERIC(18,2), Latency))
FROM ( SELECT a.MemberID,
   Latency = DATEDIFF(DAY, a.TransactDate, b.TransactDate)
  FROM #1 a
  INNER JOIN #1 b
   ON b.MemberID = a.MemberID
   AND b.RankNo = (a.RankNo + 1) 
) aa GROUP BY aa.MemberID

That's all folks!

Share:

Tuesday, September 23, 2014

Combine Rows into a Column

Happened to find a way to combine rows into a column. I find it quite useful, and it comes in handy.

For illustration, suppose we have UserGroup & Users. We want to list the users belonging to each group.

UserGroup:

Users:

If we do normal joining of the 2 tables, it will look like:

If we combine the values on the rows into a column, it will look like:

Here is the query to do so:
-- Create temporary table to store UserGroup
SELECT GroupID = 1, UserGroup = 'Group 1' INTO #UserGroup
INSERT INTO #UserGroup VALUES ( 2, 'Group 2' )
INSERT INTO #UserGroup VALUES ( 3, 'Group 3' )

-- Create temporary table to store Users
SELECT UserID = 'Superman', GroupID = 1 INTO #Users
INSERT INTO #Users VALUES ( 'Luck', 2 )
INSERT INTO #Users VALUES ( 'Himura', 1 )
INSERT INTO #Users VALUES ( 'Selvi', 1 )
INSERT INTO #Users VALUES ( 'Smarty', 2 )
INSERT INTO #Users VALUES ( 'Lucky', 3 )

-- Normal Join
SELECT a.UserGroup, b.UserID
FROM #UserGroup a
LEFT JOIN #Users b
 ON b.GroupID = a.GroupID
ORDER BY a.UserGroup

-- Combine Rows into Column
SELECT UserGroup, Users = STUFF(( SELECT ', ' + UserID FROM #Users a
    WHERE a.GroupID = b.GroupID
    ORDER BY a.UserID
    FOR XML PATH('') )
   , 1, 2, '')
FROM #UserGroup b
ORDER BY UserGroup

-- This is the key query to combine rows into column
SELECT UserID + ', ' FROM #Users
WHERE GroupID = 1
FOR XML PATH('')

If you are concatenating a field which contains some HTML tag, it would be automatically encoded. To avoid that, consider using this query:
-- Combine Rows into Column
SELECT UserGroup, Users = STUFF(( SELECT ', ' + UserID FROM #Users a
    WHERE a.GroupID = b.GroupID
    ORDER BY a.UserID
    FOR XML PATH,TYPE).value('.[1]','nvarchar(max)')
   , 1, 2, '')
FROM #UserGroup b
ORDER BY UserGroup

Update: Looks like I posted something similar quite some time ago, but using different method: Selecting several rows into one row in SQL
Share:

Thursday, July 12, 2012

Get The First and The Last Day of The Month

Just saw a question from someone on a mailing list I join asking for how to get the transactions occurring for the last three months (in other words, three months from current date).


First, you may want to do something simple.
To get the first day and the last day of the month, we can use:

SELECT 
DATEADD(month, DATEDIFF(month, 0, GETDATE()), 0),
DATEADD(month, DATEDIFF(month, 0, GETDATE())+1, 0) - 1


If you play around a little bit, you can also get the first day and the last day of the previous month:

SELECT 
DATEADD(month, DATEDIFF(month, 0, GETDATE())-1, 0),
DATEADD(month, DATEDIFF(month, 0, GETDATE()), 0) - 1


To get the transactions for the last three months, we just need to modify the number a little bit.

SELECT
DATEADD(month, DATEDIFF(month, 0, GETDATE())-2, 0),
DATEADD(month, DATEDIFF(month, 0, GETDATE())+1, 0) - 1



Then, just select the transactions occurring in between these ranges.

SELECT * FROM TableName WHERE TransactDate BETWEEN
DATEADD(month, DATEDIFF(month, 0, GETDATE())-2, 0) AND
DATEADD(month, DATEDIFF(month, 0, GETDATE())+1, 0) - 1

Share:

Wednesday, July 11, 2012

Select Several Rows into One Row in SQL

In SQL Server, there are times when we want to consolidate the content of several rows into one row.

Though it is not meant for sophisticated purposes, there is actually a simple and easy way to do this.
Suppose we have a Member table containing  MemberID and Name fields.

To select the names of the members in one row, simply use the query:


DECLARE @strResult VARCHAR(5000)
SELECT @strResult = COALESCE(@strResult + ', ', '') + Name
FROM Member

SELECT MembersName = @strResult


The result will be:


We can also add some conditions to the select statement.
e.g. To generate the members with name containing 'Himura', we can simply do:


DECLARE @strResult VARCHAR(5000)
SELECT @strResult = COALESCE(@strResult + ', ', '') + Name
FROM Member
WHERE Name LIKE '%himura%'

SELECT TheHimura = @strResult


And we'll get:

That's all folks :)
Share:

Thursday, May 5, 2011

Paging in SQL 2

Regarding my post about Paging in SQL, I have found another easier way to include a column as a running number of the records. Hence, we do not need to create a temporary table, and the code will be much simpler.

I will use the same tables for example. In case you find difficulty finding the post about Paging in SQL, I include them here.

Here is the Create table Query:
CREATE TABLE dbo.MsUser
(  UserID CHAR(20) PRIMARY KEY,
   Username VARCHAR(100),
   Address VARCHAR(50),
   DivisionID INT
)

And here is the query to insert sample data:
-- Insert Data into MsUser
INSERT INTO dbo.MsUser VALUES ( 'Himura', 'Miss Himura', 'Earth', 3 )
INSERT INTO dbo.MsUser VALUES ( 'Selvia', 'Selvia', 'Indonesia', 2 )
INSERT INTO dbo.MsUser VALUES ( 'Superman', 'Clark Kent', 'Earth', 2 )
INSERT INTO dbo.MsUser VALUES ( 'SelviaHimura', 'Selvia Himura', 'Earth', 4 )
INSERT INTO dbo.MsUser VALUES ( 'Luck', 'Steven Luck', 'Indonesia', 1 )
INSERT INTO dbo.MsUser VALUES ( 'SuperLuck', 'Super Luck', 'United States', 1 )
INSERT INTO dbo.MsUser VALUES ( 'Selvi', 'Selvi', 'Indonesia', 1 )
INSERT INTO dbo.MsUser VALUES ( 'Lucky', 'Lucky Luke', 'United States', 3 )
INSERT INTO dbo.MsUser VALUES ( 'Steven', 'Steven', 'Earth', 4 )

Here is the data inserted to the tables:

Here is the query to get the data using paging:
DECLARE @PageSize INT, @PageNo INT
SELECT a.Username FROM (
    SELECT Username, RowNumber = ROW_NUMBER() OVER (ORDER BY Username) 
    FROM MsUser
) a WHERE a.RowNumber BETWEEN (@PageNo - 1 )* @PageSize + 1 AND @PageNo * @PageSize

Note that the key is in this part of code:

ROW_NUMBER() OVER (ORDER BY Username)


If the PageSize is set to 5 item per page, then the top 5 users will be displayed on Page 1 and the next 5 users will be displayed on Page 2.

e.g. 1
PageSize = 5, PageNo = 1
DECLARE @PageSize INT, @PageNo INT
SELECT  @PageSize = 5, @PageNo = 1
SELECT a.Username FROM (
    SELECT Username, RowNumber = ROW_NUMBER() OVER (ORDER BY Username) 
    FROM MsUser
) a WHERE a.RowNumber BETWEEN (@PageNo - 1 )* @PageSize + 1 AND @PageNo * @PageSize

The data retrieved on Page 1 will be:

e.g. 2
PageSize = 5, PageNo = 2
DECLARE @PageSize INT, @PageNo INT
SELECT  @PageSize = 5, @PageNo = 2
SELECT a.Username FROM (
    SELECT Username, RowNumber = ROW_NUMBER() OVER (ORDER BY Username) 
    FROM MsUser
) a WHERE a.RowNumber BETWEEN (@PageNo - 1 )* @PageSize + 1 AND @PageNo * @PageSize

And the data retrieved on Page 2 will be:

Share:

Sunday, March 7, 2010

Paging in SQL

Haven't posted anything for quite some time. This post is about how to make paging in SQL.

If you are interested, you can follow the steps and execute the query provided. Else, I also provide the source code. You can download and observe it later.

Before getting into the code, execute this query to create the tables first. 2 Tables are used.

create table dbo.MsUser
( UserID char(20) primary key,
Username varchar(100),
Address varchar(50),
DivisionID int
)

-- Create Table MsDivision
create table dbo.MsDivision
( DivisionID int primary key,
DivisionName varchar(50)
)



And here is the query to insert sample data:

-- Insert Data into MsDivision
insert into dbo.MsDivision values ( 1, 'IT' )
insert into dbo.MsDivision values ( 2, 'IS' )
insert into dbo.MsDivision values ( 3, 'Finance' )
insert into dbo.MsDivision values ( 4, 'Management' )

-- Insert Data into MsUser
insert into dbo.MsUser values ( 'Himura', 'Miss Himura', 'Earth', 3 )
insert into dbo.MsUser values ( 'Selvia', 'Selvia', 'Indonesia', 2 )
insert into dbo.MsUser values ( 'Superman', 'Clark Kent', 'Earth', 2 )
insert into dbo.MsUser values ( 'SelviaHimura', 'Selvia Himura', 'Earth', 4 )
insert into dbo.MsUser values ( 'Luck', 'Steven Luck', 'Indonesia', 1 )
insert into dbo.MsUser values ( 'SuperLuck', 'Super Luck', 'United States', 1 )
insert into dbo.MsUser values ( 'Selvi', 'Selvi', 'Indonesia', 1 )
insert into dbo.MsUser values ( 'Lucky', 'Lucky Luke', 'United States', 3 )
insert into dbo.MsUser values ( 'Steven', 'Steven', 'Earth', 4 )



Here are the data inserted for the tables:
Table MsDivision:



Table MsUser:



After creating the tables and inserting the data, execute the following query to select the data, sorted by the field Username.

-- Select All Data Sorted by Username
select a.UserID, a.Username, a.Address, a.DivisionID, b.DivisionName
from dbo.MsUser a
left join dbo.MsDivision b
on b.DivisionID = a.DivisionID
order by a.Username



Here is the result:




Here is the part about how to create Paging in SQL.

We will use the functionality of Stored Procedure to retrieve the data.
Here is the query of the stored procedure used to retrieve the paged data.

create procedure dbo.spr_GetSimplePaging
@PageSize int, -- @PageSize = item per page
@PageNo int -- @PageNo = current page
as
set nocount on

-- Create Temporary Table to Store the Raw Data
-- The content of the temporary table: An identity field which will be used to determine the paging
-- and the columns contained in the original table
create table #temp
( UniqueID int identity, -- will be used to determine the paging
UserID char(20),
Username varchar(100),
Address varchar(50),
DivisionID int,
DivisionName varchar(50)
)

-- Select the Raw Data into temporary table #temp
insert into #temp ( UserID, Username, Address, DivisionID, DivisionName )
select a.UserID, a.Username, a.Address, a.DivisionID, b.DivisionName
from dbo.MsUser a
left join dbo.MsDivision b
on b.DivisionID = a.DivisionID
order by a.Username

-- Get Data Using Formula for Paging
select UserID, Username, Address, DivisionID, DivisionName
from #temp
where UniqueID between ( @PageNo - 1 ) * @PageSize + 1
and @PageNo * @PageSize

-- Drop temporary table
drop table #temp



There are 3 main parts in the data retrieval process:
1. Create a temporary table which will be used to store the raw data. The temporary table will consist of an identity field which will be used to determine the paging (we use a field named UniqueID in the query above) and the columns contained in the original table(s).
Here is the first part of the query:

create table #temp
( UniqueID int identity, -- will be used to determine the paging
UserID char(20),
Username varchar(100),
Address varchar(50),
DivisionID int,
DivisionName varchar(50)
)



2. Select the raw data into a temporary table. We name the temporary table #temp on the sample query above.

Here is the second part of the query:

insert into #temp ( UserID, Username, Address, DivisionID, DivisionName )
select a.UserID, a.Username, a.Address, a.DivisionID, b.DivisionName
from dbo.MsUser a
left join dbo.MsDivision b
on b.DivisionID = a.DivisionID
order by a.Username




3. Get the data using formula for Paging.

Here is the third part of the query:

select UserID, Username, Address, DivisionID, DivisionName
from #temp
where UniqueID between ( @PageNo - 1 ) * @PageSize + 1
and @PageNo * @PageSize



Notice the formula used above.
The formula used to get the data between the wanted page is:
select data where UniqueID between ( @PageNo - 1 ) * @PageSize + 1 and @PageNo * @PageSize

Therefore, if we would like to retrieve the data of PageSize = 5 on Page 1:
We will get the data with the following condition
UniqueID between ( 1 - 1 ) * 5 + 1 and 1 * 5
UniqueID between 1 and 5


If we would like to retrieve the data of PageSize = 5 on Page 2:
We will get the data with the following condition
UniqueID between ( 2 - 1 ) * 5 + 1 and 2 * 5
UniqueID between 6 and 10


Example:
The data retrived from the select query will be:


If the PageSize is set to 5 item per page, then the top 5 users will be displayed on Page 1 and the next 5 users will be displayed on Page 2.
Using the query

-- Example : PageSize = 5, PageNo = 1
spr_GetSimplePaging 5, 1


The data retrieved on Page 1 will be:



To get the data for the second page, use the query:

-- Example : PageSize = 5, PageNo = 2
spr_GetSimplePaging 5, 2


The data retrieved on Page 2 will be:



The sample query used to create simple paging above will perform quite slowly when the data is very big. To optimize the performance, therefore, we will have to make a little modification on the query.

Here is the modified query:

create procedure dbo.spr_GetModifiedPaging
@PageSize int, -- @PageSize = item per page
@PageNo int -- @PageNo = current page
as
set nocount on

-- Create Temporary Table to Store the Raw Data
-- The content of the temporary table: An identity field which will be used to determine the paging
-- and the UNIQUE COLUMN(S) - usually marked as primary key - contained in the original table
create table #temp
( UniqueID int identity, -- will be used to determine the paging
UserID char(20)
)

-- Select the Raw Data into temporary table #temp
insert into #temp ( UserID )
select a.UserID
from dbo.MsUser a
order by a.Username

-- Get Data Using Formula for Paging
-- Join the data with the original table(s)
select a.UserID, b.Username, b.Address, b.DivisionID, c.DivisionName
from #temp a
left join dbo.MsUser b
on b.UserID = a.UserID
left join dbo.MsDivision c
on c.DivisionID = b.DivisionID
where UniqueID between ( @PageNo - 1 ) * @PageSize + 1
and @PageNo * @PageSize

-- Drop temporary table
drop table #temp



Instead of storing all raw data in the temporary table, we actually only need to store the unique data, thus, we only need to create a temporary table containing UniqueID and the UserID:

create table #temp
( UniqueID int identity, -- will be used to determine the paging
UserID char(20)
)



On the second part, the query will be much simpler:

insert into #temp ( UserID )
select a.UserID
from dbo.MsUser a
order by a.Username



On the third part, we will then have to join the tables used to get the complete data to be retrieved.

select a.UserID, b.Username, b.Address, b.DivisionID, c.DivisionName
from #temp a
left join dbo.MsUser b
on b.UserID = a.UserID
left join dbo.MsDivision c
on c.DivisionID = b.DivisionID
where UniqueID between ( @PageNo - 1 ) * @PageSize + 1
and @PageNo * @PageSize



Using the modified query, the temporary table created will be smaller. Consequently, the raw data stored in the temporary table will be smaller. The complete data will then be selected by joining it with the original table(s) on the last part. This enables better performance while retrieving a very big data.


If you need the source code, click on the link to the file to download the source code: Paging in SQL
Share:

Friday, May 8, 2009

Import data from excel to table in SQL

A friend asked about how to import data from Excel to table in SQL, then I searched for it. Found this code. Having tried it, I think it's a nice idea to share it.

INSERT INTO Table_Name
SELECT column1, column2
FROM OPENROWSET ('Microsoft.Jet.OLEDB.4.0', 'Excel 8.0;Database=C:\ExcelTable.xls', 'SELECT * FROM [Sheet1$]')
Share:

Friday, May 1, 2009

Generate random characters

There is a way to generate random characters in a simple way, just use :
select NEWID()


This function is quite useful in some occasions, though may not be it's main usage.
e.g. If you're going to generate a random quote from a table named MsQuote :
SELECT TOP 1 QuoteId, QuoteText
FROM MsQuote
ORDER BY NEWID()
Share:

Stored procedure's dependencies

Find the dependencies of a stored procedure / Find tables used in a stored procedure:
use sp_depends sp_name
Share:

Sys on SQL

Not sharing query, but some tips that might be useful when looking for a needle in a haystack.
Am using Microsoft SQL Server 2000, but won't differ too much on later versions, I guess.

Find tables with the specified name

-----------------------------------
select name from sysobjects where name like '%table_name%' and type = 'u'



Find stored procedures with the specified name
----------------------------------------------
select name from sysobjects where name like '%sp_name%' and type = 'p'



Find the name of the tables which contain specified column
----------------------------------------------------------
select TableName = a.name, ColumnName = b.name from sysobjects a, syscolumns b
where a.id = b.id and b.name = 'column_name'



Find stored procedures which contain certain key / text
-------------------------------------------------------
select name from sysobjects where id in ( select id from syscomments where text like '%text_in_sp%')

This is also useful when you would like to search for stored procedures which uses a certain table.
e.g. If you would like to know the stored procedures which use the table named table_name,
select name from sysobjects where id in ( select id from syscomments where text like '%table_name%')
Share:

You may be intersted in

Related Posts

Updating Table Containing Xml Column via LinkedServer

If you are trying to update a table containing XML column via Linked Server in SQL Server, and you are not able to, you are not alone. There...

About Me

My photo
Is an ordinary man, with a little knowledge to share and high dreams to achieve. I'd be glad if I can help others, 'coz the only thing for the triumph of evil is for a good man to do nothing.

About Blog

You can find a lot of debugging and deploying problems while developing applications in .NET and Visual Basic here. There are also some querying tips in SQL and typical source codes which might be useful shared here.

Popular Posts

Blogroll

Followers

Leave a Message