General question on DB normalization - Database Discussion Boards

First Prev Next

Re: Cursor issue [modified]

Niladri_Biswas7-Dec-09 21:59

Niladri_Biswas

7-Dec-09 21:59

Hi,
Try to understand the output of NewId()

If I issue select NewId() the output will be something like

3B311450-65BB-4AF8-898B-0CD02C5086E2

.

It is a pure hyphenized alphanumeric charecter. If your destination table has the Id column as integer type, then you need to either change the column type from Integer to Varchar or you need to extract only the numeric values from NewId() for your future use.

I have created a sample for you. Look into that(id datatype is varchar) . I hope you will get some insight.

declare @tblSource table(id int identity, FirstName varchar(50),LastName varchar(50))
declare @tblDest table(id varchar(max) , FirstName varchar(50),LastName varchar(50))
insert into @tblSource 
	select 'firstname1', 'lastname1' union all
	select 'firstname2', 'lastname2' union all
	select 'firstname3', 'lastname3' union all
	select 'firstname4', 'lastname4' union all
	select 'firstname5', 'lastname5'

Query:

insert into @tblDest (id,FirstName,LastName)
select NEWID(),FirstName,LastName 
from @tblSource 

select * from @tblDest

Output:

id	FirstName	LastName
D2A3D81C-F76E-44A4-9D47-83FBE4DDB76B	firstname1	lastname1
0B827C0E-EB68-43EA-A423-B10575EAF572	firstname2	lastname2
EB4DB402-3D8F-4C09-9E37-004CE37DE1FD	firstname3	lastname3
E6CE9239-1E95-4660-B5AC-00DBFEE28474	firstname4	lastname4
345F70CC-B957-41CC-977C-9FBA7A81E912	firstname5	lastname5

In case you need to strip out only the numbers, here is an example

declare @str varchar(max)
set @str = 'D2A3D81C-F76E-44A4-9D47-83FBE4DDB76B'
;with cte as(
select 1 as rn 
union all
select rn+1 from cte where rn<LEN(@str)),
cte2 as(
select rn,chars
from cte
cross apply(select SUBSTRING(@str,rn,1) chars)X
where chars like '%[0-9]%'
)
select numericData from (
select cast(chars as varchar(max))from cte2
for xml path(''))X(numericData)

Output:

numericData
23817644494783476

Niladri Biswas

modified on Tuesday, December 8, 2009 4:12 AM

Performance tip on multiple selects to retrieve alot of BLOBS?

Thomas ST3-Dec-09 22:28

Thomas ST

3-Dec-09 22:28

I am retrieving a lot of data and want this to be as fast as possible.
Let's say I have a database with two tables; tablePerson and tableFiles. A person has one file wich is stored in a BLOB. In my solution as it is now, I select * from person table, and while reading from this I select one row from the table with the blob.. This sums up to alot of one-row-queries towards the blob-table.. This blob-table has 678000 rows, and all these queries are taking a lot of time...

So.. How can I improve performance on this? Any tips?

This is something similar to my code:

using(SqlCommand cmd = new SqlCommand("select * from tablePerson", _conn)
{
  SqlDataReader r = cmd.exeCuteReader();

  while(r.read())
  {
    Person p = new Person();
    //read some data from columns..
    
    string documentID = Convert.ToString(r["p_docid"]);
    p.LocationToDocument = ExtractDocument(documentID);
  }
  r.close();
  r.dispose();
}

...

private string ExtractDocument(string docid)
{
   string filename = "";
   using (SqlCommand cmd = new SqlCommand("select d_docid, d_title, d_BLOB from tableDocument where d_docid=@paramDocID", _conn)
   {
      SqpParameter p = new SqlParameter("@paramDocID", System.Data.SqlDbType.Text);
      p.Value = docid;

      SqlDataReader r = cmd.exeCuteReader(System.Data.CommandBehavior.SequentialAccess);

      while(r.read()) //1 row..
      {
          //extract blob to a file, and set filename=that document..
      }
   }
   return filename;
}

Re: Performance tip on multiple selects to retrieve alot of BLOBS?

Ashfield4-Dec-09 1:27

Ashfield

4-Dec-09 1:27

Why not do it in one select with a join?

Bob
Ashfield Consultants Ltd
Proud to be a 2009 Code Project MVP

Re: Performance tip on multiple selects to retrieve alot of BLOBS?

Bassam Saoud4-Dec-09 10:44

Bassam Saoud

4-Dec-09 10:44

As Asfield said, join the 2 tables so that you retrieve them once. I would add to what he said is that you probably want to add paging. So instead of processing 1 M record, you can process 1000 record at a time etc..

Adding an Identity Later T-SQL

Vimalsoft(Pty) Ltd3-Dec-09 21:17

Vimalsoft(Pty) Ltd

3-Dec-09 21:17

Good Day All

i have a Table with the ID Field that was not an identity Field. This Field has not Duplicates. Now later i want to Change this numeric Field as an Identity Field like this

--STEP 7 ADD THE IDENTITY BACK IN TABLE MTM_ACTV_STAFF
ALTER TABLE [New_Jaco].[dbo].[MTM_ACTV_STAFF]
ALTER COLUMN [ID] IDENTITY(1,1)

but i get an Error that says

Msg 156, Level 15, State 1, Line 3
Incorrect syntax near the keyword 'IDENTITY'.

Thanks

Vuyiswa Maseko,

Spoted in Daniweb-- Sorry to rant. I hate websites. They are just wierd. They don't behave like normal code.

C#/VB.NET/ASP.NET/SQL7/2000/2005/2008
http://www.vuyiswamaseko.com
vuyiswa@its.co.za
http://www.itsabacus.co.za/itsabacus/

Re: Adding an Identity Later T-SQL

Mycroft Holmes3-Dec-09 22:11

Mycroft Holmes

3-Dec-09 22:11

A quick guess, I never script these thing I use SSMS, but don't you need a data type in the alter as well as the create.

ALTER COLUMN [ID] INT IDENTITY(1,1)

Never underestimate the power of human stupidity
RAH

Re: Adding an Identity Later T-SQL

Vimalsoft(Pty) Ltd4-Dec-09 0:49

Vimalsoft(Pty) Ltd

4-Dec-09 0:49

does not work , but i found a resolution. here http://social.msdn.microsoft.com/Forums/en-US/transactsql/thread/7f24ba5a-b091-4119-9216-1564b61f4aa0/[^]

General question on DB normalization

Dewald3-Dec-09 20:59

Dewald

3-Dec-09 20:59

Hi all, I have what might well be considered a stupid question but fortunately none of you are my employer or former varsity prof so I'll ask away Smile | :)

I've always endeavored to keep my databases normalized to the best of my ability and, for the most part, I think I'm fairly well set in that line of thinking.

At my current company though some of our databases are anything but normalized. When I challenged my boss about it he responded that it was done deliberately because it is not supposed to be a relational database. It is a DB that accepts, stores and processes transactions at quite a fierce tempo and they didn't want every INSERT or SELECT statement to have to internally cross reference lots of other tables.

Fair enough, it makes sense. But then it occurred to me that I probably don't know enough about DB design because I would never have thought of that. I would simply have designed a normalized DB because that's the way I'm programmed.

So, I turned to Wikipedia but even there I can't seem to find an awful lot of information on non-normalized DB's but that's most likely because I don't really know what search terms to use.

So my question is:
1. When should a DB be normalized and when not?
2. Can anyone point me to an online article or two that deals with the topic?
3. Am I correct in assuming that a normalized DB is called a "Relational Database"?
4. What would you call a DB like ours here that is not normalized?

Re: General question on DB normalization

Mycroft Holmes3-Dec-09 22:17

Mycroft Holmes

3-Dec-09 22:17

Your boss may actually be right (I work with batch processes not transactions) because with every FK and index on a table the CRUD operation need to do the additional work to maintain the relationships. I still doubt it is a valid answer though, only a reporting database should not be normalised IMHO.

Can't help more than that and I hope you find the ammo to shoot this one down.

Never underestimate the power of human stupidity
RAH

Re: General question on DB normalization

Dewald3-Dec-09 22:44

Dewald

3-Dec-09 22:44

Thanks, but I'm not disputing that my boss is right. I think he has a valid point so I'm not really looking for ammo to shoot his argument down Smile | :)

I'd just like to make sure I really understand DB design and not blindly stick to normalized DBs if that's not always the best way to go.

Re: General question on DB normalization

Mycroft Holmes3-Dec-09 22:56

Mycroft Holmes

3-Dec-09 22:56

Ah but I think blindly sticking to normalised databases is the correct thing to do. The only exception I have found is if you are creating a summarised table for reporting, I often include the report descriptors in the summarised table. This has gone out of fashion as we are starting to use cubes to support our reporting and they are whole different matter.

Never underestimate the power of human stupidity
RAH

Re: General question on DB normalization

i.j.russell4-Dec-09 1:19

i.j.russell

4-Dec-09 1:19

The term that you are looking for is Denormalization.

In general, databases should always be normalized to at least 3NF unless there are specific areas where performance of reading/writing is a problem. However, most modern RDBMS like Sql Server are extremely efficient at joining tables using indexes so many of the reasons for denormalization are no longer applicable.

Re: General question on DB normalization

Eddy Vluggen4-Dec-09 3:01

Eddy Vluggen

4-Dec-09 3:01

Dewald wrote:
1. When should a DB be normalized and when not?

You normalize it by default, removing redundancy. Once the database becomes too slow (which is usually not due to the normalization, but lack of indexes and such) you start to optimize/de-normalize/add redundancy.

You'll waste a bit more space and gain some speed.

Dewald wrote:
2. Can anyone point me to an online article or two that deals with the topic?

The Oracle FAQ[^] also contains a link to the reverse process.

Dewald wrote:
3. Am I correct in assuming that a normalized DB is called a "Relational Database"?

The fact that there are relations between the data doesn't mean that these have been normalized.

Dewald wrote:
4. What would you call a DB like ours here that is not normalized?

That depends on what you're using it for. If it's a blob-store or a container for hashtables, then I'd say that it's an optimized datastore and I'd make sure to give credit to the optimizer.

If it's relational data that you're storing, then I'd call it an "accident waiting to happen" Smile | :)

I are Troll Suspicious | :suss:

Re: General question on DB normalization

Dewald4-Dec-09 3:07

Dewald

4-Dec-09 3:07

Great, thanks!

DB Normalization - Further reading

David Mujica4-Dec-09 3:37

David Mujica

4-Dec-09 3:37

Actually Wikipedia has a good explaination of Normalization.

http://en.wikipedia.org/wiki/Database_normalization[^]

Chris Date and Edgard Codd have written numerous books on the topic and are frequently used in Universities as the course textbook. I had to study them when I was in Graduate School.

[Message Deleted]

Uma J3-Dec-09 20:24

Uma J

3-Dec-09 20:24

[Message Deleted]

Re: Exporting data from stored procedure to excel spreadsheets

dan!sh 3-Dec-09 20:46

dan!sh

3-Dec-09 20:46

Check the link in my reply to the post below. Hope it helps.

50-50-90 rule: Anytime I have a 50-50 chance of getting something right, there's a 90% probability I'll get it wrong...!!

Export records to excel from sql

padmanabhan N3-Dec-09 19:17

padmanabhan N

3-Dec-09 19:17

Dear friends,

I need to export records from sql to excel. I know how to export the whole record to excel. But my requirement is, i have to insert 10 records in sheet1 and another 10 records in sheet2 and so on.... a help or some links would be highly appriciated..... (i dont want any packages like DTS)

Padmanabhan
My Articles:
Articles[^]
My latest Article:
Word Automation[^]

Re: Export records to excel from sql

T21023-Dec-09 19:29

T2102

3-Dec-09 19:29

The only direct way I know how to do that with separate sheets is with ADO and COM.

Re: Export records to excel from sql

dan!sh 3-Dec-09 19:39

dan!sh

3-Dec-09 19:39

Check this[^]. I think this should help you.

50-50-90 rule: Anytime I have a 50-50 chance of getting something right, there's a 90% probability I'll get it wrong...!!

How do I make a MS Access Subform, A Special Case...] [modified]

new_phoenix3-Dec-09 14:28

new_phoenix

3-Dec-09 14:28

I am working on a MS Access database that has a central table with a composite key consisting of seven elements, and another table also with the same seven composite key elements. The central table composite key is unique thereby making it a composite primary key. The other table has multiple records of the composite key. In other words, there is a one to many relationship between the main table and the other table using composite keys.

I need to create a form with the composite key elements on the top of the form and a subform with the multiple records matches to the composite key from the main form. The problem is the MS Access only provides the ability to create a subform using three elements from the seven elements of the composite key.

What would be the solution to allow me to create a subform that matches all seven elements of the composite key form the "many" table to all seven elements of the primary composite key from the central table?

Confused | :confused:

modified on Thursday, December 3, 2009 9:26 PM

Re: How do I make a MS Access Subform, A Special Case...]

Mycroft Holmes3-Dec-09 18:43

Mycroft Holmes

3-Dec-09 18:43

A seven field composite primary key Dead | X|

- sounds like a disaster has found somewhere to happen. This is a BAD idea, I have no knowledge of the performance issues with such a mess but the sheer cumberomeness of forever doing a 7 key join would drive me to drink.

I recommend that you insert a primary key into the central table and populate it, then turn autonumber on.

Now add the foreign key field to the other table(s) and use the 7 key horror to populate the field.

Make your relationship using these fields. You must make sure you support the change within your CRUD code.

Never underestimate the power of human stupidity
RAH

Options besides sql dependency?

prommel3-Dec-09 11:30

prommel

3-Dec-09 11:30

What are some options to replace sql dependency using sql server with multiple users? I like how sqlDependency works but without some changes to my database I can't determine what was added, modified , or deleted. So when I receive a notification I'm forced to reload the dataset and refresh. While this is not a major problem with a small dataset, as the dataset grows the time required to do this will be come unacceptable. I know one option is to continue using sqldependency and only load the data displayed for the current day on the calendar. So is there any other ways to keep data sync'd in a multiple user environment?

Thanks,
Paul

Some more information:

Application is a scheduling program
Use sqldependency for notification only
Use linq to sql for actual data retrieval

Re: Options besides sql dependency?

Mycroft Holmes3-Dec-09 14:13

Mycroft Holmes

3-Dec-09 14:13

I'll be interested in any responses to this one, I have never worked with SQLDependancy. I insist on minimum recordset size for the UI, any more than about 500 records gets looked at severely and usually the design is changed to meet the requirement.

Where I have to load a large volume I often allow the user to make a number of changes and then reload the dataset.

Never underestimate the power of human stupidity
RAH

database design

AndyInUK3-Dec-09 4:09

AndyInUK

3-Dec-09 4:09

Hi,

We have got hundreds of tables and SPs but no diagram. Basically we just keep on adding new tables as per requirement or delete table and there is no info.

What i want to implement now is - a whole database diagram - So what would be the best way keeping in mind that database is big if not huge.

Thank You
Andyyy

Last Visit: 31-Dec-99 18:00 Last Update: 29-Apr-24 22:58

Refresh

ᐊ Prev 1...707 708 709 710711712 713 714 715 716 Next ᐅ

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.