Click here to Skip to main content
15,893,594 members
Home / Discussions / Database
   

Database

 
QuestionHow to use FBExport tool. Pin
avvaru.murali23-Jun-08 20:45
avvaru.murali23-Jun-08 20:45 
AnswerRe: How to use FBExport tool. Pin
Ashfield23-Jun-08 21:38
Ashfield23-Jun-08 21:38 
QuestionAnother DB design question: Pin
alex3_1423-Jun-08 3:50
alex3_1423-Jun-08 3:50 
AnswerRe: Another DB design question: Pin
Ennis Ray Lynch, Jr.23-Jun-08 3:53
Ennis Ray Lynch, Jr.23-Jun-08 3:53 
AnswerRe: Another DB design question: Pin
Michael Potter23-Jun-08 9:02
Michael Potter23-Jun-08 9:02 
AnswerRe: Another DB design question: Pin
Mycroft Holmes27-Jun-08 3:59
professionalMycroft Holmes27-Jun-08 3:59 
QuestionPerformance question Pin
Ennis Ray Lynch, Jr.23-Jun-08 3:27
Ennis Ray Lynch, Jr.23-Jun-08 3:27 
AnswerRe: Performance question Pin
Mike Dimmick23-Jun-08 4:24
Mike Dimmick23-Jun-08 4:24 
Your performance will be I/O bound.

It seems to be very hard for programmers to understand this, but RAM and processor cores have very little to do with database server performance when writing. The bounding factor for writes (inserts, updates, deletes) is how fast the transaction log records can be written out to the log file. Transaction log writes are ideally sequential and are fastest when the drive is dedicated to the transaction log, as the disk head will always remain in approximately the right position to write the next batch of log records. Putting the data on the same disk as the log will cause dirty page flushes to be interleaved with log writes, causing the log writes to be held up while the head moves to and from the data pages.

It's best if you do have dedicated drives to preallocate the files to close to the capacity of the drive (array), and ensure each file is contiguous. There is a benefit to sequential I/O but file fragmentation will turn what looks sequential to SQL Server into random-access on the drive itself. File-grow operations are very expensive.

You should consider the capacity of the drives in terms of I/Os per second, not in terms of raw gigabytes writeable. Unfortunately you won't see this quoted for drives as it depends so much on the workload and pattern of access. SQL Server does batch up log records and will write them in batches of anything from 512 bytes up to 16kB and possibly even more, but remember that the log records for a transaction must be written fully to disk before SQL Server will consider the transaction committed (the Durable property, the D of the ACID properties).

In terms of selecting data, that depends on how you access it. SQL Server will cache data in RAM when it is accessed (which will probably be randomly, one 8KB page at a time as it's required but SQL Server also does 'read ahead' to perform larger I/Os in one command). Ideally you want an index that covers all of the terms in your WHERE clause. The query plan will show this as an 'index seek' (clustered index seek for the clustered index). However, with sufficient RAM, and the data already cached in RAM, you may not notice that the query is in fact simply reading every row from start to finish, shown as 'table scan' or (clustered) index scan. This will appear as 100% CPU, but adding more or faster CPUs generally won't make it go any faster and it'll still show 100% CPU: what you're actually seeing is the processor bringing the data from main memory into its caches, which is bound by the CPU and/or memory bus speeds. The OS can't account for how many CPU cycles were burnt retrieving data through the cache hierarchy, it just sees them as cycles consumed by this thread. In contrast when retrieving data from disk, it knows the thread can't be scheduled. (SQL Server actually uses asynchronous I/O, it puts a thread waiting for I/O onto another request so it doesn't incur the OS scheduling overhead, unless there's no more work to do.)

On a lightly-loaded multiprocessor/multicore system, you might also see some parallelism being invoked so each core processes one part of the data set, if the table is already cached in RAM. Don't be fooled by this: once you have a lot of concurrent transactions, the other cores will be busy processing those requests and you'll see very little parallelism.

Best case, you can get very high SELECT performance if the query uses a highly selective index (so that one unique row will be found) and the data is already cached in RAM: it will only need to follow a few page references before locating the actual row. Worst case is a non-clustered index scan of a large table not currently in memory or that won't fit in memory. (Non-clustered index lookups require a clustered index or bookmark lookup on the base table if you've SELECTed a column that doesn't appear in the non-clustered index.)

All you can really do is try to provide a representative data set and workload for the system and try it out.


DoEvents: Generating unexpected recursion since 1991

GeneralRe: Performance question Pin
Ennis Ray Lynch, Jr.23-Jun-08 4:35
Ennis Ray Lynch, Jr.23-Jun-08 4:35 
GeneralRe: Performance question Pin
Andy Brummer26-Jun-08 11:03
sitebuilderAndy Brummer26-Jun-08 11:03 
GeneralRe: Performance question Pin
Ennis Ray Lynch, Jr.26-Jun-08 11:18
Ennis Ray Lynch, Jr.26-Jun-08 11:18 
GeneralRe: Performance question Pin
Andy Brummer26-Jun-08 11:34
sitebuilderAndy Brummer26-Jun-08 11:34 
GeneralRe: Performance question Pin
Ennis Ray Lynch, Jr.26-Jun-08 11:40
Ennis Ray Lynch, Jr.26-Jun-08 11:40 
GeneralRe: Performance question Pin
Andy Brummer26-Jun-08 12:11
sitebuilderAndy Brummer26-Jun-08 12:11 
GeneralRe: Performance question Pin
Ennis Ray Lynch, Jr.26-Jun-08 12:21
Ennis Ray Lynch, Jr.26-Jun-08 12:21 
GeneralRe: Performance question Pin
Andy Brummer27-Jun-08 2:44
sitebuilderAndy Brummer27-Jun-08 2:44 
Questiondatetime query in SQL server Pin
laziale22-Jun-08 23:27
laziale22-Jun-08 23:27 
AnswerRe: datetime query in SQL server Pin
r aa j23-Jun-08 0:01
r aa j23-Jun-08 0:01 
AnswerRe: datetime query in SQL server Pin
Ashfield23-Jun-08 0:08
Ashfield23-Jun-08 0:08 
GeneralRe: datetime query in SQL server Pin
laziale23-Jun-08 0:19
laziale23-Jun-08 0:19 
GeneralRe: datetime query in SQL server Pin
Ashfield23-Jun-08 1:16
Ashfield23-Jun-08 1:16 
GeneralRe: datetime query in SQL server Pin
laziale23-Jun-08 1:39
laziale23-Jun-08 1:39 
GeneralRe: datetime query in SQL server Pin
Ashfield23-Jun-08 1:54
Ashfield23-Jun-08 1:54 
GeneralRe: datetime query in SQL server Pin
laziale23-Jun-08 2:01
laziale23-Jun-08 2:01 
GeneralRe: datetime query in SQL server Pin
Ashfield23-Jun-08 3:30
Ashfield23-Jun-08 3:30 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.