A traditional transactional database is made for storing data. These databases are generally highly normalized to keep the referential integrity, but might be to slow for analytical queries.
A data warehouse is optimized for analysis of the data. The data is therefore often stored in a denormalized and/or aggregated form. To make sure the referential integrity is kept intact they usually pull their data from a transactional database.
There is not a straight answer to that question as there are many different forms of data warehouses.
Also a data warehouse is a database so the question does not really make sense.
The data warehouses I have worked with differ from non data warehouse databases in the following manner:
A data warehouse keeps transactional data so that the state of a database at any point in time can be recreated.
As a consequence data warehouses contain a lot of data as every single change, at the column level, illicits the creation of a new row.
Data warehouses can contain raw, staging and warehouse schemas to enable the verifying of data before it is placed in the data warehouse.
A data warehouse is generally used where one wants to be able to have an audit trail of the contents of the database.
That's my t'penneth.
“That which can be asserted without evidence, can be dismissed without evidence.”
Currently we are using crystal report to display the data whihc is internally fetching almost 55,234,22 records from DB. Since it the no of records are huge it is taking almost 7 minutes to display the report which is very poor performance in nature.
Can anyone suggest me some useful sites to make it faster and can recommend some free tool to measure the exact point which is causing the issue?
Since it the no of records are huge it is taking almost 7 minutes to display the report which is very poor performance in nature.
Stop right there; that is not poor performance. Something taking a lot of time does not mean that it does not perform.
What's the size of a single record? Just use Google to look up the size of each field that has no length. Now multiply that by the amount of records. Now take into consideration that it is a Crystal Report, not some text-only output.
You could start with pasting the query here; you'd also want to download a profiler for Sql, and/or use SQL Management Studio to see if you can optimize the query.
That's the fetching-the-data part. The other part would be getting the fetched data into a report-format. If you do not need all the fancy stuff that CR provides, then try writing a simple HTML-report.
Bastard Programmer from Hell
If you can't read my code, try converting it here[^]
I agree with Eddy, this is not a performance problem, seems more like a poor implementation of user requirements.
Am I correct in understanding that the report fetches 55 million records, but does it display that much data ?
Displaying 55 detail records per page would generate a 1 million page report. Really ? Does the user need to see all of that data ?
Is there any way to create summary data at an off-peak time so the report can generate from that summary data ?
What about implementing an asynchronous report generator where the user selects the input parameters, then "kicks off" the report in the background. The report could generate an HTML or PDF report, then send an email to the requestor with a hyperlink to the report when it is complete.
You would be surprised how agreeable a user would be to a solution like this. They kick off a report, then they can move on to something else while it generates. The email tells them when it is done.
In programming, Public, Private (and friends) are called access modifiers, the state achieved by assigning one of them called accessibility level - that may give you a name, like:
I'm not questioning your powers of observation; I'm merely remarking upon the paradox of asking a masked man who he is. (V)