Click here to Skip to main content
15,922,407 members
Home / Discussions / Database
   

Database

 
GeneralRe: Conversion failed when converting the varchar value in SQLSERVER 2008 Pin
ChandrakanthGaddam22-May-12 6:20
ChandrakanthGaddam22-May-12 6:20 
QuestionHandling big tables Pin
V.21-May-12 23:28
professionalV.21-May-12 23:28 
AnswerRe: Handling big tables Pin
Mycroft Holmes21-May-12 23:56
professionalMycroft Holmes21-May-12 23:56 
GeneralRe: Handling big tables Pin
V.22-May-12 0:01
professionalV.22-May-12 0:01 
GeneralRe: Handling big tables Pin
Mycroft Holmes22-May-12 14:15
professionalMycroft Holmes22-May-12 14:15 
GeneralRe: Handling big tables Pin
R. Giskard Reventlov22-May-12 6:26
R. Giskard Reventlov22-May-12 6:26 
AnswerRe: Handling big tables Pin
Eddy Vluggen22-May-12 9:32
professionalEddy Vluggen22-May-12 9:32 
AnswerRe: Handling big tables Pin
jschell23-May-12 10:46
jschell23-May-12 10:46 
V. wrote:
The user can choose to view just a day a year or even the entire history. (eg.
the last 2 years wich still means 1 500 000 000 records)


Nonsense.

Simple math will demonstrate that a user (person) will NEVER look at that many records at once.

They will either look at a summary, or a summary with drill downs or a will be looking for a very, very small subset of that.


V. wrote:
Has anyone got any experience with this size of database and how to solve the
issue of getting the data quickly?


Get real requirements.


V. wrote:
The data are datapoints that need to be plotted on a graph in a certain time
period.


Excellent example. Graph 1.5 billion data points (from your first requirement) - so xactly how many pixels are on your screen? Again simple math will demonstrate that you can't view that many data points on a graph. So either there will be a much smaller time period or a summary of the entire period.

If a summary one solution is to build summary tables. So for example at the end of every data you create summary data of the day. Then a graph that display every day, uses the summary table rather than the raw data.


V. wrote:
1. Keep table size reasonable, so how to split the data into different tables
easily?


You start with real usage patterns. For example what percentage of the time does a user want to look at data for the last week, month, year? Or hour, day, week? Or by collection point via week. Etc, etc, etc.

Second you get realistic estimates and anticipated growth. So is 3 billion that average or the maximum? Will it be 3 billion next year or 30 billion. Keep in mind it must be realistic, not pie in the sky.

Third how long must you keep it? 1 year, 10 years 100 years?

And if it no one seems willing to discuss reality then you might want to look into price tags for a really hug SAN system, and submit a request to buy it because you will need it for testing - because the system will need that to run.
GeneralRe: Handling big tables Pin
V.23-May-12 20:45
professionalV.23-May-12 20:45 
Questionerd concepts Pin
mido_h_8921-May-12 22:34
mido_h_8921-May-12 22:34 
AnswerRe: erd concepts Pin
Mycroft Holmes21-May-12 23:53
professionalMycroft Holmes21-May-12 23:53 
QuestionError while declaring a cursor Pin
VC_RYK21-May-12 19:16
VC_RYK21-May-12 19:16 
AnswerRe: Error while declaring a cursor Pin
R. Giskard Reventlov22-May-12 6:23
R. Giskard Reventlov22-May-12 6:23 
QuestionHow to get delete/Update row + SQL Pin
jojoba201118-May-12 22:00
jojoba201118-May-12 22:00 
AnswerRe: How to get delete/Update row + SQL Pin
Eddy Vluggen19-May-12 0:42
professionalEddy Vluggen19-May-12 0:42 
QuestionRe: How to get delete/Update row + SQL Pin
jojoba201119-May-12 4:38
jojoba201119-May-12 4:38 
AnswerRe: How to get delete/Update row + SQL Pin
Eddy Vluggen19-May-12 5:05
professionalEddy Vluggen19-May-12 5:05 
QuestionRe: How to get delete/Update row + SQL Pin
jojoba201120-May-12 2:42
jojoba201120-May-12 2:42 
AnswerRe: How to get delete/Update row + SQL Pin
Eddy Vluggen20-May-12 3:03
professionalEddy Vluggen20-May-12 3:03 
QuestionRe: How to get delete/Update row + SQL Pin
jojoba201120-May-12 17:38
jojoba201120-May-12 17:38 
AnswerRe: How to get delete/Update row + SQL Pin
Eddy Vluggen21-May-12 4:30
professionalEddy Vluggen21-May-12 4:30 
QuestionRe: How to get delete/Update row + SQL Pin
jojoba201121-May-12 20:11
jojoba201121-May-12 20:11 
AnswerRe: How to get delete/Update row + SQL Pin
Eddy Vluggen22-May-12 0:52
professionalEddy Vluggen22-May-12 0:52 
QuestionRe: How to get delete/Update row + SQL Pin
jojoba201122-May-12 3:22
jojoba201122-May-12 3:22 
AnswerRe: How to get delete/Update row + SQL Pin
Eddy Vluggen22-May-12 8:50
professionalEddy Vluggen22-May-12 8:50 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.