|
Hi Everybody,
Please note that I am not a very senior developer so want help from you guys...
We are going to start a new project which is very similar to Shopify. The proposed solution will include the following,
1. A admin site which manages Merchants. Here the merchants will have multiple stores under him.
2. A Tablet client which downloads/updates the data from/to the website using web services.
My questions is,
1. So while designing the database model whether is it a good idea to keep separate databases for each merchant or keep a single database?.
2. How much data can be kept in a single table?.
3. Regarding the scalability and maintainability which is the proper design, single database or multiple database?
4. Is there a bottle neck in MS SQL server when creating multiple databases instead of one?.
5. Which kind of sever solution in this case we need to propose, Cloud based database or Godaddy like hosting server?. Can a godaddy like hosting service provide a performance like Amazon.
Please note that we are planning to use MS SQL server for keeping the data.
Thanks & Regards,
Nitheesh George
Nitheesh George
http://www.simpletools.co.in
|
|
|
|
|
1. Single database is the better design, there are strategies to handle large volume. Extreme data (think Google, facebook, Amazon etc) requires a different strategy but it is probably too early to begin taking that into account.
2. Much more than a startup site can expect to see.
3. See 1 - same question
4. Don't create multiple databases - removes the question.
5. Performance like Amazon is above and beyond your short/medium term expectations, I imagine they have a large dedicated team simply looking after their database systems. Any of the hosting companies will be delighted to meet you short/medium term needs.
If your database is designed properly it will be scalable, I would suggest getting a professional to design your system. An excellent specification will facilitate the design.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Nitheesh George wrote: How much data can be kept in a single table?
524,272 TB (Source[^])
Please do notice that 16 TB is the max file size.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
|
|
|
|
|
Quote: 1. So while designing the database model whether is it a good idea to keep separate databases for each merchant or keep a single database?.
Separate databases with separate security credentials. You really don't want a data breach if you can possibly avoid it.
Quote: 5. Which kind of sever solution in this case we need to propose, Cloud based database or Godaddy like hosting server?.
Cloud database - as you have already specified SQL Server I recommend SQL Azure.
|
|
|
|
|
Hi can anyone help me I don't necessarily want a solution just an idea.
I have a table e.g
ID - Start Date - End Date - Difference(Duration) - SessTotal - SessID
Now the Id is per user. and the date difference is the time spent doing something. If the difference is greater than 30 minutes I need to increment the session ID by 1.
Second I need to create a running total for session total per session.
1 session ID must also not be able to have more than 1 UserID [ID] in it's row.
|
|
|
|
|
If the Difference field is the difference between Start and End dates, then it should not be stored in the database; it should be calculated as and when required. The same goes for Session Total.
- You can modify the session Id when you insert the records.
- Calculated values can be done by program or SQL, it all depends on where you are displaying the information.
- Not sure I understand the last question
|
|
|
|
|
I absolutely have to display the date difference that is how the specification is made.
Let me try saying it this way:
I have to have a running total for the times until there is a 30 minute or larger gap or if a new user ID which is a different person is the next row of the dataset. then that is a session then I have to increment the session ID for each Session.
like
Update
SET [Session ID] = [Session ID] + 1
WHERE [Date difference] < 30 Minutes
OR If dates run out for this person presenting a new person.
And for then the running total must start again.  It is allot to ask I am sorry but any help would be greatly appreciated.
|
|
|
|
|
Martin Niemandt wrote: I absolutely have to display the date difference
Fine, but you should calculate it at the time you need to display it. Storing calculated values in the database is bad design and prone to error. I would suggest reading http://www.w3schools.com/sql/func_datediff.asp[^].
|
|
|
|
|
Thank you for the function, but it is not quite solving my problem. If you could maybe give me a guideline on how I can update using this row combined with the following row. and then also group by as to not overlap users? if not then thank you for your effort.
|
|
|
|
|
Sorry, but my SQL skills are not very advanced. And I'm not sure that I fully understand what you are trying to achieve.
|
|
|
|
|
Yes it is quite difficult to explain as well, I would show you my query but you might puke! Thank you for your effort though! 
|
|
|
|
|
Martin Niemandt wrote: I would show you my query but you might puke!
Which suggests that your design needs looking at.
|
|
|
|
|
It is not the design really it is the indents and spacing that is terrible at the moment. thanks any way I figured it out!
|
|
|
|
|
Richard is correct in that the calcs should not be stored. Look into creating a view to service your requirements.
Where you need 2 rows to interact you can create a left join back to the same table on A.UserID = B.UserID and B.ID = A.ID + 1. Do not forget to test for null ISNULL(B.Value,0)
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Thank you that helps. I do understand when you guys say calculated fields should not be stored but I have to store summary's of 80 million rows. and when they are stored they will not change but get added to. I could show you my query then you could tell me what to improve if you want. but thank you any way
|
|
|
|
|
For the query where you need to access the 2nd row I would store the results during a process run.
A summary of 80 million implies less than! Most people dealing with this sort of volume create OLAP cubes for reporting purposes (summaries optomised for reporting purposes).
If you have a query where an calculation is being performed within the row data there is no need to store it simply do the calc in your select procedure.
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
Interesting I will go take a peek at OLAP, The only problem is I am moulding the data in parts updating parts after the initial insert. I just need to get a running total of the date difference (Which is used as time) and I need to restart the counting when the difference (not the total) is more than 30 minutes and then I need to assign that session an ID a session ID must not lap over different users. the part I am stuck with is 1. the running total of the time difference 2. assigning incremented ID's based on this logic
[Edit]: Your left join idea helped already thank you
|
|
|
|
|
Do some research into ROW_NUMBER and PARTITION OVER these may be the keywords you are looking for. Oh no I'm channelling POH
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
channelling POH? :P I am already using DENSE_RANK(). but I didn't think of partition thank you
|
|
|
|
|
Mycroft and Richard are completely correct in that you should never store duplicated data.
But theres a way to automize it, on SQLServer it's called a Computed Column[^] and if you're using Oracle it's a Virtual Column[^].
The best part is that you can put an index on them.
|
|
|
|
|
Haha I realize that they are correct. and it is not going to store duplicate data it will always change. see this is to analyse user actions and time they spent lingering about on certain pages. and I know about Computed columns I am a bit more intermediate with sql but thanks any way
|
|
|
|
|
Hi,
I want to know if there is a simple and efficient way to select a value from a row that has a column with minimum creation time for the given parameters.
Let us take a sample table named Patient
-----------------------------------------------------------
PatientId Observation Time Value
-----------------------------------------------------------
1 Temp 2014-02-19 03:55:00 35
1 Temp 2014-02-19 03:45:00 37
1 Weight 2014-02-19 03:40:00 60
If i am given the PatientId and Observation, I need to retrieve the value field with minimum creation time
For patient id 1 and observation Temp this would be Value 37
since it has 2014-02-19 03:45:00 as minimum creation time
|
|
|
|
|
This should work:
select top 1 Value
from Patient
where PatientId = 1
and Observation = 'Temp'
and Time in
(
select min(Time)
from Patient
where PatientId = 1
and Observation = 'Temp'
)
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
|
|
|
|
|
Thanks. That seems to be simple. If i have numerous entries for the specific patiend id and observation, will the use of top be performance efficient ?
|
|
|
|
|
Top ensures that only one line is returned.
You can remove the top 1 - however if more than one line is returned you will have to make a choice at some point if the values are different.
Simply replace the top 1 with distinct if you know there will only be one value returned.
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
