|
I never could get the hang of Thursdays.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
I did not get you
Life is all about share and care...
public class Life : ICareable,IShareable
{
// implements yours...
}
|
|
|
|
|
|
But do you know where you towel is?
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Probably in the wash.
That's where they always are when I get out of the shower.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
Just kidding about the rock stars but thought it might get some attention. But I do have a question. I have read all about the databases and really don't care which one is better or more efficient and so on, what I want to know and have always wanted to ask: If you have a simple program that needs to store data, what is the data size cut off that it is more efficient to use a database or just serialize to files (xml or other)?
I have tested xml files holding as many as 10,000 elements, and have not noticed much of a delay putting same elements in database tables. I have tried several different approaches and tried to complicate the tables and queries and didn't see much difference in performance.
Has anybody ever done any load testing or any research on what would be the cutoff point on data size, or evaluated the overhead over XML or a Database?
I also know that there is a lot of support for working with databases and cloud storage which simplifies the DAL & LOB data access and storage, but it also leaves a large footprint on your device for something where the data load might not be so large as to warrant such a large footprint, such as mobile devices. Or does it? I don't know that's why I am asking.
Please enlighten me on the subject.
The difference between a question and a stupid question depends on who you ask to answer it!
wreckless
|
|
|
|
|
I think you that be looking for a silver bullet number when the decision is based on a range of factors. I would place platform as the first criteria and would consider xml for mobile.
Complexity of the data structure would be the next criteria, 3-5 small tables and I might use XML (unlikely as I am very database centric) much more than that and I would not even consider xml.
As for volume of data, it would probably not even come into the equation.
This discussion should also probably not be in the Lounge, expect some left field answers!
Never underestimate the power of human stupidity
RAH
|
|
|
|
|
There are no database rockstars, but those who master databases are a SELECT group.
|
|
|
|
|
ORDER, please!
No object is so beautiful that, under certain conditions, it will not look ugly. - Oscar Wilde
|
|
|
|
|
Duncan Edwards Jones wrote: There are no database rockstars,
Well, there is "the One". Oracle told me so.
Ok, I am out.
You have just been Sharapova'd.
|
|
|
|
|
Yeah, it's pretty hard to JOIN their RANKS! 
|
|
|
|
|
Indeed, a GROUP HAVING few members.
I'd INSERT myself in the ranks, but I am not worthy to JOIN...
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
It looks as I am BETWEEN monsters, WHERE was my head when I decided to JOIN? I'm about to DROP the USE of this GROUP BY heaven's sake!
BTW I dislike SQL and DBs
Geek code v 3.12
GCS d--- s-/++ a- C++++ U+++ P- L- E-- W++ N++ o+ K- w+++ O? M-- V? PS+ PE- Y+ PGP t++ 5? X R++ tv-- b+ DI+++ D++ G e++>+++ h--- r++>+++ y+++*
Weapons extension: ma- k++ F+2 X
I use 1TBS
|
|
|
|
|
Try using the KEY (the whole key, and nothing but the key)
|
|
|
|
|
I'm feeling left outer this conversation
PooperPig - Coming Soon
|
|
|
|
|
He could try, but he is not the right CANDIDATE to have the KEY.
You have just been Sharapova'd.
|
|
|
|
|
|
If you're not clever enough to join the RANKs[^], you could try the DENSE_RANKs[^]. 
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Yup, and we hang out in HEAPs and CLUSTERS.
If it's not broken, fix it until it is
|
|
|
|
|
here we go.... 
Charlie Gilley
<italic>Stuck in a dysfunctional matrix from which I must escape...
"Where liberty dwells, there is my country." B. Franklin, 1783
“They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759
|
|
|
|
|
Databases do lots more than just store data. It's a centralized point for transactional data storage, modification and retrieval.
Try using some XML files in an enterprise environment. That's simply not going to work.
If you're just building some consumer app I'd say go for XML, as it saves you, and the consumer, all that trouble of installing a database.
On modern machines size really doesn't matter.
|
|
|
|
|
Even for a consumer app, it's not so cut and dried.
I'd put the XML / DB cutoff as a table complexity and INSERT / UPDATE / DELETE criteria myself. XML is text based, so frequent changes are going to mean a lot of file copying.
Even for mobile there is SQLCE or SqLite instead, both of which have a relatively small footprint.
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
For me the question is not quite right.
I don't think row count, or class count should determine whether a database is used or not.
I think it's more a case of what you are going to do with the data.
If you think you may need to 'look' at the data occasionally, manipulate it or run some form of reports(statistics, counts, aggregation...) on the data than a database is the right choice even if you have as few as 100 rows.
If you are just reading in data that has already been 'cleaned' and all you are doing is transforming the data then perhaps a database is not necessary.
As others have mentioned there is no correct answer - whichever path you take there will be advantages and disadvantages.
My rule of thumb is not so much the volume of data, but what is going to happen to the data.
Regarding footprint - there are some databases which leave a reasonably small(suitably vague statement of the day...) footprint, particularly those where the database is structured as one file.
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
modified 16-Apr-15 3:35am.
|
|
|
|
|
10,000 is very low number to compare anything for performance.
When you compare XML / Flat file with database the only advantage you get is write performance and cheaper cost. Which is never a deciding factor for any application, whether its enterprise or consumer.
The things you can get with database and not with xml are:
- Concurrency (You can control if you have insert/update at the same time on your data)
- Transaction Reliability (You can go back to an older data/snapshot to correct a mistake)
- Backup (You can configure very efficient backup policy i.e. log shipping / replication which is not possible with XML)
- Partitioning (You can store data to different database/location based on region and then retrieve it easily)
- Archiving (Transfer data to archive in realtime and then access it in realtime with linked database)
- Security (fine grain control over who can access what)
- And the most important, Well Structured and Connected Data (makes complex query for different reports very easy)
So I would suggest you to use XML for intermediate stuff, or to store settings. but other than that don't waste your time and efforts. Just go with the RDBMS.
And for Mobile device to store offline stuff you can use XML / JSON Documents (JSON is better than XML). But at the end you should have a RDBMS server and Sync your offline data to it.
|
|
|
|
|
I'd go for XML with anything that is just a few lines, like configuration. If it contains a lot of relations, I'd switch to SQLite.
If multiple users are to access the datasource, it would also become a real database. I don't want to have to implement things like table-locking.
Bastard Programmer from Hell 
If you can't read my code, try converting it here[^]
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
