|
Hi All,
I'm been task to migrate a classic ASP website to ASP.NET - I intend to migrate to ASP.NET 2.0 (bare min.) using C#.
There is a processing engine which is causing much problems and dims the lights out when this is run.
What the processing engine does:
From a conceptual level it's a very straightforward process. What this processing page does is reconciles data against two data sources and if there is match on the first level of Ids then it goes to the next level of Ids and checks if there are mismatches for that level. If there are then it records those mismatch records. If there are matches at this level then it goes to the next level and check for mismatches between records at this level. Keep in mind that all these ids need to be linked together back to the master Id. My Id structure is like this:
Master ID - check datasource 1 against datasource 2
Match goto to second level
Child1 ID- check ds1 against ds2
Match goto to third level
GrandChildID - check ds1 against ds2
Match goto to fourth level
GreatGrandChildID - check ds1 against ds2
How this is handle in classicASP:
Currently this contains a huge initial openrecordset which is never closed. It does a 4000 code line do loop and creates embedded SQL with "Select * From" queries and creates recordsets and opens the recordsets but never closes those recordsets. Mind you when a user has to process say 50k to 100k records this thing absoultely bombs out. And in the creation of SQL on the fly it creates an IN function with over 1,000 ids in that IN clause which maximizes the the total number of Ids you can have in IN functions.
Design Questions:
1. Where to begin with this?
2. Should I go database processing or business rule engine?
3. How to improve workflow and performance for records reaching close to 1 million?
4. Should I create an interface for the user to add more rules?
Any starting points or case studies to at least get me to design mode. I would be most grateful.
Thanks,
|
|
|
|
|
I'm sorry, but I'm not really seeing your design here. Where do these Ids come from? Could you give a sample of data so that we can see what the problem really is. Instinctively, I can see that there are easier ways to achieve this but without more information from yourself there's not a lot I can do to help (except state that a BRE isn't going to help you here).
|
|
|
|
|
Sounds almost like a search engine or document searching app that is querying a dictionary and matching document ID's.
I've written one of those before although that's not entirely relevant, but what is is that all the hard work should be at the sql server, also it's potentially possible to do almost any query in one trip, not in a loop if you do some hard refactoring of the data and or add speciality tables just for this process. Those thousands of id's, that just sounds..well wrong. I would look first at the database and potentially refactoring it before I'd even consider anything else.
You say you've been tasked with going to asp.net from classic asp, to be honest that's not related at all to the real problem, you need to find a solution that *doesn't* involve looping, separate queries with thousands of id's in the IN clause before you do anything else.
And don't say you can't touch the data, there's always a way to simplify this stuff, sometimes building a separate table entirely from the existing data, who knows but what you are describing is just wrong. 
"The pursuit of excellence is less profitable than the pursuit of bigness, but it can be more satisfying."
- David Ogilvy
|
|
|
|
|
I want to develop an application in .NET. Basically i am answering one question but using different algorithms. So therefore, i prefer it doing it seperately. And once i get all the algorithms working, then i can bring all the implementation together under one single project. Yeh i will be defining classes not assemblies.
So what is this design methodolgy called so that i know what i am doing. I can see so many designing methods and i am confused like which one is best for me..I have read waterfall approach, agile methodology, etc etc...
Thank You
|
|
|
|
|
What you need to do is define an interface that exposes the public face of the algorithms you want to develop. Then you could create individual classes that implement the interface using something like Test Driven Development. Bringing them together is the easy part - and you may want to look at something like a Class Factory to choose the relevant implementation at run time.
|
|
|
|
|
|
ok Test Driven Development
|
|
|
|
|
Julia4u wrote: Basically i am answering one question but using different algorithms.
Sounds like a classic case of designing by contract (a la plug-ins).
See the Bridge[^] pattern and for a concrete implementation of plug-ins, see .NET 3.5's new System.AddIn[^] namespace, although I think that may be overkill at this point (unless you want discoverability, versioning, etc. for free).
/ravi
|
|
|
|
|
don't think so
Luc Pattyn [Forum Guidelines] [My Articles]
This month's tips:
- before you ask a question here, search CodeProject, then Google;
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get;
- use PRE tags to preserve formatting when showing multi-line code snippets.
|
|
|
|
|
Sorry to say but the answer to my question is
AGILE METHODOLOGIES like XP(Incremental development)
Thanks for trying at least. It took me some time to figure out but i am surprised to see that none of you know this easy answer. Strange...
Anyway
|
|
|
|
|
Julia4u wrote: AGILE METHODOLOGIES like XP(Incremental development)
WRONG! 
Citizen 20.1.01 'The question is,' said Humpty Dumpty, 'which is to be master - that's all.'
|
|
|
|
|
Incremental development is a scheduling and staging strategy in which the various parts of the system are developed at different times or rates, and integrated as they are completed.
This might help you a bit to calm down. If it's not correct show me the proof like i did.
ta
|
|
|
|
|
Julia4u wrote: Incremental development is a scheduling and staging strategy in which the various parts of the system are developed at different times or rates, and integrated as they are completed.
This might help you a bit to calm down. If it's not correct show me the proof like i did.
Every bloody development has various bits developed at different times and added together - there's nothing Agile about this as it applies equally to waterfall, RAD, SCRUM and the like. What we told you were the ways you should look at doing it, not the name of a methodology because we thought you were a professional developer, not a troll wanting somebody to do her homework for her. That's not what we do here.
|
|
|
|
|
Julia4u wrote: Sorry to say but the answer to my question is
AGILE METHODOLOGIES like XP(Incremental development)
Thanks for trying at least. It took me some time to figure out but i am surprised to see that none of you know this easy answer. Strange...
"easy answer"
This is what you originally posted:
I want to develop an application in .NET. Basically i am answering one question but using different algorithms. So therefore, i prefer it doing it seperately. And once i get all the algorithms working, then i can bring all the implementation together under one single project. Yeh i will be defining classes not assemblies.
From reading your initial description, how would anyone know that what you were talking about is the Agile Method? It's very vague. If I had to call it anything, it would be bottom-up design.
|
|
|
|
|
Julia4U - you seem to have a little bit of an attitude on you, so let me explain a couple of things to you:
Julia4u wrote: I want to develop an application in .NET. Basically i am answering one question but using different algorithms. So therefore, i prefer it doing it seperately. And once i get all the algorithms working, then i can bring all the implementation together under one single project. Yeh i will be defining classes not assemblies.
So what is this design methodolgy called so that i know what i am doing. I can see so many designing methods and i am confused like which one is best for me..I have read waterfall approach, agile methodology, etc etc...
This question is open to interpretation - you asked what the design methodology was here, but that's totally unimportant. You were told the best approach to take developing it - as I explained above the methodology is largely irrelevant if you don't know how to actually develop the code. That's why people told you how to actually develop the software - which is far more important to a real developer than the name of a methodology. BTW - which Agile method are you referring to? There are so many.
Now - I assume you want a career as a developer. You're going to have to learn how to take criticism and heat. Don't play the gender card, because it just demeans you and other women. Learn how to listen and respond politely - don't get into flame wars because YOU WILL LOSE.
|
|
|
|
|
Oh brother, I just read "her" replies in the other thread that got moved over here. As far as I'm concerned this person has burned "her" bridges here.
|
|
|
|
|
|
Pete O'Hanlon wrote:
Now - I assume you want a career as a developer.
Looks to me not developer, more like a PM, you know, those type who draws a lot of charts and schedules. 
|
|
|
|
|
Hi,
As a newbie, i am wondering which design methodology should i follow...
I want to develop a .Net application in modules. Each module will not depend on other module. So basically i want to code for each module seperately and once all of them are completed. I will bring them together under a single project.
So i am just wondering which design methodolgy this will be? I am looking over the google for design methods and there are so many and i am confused. 
Thank You
Julia
modified on Wednesday, April 9, 2008 9:27 PM
|
|
|
|
|
Why not mosey on over to the Design forum and ask this before the oven is turned up under your pants? 
|
|
|
|
|
ooops i never noticed design forum here.
Ill post it there then but that forum seems to be dead and i am looking for a quick reply. As it is just a one word answer for all of you as you are so experienced and you can just name the design method.
But you are right it's not the right place to ask.
Sorry for that.
Thank You
|
|
|
|
|
Hey - I just answered a question there. It's not a dead forum - it's just people don't ask many questions there.
|
|
|
|
|
Yeh i read your answer and i am sure you know the answer of my question as well. Can you not answer here because now i have posted this thread and if ill post it again it will be double post.I will be careful from next time for sure.
But if there are any issues, then ill post it there. Not a problem.
Thank You
|
|
|
|
|
The reason for posting in the correct forums is that they are the places that people will search when they are looking for an answer to a specific problem, not the lounge which is specifically NOT for programming questions. It's that quaint bit at the top of this page (just above the FAQ) that states Please do not post programming questions here. If you have a programming question then click here!.
BTW - when you post your problem in the Design forum you need to post a more detailed description. You need to develop in modules without any dependencies - how are you defining modules, Assemblies, classes?
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
