|
You need to understand your client's needs, technology to archieve their needs, and design a solution of their needs.
"Throughout human history, we have been dependent on machines to survive. Fate, it seems, is not without a sense of irony. " - Morpheus
"Real men use mspaint for writing code and notepad for designing graphics." - Anna-Jayne Metcalfe
|
|
|
|
|
|
Almost any application in our company interacts with data sources. As the number of applications and data sources grows it becomes very difficult to support our information system. To avoid this we decided to develop some "common gateway" - middle tier that will be used by all applications that need to access any data sources. Also we plan to implement WCF for this task.
What do you think about this?
Does anybody know any existing solutions?
Thanks.
|
|
|
|
|
I don't have an answer for you but a question:
A single "gateway" will surely simply the "layout" of your system (applications, data sources, ...) but will probably introduce a bottleneck, because all applications have to communicate with this gateway.
When performance is critical then a more SOA like approach (not exactly sure if SOA is the right term, though) by building a system where services interact on there own. When each service has a clearly defined task then this approach is clearer and more scalable then a Gateway approach.
But it's very difficult to give a perfect solution for all possible systems 
-^-^-^-^-^-
no risk no funk
|
|
|
|
|
“A single "gateway" will surely simply the "layout" of your system (applications, data sources, ...) but will probably introduce a bottleneck, because all applications have to communicate with this gateway. “
Our test application, built on WCF processes about 200 client requests per second. Average number of rows returned for one request is about 100. Hardware – is our oldest server based on 2 CPU’s Pentium3. It does not seem to me to be a bottleneck.
“When performance is critical then a more SOA like approach (not exactly sure if SOA is the right term, though) by building a system where services interact on there own. When each service has a clearly defined task then this approach is clearer and more scalable then a Gateway approach.”
If we take the approach where a numerous services will interact on there own and will have a clearly defined task –there will be almost the same problems that we have now: different applications (or services) will access different data sources at anyway they like. At the same time “Gateway approach” provides the full control over interaction with data sources – very useful in case of finding bugs or performance issues. This approach also gives an opportunity to implement some kind of “business layer” –that will be common for all developers.
I consider WCF to be the perfect platform for building SOA-oriented applications. So “Gateway” is not necessary the “one” and “only” service in the information system. If there will be some central repository, where the elements of business logic will be held – we can use any number of this “Gateway” services according to our needs.
|
|
|
|
|
You almost convinced me, but one last concern:
With a single gateway you risk to get a monolythic solution (everything is stuffed into one place).
What do you plan to prevent this? How to you keep good extensibility and maintainability?
-^-^-^-^-^-
no risk no funk
|
|
|
|
|
For this reasons (performance, security,maintainability) we can use several instances of services. They can share common repository or have it's own. Besides data sources these services will be able to communicate with each other or any external web or wcf-services.
In general we are going to achieve extensibility and maintainability by using some common model of business logic that will accommodate our needs.
|
|
|
|
|
After reading the other posts, too, I think that you have a nice concept for your solution. Reminds me somewhat of CAB (Composite UI Application Block) where the CAB is the central part that coordinates all others, just like your Gateway coordinates DataSources and client applications.
Best luck with your approach. And maybe you can provide us with some information how it worked out in a CodeProject article - would be very interesting.
-^-^-^-^-^-
no risk no funk
|
|
|
|
|
I think you should see this: www.datumnode.com
|
|
|
|
|
Thanks for the interesting link.
-^-^-^-^-^-
no risk no funk
|
|
|
|
|
The way I think:
You need a data provider factory, which returns an IDataSource. Any new DataSource provider would be implement the IDataSource. Once you have a default implemention (base class) of the the IDataSource, you could effectively use the provider pattern to return the datasource.
Effectively, the client would talk with the factory to return the appropriate data source provider.
Now, how you talk with the factory host, could be decided on the performance, deployment plans, ease of applying updates etc. A simple webservice hosting the above mentioned factory should get your started ?
But again, it does not sound like a good idea to have one webservice datasource provider for all the applications you use unless its the same datasource which is being used.
|
|
|
|
|
With this “gateway” we plan to implement some kind of business logic – that will be common to all developers. In this case, not only “data sources will be the same” but developers will even have no idea with what database they are actually interacting with.
|
|
|
|
|
rokford wrote: To avoid this we decided to develop some "common gateway" - middle tier that will be used by all applications that need to access any data sources. Also we plan to implement WCF for this task.
I don't think I understand. You are going to run all queries and result sets through the gateway machine?
|
|
|
|
|
Not machine but by common service (or services).
|
|
|
|
|
I like WCF for the communication, but I am concerned that you are looking to roll your own gateway mechanism. The problem with this approach is that you have to cater for all situations because you can guarantee that, sooner or later, somebody will want to do something that your gateway just isn't designed to do. More importantly, they will have a valid reason for doing this, and you will either have to extend your gateway or you will constrain what they can do.
Another issue, is that there really does come a time when your application needs to know what the data source is. For instance, you may want to insert a record into a database that has an automatically incremented primary key - but not all databases support this, so you need to have some mechanism to generate the key for this database.
|
|
|
|
|
From my point of view accessing a data source and retrieving data is the task that can be formalized quite well. ADO.NET is a good example of this. Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model. You should not think that application "knows nothing" about database - it knows enough to get required information from database. But there will not be connection strings, stored procedures ,etc.
"Another issue, is that there really does come a time when your application needs to know what the data source is. For instance, you may want to insert a record into a database that has an automatically incremented primary key - but not all databases support this, so you need to have some mechanism to generate the key for this database."
In our company we always use stored procedure instead of direct sql statements. So in this case there must be a procedure like "insert_something" that hides away how primary key is generated - with auto increment or using sequences.Of course,it's implementation will depend on what database is being using, but at the same time it will transparent to client applications.
So we do not want to create some universal language for all databases- but organize all these databases (connection strings,user credentials,stored procedures,ets) is some abstract model and make it accessible by all applications using different protocols and communication technologies (due to WCF)
|
|
|
|
|
But you still haven't answered the question about what happens when somebody legitimately needs to do something that your gateway just isn't designed to do?
I haven't got a problem with the idea of formalizing the way you communicate with databases, this is just good design, but there will come a time when the gateway just won't cope 100%. The question I'm asking is, what is your strategy going to be at this point in time? Are you going to have a team to manage the gateway separate from the team who write the applications? It's not a good idea for the two groups to be one and the same.
|
|
|
|
|
Actually I don't understand your question. Can you give some examples?
Without it your it seems to me something like "What you are going to do if ADO.NET (.NET,Oracle,MS SQL.) just won't cope 100%"
|
|
|
|
|
rokford wrote: Unlike ADO.NET we plan to implement some middle tier that will common for all applications (developers). So the main task is to access different data sources using some abstract model.
That is what a DAL is for. There is a very good reason that a DAL is a library and not a tier. By making it a tier you are adding significant application performance penalties for a perceived benefit that is still completely unclear to me.
|
|
|
|
|
We found some interesting solution but not sure it will accommodate our needs.
http://wcf.netfx3.com/files/folders/development_tools/entry11198.aspx
|
|
|
|
|
Hi,
I'm designing a processing engine that will do the following:
1. Determine the file type, i.e .TXT;.CSV;.XML
2. Check to make sure the file being review is valid
3. Make sure the data is valid based on delimiter and based on XML parsing
4. Load to a database server or pass to some type service
The purpose of this is to enable IS to utilize this "Framework" and extend the use of File I/O functionality or XML functionality dependending on the file type selected.
My first question is where do I start?
I'm looking at the Decorator/Iterator patterns but I feel I'm only extending the File I/O.
Thanks,
|
|
|
|
|
I might have done it this way:
Have a class for each file type.
1.) Find out the class/provider which knows how to handle the given file type from the list of registerd handlers (maintained in a global variable list).
eg:= FileProcessorTXT, FileProcessorCSV etc, all deriving from FileProcessorBase.
2.) If there is no registered handlers, error!
3.) If there is a registered handler, call the virtual function Validate() which should check the format. eg:- FileProcessorTXT's Validate() needs to bother about TXT file validation only.
The above classes job is to validate, read the special file types. Now we might need to introduce a common interface to talk with the DB. This could be a class/function on the FileProcessorBase such that each of the descendants would return data in the accepted format; say based on an XML template.
O my god.. I think I just wrote a design document!
|
|
|
|
|
Thanks I appreciate your input
|
|
|
|
|
Hi,
I've been surfing around for articles on best approaches to library architecture for multiple applications used by multiple developers and so far have come up with very little so I thought I may as well just ask some questions.
Our company has been developing solutions for a number of years using Borlands Delphi and a FAT Client-style architecture where most of the code gets compiled and linked into one executable per application (There were historical reasons behind this style that I won't go into). So structuring our code library has been fairly easy, using a Visual Source Safe as our primary source control, we simply share the code into the application folder, then we reference all the code as relative path.
However, since starting development in C# (and the dotNet Framework) its obvious that this approach was not going to cut it.
So then development started and progressed down the path of separate, specific assemblies for each specific algorithm/datastore/UI etc. Now I/We have the problem of structuring these assemblies/libraries so that multiple applications reference them both in development and deployment. Deployment isn't much of a worry as we are placing all the assemblies in a folder on the target platform but our major problem is referencing these assemblies during development. Ie in Solution/Project files, VSS and also in our Build System (FinalBuilder)
I'll give you an example.
LibraryA and LibraryB are two assemblies that contain algorithms for two separate/different situations. They have there own Project and Solution files, their own specific tests, are in their own separate folders with our source control and are built separately within our Build System.
Now Application1 needs to use both LibraryA and LibraryB. Application1 is laid out in the same way that both libraries are (VSS, BuildSystem etc) but there needs to be a reference to both assemblies within the Project file. And it needs to be available at Design time (within VS) and during automated Build time (FinalBuilder).
I know about the GAC and as a company we have decided against using it.
Our preliminary idea is to use a Global assembly area of our own, that each developer has (using an environment variable) and it contains all the assemblies within our library, so all references are directed at this location. So our Build System just needs to ensure that these assemblies are available when building each application. Other ideas that have been discussed was to setup our Build System to build all dependent projects before the main project and reference each dependent project within the main projects Solution file with the references to the dependent projects build locations.
So my question(s) is/are:
What are some of the approaches that other organisations/people use to reference assemblies at design/development time across multiple applications and multiple developers?
Hopefully I have explained our problem sufficiently and I'm betting that this is a issue in more than one place of business, so I'm sure the answer(s) is out there. Please share your thoughts and ideas.
Thanks
Brett Morris
-- modified at 21:50 Tuesday 26th June, 2007
|
|
|
|
|
Why don't each of you just assemble your own custom solutions of the pieces you need and allow the build machine to do the same?
This statement was never false.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
