Design and Architecture

Re: Release implementation

18-Sep-07 5:07

led mike wrote:
Just a crazy idea but maybe you could use Google

Oh you crazy so and so. It's zany ideas like this that make people chuckle. You're mad you are.

Deja View - the feeling that you've seen this post before.

etkid8421-Sep-07 2:50

etkid84

21-Sep-07 2:50

use subversion for your cm tool, you can use it from the command line, or tortoisesvn (windows gui version), or subclipse (eclipse plugin).

use trac to track progress, tasks, bugfixes, user docs etc... http://trac.edgewall.org

setup subversion and trac on an apache webserver, and you should have no problem.

make sure you identify two people to handle all the CM work, and appoint them as your official "build-meisters", set up nightly cron jobs for builds, etc.

have two branches a delivery branch, integration branch, and make sure the developers create their own branches when they work on code.

set periodic dates for builds, do code reviews and make sure your developers unit test their stuff before checking their code into to the integration branch. test everyones stuff together using the integration branch...make corrections, retest, repeat until you have something good...then merge it into the deliverable branch.

reset the integration branch from that point on the delivery branch... and repeat...

make sure the developers continually grab the latest from the integration branch if their stuff doesn't make the deadline or cut... they need to always play with the current stuff..

kind regards,

David

Low Level Design

hari.babu16-Sep-07 22:50

hari.babu

16-Sep-07 22:50

Hi all,

am working on pharmacy related project in which now working on Low Level Design document, want to know what no clearly what comes under below stated section in LLD:

Design Alternatives:
Brief description of the design alternatives considered for this module should be stated. Reasons for selection of a particular design from the alternatives.

hari.k

Re: Low Level Design

Pete O'Hanlon17-Sep-07 0:29

17-Sep-07 0:29

This is just asking what you considered in your design, and why you chose the particular design you did. It's normally in there to show that you did consider alternatives and the design isn't just something that you threw together.

Deja View - the feeling that you've seen this post before.

Binary Data Storage

rcardare14-Sep-07 11:17

rcardare

14-Sep-07 11:17

I need some ideas or pointers on storing several hundred gigabytes of data. Currently, the files transferred range from 1 KB up to 600 MB and vary in sizes.

What I would like to do is break the files down into small (8K?) blocks and index them with a hash. Then in a database, create a chain so the files can be reconstructed. I like this because it will allow duplicated blocks of data to be identified and reduce the size of the storage. Bad idea? How might a directory structure look to accomplish this so it isn’t impossible to enumerate through the files?

I thought about storing the files blocks in a database as blobs but I think that would be too much strain on the database resources. SQL 2008 will have some nice features to accomplish this but it will be a year or two before we get there.

Any ideas? Thanks in advance.

Mark Churchill17-Sep-07 4:38

Mark Churchill

17-Sep-07 4:38

Whats wrong with the filesystem for storing your data, with a seperate index if necessary? We probably need more information on what type of data you have and how you intend to index it.

When you said "I have a few hundred gig of files and I need to store them somehow" the first thing that comes to mind is NTFS Wink | ;)

Mark Churchill
Director
Dunn & Churchill

rcardare17-Sep-07 6:08

rcardare

17-Sep-07 6:08

Thanks for the response Mark.

The system I am working on is a pub/sub that distributes files to multiple subscribers. It is an in-house system that transfers manufacturing data – application, documents, collected data, test results, etc. I currently have it where the data is uploaded to a file server in the sky and the subscribers then download – nothing too complicated.

It is currently architected where the subscribers connect through a web farm (load balanced) to download small chunks of 64K until the transfer is complete. Each chunk is a new http request which is causing heavy I/O between the web servers and file server as it opens the file, reads until the requested chunk and then returns the data. It would be ideal to just stream the file using the same connection until the transfer is complete and resume with network hiccups. The problem here is that some of the third-party sites use older proxy servers which won't allow for that. In addition, there is a firewall(beyond my control) which limit the amount of time a connection is allowed to remain open.

My new plan is to store the files in smaller chunks so it is more efficient when downloading so it doesn't have to navigate through the large files up to the position of the chunk being downloaded. I could also leverage this to reduce duplications of data stored on the server. Unfortunately with this design, I would then have to open a database connection to identify where the blocks of data are stored and how to piece them together. I would probably end up in a worse scenario with the IO to the database and calculating what to return. This is where I thought storing the data in the database might be better as I would already have a connection and I can query the data and return the exact requested chunk. I am torn here because of the associated costs for storing that much data in a database.

I am just curious if there are any ideas. I probably should not worry about how the data is stored and work on solutions to solve the number of http requests.

Thanks

led mike17-Sep-07 6:47

17-Sep-07 6:47

rcardare wrote:
I probably should not worry about how the data is stored and work on solutions to solve the number of http requests.

Yes, it sounds to me your current solution is not using chunked-encoding, or not using it correctly.

Mark Churchill17-Sep-07 23:22

Mark Churchill

17-Sep-07 23:22

Hi,

It seems like the issue with some legacy clients not handling large downloads may be solvable by the clients requesting the data in chunks. This doesnt mean the data needs to be stored in chunks though Wink | ;)

I'd solve the issues with how the clients retrieve the files first. Then you can see how appropriate your storage mechanism is (I'd suggest NTFS with additionally indexing in SQL Server would work fine.

Mark Churchill
Director
Dunn & Churchill

etkid8421-Sep-07 2:55

etkid84

21-Sep-07 2:55

Here is my two cents for what it's worth:

avoid moving the data around, why not use the database to tell the client where the data is located?

whereever there is data there should be services to provide the client with the "knowledge" about the data he is looking for.

when you are working with large chunks of data, avoid moving it around, and especially across a network.

make the local service perform as much work as possible and return an "answer".

David

Message on user ingerface!

ky_rerun27-Sep-07 0:23

ky_rerun

27-Sep-07 0:23

We had a similar problem with large file distribution.

I wrote a system where a client would download the file and then send broadcast messages out so that other local hosts could pick up the file and download it from the lan. Also if your using windows clients look at the Background intelligent file transfer.

a programmer traped in a thugs body

Jwalant Natvarlal Soneji13-Sep-07 21:56

Jwalant Natvarlal Soneji

13-Sep-07 21:56

Dear friends,
I guess and must have this thing been discussed many times before also, but now a fresh question. Smile | :)

What should messages on user interface say to user while these situations (and many more)(user is in US).
Also before, at the moment of doing and after performing the action...
File Copy
File Not found
File Delete
File Send
Database Update
Database Insert
Database Delete

Please help with the syntax and approprite sentences.
thanks in advance,

Jwalant Soneji (BE IT) (India)
Mobile: +91 9969059127
At Winodows Live Spaces
At Yahoo! 360
At BlogSpot

Re: Message on user ingerface!

Pete O'Hanlon14-Sep-07 0:15

14-Sep-07 0:15

The convention for a file copy is to show a file being copied animation, and to provide feedback on the file being copied. For instance, if there are multiple files being copied you would normally tell the user where they are in the copying process, e.g. Copying 10 of 24 and it is common to show a progressbar detailing performance.

Anyway, the standard way to design your application is to write the messages out in your own language (stored in a resource DLL), and then have someone translate the commands out in the relevant language.

When it comes to standards, take a look at what the tools you use on a day to day basis tell you. What does Word say in these situations? Visual Studio? Explorer? It's generally best to follow what others have done rather than reinventing things yourself.

Deja View - the feeling that you've seen this post before.

Mass mail sender app

firestoper12-Sep-07 19:20

firestoper

12-Sep-07 19:20

Hi Guys,

I'll be starting to work on web app that would send out massive mails for our clients campaign (approx 20k per batch), the webserver's and mail server are being hosted on our end so we have the complete control of the platform.

I'm just wondering:

1. What would it takes me to maintain a high volume of mails being received by our mail server
2. Would it be possible to have another mail server where we could crate a load balance for the volumes of mails?
3. Can you give reference / URL where I could check on the design for this kind of web apps.

Thanks
Dom

Re: Mass mail sender app

led mike13-Sep-07 4:44

13-Sep-07 4:44

1) cross posting is discouraged here ( that means you posted this same message in more than one CP forum)

2) people don't like SPAM including the people at CodeProject

However hope is not lost, since you think this is a "Web App" with any luck you will never get it to work.

The "EGG" task

Saikek12-Sep-07 11:16

Saikek

12-Sep-07 11:16

Hello comrades !!! I need your help. Here is what we have:

We have a jpg image that contains gray background and a lot of white eggs on it. We need to get the number of eggs and the size of each eggs. (i think in pixels).

How can i do it ?

The hardest problem is to get the whole egg (To mark the edges) and smart divide screen into some parts, for easiest count.

Thanks for help.

One nation - underground

Re: The "EGG" task

Pete O'Hanlon13-Sep-07 10:26

Technologies for multiple-question application form.

13-Sep-07 10:26

Gosh. Cross posting homework. Excellent.

Deja View - the feeling that you've seen this post before.

walkeraj11-Sep-07 8:24

walkeraj

11-Sep-07 8:24

I work for a state agency that has a multiple page, multiple question application form. Because the application centers around eligibility for a payoff, there are different paths that an applicant may take through the application. For instance, depending upon their answers, they may only need to fill out one or two pages of questions, but could conceivably need to fill out more pages depending upon whether or not further information is required.

What sort of approach/technology would be ideal to handing this sort of thing in .NET? Currently, it is handled in a very clumsy manner involving multiple .aspx pages and a lot of unnecessary, hard-coded session variables and logic checks. In this environment it's easy for things to become unnecessarily complicated, and changes to a single question might require changes in multiple places throughout the application (especially to database calls).

One method I was thinking of would be a single .aspx page with a multiview control backed by XML files for each page of questions where the questions are given attributes that define what type of question they are (i.e. yes/no, checkbox, text), as well as the name they are to be given in the database (i.e. "absence_eligibility1"), but I am unsure of the best way to implement the dependencies.

Any ideas?

Re: Technologies for multiple-question application form.

ncushing26-Sep-07 20:16

ncushing

26-Sep-07 20:16

Depending on how much time you are planning to invest in this, it may be more cost effective to get an off-the-shelf solution that can handle this type of skip logic. These types of product range in price from hundreds to thousands of dollars.

*DISCLAIMER* I am the architect for a company that produces this type of software, but there are many options so you might want to check around. I can tell you from first hand experience that once you get into conditional logic and/or have to create and deploy multiple questionnaires you've got quite a bit of work to do.

Basic Class Design

rb15710-Sep-07 0:42

rb157

10-Sep-07 0:42

I am just trying to learn OOAD using c# at present, by designing a simple song management system.

Within this system there are songs which have one to many artists. I therefore designed classes as follows:

<br />
class Song<br />
{<br />
  Artist[] artists; <br />
  // various other attributes<br />
}<br />
<br />
class Artist<br />
{<br />
   String name;<br />
   //various other attributes<br />
}<br />
<br />
class SongList<br />
{<br />
  List<Song> songs;<br />
}<br />
<br />

This seems to work OK I could in my output list each 'Song in songs' and show the artist by looping through each one in that song.

Where I have got really confused is another requirement which I might have to list all Artists along with their songs. How can I build a collection of artists and list the songs for each artist if artist is an attribute of a song. I can after all do song.artist, but I cannot access song from the artist: artist.song.

On thinking further I decided that an artist is not actually an attribute of song, but just a loose relationship. If they are just to seperate objects in a collection how do I implement this loose relationship between the two objects.

Any help, including references to books or other material which may be of help would be greatly appreciated. I son't think that I am thinking correctly to understand this, so any help that will assist me in thinking the right way would be greatly appreciated.

Thanks,
Richard

Ed.Poore10-Sep-07 0:54

Ed.Poore

10-Sep-07 0:54

rb157 wrote:
How can I build a collection of artists and list the songs for each artist if artist is an attribute of a song. I can after all do song.artist, but I cannot access song from the artist: artist.song.

You could have a dictionary with the artist as the key and a list of songs as the value, e.g.

Dictionary<Artist, List<Song>> songsByArtist = new Dictionary<Artist, List<Song>>();

You'd then have to loop through all the songs, loop through the artists, check if they already exists in the dictionary (if not add the key-value pair) and then add the song to the List in the value part.

As for the further thinking I'm also having a think about it (too early in the morning (oops, afternoon Blush | :O

))

led mike10-Sep-07 5:09

10-Sep-07 5:09

rb157 wrote:
How can I build a collection of artists and list the songs for each artist if artist is an attribute of a song. I can after all do song.artist, but I cannot access song from the artist: artist.song.

Sure you can

<Artist><Song><Song>class Song
{
	List<Artist> artists;
	// various other attributes
}

class Artist
{
	String name;
	List<Song> songs;
	//various other attributes
}

class SongList
{
	List<Song> songs;
}

</Song></Song></Artist>

rb15710-Sep-07 6:28

rb157

10-Sep-07 6:28

Thanks, I did wonder about these sort of circular relationships and whether they are wise... i.e. if for some reason the Song and Artist objects do not replicate each other correctly then a different result is obtained when outputing song.artist[i] than artist.song[i]. However if circular relationships are generally considered appropriate, then I am quite happy with that approach.

My concern is that in the real world the artist is probably not an attribute of the song and the Song not of the artist, in that one is not part of the other, just has an other type of relationship: In the same way that the Driver is not an attribute of a car. However if a driver is modelled in the same way as a steering wheel in its relationship then that is fine. I would appreciate comment on this point.

The approach provided by Ed... This would work artist to Song, but I think that another list would be required to output song by artist:

Dictionary <Song, List <Artist>>

Which is fine but would appear to be an amount of processing which is far in excess of mike's suggestion.

Thanks,
Richard

led mike10-Sep-07 8:02

10-Sep-07 8:02

rb157 wrote:
I did wonder about these sort of circular relationships and whether they are wise

I apologize. I was in no way suggesting it was wise, only that it could be done. In order to know if it is wise much more information is required. I would also guess that a design utilizing one or more interfaces would most likely be superior.