|
I want to be downloading 5 files simultaneously. I have an array of threads created, each downloading a file. These files though, are actually pages of a book. So I want the thread to download a page only once (of course!). What I wanted to do was to maintain a shared global page number in the main thread, so that when each of the worker thread finishes it's current page, it reads this number, decrements it, and downloads it. Only, two things:
1- I want this number to somehow be stored on disk, so that if a power cut happens, or windows restarts or any of these issues, I'd be able to resume.
2- Of course, I don't want the threads to access the global number at the same time, so that they wouldn't corrupt it. How can I enable this lock?
Thank you very much.
Sammy
"A good friend, is like a good book: the inside is better than the cover..."
|
|
|
|
|
Don't use an array of threads. Use async IO or ThreadPool.QueueUserWorkItem() instead.
Use the C# lock statement or possibly Interlocked.Decrement() to protect your shared state.
Think about the fact that what your just described will not correction resume because you describe decrementing a counter when the download is started. You need to maintain state about which ones are started and which ones are completed. You can't rely on them completing in the order the downloads were started.
Why do you think you need to download five pages simultaneously anyway? That sounds suspect to start with.
--
-Blake (com/bcdev/blake)
|
|
|
|
|
Blake Coverett wrote:
You can't rely on them completing in the order the downloads were started.
Actually, what I was planning to do is, if I detected that the download was interrupted, I would increment this counter by 5, and re-start. This way, all pages that were downloading at that time, will be resumed.
Blake Coverett wrote:
Why do you think you need to download five pages simultaneously anyway? That sounds suspect to start with.
I'm writing a program for someone to parse his daily online reports. It's much faster if I download the html pages simultaneously than to get them one by one. Also, I can parse one page while the other is downloading.
Sammy
"A good friend, is like a good book: the inside is better than the cover..."
|
|
|
|
|
profoundwhispers wrote:
Actually, what I was planning to do is, if I detected that the download was interrupted, I would increment this counter by 5, and re-start. This way, all pages that were downloading at that time, will be resumed.
You can't get away with just a single counter like that. That's the class problem I was describing when I said you couldn't count on them completing in order. Consider this sort of pattern:
counter = 10
start 1, counter = 9
start 2, counter = 8
start 3, counter = 7
start 4, counter = 6
start 5, counter = 5
end 2
start 6, counter = 4
If it aborts here, incrementing the counter by 5 puts it back at 9, meaning page 2 will be downloaded twice and page 1 lost.
You need to track completions separately.
profoundwhispers wrote:
I'm writing a program for someone to parse his daily online reports. It's much faster if I download the html pages simultaneously than to get them one by one. Also, I can parse one page while the other is downloading.
Right, but why particularly 5 at a time? What you probably want to do is just fire off all your WebRequest.BeginGetResponse and let the normal infrastructure dispatch them a few at a time. It already knows how to balance such things.
--
-Blake (com/bcdev/blake)
|
|
|
|
|
Hi!
I'm working on a P2P app, with security built in, it's going to be released as open source as soon as I have a working prototype...
I've gotten as far as exchanging messages, encrypting username password with RSA and exchanging a common symmetric key. But when I try to decrypt messages encrypted with the sym-key I get an error:
Binary stream does not contain a valid BinaryHeader, 95 possible causes...
I have the following code:
<br />
public byte[] SecureSerialize(byte[] key, byte[] iv)<br />
{<br />
if (encrypted == Encryption.NotEncrypted) throw new Exception("Message type should be serialized with Serialize()");<br />
<br />
byte[] streamBuffer = new byte[BufferSize];<br />
<br />
streamBuffer[0] = (byte)msgType;<br />
streamBuffer[1] = (byte)encrypted;<br />
<br />
MemoryStream ms = new MemoryStream(streamBuffer, sizeOffset, BufferSize - sizeOffset);<br />
<br />
RijndaelManaged rij = new RijndaelManaged();<br />
CryptoStream cs = new CryptoStream(ms, rij.CreateEncryptor(key, iv), CryptoStreamMode.Write);<br />
<br />
BinaryFormatter bf = new BinaryFormatter();<br />
bf.Serialize(cs, this);<br />
<br />
cs.Flush();<br />
ms.Flush();<br />
<br />
SHA1Managed hashAlg = new SHA1Managed();<br />
byte[] hash = hashAlg.ComputeHash(streamBuffer, 0, (int)ms.Position + sizeOffset);<br />
bf.Serialize(ms, hash);<br />
<br />
byte[] returnBuffer = new byte[(int)ms.Position + sizeOffset];<br />
Array.Copy(streamBuffer, 0, returnBuffer, 0, (int)ms.Position + sizeOffset);<br />
<br />
return returnBuffer;<br />
}<br />
<br />
public static UdpMessage SecureDeserialize(byte[] msg, byte[] key, byte[] iv)<br />
{<br />
if ((Encryption)msg[1] == Encryption.NotEncrypted) throw new Exception("Message type should be deserialized with Deserialize()");<br />
<br />
byte[] decryptedBuffer = msg;<br />
<br />
MemoryStream ms = new MemoryStream(decryptedBuffer, sizeOffset, decryptedBuffer.Length - sizeOffset);<br />
<br />
RijndaelManaged rij = new RijndaelManaged();<br />
CryptoStream cs = new CryptoStream(ms, rij.CreateDecryptor(key, iv), CryptoStreamMode.Read);<br />
<br />
BinaryFormatter bf = new BinaryFormatter();<br />
UdpMessage message = (UdpMessage)bf.Deserialize(cs);<br />
<br />
cs.Flush();<br />
ms.Flush();<br />
<br />
message.msgType = (MessageType)msg[0];<br />
message.encrypted = (Encryption)msg[1];<br />
<br />
SHA1Managed hashAlg = new SHA1Managed();<br />
byte[] calculatedHash = hashAlg.ComputeHash(decryptedBuffer, 0, (int)ms.Position + sizeOffset);<br />
byte[] msgHash = (byte[])bf.Deserialize(ms);<br />
for (int i = 0 ; i < calculatedHash.Length ; i++)<br />
if (calculatedHash[i] != msgHash[i])<br />
throw new CryptographicException("Hash missmatch");<br />
<br />
return message;<br />
}<br />
The same code without the encryption stuff works fine.
Anybody has a clue what's wrong?
And while I'm at it... Is there any good way to protect against man in the middle attacks when exchanging keys? Is the application very vunerable with out that protection?
/Regards 3ddA
|
|
|
|
|
Just a quick reply to your final question (I'm afraid I don't have time at the moment to figure out your code): SSL was designed to protect against man-in-the-middle attacks. Check out the specification of SSL to understand how it works (it involves the exchange of certificates for example).
Cheers, Julian
Program Manager, C#
This posting is provided "AS IS" with no warranties, and confers no rights.
|
|
|
|
|
What I've gathered so far is that it all comes down to having some trusted site to sign a certificate for you. I guess using that service will cost money and will not be that interesting for this type of friend2friend communication application.
I think I'll settle for a hash of the public key encoded in base64. This could be calculated on the client side too and then compared by using the telephone or something if MITM-attack is suspected. I guess MITM-attack of voice communication still is very hard to do 
I guess problems like this will have to be solved by people that really knows security when the app is available to the public.
Thanx for the help. I now realize how much more complex this is than exchanging public keys.
|
|
|
|
|
Key and iv are double checked and correct data is received as it was sent.
<br />
public byte[] SecureSerialize(byte[] key, byte[] iv)<br />
{<br />
byte[] streamBuffer = new byte[BufferSize];<br />
<br />
MemoryStream ms = new MemoryStream(streamBuffer);<br />
ms.WriteByte((byte)encrypted);<br />
<br />
RijndaelManaged rij = new RijndaelManaged();<br />
CryptoStream cs = new CryptoStream(ms, rij.CreateEncryptor(key, iv), CryptoStreamMode.Write);<br />
<br />
BinaryFormatter bf = new BinaryFormatter();<br />
bf.Serialize(cs, this);<br />
<br />
SHA1Managed hashAlg = new SHA1Managed();<br />
byte[] hash = hashAlg.ComputeHash(streamBuffer, 0, (int)ms.Position);<br />
bf.Serialize(ms, hash);<br />
<br />
byte[] returnBuffer = new byte[(int)ms.Position];<br />
Array.Copy(streamBuffer, 0, returnBuffer, 0, (int)ms.Position);<br />
<br />
return returnBuffer;<br />
}<br />
<br />
public static UdpMessage SecureDeserialize(byte[] msg, byte[] key, byte[] iv)<br />
{<br />
MemoryStream ms = new MemoryStream(msg);<br />
byte enc = (byte)ms.ReadByte();<br />
<br />
RijndaelManaged rij = new RijndaelManaged();<br />
CryptoStream cs = new CryptoStream(ms, rij.CreateDecryptor(key, iv), CryptoStreamMode.Read);<br />
<br />
BinaryFormatter bf = new BinaryFormatter();<br />
UdpMessage message = (UdpMessage)bf.Deserialize(cs);
<br />
message.encrypted = (Encryption)enc;<br />
<br />
SHA1Managed hashAlg = new SHA1Managed();<br />
byte[] calculatedHash = hashAlg.ComputeHash(msg, 0, (int)ms.Position);<br />
byte[] msgHash = (byte[])bf.Deserialize(ms);<br />
for (int i = 0 ; i < calculatedHash.Length ; i++)<br />
if (calculatedHash[i] != msgHash[i])<br />
throw new CryptographicException("Hash missmatch");<br />
<br />
return message;<br />
}<br />
Regards 3ddA
|
|
|
|
|
Hey guys...
I am writing an SMTP server and am interested in adding support for open relay blacklists.
However, beyond the implementation, what I need to achieve is retrieving the remote IP from the TcpClient object.
I know that this can be retrieved from a raw socket object (which can be generated through TcpListenet.AcceptSocket() as opposed to TcpListener.AcceptTcpClient()), although this doesn't support streaming.
Any ideas?
Tatham Oddie (VB.NET/C#/ASP.NET/VB6/ASP/JavaScript)
tatham@e-oddie.com
+61 414 275 989
|
|
|
|
|
You don't want to use TcpListener at all for another even remotely performant. Stick to Socket.Accept().
--
-Blake (com/bcdev/blake)
|
|
|
|
|
Why do you say this? I would have though that something relating to TCP would be best performed by a TcpListener?
Is it performance issues, functionality (like what i'm trying to do) or what?
Tatham Oddie (VB.NET/C#/ASP.NET/VB6/ASP/JavaScript)
tatham@e-oddie.com
+61 414 275 989
|
|
|
|
|
TcpListener is just a wrapper around Socket. It doesn't give enough flexibility for anything but the most simple applications. Certainly not enough to implement an SMTP server.
--
-Blake (com/bcdev/blake)
|
|
|
|
|
Blake Coverett wrote:
Certainly not enough to implement an SMTP server.
 I'll have to debate you on this - the entire serhas has been written and deployed. The only functionality restriction I have discovered was this one problem of determining the remote endpoint.
Blake Coverett wrote:
It doesn't give enough flexibility for anything but the most simple applications.
SMTP, POP3, IMAP and the like are all extremely basic protocols.  For more complex protocol implementations (such as when I implemented the DNS client) I will agree that sockets are more appropriate. Although, I do not value the assosciated coding overheads required to implement the more complex solution when an appropriate wrapper is available. Wrappers are designed to simplfy integration.
While I will agree that raw sockets present greater funtionality, I think it is critical that when greater functionality is not required, the appropraite (and probably more reliable) wrapper should be utilised.
Anyway, thanks for the help - it certainly spurred some research and did solve the problem. 
Tatham Oddie (VB.NET/C#/ASP.NET/VB6/ASP/JavaScript)
tatham@e-oddie.com
+61 414 275 989
|
|
|
|
|
I'm glad to hear the problem is solved.
On the merits of TcpListener as a wrapper:
If you look through the source for it (isn't Reflector a wonderful tool) you'll see it's all of about 40 lines of code. Really all it does is wrap Bind/Listen in one method and, create TcpClients around accepted sockets, and translate from one set of exceptions to a different set if you try to accept on a closed socket.
What does this actually really buy, except hiding the real Socket's based programming model? Perhaps the thought is it buy's having TcpClients rather than sockets after the connect, but they don't add any value in a server scenario at all. The only real abstraction they provide is in the Connect process, which isn't used there.
Perhaps what you want is the NetworkStream model for the actual IO. I might argue that too on different grounds, but I won't today. If that's what you want, then then just do a new NetworkStream(listeningSocket.Accept(), true).
Of course, better yet is to use the async Socket.BeginAccept/BeginRead/BeginWrite methods. This is where you'll get scalability.
regards,
-Blake
|
|
|
|
|
Does anyone know how I can get (and eventually set) the properties of a file.
I have some pictures take with a digital camera. In all the pictures properties (attributes?) are set for lot o things. E.g. Widht, Height, Bit Depth, Flash Mode, Focal Length, etc...
I guess that the attributes are dynamic. But how can I read them using the Framework?
I have not been able to find the attributes using a FileInfo-object.
I have seen someone doing this on the net using a COM-object with prog-id "DSOLEFILE.PropertyReader".
Do I have to use this in stead? And in that case where can I find it?
  
Thanks in advance,
Mattias
|
|
|
|
|
What you're talking about is metadata, such as Exif image file attributes, which have nothing to do with normal attributes. Use the PropertyItems[] property of the Image class for this.
"Blessed are the peacemakers, for they shall be called sons of God." - Jesus
"You must be the change you wish to see in the world." - Mahatma Gandhi
|
|
|
|
|
Tanks a lot. 
(Even though it was not a trivail thing to get the metadata from the EXIF-file)
But what is not wasy, is fun! 
/Mattias
|
|
|
|
|
I'm having trouble locating C# source code to create transparent forms for Pocket PC 2002.
I've located code for c++ but I haven't had experience with c++.
Any help would be great !
Rowdy
|
|
|
|
|
Does the CE version of .NET have the TransparencyKey property on forms?
"Blessed are the peacemakers, for they shall be called sons of God." - Jesus
"You must be the change you wish to see in the world." - Mahatma Gandhi
|
|
|
|
|
You can set the background colour of the form to "Transparent" but it displays the following error:
---
Invalid Property Value
This control does not support transparent background colors.
---
Rowdy
|
|
|
|
|
Tansparent background color just means it lets the parent window draw the background instead of doing it itself. Forms have no parent; therefore, they can't do this. But you can still set the TransparencyKey property to the degree of transparency you want the form to have.
"Blessed are the peacemakers, for they shall be called sons of God." - Jesus
"You must be the change you wish to see in the world." - Mahatma Gandhi
|
|
|
|
|
Hello gurus,
My C# app runs into too much trouble with existing file and directory names. It crashed while deleting and copying on
1) File Names like:
"User's Manual" // astrophy sign
"User Maual 5.4.pdf" // the extra "."
2) Folder names like:
"User Manual 1" // the number "1"
I prefixed the variable name with "@" but did not help.
e.g. @sFilename, @sDirName;
Do you know some way to get around these tideous and painful problems?
Thanks in advance
Khang Nguyen
|
|
|
|
|
If you don't have problem with renaming those folders, first rename them to an appropriate name, then copy or delete them. hum?
Don't forget, that's Persian Gulf not Arabian gulf!
|
|
|
|
|
Hi,
I have been stuck with a problem a couple of days now and now when I finally thought that I don't have to do anything more to get the correct pages then suddenly a wrong result was retrieved. Ok what I'm trying to do is to get a web-page and when using Swedish characters it fails to get the same answer as
internet explorer does. I have tried to read the http spec but I still doesnt get this right. Now I call for real experts on this. I would be very grateful for any help on this.
This is the url:
http://www.altavista.com/web/results?q=%22l%C3%A4gg+till%22&kgs=0&kls=1&avkw=aapt
It will display a page on altavista with some hits to Swedish pages. When I run this to my program I got a page on altavista where it says that I got zero hits.
Please let me know if I can give you more detail of the problem.
Yours sincerely
Andla
I Post the source code here for you:
//lnk is the url I want to recieve.
int start=lnk.IndexOf("://");
if(start==-1)
return;
start+="://".Length;
int stop=lnk.IndexOf("/",start);
if(stop==-1)
stop=lnk.Length;
string domain=lnk.Substring(start,stop-start);
string path=lnk.Substring(stop,lnk.Length-stop).TrimEnd('/');
TcpClient cli = new TcpClient();
cli.Connect(domain,80);
NetworkStream stream = cli.GetStream();
UTF8Encoding encoder2 = new UTF8Encoding(false);
/*Encoding encoder = Encoding.GetEncoding( "iso-8859-1" );
encoder = Encoding.GetEncoding( "unicode-1-1-utf-8" );
encoder = Encoding.GetEncoding( "ascii" );
encoder = Encoding.GetEncoding( "unicode-2-0-utf-8" );*/
Byte[] request;
if(path=="")
{
request = encoder2.GetBytes("GET / HTTP/1.0\r\nHost: "+domain+"\r\n\r\n");
}
else
{
request = encoder2.GetBytes("GET "+path+" HTTP/1.0\r\nHost: "+domain+"\r\ncharset = \"UNICODE-1-1-UTF-8\"\r\n\r\n");
string fan=encoder2.GetString(request);
fan="";
}
stream.Flush();
stream.Write(request, 0, request.Length);
while(!stream.DataAvailable)
{
Console.WriteLine("WAIT");
Thread.Sleep(1000);
}
int buffcount = 0;
Byte[] buffer = new byte[1024];
String response = String.Empty;
Encoding encoder = Encoding.GetEncoding( "iso-8859-1" );
while((buffcount = stream.Read(buffer, 0, buffer.Length)) > 0)
response += encoder.GetString(buffer, 0, buffcount);
stream.Close();
cli.Close();
|
|
|
|
|
I'm needing a specialized list class that meets these criteria:
* Small memory footprint
* Uses keys and allows duplicate keys
* Fast lookup and allocation.
* Allows retrieval by index (much preferred, but no absolutely required)
Something like the HybridDictionary class would be perfect, except that it doesn't allow duplicate keys, which is essential.
I'm wanting to keep away from Hashtables if possible, because they have a large memory footprint.
Do you know where I can get a class like this? Otherwise, how is the best way to go about making one? Shall I just start completely from scratch, using two ArrayLists, or is there something intermediate that I can build on?
"Blessed are the peacemakers, for they shall be called sons of God." - Jesus
"You must be the change you wish to see in the world." - Mahatma Gandhi
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
