Re: Playing smoothly small pieces with DirectSound - .NET (Core and Framework) Discussion Boards

Moreno:
Is that your Name?

What you suggest is the first thing I tried, but It only worked with the object that is getting called clone.

It doesnt work if that object contains references to other chils of the same base class. I mean, it works for the object being cloned, but not for the references it has.

I´m working now in a solution suggested by a collegue by implementing the IXMLSerializable interface.

I´ll let you know what comes up.

Sebastián

Yep it's my name, I know it's a common family name for Spanish and latinos.

It sure works only for the object on which you call clone, and not for references.
Best of luck with implementing the interface then.

At first I also reasoned on the line of parsing the XML and getting the status directly from there, in order to apply it to the cloned object(s) after de-serialization, but I find it too much of a patchy solution.

Well, best of luck again, and I'll be waiting to hear how it goes. Wink | ;)

2+2=5 for very large amounts of 2
(always loved that one hehe!)

Hi
I have a problem. I am developing a VoIP application using DirectSound to capture and play voice. Everything is OK with the capturing, but I can't play the received data smoothly when it is a small piece. It should be small for preventing latency. Currently I capture 320 bytes and send it via UDP. The wave format is 8000 Hz and 8000 B/S.
For reproducing the problem I use the following code:

int step = 320, c = 0, N = 2;
offset = data.Length % step;

// (SecondaryBuffer) buf size is N * step;
buf.Play(0, BufferPlayFlags.Looping);

byte[] smallArr = new byte[step];

double sleep = (double)step / waveFormat.AverageBytesPerSecond * 1000;

//data is a captured sound saved to a file previously
while (data.Length > offset)
{
buf.SetCurrentPosition(((c + 1) % N) * step);

Array.Copy(data, offset, smallArr, 0, step);
buf.Write(c*step, smallArr, LockFlag.None);
Thread.Sleep((int)sleep);
offset += step;
c++;
c%=N;
}

While debugging I found out that the PlayPosition cursor doesn't reach the expected point. I move it forward and the result is that not all data is played and the sound is not smooth.

There is no problem when the playing portion is bigger but I can't afford latency.
When I use a lot higher quality(and play bigger portion of data) there is no problem again but the traffic is large!
I can't use more than one buffers because the noise is awful when play/stop/mute.

So, would you help me to solve the problem with the playing buffer?

Thank you in advance!

Andrey U wrote:
While debugging I found out that the PlayPosition cursor doesn't reach the expected point. I move it forward and the result is that not all data is played and the sound is not smooth.

That's not a good idea. Any change to the play position will affect the output. The results from GetCurrentPosition may be incorrect / inaccurate due to the way DirectSound works (especially on XP).

Your basic idea of double buffering is good, but you need to manage it in a more reliable way. The "Sleep" in particular, since it's the same length as the time to play one buffer-full, doesn't allow any time for the data copying. (You should also try to only have one copy / write for efficiency.)

My approach is similar, but I check when the play cursor switches from one buffer to the next (either half way along the double buffer, or looping back to the start) and then I copy into the other half. Use a smaller sleep time, and check regularly.

I should say "my approach was similar..." though, as I've now switched over to using XAudio2, which is much better for managing streaming.

Andrey U wrote:
I can't use more than one buffers because the noise is awful when play/stop/mute.

That shouldn't happen - it sounds like something else is going wrong in that case.

There are three kinds of people in the world - those who can count and those who can't...

molesworth wrote:
That's not a good idea. Any change to the play position will affect the output. The results from GetCurrentPosition may be incorrect / inaccurate due to the way DirectSound works (especially on XP).

I know that the play position affects the output, but in my real application I receive packets and I can't wait.
I use PlayPosition property only for finding out what happens, so I don't rely on it.

molesworth wrote:
Your basic idea of double buffering is good, but you need to manage it in a more reliable way. The "Sleep" in particular, since it's the same length as the time to play one buffer-full, doesn't allow any time for the data copying. (You should also try to only have one copy / write for efficiency.)

I've tried different types of buffering and the only way to improve the output is to use bigger buffer, but this means more latency...

molesworth wrote:
That shouldn't happen - it sounds like something else is going wrong in that case.

It happens! I've tried with an empty buffer and when I call play,stop,etc. I hear this awful pop.

I've just read in another forum that this problem migth be caused by "The Windows resampler", which works better at certain rates and bad at 8 kHz. But how could I avoid this?

Andrey U wrote:
I know that the play position affects the output, but in my real application I receive packets and I can't wait.
I use PlayPosition property only for finding out what happens, so I don't rely on it.

Yes, but when you call "SetCurrentPosition" you're moving the play cursor, which will cause jumps / glitches in your audio output. If you're playing at the same rate you're receiving data (which you should be) then you should be able to let the buffer play continually without having to move the cursor.

Andrey U wrote:
I've tried different types of buffering and the only way to improve the output is to use bigger buffer, but this means more latency...

Well, your buffers are very small, and about 40ms (1/25th sec) long. For a voice chat or VoIP application you should be able to go up to 100ms or even 200ms without any problem. Nobody is going to notice those kinds of delays in voice only communications.

I'm not sure why you're getting popping on playing empty buffers - if you've zero-filled them they should play silence. Starting and stopping waves which aren't on zero-crossings can cause pops, but dealing with that is another matter.

Andrey U wrote:
I've just read in another forum that this problem migth be caused by "The Windows resampler", which works better at certain rates and bad at 8 kHz. But how could I avoid this?

There shouldn't be any problem playing at 8 KHz, and I've just run a quick test in my app which works fine. It sounds pretty bad playing music, but it does play smoothly. I suspect your problems are more to do with your Sleep timer and moving the play cursor.

I'd go for something more like :-

while (stillPlaying)
{
    CheckForInputData();

    playPos = GetCurrentPosition();

    if (playPos just crossed half-buffer)
    {
        FillOtherHalfBuffer();    // hopefully data has arrived, if not more complex logic needed
    }

    Sleep(10);    // some short-ish time
}

There are three kinds of people in the world - those who can count and those who can't...

I can pretty much guarantee you will have problems with this approach.

First, looping and using Sleep for synchronization is going to be nothing
but problematic. You can use SetNotificationPositions() to have an event
signaled when the playback buffer reaches certain positions - that event can wake
a waiting thread which can take appropriate action to provide more playback data.
Much more efficient and no Sleep required.

Second, on the playback end, you need to deal with jitter from at least two main sources:

1) Network. If you're transferring audio over a network it is NOT always going to
arrive in an evenly timed stream - it will come in chunks and at different intervals,
often with variations larger than 25ms, which I'm assuming is your "packet" time.

2) Clock drift. Clocks on different machines are never the same. Close, but close
doesn't count here. As one machine streams data at its 8000KHz, the playback machine
tries to play the data at its 8000Hz, but with different clocks, they (usually pretty
rapidly) get out of sync - the playback machine either gets data faster than it is consuming
it, or consumes data faster than it is receiving it. This HAS to be accounted for, unless
all conversations are going to be very short - like a few seconds.

I would really recommend redesigning your playback end. Use directsound buffer notifications
and use threads smartly. I personally have data read from the network on one thread which is
pushed into a jitter buffer. Another thread waits on buffer notifications, and when it wakes,
pulls data from the jitter buffer and copies it to the playback buffer. The jitter buffer
can be implemented any way you want - it just needs to provide data when none is available, or
deal with too much data available.

Good luck.

Mark Salsbery
Microsoft MVP - Visual C++
Java | [Coffee]

Thank you very much!

Let me show you what I did.

Playing method:
(I've set the buffer to notify me at the half and at the end)

public void PlayingInThread()
{
    NotifyPlaying();
    playBuffer.Play(0, BufferPlayFlags.Looping);
    bool flag = true;
    byte[] data;
    while (true)
    {
        arePlaying.WaitOne();
        data = jitter.Get();

        if (flag)
        {
            playBuffer.Write(playBufferBytes / 2, data, LockFlag.None);
        }
        else
        {
            playBuffer.Write(0, data, LockFlag.None);
        }

        flag = !flag;
    }
}

And I wrote a very simple jitter class, which uses a queue:

class JitterBuffer
    {
        Queue<byte[]> mBuffer = new Queue<byte[]>();
        int mItemLength;

        public JitterBuffer(int length)
        {
            mItemLength = length;
        }

        public void Put(byte[] data)
        {
            mBuffer.Enqueue(data);
        }

        public byte[] Get()
        {
            if (mBuffer.Count == 0)
                return new byte[mItemLength];
            else
                return mBuffer.Dequeue();
        }
    }

Am I right up to this point?

Testing this code I saw that there is aproximately one more packet (which can't be played because of the slower playing!) received every second.
So my next question is how to deal with 'too much data available'?

To be honest, I'm not very good at threads and I don't know if it is a good solution but I've tried the following code:

public void CatchUp()
{
    if (playBuffer.PlayPosition >= playBufferBytes / 2)
        playBuffer.SetCurrentPosition(playBufferBytes-2);
    else
        playBuffer.SetCurrentPosition(playBufferBytes / 2-2);
}

And I call this method from JitterBuffer.Put() when there are more then 2 packets in the queue. As you can guess in this case I face my initial problems with 'not smooth output'. Maybe I should only drop a packet from the queue when there are a lot?
(What bothers me about the threads is that I manipulate playBuffer from 2 threads in this case. Am I supposed to do it?)

And my last question: should I rely on the notification from a secondary buffer? I've read several times that there are bugs with it.

Regards

Andrey U wrote:
Am I right up to this point?

Yep, that looks like a good way to do things. As Mark says, threading is definitely the right approach (I previously just posted a simplistic method since your original post just had a simple closed play loop). Threading may look complicated, but once you've done a couple of trials you'll probably see that it's fairly simple really Smile | :)

You should perhaps zero-fill your empty buffer in the case where there's no received data, however. It may be cleared in debug builds, but probably will be full of random data (noise) in release.

Andrey U wrote:
Maybe I should only drop a packet from the queue when there are a lot? (What bothers me about the threads is that I manipulate playBuffer from 2 threads in this case. Am I supposed to do it?)

You definitely shouldn't operate on the play buffer in anything other than your play thread. I'd also advise against moving the play cursor, since this will definitely cause glitches, and may also cause other problems in playback.

For overruns you can either drop packets, which will also give glitches, although more manageable ones, or you can just play all data - ideally your transmitting app should detect silence and stop sending data, giving the receiver time to catch up during those pauses.

Your buffer size and the reliability of the connection will determine how often you get overruns or underruns, and it's a trade-off between several factors to determine the best size to use. As Mark mentioned, you need to deal with all of these to make a reliable application.

Andrey U wrote:
And my last question: should I rely on the notification from a secondary buffer? I've read several times that there are bugs with it.

I've had problems with notifications myself. My last DirectSound system used GetCurrentPosition to check the play cursor at regular intervals, and I found this was much more reliable. (My current project is using XAudio2, which seems to be a lot better.)

There are three kinds of people in the world - those who can count and those who can't...

Will all modern versions of Windows honor small changes in playback speed, or are they apt to be rounded up to larger increments? If the sampling PC samples at a constant 8,000Hz, would it be possible to have the playback PC output at a rate slightly faster or slower based upon whether it is getting ahead of or behind the data stream? One would have to be careful, of course, to avoid varying the speed so much as to cause warbling, but I would expect two system clocks should be easily within 0.1% of each other.

Playback frequency on DSound buffers can be set to integer accuracy, and they'll play properly at whatever pitch is selected.

I'm not sure this is a good approach however. Accurately measuring the rate at which network data is arriving could be pretty difficult, especially since it's arriving in reasonable sized packets, and making continual changes to the frequency could, as you say, cause noticeable unpleasant artefacts.

I've never written a VoIP app myself*, so can only advise in a theoretical way, but I'd go with the approach of playing all data as it arrives, and using silence detection on the transmitting side to signal gaps in the stream during which the playing side can "catch up".

That would be for small (e.g. < 250ms) gaps - for larger ones I'd probably drop some packets, and live with the glitches. VoIP isn't trying to achieve CD quality sound, just reasonably comprehensible speech. Sampling at 8 bit, 8KHz is going to reduce quality quite a lot anyway... Smile | :)

[* My current system is dealing with horrible latencies, multiple streams and multiple buffers, but I'm taking a different approach to dealing with these problems.]

There are three kinds of people in the world - those who can count and those who can't...

supercat9 wrote:
would expect two system clocks should be easily within 0.1% of each other.

I can only smile Smile | :)

I don't know what the percentage margin is, but on pretty
much any two systems I conversed between with voip, playing every audio data
byte sent, the latency would increase or decrease to an unacceptable level within
a minute or two, sometimes less. I'll let you do the math. Maybe it is within 0.1%,
but it's still unacceptable for real time communication.

Point is, they're not even close Smile | :)

Don't take my word for it - try it...you'll see. You don't even need to playback data...
every time a packet arrives, record the playback buffer cursor position in an array
(using an adjusted value that converts the current circular buffer cursor position into
a linear position is easier to read in the results - i.e. whenever the cursor wraps, add the
buffer length to a running total).
Run it for a minute or two (network fluctuations will average out so they're a non-issue at
low bandwidth), then break and look at the array in a watch window...

I did tests like this for months trying to figure out decent statistics to use for a good jitter
buffer implementation...I should have hired a PhD for that.

Moral: Real-time voip streaming is quite different than streaming static media...

Mark Salsbery
Microsoft MVP - Visual C++
Java | [Coffee]

In addition to molesworth's reply...

1:
I use buffer notifications at 25ms intervals and have had no problems with reliability,
especially XP SP2+. (I've heard no complaints on Windows 2000, but I struggled with network
chunkiness on that OS Smile | :)

)

2:
jitter handling:
Dropping packets if necessary is fine if the packets are 25ms - that shouldn't be audible.
If you run out of packets, repeating the last packet written works ok. If you want to go crazy, you can
use interpolation to stretch or compress audio data....kinda overkill for voice.

Molesworth mentioned "Your buffer size and the reliability of the connection will determine
how often you get overruns or underruns". There's also clock drift - two PC's clocks do NOT tick
at the same rate - you'll see (hear)! Over (wall clock) time, the buffer cursors at the endpoints are going to
drift apart. You can hear this if you try to just write every byte of every packet sent to the playback buffer -
either the latency will increase over time or the latency will decrease over time. In both
cases playback eventually turns to garbage when the cursors wrap and/or get out of sync with where you're stuffing
the data.

3:
You should only (and only need to) manipulate the play buffer from one thread - the thread that
responds to buffer notifications and stuffs more data in the buffer.

4:
Let the buffer play and always provide data just ahead of the cursor - you shouldn't and don't need
to adjust the play position during playback. A robust jitter buffer implementation should provide data
whenever requested by the playback thread.

Just tips based on years of doing live multimedia streaming code - hopefully helpful.

Mark Salsbery
Microsoft MVP - Visual C++
Java | [Coffee]

Aye - this sort of thing is exactly why we've gone for a third-party solution for voice chat in our next game, rather than dealing with all these complications developing our own... OMG | :OMG:

[ obadvert : APB, coming to a store near you early next year Big Grin | :-D

]

There are three kinds of people in the world - those who can count and those who can't...

Andrey,

I am trying to do something similar for a cell phone application.
Would you mind possibly sharing your c# code for recording and
playing back? Thanks.

Benjo

In a programme there is a follwing error message from the system..

message

"The .Net Data OLE DB Provider(System.Oledb.Data) requires Microsoft Data Access Components(MDAC)version 2.6 or later, version 2.10.3711.9 was found currently installed"

This result the database connection lost,how this problem solve

rajagopal
rajkottayam@rediffmail.com
Frown | :(

based on the error message it looks like you need to install MDAC version 2.6 or later.

Badger | [badger,badger,badger,badger...]

thanks

Hi Dears,

I have downloaded the code from Code project for Network Sniffer could anyone help me in running the the same.
I am getting the following error in running N/W sniffer.

"An unhandled exception of type 'System.InvalidOperationException' occurred in system.serviceprocess.dll

Additional information: Service SMTPSVC was not found on computer '.'."

Thanks,
Amit

If you can't figure the problem out by looking at the code yourself, the best thing to do is post in the forum at the bottom of the article that you downloaded the code from, that will get the attention of the person who wrote it. They are more likely to be able to help you.

(On the other hand, if you are just after a network packet sniffer, you could try something like wireshark[^], or one of the ones listed here[^])

Simon

Hello every one,
I am developing an application using asp.net
my requirements like

* on server there will be web application and at client side there will be an device

* web application is using asp.net and client application is using C++

* server and client are connected using GPRS.

* when user will login in web application, then there will be list of device which is present on different different locations.

* user can select particular device and after click on connect button, communication will start between server and client.

* after establishing the connection client can send log file to server or there is both way communication

my issue is only for web application. Like how can i make connection and how to communicate with client
should i use webservice for this? or how to achive this..!

Please help me on this

Thanks in advance

hello

If u want to make the communication with device(i hope ur device will be physically connected with machine so they would b able to work remotely)you should do the port programming. because ports are areas where from connection can be established.

Rohit Sharma

Hi,

I am having a project to build a toolbar for IE that doesn't require any installation, a bit like Digg Bar. Does anybody know how to do it and what technologies should be used?

Thanks for your help.

ZL

Hello

I have an iPAQ 114 with Windows Mobile 6 Classic.
I have developped an app with NETCF 3.5
If I use Visual Studio 2008 and debug, it implements NETCF 3.5 on the device without error.
If I copy NETCFv35.wm.armv4i.cab on the device and I try to run it, it shows me an error message because it is not digitally signed with a trusted certificate (it's Micosoft program on Microsoft OS Confused | :confused:

).
I have tried to reduce the security configuration with Visual Studio 2008 and I have left no security. But I have the same error. Frown | :(

Only if I debug, I can install NETCF 3.5
I need to make it without Visual Studio because the final user don't have Visual Studio and I need he can make click on the NETCFv35.wm.armv4i.cab file to install it.

Any suggestion?

Thanks in advance

modified on Monday, June 1, 2009 9:08 AM