Re: 64bit fixed point timestamp to and from DateTime - C# Discussion Boards

Re: 64bit fixed point timestamp to and from DateTime

9-Jul-09 11:32

I've been battling this for a few hours so now I need help!

I'm converting a 64bit fixed point timestamp (32bit integer part, 32bit fractional part) to a DateTime by converting it to a TimeSpan adding that to an epoch, as the timestamp represents the number of seconds since the epoch.

The problem is I'm losing precision (I assume because of the cast to/from double and UInt64 - see code), so converting back from DateTime to timestamp gets a different value.

Any ideas?

using System;

class Program
{
    static void Main(string[] args)
    {
        UInt64 timestampA = 14844064336383267476;
        Console.WriteLine("TimestampA: {0}", timestampA);

        DateTime dateTime = NTPConversion.TimestampToDateTime(timestampA);
        Console.WriteLine("DateTime: {0}", dateTime);

        UInt64 timestampB = NTPConversion.DateTimeToTimestamp(dateTime);
        Console.WriteLine("TimestampB: {0}", timestampB);

        Console.WriteLine("Discrepancy: {0}", timestampB - timestampA);

        Console.ReadKey();
    }
}

/// <summary>
/// NTP Timestamp is 64 bit fixed point.
/// The upper 32 bits are the integer part and the lower 32 bits are the fractional part.
/// A timestamp is the time in seconds since 1st Jan 1900 (Epoch).
/// </summary>
public static class NTPConversion
{
    public static readonly DateTime Epoch = new DateTime(1900, 1, 1);

    public static DateTime TimestampToDateTime(UInt64 timestamp)
    {
        // effectively shift right 32 bits
        TimeSpan timeSpan = TimeSpan.FromSeconds((double)timestamp / 4294967296);
        DateTime dateTime = Epoch + timeSpan;
        return dateTime;
    }

    public static UInt64 DateTimeToTimestamp(DateTime dateTime)
    {
        TimeSpan timeSpan = dateTime - Epoch;
        // effectively shift left 32 bits
        UInt64 timestamp = (UInt64)((timeSpan.TotalSeconds * 4294967296d));
        return timestamp;
    }
}

Dave
BTW, in software, hope and pray is not a viable strategy. (Luc Pattyn)
Visual Basic is not used by normal people so we're not covering it here. (Uncyclopedia)
Why are you using VB6? Do you hate yourself? (Christian Graus)

Luc Pattyn9-Jul-09 12:24

Re: 64bit fixed point timestamp to and from DateTime

9-Jul-09 12:24

Hi Dave,

the problem is here: (double)timestamp

uint64 (aka long) has a 63-bit resolution, whereas double only has some 52 bits of mantissa, the remaining bits required to hold the exponent.

You'll have to smarten up your TimestampToDateTime() method by splitting the uint64 in two parts, and adding them separately to get the DateTime. Example:

long hipart=timestamp&0xFFFF0000;
long lopart=timestamp&0x0000FFFF;
DateTime dt=TimeSpan.FromSeconds(hipart*...);
dt.AddSeconds(lopart*...);

I strongly suggest you don''t use those magic numbers of yours either; they are incromprehensible; what I would do is write them as a formula, the compiler will multiply them for you, and better yet also make them named constants so their name indicates their meaning.

Cheers.

Luc Pattyn [Forum Guidelines] [My Articles]

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.

DaveyM699-Jul-09 22:28

Re: 64bit fixed point timestamp to and from DateTime

9-Jul-09 22:28

Luc Pattyn wrote:
I strongly suggest you don''t use those magic numbers of yours either

lol, in the actual class I have been a little more elegant and logical. What I've posted is a separate quicky console app with this stuff hardcoded so I could use it like a sledgehammer to solve/analyse the problem quickly without messing with the real source.

Sorry for the delay in replying, this is just one in a series of problems (I had solved the others) and my brain was hurting Mad | :mad:

After posting I turned the PC off in disgust with myself and went to bed (after a Beer | [beer]

of course)!

Henry Minute9-Jul-09 12:29

Henry Minute

9-Jul-09 12:29

Dave, I have just tried something for your TimestampToDateTime method and I got 09/07/2009 18:30:53 (dd/mm/yyyy), is that what you were hoping for?

If so I will post it for you. Smile | :)

Henry Minute

Do not read medical books! You could die of a misprint. - Mark Twain
Girl: (staring) "Why do you need an icy cucumber?"
“I want to report a fraud. The government is lying to us all.”

Re: 64bit fixed point timestamp to and from DateTime

DaveyM699-Jul-09 22:20

Re: 64bit fixed point timestamp to and from DateTime [modified]

9-Jul-09 22:20

Hi Henry,

Happy to look at anything, some fresh eyes/new perspective is always good!

I can retrieve the same DateTime no problem when just looking down to the second, but it changes when using milli/micro/pico second precision. That isn't necessarily a problem, so long as I can get the Timestamp data consistant after going through the conversion processes - see my reply to Luc below.

Luc Pattyn9-Jul-09 13:41

Re: 64bit fixed point timestamp to and from DateTime

9-Jul-09 13:41

Hi Dave,

I have investigated a bit more, and there is a fundamental problem: DateTime and TimeSpan basically count ticks, there are 10 million ticks per second.

Your timestamp has much higher accuracy (4 billion parts in one second), so the extra information is lost forever when you turn them into DateTime/TimeSpan structs.

To convince yourself, make discrepancy a signed number, and try several timestamp values (even small numbers will show the problem, I tried numbers from 0.2 to 2.0 milliseconds where

ulong Shift32 = 0x0000000100000000UL;
ulong msec = Shift32 / 1000;

or use Reflector to peek into the code for DateTime properties Ticks and Seconds, and its AddSeconds() method.

Furthermore your unit changes from a binary one to a decimal one, and back; which is bound to cause round-off problems, similar to float and double having trouble representing 1.0/3.0*3 and 1.0/7.0*7

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.

modified on Thursday, July 9, 2009 8:02 PM

DaveyM699-Jul-09 22:16

Re: 64bit fixed point timestamp to and from DateTime

9-Jul-09 22:16

Thanks Luc, I figured that there would be problems with the internal resolution of DateTime vs the timestamp

Unfortunately, that is probably not solvable, as the Timestamp format is not one of my own, but part of the NTP/SNTP[^].

I think my best solution is to round any timestamps to the acheivable precision of DateTime so I can at least get consistant data. The level of precision provided in the protocol is for the most part unecessary IMO.

DaveyM699-Jul-09 23:05

Re: 64bit fixed point timestamp to and from DateTime

9-Jul-09 23:05

Further to my earlier reply, and after a little more thought, accuracy of less than 1ms is probably not required so I only need 40 of the 64 bits - the 32 integer bits, and 8 fractional ones. This should make it a little easier I hope.

Luc Pattyn9-Jul-09 23:22

Re: 64bit fixed point timestamp to and from DateTime [modified]

9-Jul-09 23:22

Sure. Due to the different bases (1 in 1000 and 1 in 2^32) you still are bound to encounter some odd values that get modified when applying a round-trip conversion.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.

DaveyM6910-Jul-09 4:00

Re: 64bit fixed point timestamp to and from DateTime

10-Jul-09 4:00

I think I've sorted it Luc. Part of the problem was TimeSpan.FromSeconds and FromMilliseconds are only accurate to 1 millisecond so it was bound to do wierd stuff!

The answer to that was to use TimeSpan.FromTicks, in combination with Math.Round(), and now I appear get ~~1 picosecond~~ 10 nanosecond precision. [Edit] The resolution of Ticks per second [/Edit]

Obviously the first pass results in a different timestamp still - but it's acceptably close. I just run it through a TimestampRound method first so future comparisons are made against that.

public static class NTPConversion
{
    public static readonly DateTime Epoch = new DateTime(1900, 1, 1);

    public static UInt64 TimestampRound(UInt64 timestamp)
    {
        long ticks = (long)(Math.Round(
            (double)timestamp / 0x100000000, 7, MidpointRounding.AwayFromZero)
            * TimeSpan.TicksPerSecond);
        UInt64 newTimestamp = (UInt64)(((double)ticks / TimeSpan.TicksPerSecond) * 0x100000000);
        return newTimestamp;
    }

    public static DateTime TimestampToDateTime(UInt64 timestamp)
    {
        long ticks = (long)(Math.Round(
            (double)timestamp / 0x100000000, 7, MidpointRounding.AwayFromZero)
            * TimeSpan.TicksPerSecond);
        TimeSpan timeSpan = TimeSpan.FromTicks(ticks);
        DateTime dateTime = Epoch + timeSpan;
        return dateTime;
    }

    public static UInt64 DateTimeToTimestamp(DateTime dateTime)
    {
        TimeSpan timeSpan = dateTime - Epoch;
        long ticks = timeSpan.Ticks;
        UInt64 timestamp = (UInt64)(((double)ticks / TimeSpan.TicksPerSecond) * 0x100000000);
        return timestamp;
    }
}

Thanks for your help! If you see any obvious flaws in this - let me know!

modified on Friday, July 10, 2009 2:48 PM

Luc Pattyn10-Jul-09 7:28

Re: 64bit fixed point timestamp to and from DateTime [modified]

10-Jul-09 7:28

Hi Dave,

I am confused, I used Reflector and got convinced ticks was the best you could ever get from DateTime/TimeSpan. I will need more time and a bigger screen to investigate further, I am on a tiny netbook right now, making some things harder. I'll come back on this next week.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.

DaveyM6910-Jul-09 8:48

IsolatedStorageFileStream

10-Jul-09 8:48

You're right - I was getting confused between nano and pico, this stuff has really fried my brain this week!

10ns is the smallest achievable resolution - good enough I think.

[Edit] After all that, once moved over to the real application, there were some serious errors in that code. I ended up dividing into the whole part and fractional part as you suggested initially. The code below works well Big Grin | :-D

(In the actual app the ulong's are converted to a big endian byte array)[/Edit]

private void DateTimeToTimestamp(DateTime dateTime, int startIndex)
{
    UInt64 ticks = (UInt64)(dateTime - Epoch).Ticks;
    UInt64 seconds = ticks / TicksPerSecond;
    UInt64 fractions = ((ticks % TicksPerSecond) * 0x100000000L) / TicksPerSecond;
    for (int i = 3; i >= 0; i--)
    {
        data[startIndex + i] = (byte)seconds;
        seconds = seconds >> 8;
    }
    for (int i = 7; i >= 4; i--)
    {
        data[startIndex + i] = (byte)fractions;
        fractions = fractions >> 8;
    }
}

private DateTime TimestampToDateTime(int startIndex)
{
    UInt64 seconds = 0;
    for (int i = 0; i <= 3; i++)
        seconds = (seconds << 8) | data[startIndex + i];
    UInt64 fractions = 0;
    for (int i = 4; i <= 7; i++)
        fractions = (fractions << 8) | data[startIndex + i];
    UInt64 ticks = (seconds * TicksPerSecond) + ((fractions * TicksPerSecond) / 0x100000000L);
    return Epoch + TimeSpan.FromTicks((Int64)ticks);
}

modified on Saturday, July 11, 2009 8:11 PM

AAKAra9-Jul-09 10:32

AAKAra

9-Jul-09 10:32

Hi,
I use the Stream class in a framework.dll to read and write a setting like this,

get
{
Stream stream = GetStreamForRead(ClientHelpFile);
if (stream == null)
return HelpOption.Automatic;
int HelpOptionRead = stream.ReadByte();
stream.Close();

}
set
{
Stream stream = GetStreamForWrite(ClientHelpFile);
stream.WriteByte((byte)value);
stream.Close();
}

private const string ClientHelpFile = "TAC.ClientHelpProfile.bin";
private const IsolatedStorageScope scope =
IsolatedStorageScope.Roaming |
IsolatedStorageScope.User |
IsolatedStorageScope.Assembly |
IsolatedStorageScope.Domain;
private static IsolatedStorageFile GetIsolatedStore()
{
return IsolatedStorageFile.GetStore(scope, null, null);
}

private static Stream GetStreamForWrite(string filename)
{
IsolatedStorageFile storage = GetIsolatedStore();
return new IsolatedStorageFileStream(filename, FileMode.Create, storage);
}

private static Stream GetStreamForRead(string filename)
{
IsolatedStorageFile storage = GetIsolatedStore();
if (storage.GetFileNames(filename).Length == 0)
return null;
return new IsolatedStorageFileStream(filename, FileMode.OpenOrCreate, storage);
}

the problem I am facing is I am using another application and referencing the same dll(FrameWork.dll).
the storage.GetFileNames returns null and I wish it return me whatever the file has saved in it. Am I missing something?

Re: IsolatedStorageFileStream

Henry Minute9-Jul-09 11:54

Henry Minute

9-Jul-09 11:54

AnilUnni wrote:
the problem I am facing is I am using another application and referencing the same dll(FrameWork.dll).
the storage.GetFileNames returns null and I wish it return me whatever the file has saved in it. Am I missing something?

I think, note that is bold, that the problem might be IsolatedStorageScope.Domain. Each application runs in its own domain, so the path to the isolated storage will be different from different applications.

Once again, this is not a certainty, just my interpretation of how it works.

If it were my app, I would start with just IsolatedStorageScope.User, see if it works for both apps add another scope see if it works etc until it fails. That way you will identify which scope(s) will break it and can avoid them.

Mail Merged Word document in IE , Need Control and want to update it ..Please help

Sandumone9-Jul-09 10:03

Sandumone

9-Jul-09 10:03

Hi all,

Thanks a lot for your help in advance.

I am doing a Mail Merging Application. I can get all data from the source (database, file or anything) and using mail Merge fields in Word Document from C# code, I open up the word which is merged with data from database.

MS Word Document Opens –

--- should this be open in IE

--- Or should this be open as ordinary MS Word on Desktop

The problem is if I open it anyway, I loose all control over the Word Document which is opened. I don’t want to do that. I want to keep control over it and allow the Users to update the document with some standard messages which we have in the project. What that means is, the messages will be selected and those messages selected will be put at some point where cursor is kept in the document.

So, when mail merging happens

MS Word Document is opened.
A pop up with standard messages is opened.
User selects the messages he/ she wants to insert into the MS Word Document which is opened.

For these things to happen, I should have control over the MS Word Document which is opened on the client side. Do selection of the messages and populate in Word document (this is my understanding, is there any way to do it easily).

The Issues are:

I need a way to open the MS Word Document in a controlled manner where I can still control the User editing or help user in editing when he / she clicks on my standard messages and wants them to populate on the MS Word Document.
How can I save the MS Word document and make sure that user is not saving it locally and has to upload it again to save it on server.

I Hope I am clear in explaining the situation, please find the attachments which show the MS Word Doc and the popup on side .

ANY RUNNING CODE WILL BE OF GREAT HELP.

Re: Mail Merged Word document in IE , Need Control and want to update it ..Please help

Mycroft Holmes9-Jul-09 11:45

Mycroft Holmes

9-Jul-09 11:45

Sandumone wrote:
ANY RUNNING CODE WILL BE OF GREAT HELP.

So this is a new way to say snd me the codz. This will get you flamed severely for being rude and shouting it is just plain stupid.

You have a design problem, why do you open the word document before asking the user for the message. Get the message text in your app before you start the word merge that way you do not need to relinquish control.

Never underestimate the power of human stupidity
RAH

LDAP Question

mypicturefaded9-Jul-09 8:23

mypicturefaded

9-Jul-09 8:23

I need to get the full name and email of a user using nothing more than the 2.0 Framework.
I figured LDAP would be the best way to go, but I cannot get my code to work.
I am not using the syntax correctly. I need to get the information using the logged in user name.
if my username is "mypicturefaded" and my email for the company is mypicturefaded@bbt.gov, how would I get the full name or email address. Is there an easier way of doing this? It needs to be from Active Directory though. I can do it using 3.5 framework, but it needs to be rolled back. Here is the code I have:

<br />
<br />
            string strValue;<br />
            DirectoryEntry ent = new DirectoryEntry("LDAP://CN=mypicturefaded");<br />
            strValue = ent.Properties["displayName"].Value.ToString();<br />
            ent.Close();<br />
<br />
            Messagebox.Show(strValue);<br />
<br />

Jeremy Likness9-Jul-09 8:27

Jeremy Likness

9-Jul-09 8:27

You probably need to search for the entry first, then act on it:

http://msdn.microsoft.com/en-us/library/system.directoryservices.directorysearcher.aspx[^]

Jeremy Likness
http://csharperimage.jeremylikness.com/

Manas Bhardwaj9-Jul-09 8:29

Manas Bhardwaj

9-Jul-09 8:29

Here [^]you go.

Manas Bhardwaj
Please remember to rate helpful or unhelpful answers, it lets us and people reading the forums know if our answers are any good.

mypicturefaded9-Jul-09 8:37

mypicturefaded

9-Jul-09 8:37

So how come something like this doesn't work?

<br />
<br />
            DirectoryEntry entry = new DirectoryEntry("LDAP://DC=bbt,DC=com");<br />
            DirectorySearcher mySearcher = new DirectorySearcher(entry);<br />
            mySearcher.Filter = "(&(objectClass=user)(anr=" + "mypicturefaded" + "))";<br />
            SearchResult searchResult = mySearcher.FindOne();<br />
            string email = searchResult.Properties["mail"].ToString();<br />
            MessageBox.Show(email);<br />

I just need the one field. I really don't need all of them.

I get a NullObjectException error.

Manas Bhardwaj9-Jul-09 8:41

Manas Bhardwaj

9-Jul-09 8:41

Should be like this:

DirectoryEntry entry = new DirectoryEntry("LDAP://DC=Domain,DC=com");
DirectorySearcher mySearcher = new DirectorySearcher(entry);
mySearcher.Filter = "(&(objectClass=user)(anr="+ userID +"))";

mySearcher.PropertiesToLoad.Add("mail");

SearchResult searchResult = mySearcher.FindOne();

string email = searchResult.Properties["mail"];;
MessageBox.Show(email);

You did not add mail as a property to be loaded. Poke tongue | ;-P

Manas Bhardwaj
Please remember to rate helpful or unhelpful answers, it lets us and people reading the forums know if our answers are any good.