How do I allocate gt 2GB buffer for PInvoke

Question

0.00/5 (No votes)

See more:

From a C# application, how can I allocate (and free) a huge (>2GB) buffer from unmanaged memory (since I can't allocate a managed buffer >2GB) that I can pass via PInvoke to a C++ dll and then access it within the C# after the dll has filled it.
The function signature of the C++ dll function is:

C++

WIN32DLL_API XyzzyFunc(unsigned short* dataPtr, long buffSize, /* other irrelevant params*/);

I think that I'd use unsafe pointers in C# to work with this huge buffer.

I assume in a 32 bit application it is not possible, so how in a 64 bit?

(Re: WIN32DLL_API right now the dll is built for 32 bit, I'm expecting to be able to get the vendor to provide a 64 bit version...)

Posted 6-Feb-15 13:00pm

Matt T Heffron

Updated 6-Feb-15 13:01pm

v2

Add a Solution

Comments

Sergey Alexandrovich Kryukov 7-Feb-15 0:57am

32-bit translates to 4G address space size limit. By default, 2G is taken by OS, which can be configured to "1 for OS to 3GB for application" ratio, as far as I remember (never tried). This leaves a little hope to get GB just for one buffer. 32-bit architecture is no-go.

There is no reason to fail in 64-bit architectures. Moreover, via WOW64, even a 32-bit application can get almost all 4G (or each of 32-bit applications)...

—SA

1 solution

Add a Solution

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

DelphiCoder · Answer 1 · 2015-02-06T15:24:00

Is there a function on the Win32 DLL to save this data to a file? If so, then you could save the file to a temporary location, then use a .NET Reader or even memory map the file for random access.

I wrote a log reader once that quickly read and scrolled through logs greater than 4 Terabytes in size. I never had more than 2 Megabytes in data contained within the application's memory at any one time. Even though I could place the entire file within memory, I found that Windows generates many page faults when using that method which slowed my application down to a crawl. It is better to use a reader or memory map technique on large amounts of data that you save to a file first to avoid paging and the whole paging fault issue.

That is if your DLL will let you save the data first.

How do I allocate gt 2GB buffer for PInvoke

1 solution

Solution 1

Add your solution here

Preview 0

How do I allocate gt 2GB buffer for PInvoke

1 solution

Solution 1

Add your solution here

Preview 0

Existing Members

...or Join us