As Mehdi says, never assign a buffer to the maximum size!
In .NET the maximum size of any single object is 2Gb, so the Max size of a bytes array is 2147483648 bytes. If you keep on allocating this much memory without actually needing it, you will slow your computer to a crawl very, very quickly...
Initially, define it as
Dim buffer() As Byte = nothing
And then re-assign it when you know how many bytes you need.