|
Once bitten, twice shy...I got hit hard a long time ago by some proprietary backup software that created files only it could read, and (I guess) one misplaced bit at the worst possible location was enough to make the entire file unusable. Of course that's when I needed to use the backup.
Since that time I've been sticking with robocopy; the entire tree is identical to the source, and no third-party software whatsoever is needed to access any specific file and without mounting anything. Of course that does nothing for bootable disks and the like, but considering I'm primarily interested in backing up VHD/VHDX files (and not physical disks), I can get away with it.
The entire "disk" is encrypted with VeraCrypt which, in a case like this, is preferable over a file container (.VC files). Since this is a hardware RAID, neither Windows nor VeraCrypt are really aware the underlying disk is actually a RAID (maybe they are, but since this is "supposed" to be transparent, I'm guessing they don't do anything special about it.
|
|
|
|
|
To add to the discussion, here are some thoughts and information:
- Poor throughput could be linked to the RAID5 being either degraded or still being built. I found some Youtube videos showing that (for your model) you would get about max 40 MB/s write speed in such a case, opposed to about 80 MB/s with a healthy array.
- As mentioned by others, these entry level enclosures have pretty low hardware specs and most often do software RAID inside the box. This can seriously impact performance.
- USB 3.0 throughput can reach 5 Gb/s, or 500 MB/s after overhead etc. USB 3.1 can do double that. USB 2 can do about 1/10 of that. So you are getting USB2 speeds at best.
- Have you checked transfer speeds on another PC?
- Finally, your NAS initially came out in 2010, which is a century ago in IT terms.
I run a pair of Asustor NAS with 2.5Gb/s ethernet interfaces. I get 250 MB/s read and write speeds with a simple Windows Explorer copy [^]. These NAS do however have a couple of NVMe SSD cache drives in them. A third NAS with 1 Gb/s and no cache drives gives a solid 100 MB/s. I have yet to test VM perfomance, but the NAS can expose iSCSI volumes so I am expecting to see similar performance from inside a VM.
So old that I did my first coding in octal via switches on a DEC PDP 8
|
|
|
|
|
As part of my analysis, I looked at the manufacturer's web page for the NAS box. It claims that the RAID is done in hardware.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
I have a WD PR4100 NAS configured to use RAID5 with 8TB WD Red drives. When writing to the NAS over a 1GB/s network, I get speeds of at least 60-100 Mbps. Unless you are writing large amounts of tiny files, I would expect at least that sort of performance out of any NAS.
USB 3 has a maximum transfer rate of 5 Gbps, and USB 2 has a maximum transfer rate of 480 Mbps. Even using USB 2, the transfer rates are very low. I would try each single drive in a different enclosure, if you have one, and then try one of the single drives in the NAS enclosure. This should give you enough information to determine whether it is one of the drives or the NAS that is the bottleneck.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
Daniel Pfeffer wrote: Unless you are writing large amounts of tiny files, I would expect at least that sort of performance out of any NAS.
So far I've only been trying to backup VM files - so, very few, but rather large files (multiple dozen GBs in size).
Daniel Pfeffer wrote: Even using USB 2, the transfer rates are very low
Agreed, I've been using other single drives using the same USB3 port and connector, and get much better speeds. So I have to rule out this beign stuck in USB 2 mode.
Your idea of testing out each drive separately is a good one. I wonder if I could power down the RAID, take one drive, put it in an enclosure and, without reformating it, run some sort of non-destructive speed test, then put the drive back in the RAID. I'd hate to rebuild it all over again at this point.
|
|
|
|
|
dandy72 wrote: I wonder if I could power down the RAID, take one drive, put it in an enclosure and, without reformating it, run some sort of non-destructive speed test
Given that you want to test WRITE speed I would say no, not unless you can find a non-destructive write test. I don't know of any test program that works like that.
You may also have to take into account the behaviour of the NAS. It may detect that the drive is part of a very degraded RAID-5 set and refuse to mount it. You may have to tell the NAS that this is a JBOD and have it reconfigure the drive before it can be accessed.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
First, RAID 5 with 8 TB drives is asking for complete, 100% data loss. The reason is that if you have one drive fail, when you replace it the entire RAID set has to be restored and during that restore even a single burp from one of the other three drives will double fault your data store, causing the entire store to corrupt. RAID 5 is also very computation expensive in that the data has to be split up across the drives during writes and then recombined on reads.
Second, if you want performance from large drives in a RAID environment, use RAID 1 or RAID 10. These are striped. Personally, I would use RAID 10 in your case, resulting in 16 TB of resilient storage. RAID 1 and 10 are mirrored so there is no computation involved, just two writes. During read either drive in the pair may be read depending on drive availability.
Finally, veracrypt is a good product, but if your system supports BitLocker at the hyper-v level, use it instead. There is a lot less overhead with BitLocker relative to veracrypt. I don't think this is causing your performance problems, though. I suspect one of your drives is on the way out and the RAID controller is having to do a lot of extra reads and writes to keep that drive responding.
|
|
|
|
|
Very interesting thoughts from someone who obviously has a lot more RAID experience than I do. I appreciate you sharing that.
Are failures during rebuilds so common that RAID admins consider them to be a real concern? I don't question the possibility of having two failures occurring close to each other in time...I'm wondering if you're suggesting that a RAID 5 config exacerbates the likelihood of multiple failures. I suppose all drives are working harder while the whole thing is being rebuilt...
In a way, I'm ok(-ish) with the possible loss - the whole RAID is intended to only act as an additional backup set (not my only backup set). So long as it doesn't take a week to rebuild if I do ever encounter a failure.
And since this is only for backups, I'm not terribly concerned about squeezing every last iota of performance - but again, not sacrificing performance to the extent that I'm seeing right now.
As mentioned elsewhere in this thread, I thought RAID5 was a good compromise between redundancy and capacity. I'm not a huge fan of the thought of pure duplication (eg, losing 50% of total capacity). OTOH, I'd be willing to do it if it guaranteed it brought performance back to where it ought to be. But if it only made things marginally faster, I don't think I'd be in a much better place.
|
|
|
|
|
dandy72 wrote: suggesting that a RAID 5 config exacerbates
IMO, it's the drive count. For RAID 5, it's just a sort of high barrier to entry where it's just not worth doing unless you're using a fairly large number of drives (9+) because fewer drives tends inherently mean the recoverability/performance benefits would be better in a different RAID mode.
You're right though. Something with that enclosure or one of those drives or something about the NAS networking is not right.
|
|
|
|
|
Just a thought, have you tried the enclosure on a USB 2 socket?
So old that I did my first coding in octal via switches on a DEC PDP 8
|
|
|
|
|
Wordle 1,032 3/6
⬛⬛🟩🟩⬛
🟩⬛🟩🟩⬛
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 1,032 5/6
⬜🟨⬜🟨⬜
🟩⬜🟩🟩⬜
🟩⬜🟩🟩🟩
🟩⬜🟩🟩🟩
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 1,032 3/6
🟨⬛⬛🟩⬛
⬛🟩🟩🟩🟩
🟩🟩🟩🟩🟩
Ok, I have had my coffee, so you can all come out now!
|
|
|
|
|
Wordle 1,032 3/6*
🟨⬜⬜🟨⬜
🟩⬜🟩🟩⬜
🟩🟩🟩🟩🟩
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
⬜⬜🟩🟩⬜
⬜⬜🟩🟩⬜
🟩🟩🟩🟩🟩
In a closed society where everybody's guilty, the only crime is getting caught. In a world of thieves, the only final sin is stupidity. - Hunter S Thompson - RIP
|
|
|
|
|
Wordle 1,032 4/6
🟨⬜⬜⬜⬜
⬜🟨🟨⬜🟨
🟩⬜🟩🟩⬜
🟩🟩🟩🟩🟩
|
|
|
|
|
(16. April 2024) 4/6
⬛⬛⬛⬛🟩
⬛⬛⬛⬛🟩
⬛🟩⬛🟩🟩
🟩🟩🟩🟩🟩
Rules for the FOSW ![ ^]
MessageBox.Show(!string.IsNullOrWhiteSpace(_signature)
? $"This is my signature:{Environment.NewLine}{_signature}": "404-Signature not found");
|
|
|
|
|
Wordle 1,032 4/6*
🟩⬜🟩⬜⬜
🟩⬜🟩⬜⬜
🟩⬜🟩⬜⬜
🟩🟩🟩🟩🟩
Happiness will never come to those who fail to appreciate what they already have. -Anon
And those who were seen dancing were thought to be insane by those who could not hear the music. -Frederick Nietzsche
|
|
|
|
|
Wordle 1,032 5/6
⬜⬜⬜🟩🟨
🟩⬜⬜🟩⬜
🟩⬜🟩🟩🟩
🟩⬜🟩🟩🟩
🟩🟩🟩🟩🟩
I must confess my mind was in quite another place with those earlier two guesses
|
|
|
|
|
Wordle 1,032 3/6
🟩⬜🟩⬜⬜
🟩🟩🟩⬜⬜
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 1,032 3/6
⬛⬛🟩🟩⬛
🟩⬛🟩🟩🟩
🟩🟩🟩🟩🟩
Skill 82/99
Luck 74/99
If you can't explain something to a six year old, you really don't understand it yourself. (Albert Einstein)
|
|
|
|
|
Wordle 1,032 3/6*
🟨⬛⬛🟩⬛
⬛🟨⬛🟩⬛
🟩🟩🟩🟩🟩
|
|
|
|
|
Wordle 1,032 2/6
🟨⬛🟩⬛⬛
🟩🟩🟩🟩🟩
Jeremy Falcon
|
|
|
|
|
|
jschell wrote: self driving car
How many humans do you know who always drive safely, and according to the traffic regulations? If the cars learn from us...
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|