The Lounge - CodeProject

Much appreciated

OriginalGriff wrote:
then the Surface can pick them up and display them while I'm cooking

Most restaurants here don't have that luxury. I imagine your kitchen to sport USB sockets Big Grin | :-D

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Nah, it picks up via WiFi. Big Grin | :-D

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!

Have you measured the real sustained transfer rate from the application to the network?
And the average and variation in access time? The time it takes to create ten thousand tiny files? How does it compare to an M.2 disk?

Obviously you won't have an M.2 disk of ten terabytes; I am not proposing that as viable alternative Smile | :)

I ask the questions just to make people aware of the cost (in time) of the solutions that are capable of handling tens of terabytes of data. You can't just scale up linearly with the data volume / file count the expected time to complete, from your M.2 disk!

There's a 1 Gb (don't laugh at the size) drive; an old fashioned external drive, connected to the router. It's available on the LAN as a network drive.

It's nice to have shared space between PC's.

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Interesting.

I'm looking for +10TB, though.

Cheap and ideal, since the router doesn't get turned of. Ever.

..but +10Tb is not going to work over USB. My router has USB2, but even USB3 would feel slow. If you can afford it, go OG and RAID5.

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Not getting it. Why should a storage capacity of +10 TB not work over USB? The access time of large-capacity disks is not significantly longer than that of medium or small capacity disks.

Obviously, if you are going to fill up (or back up) 10 TB in one go, the sheer volume will make it take time, regardless of interface technology. With the 22 years old USB2 and its 53 Mbyte/sec max transfer rate, it will take even more time. The 4,5 years old USB 3.2 can reach 2400 Mbyte/sec transfer rate is not likely to be the bottleneck - the rotating disk is not delivering data that fast to the interface anyway.

20+ years ago, when people defragmented their disks twice a week and twiddled around with interleave factors (that was on FAT disks!), I had several friends who seriously believed the claims that they could significantly speed up their computer by deleting unused files, even if they had plenty of free disk space. A couple of them insisted that it was indeed noticeable at the user level. I tried to make them explain why it would make a speed difference if a disk block is in a never accessed file, so it lies untouched on the disk, or lied untouched on the disk because it wasn't allocated to any file. Even if my friends could not provide any explanation (they were not computer professionals), they trusted the claims in the computer magazines more than they trusted me ... Smile | :)

I see "+10 TB disks are slow" as a variation on the same theme. There is no technical reason why a large disk would be significantly slower. The rotational speed is the same, the arm moves equally fast, and the maximum arm movement is limited by the disk size but unaffected by coding density and track density.

Both for the FAT disks 20 years ago and modern disks, you may bring up some extremely marginal points that might, under special circumstances, possibly have a measurable effect on the speed, but never so that the user would notice. E.g. in an almost empty disk, the search for a free block (when managed by a bit map) would succeed faster than if the entire bit map must be searched to find the very last free block. Such operations make up just a tiny little speck of the application run time that it can't possibly be noticed by the user. The placebo effect can be quite strong, though (and I do know that there exists placebo so strong that it works even if you don't believe in it Smile | :)

).

trønderen wrote:
Not getting it. Why should a storage capacity of +10 TB not work over USB?

Dunno. Explain me why Heroes of the Storm lags when I run it from there? Smile | :)

Router and external spinning HD are from the same era as USB2.

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

If everything is USB2 and not USB3, and you want to access 4K 60Hz video, you may be pushing USB2 limitations. You can't assume that those limitations apply to USB3 as well. They do not.

When I got my first USB2 disk, twenty years ago, the disk was the bottleneck, not USB. But it wasn't really: In video editing, I was still on SD, 540*720 resolution, 25 fps, for which USB2 has plenty of capacity.

I am not familiar with Heroes of the Storm, and have no idea what sort of data is transported from the disk, or to and from other units. So I can't tell why it lags. The bottleneck isn't necessarily the capacity of the USB line as such, it could be the software driving the interface. Or higher layers. You may use Task Manager or Resource Monitor to watch the disk load. At the "cable level", USB2 should be able to provide a little over 50 Mbytes/sec. If the traffic is significantly below this, it is not the fault of the physical USB2, but rather those processes using the USB interface.

trønderen wrote:
If everything is USB2 and not USB3

Most stuff here is.

trønderen wrote:
I am not familiar with Heroes of the Storm

A modern game, like Warcraft in terms of size.

trønderen wrote:
it is not the fault of the physical USB2, but rather those processes using the USB interface.

Did I mention it not a real PC but a router?

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

My router has a built in storage (backed up to the cloud)... almost empty...

"The only place where Success comes before Work is in the dictionary." Vidal Sassoon, 1928 - 2012

I have a Windows 2012R2 headless server here with 4 x 4TB drives (SATA) and 4 x 3TB drives (SAS - cheap as chips on e-bay). Drives are on a Dell PERC H700 (needs a small physical mod to work in non-Dell machines) with battery backup.

This server is fast and (apparently) highly reliable, the only failure I have had was one of the WD Red SATA drives that crapped out after about 1 year - the MegaRaid software sent me an alert and I did a cold-swap and it rebuilt nicely. The s/h SAS drives seem to be bullet-proof (they should be, they are enterprise quality IBM/Seagate).

This is my first line of defense: All our important files are stored directly to the server plus all our media files (having a fast server is great when you are copying 50GB at a time). My workstations do a Macrium GFS backup to this server. There is also a NextCloud VM which shares folders and workspace with other (remote) family members.

Additional to this server I have 2x4TB NAS (Netgear) which does a 'pull' backup of data from 3 workstations (documents, mail folder etc.) plus backups of the really important folders on the Windows 2012 server including 350GB of photos). This data is mirrored to Amazon Cloud Drive via a Netgear plug-in.

The Windows server also host a few VMs, mainly old Windows builds and temporary Linux projects.
The NAS is also used for backups of various Linux and Android devices via NFS shares.

Overkill? Maybe, maybe not - but life would be a lot more complicated without the server.

So old that I did my first coding in octal via switches on a DEC PDP 8

Thanks

I have two old machines running on my home network serving up ~110TB of disk between them. I ripped all my DVD collection (600+) onto these disks (multiple copies across multiple disks) and can access them from any machine on my network (kitchen, one in each bedroom plus my office and the TV room). I also download recipes and how-tos from YouTube and scan all my documents onto other disks. Important documents I encrypt and store them additionally in multiple clouds. Finally, all the software I develop at home, games, graphics and other fun stuff go on there as well. I have had to replace about 7 drives so far over the last ten years or so ranging from 2TB to 8TB. I have decided 8TB is the maximum size I will use as it takes a long time to duplicate to two or three copies and there is too much to lose in one failure.

NAS? What's that? Wink | ;-)

- I would love to change the world, but they won’t give me the source code.

� Forogar � wrote:
NAS? What's that?

A file server, but cheap.

Wrong is evil and must be defeated. - Jeff Ello

I have a (retired) Dell Precision workstation running TrueNAS with mirrored disks. Very flexible and configurable. FWIIW: The data on my system is kept in a RAID mirror (hardware addon not built in). Protects from hardware fails but not idiots (me) or malware.
Of course, there is always the cloud. D&R. Smile | :)

>64
Some days the dragon wins. Suck it up.

I have a Synology 1621 with 4x6TB drives in RAID5ish configuration as a backup/etc storage host. In most ways it's serious overkill for my needs, but is about the minimum spec level that supports a PCIe slot that I can stuff a 10GB network card into in a year or so when I build my new PC and upgrade my lan.

Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason?
Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful?
--Zachris Topelius

Nice, thanks

Yes, a small Synology NAS (DS220j) with 2 4tb drives.

Accessible from PC and Mac at home.

Worth it ? depends on your usage.

CI/CD = Continuous Impediment/Continuous Despair

Franc Morales wrote:
Do any of you have a file server in your home LAN?

One of my company's servers (our only remaining physical server) resides here in my home office. The current h/w (excluding SSDs) is around 13 years old and still works great.

Roles:
0: File Server - All development projects/files/documents are kept here.
1: Web/FTP Server - This server hosts one of the main company web sites as well as a half-dozen web company use web apps, a few customer web apps, and a dozen or so demonstration web apps.
2: Database Server - SQL Server/MySQL including SSRS
3: Mail Server

Franc Morales wrote:
If so, is it worth it?

In recent years, we have moved most of the customer web apps to an Azure VM. At this point, I'm weighing the options of replacing the aging on-premise server or moving it all to the cloud.

"Go forth into the source" - Neal Morse
"Hope is contagious"

Small Synology NAS (DS213j) configured with mirrored (RAID 1?) 4TB WD drives.

Worth it for me - for now. It's setup as a Plex server for media plus storage of quite a bit of data.

The data is slowly being migrated to cloud storage where possible.

I might add that one might want to be aware of SMR (slower write times) versus CMR drives when it comes to NAS or server drives, if you are going to be writing large amounts of data. Some vendors are using SMR which is slower. I use the plus drives from one of the manufacturers, supposed to be CMR. I think most, if not all, drives 6TB+ are CMR. There was much ado about this on truenas, it seems the zfs file system does not get along well with SMR drives. Don't know what file system Synology uses, never looked. We have a large one at a clients site, been running for 4 or 5 years without problems. I get an email from it every month, telling me it is happy.

>64
Some days the dragon wins. Suck it up.

Synology also does not like SMR drives, slows down writing a lot.

It doesn't help that some vendors were selling SMR drives as 'For use in NAS'.

I found all this out when trying to repopulate a donated Synology NAS with drives from my spares, and finding most of them were SMR.

SMR never should have been marketed to consumers at all. For datacenter scale write once archival storage it's limitations don't matter. Anywhere else they can be crippling at times.

Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason?
Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful?
--Zachris Topelius

Welcome to the Lounge