Steam and Blockdevices need directions.

[domrockt]

Hello there, i play a few years now with homelabbing, i got interested in the Idea of a one big PC in the basement and serve all the data to smaller PCs in the House.

Sure the NAS part of that is quite easy and done. But what i dont understand ,and the internet is not that clear on that in one Post, is how can i use one big created Blockdevice to any PC in the House, like 3x 8TB Kioxias with my Steam library on it.

I tried ISCI, there is the Internet clear, i can use one Target to as many Initiators i want but that will and does makecurruption, and i understand why.
Sure the Internet blabbers about "using a cluster aware filesystem" and i tried CEPH but as far i went i can still only do one Target and one Initiator, when i try a second Initiator i got the same behavior as regular ISCI without CEPH

I tried Starwind, same.

So how can i achieve this?
a few specs on how i can transfer Data 2x 100GbE Mellanox Cards and a Dell 100GbE switch

What i did to work around is:

I use ZFS and use a cloned snapshot that i mount as a ISCI Blockdevice and make a new Target from that and repeat that for any device in the House. (it works but i need to redo that everytime some mayor games come out so i can save some space. By now i use some user scrips for that but it is not the Ideal solution.


Can someone please give me a direction i can research further? Thanks in advance!

regards Dom

 

[ano]

smb cifs, nfs will be your best bet depending on clients

or.. clustering filesystem on iscsi.. I know what I would reccomend

 

[domrockt]

depending on clients

the use case is, Steam games most of them need to be on a block device, just a given fact. I need a block device shared between Clients and dont want to make multiple Block devices with the same Data on it.

clustering filesystem on iscsi

that would be? how in that sense i wrote? iam sure i dont understand something about sharing a Block device

i found out that i can share Datasets via Clustering but that is in the giving Environment like Hyper-V or the Ceph cluster or Proxmox Cluster or ESXI cluster. That sharing is mostly explained for its High Availability. But not in my Share a Block Device around multiple clients on the Network use case.

The Clients are all Windows PCs.

 

[nexox]

Your current method with snapshots is about as good as you're probably going to get without a 'shared disk filesystem', which uses some form of out-of-band communication to coordinate between the block-access clients to avoid conflicts, this wiki page gives a list of options, most of which are proprietary or outdated: Clustered file system - Wikipedia

Everything I have heard about making OCFS or GFS2 actually work right has been pretty unpleasant, adding more automation and/or sophistication to your snapshot setup is probably going to be better in the end.

 

[mattventura]

> the use case is, Steam games most of them need to be on a block device, just a given fact. I need a block device shared between Clients and dont want to make multiple Block devices with the same Data on it.

Which games are those? I have most of my library on an SMB share, and I haven't run into any issues that I can confirm to be related to it being on a network drive as long as you take certain steps (like the setting to allow applications running as admin to see the user's network drives).

 

[i386]

What i do: My entire Steam library (6+ tb) is hosted on a nas and the share is mapped to a Drive Letter.


For Game clients that don't Support Network shares (Epic, uplay, es, iracing etc) i've created a virtual Disk (*.vdhx, currently 16+ tb) that is hosted from the Same nas. Windows sees a storage device Like it's directly connected to the host.

What's not clear to me: do you want to have just one place for multiple steam accounts or one place per account?

 

[domrockt]

What's not clear to me: do you want to have just one place for multiple steam accounts or one place per account?

Yes we all play from the same account, most of the time the saves are on the Disk on the client side or in the steam cloud. There are a bunch of games on steam that dont start over a Network share. But both options would be as valid.

Which games are those? I have most of my library on an SMB share, and I haven't run into any issues that I can confirm to be related to it being on a network drive as long as you take certain steps (like the setting to allow applications running as admin to see the user's network drives).

to be honest, the last time i checked with smb was 3 years ago.. ok creating one right now and start a copy and will test again and do a report. Will take a while

 

[i386]

There are a bunch of games on steam that dont start over a Network share.

I'm using that approach for years now and didn't find games, from aaa to indy, that didn't work.
Do you have examples?

 

[ttabbal]

For games that don't like network installs (stupid but whatever)... You could run Linux locally and use KVM to run Windows with disk images on network shares. Then they look like local disks and the VM abstracts away that change. With PCI passthrough, performance is very near native.

What it doesn't give you is the ability to share that file. What you're asking for is like connecting the same drive to 2 computers simultaneously. Windows applications have a bad habit of not playing well in those types of situations. They assume the whole drive is theirs and nobody else will mess with it.

If you have the RAM for it, you might get most of the way there with ZFS deduplication. Read up on it before messing with it. You have to re-create the filesystem to remove it. And you need a LOT of RAM to make it work well.

Cluster systems might work, CEPH seems to have the ability. You would need to install the client on every machine accessing that storage.

Or you could acknowledge that spinning rust is cheap and just get more. Even with 100Gb networking, you are using a 3 disk raidz. That's the bottleneck. Adding spindles is a cheap way to get more speed. And spinning drives with multiple clients is going to cause a lot of seek time.

 

[mattventura]

to be honest, the last time i checked with smb was 3 years ago.. ok creating one right now and start a copy and will test again and do a report. Will take a while

Don't forget to set HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\EnableLinkedConnections to 1 (DWORD).

 

[domrockt]

Cluster systems might work, CEPH seems to have the ability. You would need to install the client on every machine accessing that storage.

i gues thats a missing Part in my Ceph adventures. I read on that tool

Or you could acknowledge that spinning rust is cheap and just get more. Even with 100Gb networking, you are using a 3 disk raidz. That's the bottleneck. Adding spindles is a cheap way to get more speed. And spinning drives with multiple clients is going to cause a lot of seek time.

thats why my games live on a striped Kioxia CD6 VDEV

Don't forget to set HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\EnableLinkedConnections to 1 (DWORD).

started the copy a few minutes ago, i start testen all games in about 1 Hour and will report, thanks for the regedit

 

[nabsltd]

the use case is, Steam games most of them need to be on a block device, just a given fact. I need a block device shared between Clients and dont want to make multiple Block devices with the same Data on it.

No, Steam games need to be on a path that can be found by the Steam client.

You can trivially add any SMB path by going to Settings->Storage->Add Drive->Let Me Choose Another Location and then browsing to the empty folder where you want to place the Steam library. The gotcha, however, is you really can't share this location between multiple Steam clients, because the Steam client expects the files in the location to not be changed by an outside source, so that they can be properly updated in place. You might not have any problems if you set each client to only do installs at very different times, but it still might break.

That said, if you want to work around the "doesn't work on a network drive" bug, just create another Steam library on any disk on your machine...it doesn't even need to be a large drive. Then, robocopy the directory structure to whatever network location you want to use. Next, delete the local folder you just copied from. Last, create a symbolic link with the same name/location as the top level directory that points to the where you copied the files on the network. Unlike mapping a network location to a drive letter, very few programs will do the necessary checks to realize that the final location is really a network drive.

thats why my games live on a striped Kioxia CD6 VDEV

Mine live on a 14TB spinning rust array fronted by a two level cache consisting of 64GB of RAM and a 3.2TB Intel NVMe. After one play, 4GB/sec reads at worst, but mostly closer to 20GB/sec.