Biggest personal file management pain points?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

exfat_fingers

New Member
May 26, 2021
1
0
1
Ladies and gentlemen --

My name is exfat_fingers and I am a data hoarder and a home server enthusiast.

I've been professionally working on data storage for the last 15 years (all over the stack, from device drivers to cloud storage, including the ill-fated Windows Home Server among other things). A few months ago I decided to abandon the big tech world and all its office politics, perverse incentives, appalling privacy practices (and, sadly, good benefits and fat paychecks) and fully devote myself to a life-long hobby and passion of mine - namely, trying to make some sense of all the digital crap accumulated over the years. And, hopefully, help others to do the same and build a business out of it.

(Relevant xkcd.)

I have a list of things I'd like to do to solve the issues I am personally facing with my own environment (to start, having a unified view of all the files across all my devices, external drives and cloud stores) but I obviously would like to avoid building a product for myself and as such I am seeking the community's answers to the following question:

Thinking about management and storage of your personal files, what is the #1 thing that you wish somebody would do/fix/solve for you?

Cheers and thanks for your insight in advance.

exfat_fingers
130 TB raw disk space
1.5 million files
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,639
2,054
113
In my experience most people have segmented their files in general for personal and business use. For instance I don't need to see my family pictures in the same area I'm looking at my movies. I also don't need to see my business documents where I see my business code\project files. I don't need to see my business backup files when looking at business documents, etc... I go to where I need to see the resources. '

This not only reduces what I'm looking at but adds a layer of security by not centralizing all storage \ not centrally managing all storage.

My 02 of course
 
  • Like
Reactions: Blinky 42

tinfoil3d

QSFP28
May 11, 2020
876
403
63
Japan
130 TB and 1.5 m files looks like a lot of movies and music there. Sort that out first, continue with photos and then the rest is just a couple of documents, which you probably don't even need anymore? :) That's how it usually goes.
In general you'd get into a time machine and just go back and put the stuff into the relevant directories once you get the stuff on your PC.
 

Blinky 42

Active Member
Aug 6, 2015
615
232
43
48
PA, USA
In addition to "work" and "home" / "family" type groups I also have things broken up by broad categories like code, photos, movies, backups, etc.
Looking across everything without some of the top-level filters is almost never what I want to do, and running "locate foo" is too slow on most of my boxes anyway ;)
Also the filename itself isn't usually enough to find things - I use deep hierarchies so the last few elements of the path are needed to find what I want.

I was curious so did a sample of ends of the spectrum on computers here:
In ~ on my main dev box are 7.6M files across 700G
Main windoze laptop 1.2M files over 1.8TB (wow - another 900k files in C:\Program* and C:\WIndows* - never checked before!)
On external USB storage drives: 1.8M files over 320T of drives

For all of the external drives, when one is full / moved offsite or offline, I grab all the dir listings and store that in a file per physical device so I can grep through those to find what drive has the info on it down the road. Sort of uber-manual tiered storage.