the other day I was uploading some files to my Google Drive like I do sometimes and I noticed this little Banner at the top warning me that I was getting close to my free data allocation and encouraging me to give Google just a little bit of money for some more storage looking at their plans these are actually not too terribly expensive and storage prices are currently at historic low so I could even buy myself a terab or so for a pretty reasonable price but you know what's an even more reasonable price $0 so using a
bit of software engineering knowhow I wisely traded tens of hours of my life and much of the sanity I had left to avoid spending probably $50 here's how I managed to store hundreds of gigabytes of my files on Discord for [Music] free if you're like me you have a lot of files and you're probably getting pretty close to the free limits for iCloud or Google drive if you're not already paying for extra storage but I've been thinking if you can get a little bit of storage for free why can't you get a lot of free
storage if you think about it every social media platform is really just a place where you can upload your files or content if you prefer and have other people look at them right Facebook YouTube Instagram they're all the same you upload your files and share them with people the key part to keep in mind here is that mostly these companies might set limits on file sizes or quantities per post but they usually don't limit the amount of data a single user or account can upload and we can absolutely abuse this feature to get as much
storage as we want for free my first candidate for a platform to steal storage from was YouTube YouTube lets you upload videos for free and each one can be 256 GB or 12 hours long whichever comes first so this seemed like a solid way of getting lots of 256 GB size trunks of storage obviously this would only work for video files so I need some way of converting all my data like word docs text files images Etc to video fortunately somebody smarter than me has already figured out a way to do this with FFM Peg
and posted the script for it on GitHub storing files on YouTube with this script would probably work but there were a few drawbacks some issues are that it's slow to convert to and from video it's also pretty slow to upload a video wait for YouTube to process it and then download it later the video file size ends up being several times the size of the original file but the big issue would probably be compression YouTube processes and compresses its videos to save storage which is great for them but potentially bad for being able to read
my data later compression could lead to losing some information and ultimately corrupt my files all of these problems were probably solvable it would just make things slow but speed's not a huge concern for me anyway because I mostly just want to store a bunch of files I don't access frequently but I decided pretty early on to give up on the YouTube route because I realized there's a much better solution there's another service that lets you upload files for free it's called Discord you may have heard of it sure most people use discord's text chat and
voice chat features to do boring things like play games with friends but you can also upload files to Discord as message attachments unfortunately there's a 25 megabyte limit but fun fact one big file is really just a bunch of small files all put together in a row so all I need to do is split up my files larger than 25 megab into 25 megab chunks so that I can upload them and even though there are different types of files like text photos videos with all sorts of file formats when it comes down to it it's
all just data as long as I have all the original bits of data in the right order and I know what the original name of the file was I can chop up any file into tiny bits store them in Discord then download them and reassemble the original file later so I got to work I created a Discord application in the developer portal and wrote a server in noj as to handle uploading and downloading files from Discord I wrote a very basic front end in react to give me a simple way of interacting with the server
when I want to upload a file I can go to the front end website and hit upload like you would on any regular website that accepts file uploads the files then transferred to my backend server which splits it up into 25 megabyte chunks then uploads each chunk as a message attachment in a Discord server the bot has joined when the bot sends a file attached to a message to Discord Discord publishes that message to the channel and replies to the bot with a unique ID of that message the bot keeps track of the message IDs
of each chunk and stores them in a database with some extra information about the file like the original name size and format later when I want to download a file I can send the server the name of the file I want to download the server can then use the file name to look up which chunks are associated with that file get the message IDs of those chunks and use the Discord API to download all those attachments then it can put them all in the right order and write each chunk one after another back to disk
with the original file name in this way we can upload an download files of arbitrary size by breaking them into small 25 megab chunks but there's still a pretty big problem here privacy Discord is of course public so anyone on the server can see the messages and attachments that the bot sends the easiest way to keep my files private would be to just create a Discord server where the bot and I are the only members and that way no one else would be able to read my data this would probably work although anybody can download
an attachment from Discord CDN you would need to know the exact Channel ID message ID and even attachment name associated with an attachment in order to properly download it which I think is pretty unlikely for anyone to guess on their own but it doesn't hurt to be extra safe even if nobody's in my server right now it's possible hypothetically for instance that the IRS could join later this year when tax season comes around and they'd be able to see all my sensitive tax documents with my plans to commit tax Rod I of course do not
want this so it would be good to conceal my data fortunately I can do that pretty quickly each chunk of file I encode will be encrypted then when I read the file back I can decrypt each chunk before putting it back together now nobody can see what the data I'm writing actually is if they look at the message attachments all they'll see is a bunch of random symbols and characters now that I had it all coded up the last thing left to do was to just upload all my data this is not particularly fast first
I have to upload the file through my browser to the backend server which takes some time next the server has to read the whole file break it into Trunks and encrypt it which also takes time then it has to upload all the trunks to Discord which takes even more time I added a status bar so I'd be able to tell if anything was actually happening because uploads and downloads can take several hours but ultimately I did get all my files uploaded without too much effort now let's talk about performance if I had to describe it
in one word it would be eh I did a speed test and for a 55 GB file it took me 2 and 1/2 hours to upload that's about 2 minutes and 45 seconds per gigabyte which is definitely slow but not actually too bad I guess downloads are a little bit faster we're looking at about a minute and 40 seconds per gigabyte to download which again is pretty okay and probably much faster than this deeply cursed project really deserves in terms of data integrity and Longevity so far everything seems good it doesn't seem like any of
my data has expired it's taken me a couple months to get this all done and in that time the first file I uploaded for testing still downloads and decodes without issue quick note here it's a few months later as I'm editing this and Discord just announced that they're going to change how external CDN links work because believe it or not people have been hosting malware on Discord now CDN links from outside of Discord will break after 24 hours but links within Discord will still function I'm planning a follow-up video after they make this change because
it's possible it'll break what I'm doing here but I actually think it might not so keep an eye out for that in the near future so that's how I'm storing all my files on Discord thanks for watching
