SteamID
patreon
Hardware changes and a bit of info about running SteamID
🕑 Added 2020-10-17 15:50:43 +0000 UTCSomeone mentioned I should film changes but to be quite honest i don’t have a good camera, stand or space around the rack and other kit I have or not yet at least. I see people complain about the “paywall” which SteamID has and I’m only in it for the money. But I want to write a short post explaining what really goes into running SteamID / SteamArchive. While they are two websites there’s a fair amount of kit and servers behind it. With the size of the database(s) any managed “cloud” provider is out of the question we are talking thousands monthly for that one “server” in some cases.
Apart from travelplays who helps admin SteamID which saves me a lot of time better spent in other areas, I solely write the website, manage its network , ensure backups are ticking over (more on that later)… manage all rented servers , manage all physical servers both of which need configuration changes as it grows. I manage the database and associated services.
There are other costs behind SteamID which some people may not realize aside from server rental costs and buying / building the physical servers, Electric bill for the servers, CCTV kit, Alarms, Aircon to cool the servers in the summer, running cost for aircon, Drive failures, Batteries for battery backup… the list is endless.
I’ve been up in the early hours of the morning to recover from issues, Servers going down or things changing without warning, alarms tripping (cats…) SteamID started out as a project but suddenly with patrons supporting the site there is a need to keep this service up and online and not be so relaxed about it, I mean if you pay to support a project and there’s no effort there then what’s the point right?. Very recently, last week in fact 2 NVME drives failed in quick succession which resulted in worst case scenario – restore from backup which took me 3 days as all config etc had to be done from scratch, guess who was more focused on backing up just the primary database… the service was backup I think in less than 20 hours without names and friends but came back online over time.
The above leads me onto backup costs, Sure an external drive maybe ok for some but SteamID, My hobby I never want too loose, So we have extra servers for replication, extra servers for backups, external network storage for backups, off site backups (paid service). I’ve just recently given a family member the T320 you will see in some pictures so I can use that for storage off site - I’ve seen enough horror stories over the years. If the house caught fire, I would need to be able to recover.
I have development servers running, backup SQL servers so I can restore my backups to to verify they are valid, Low end servers for this task took forever! , Mail senders and a vast amount of other virtual machines which you may not realize run to support SteamID.
Previous name text searching performance isn’t great, I moved it out onto rented kit when Patreon took off but it looks like I might need to bring it back where I can give it more resources.
The attached pictures are from the last 24 hours which are part of the plan towards a better connection which is hopefully coming within the next few months. The main host has been swapped out with 16Gb sticks of ram and I’m trying to get hold of another set, there is now 192GB (8Gb sticks) of ram in the R620 which I’m selling off as I’m in need of something a bit better, The 420 arrived yesterday and will become the new firewall, I’ve ordered a better CPU for it and the SSD came today which is overkill but I needed something more “proper” and redundant. The Dell switch needs another 10GB card and I need two more SFP+ cards, I want to get everything in the rack running on 10GB I’ve seen a huge benefit of 10GB mainly with SteamArchive. The batteries in the battery backup need replacing which I’ve been putting off. I’m swapping out consumer grade hardware with enterprise kit and keeping spares as most of my stuff is used from recyclers or end of office life. (Which is also why there are extensive backups!) I much rather used enterprise gear than new consumer.
I don’t think I’ve scratched the surface but that’s just a little insight in to where the patreon support goes. I am so grateful for everyone’s support with SteamID and I’m sure there are better ways of doing things but at the moment I feel like its as cost efficient as it can be while kicking out the performance to allow 10s of thousands of queries to the database every second. Even if the advert revenue quadrupled, I would still have to shut SteamID down so without patreon support there would be no SteamID.
So with that said, If anyone has any connections or companies that would do good deals or want to sponsor SteamID please get in contact I’m after in the short term:
9 sticks of 16GB 10600R ECC memory
SPF+ module for a PowerConnect 6248
2tb enterprise nvme
R730 chassis
3 x SPF+ pcie cards + DAC cables
batteries for the APC unit – need to confirm size
Long term:
Dell R730 – H730 – 256GB ram+
Layer 3 switch with more than 4 SPF+ ports or some kind of solution.
