A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
I'm using a couple of cheap Kingston A400 for my setup (120 and 480 GB) and they work just fine. One thing I noticed is that the 120 GB one's health went down to 92%, from 100%, in "just" one year (smart parameter). But that's implies a lifetime of more that 12 years, so I'm not excessively concerned.
EDIT: of course, just after writing this comment the smaller SSD began to behave strangely (errors in dmesg).
Are you using SATA to USB adapters? If yes, which one work well?
You can go with something like this if you want a clean solution.
I use a drive dock station for my backup drives, and I have a few of these for one-offs too.
Yeah, I use something like this, a no-brand enclosure.
Thank you, I think I will go with this and a SSD from crucial to start off and then add some HDDs for backup and NAS. Do you think this kind of enclosure is reliable enough for a NAS? And is there enough airflow if you add two 3.5" HDDs together in the dock station?
I have a couple of no-brand SATA-USB enclosers with some jmicron chipset. Can't remember the exact chipset right now, as my RPi is not working ATM (see edit).
I'm not 100% sure but this might also depend which OS you are running on your pi. Did change from SD card to SSD few years back for my home assistant setup (now running with a mini pc) at the time I went for a geekpi adapter (can't recall exact model).
Try a quick ducduck search "sata adapter #yourOS#" and see if a list come up. Here is one for HA (but surely more models are compatible than the ones listed here) https://community.home-assistant.io/t/working-usb-enclosures-and-adapters-with-hassos/212763
Edit: as for SSD, I went for Samsung EVO model
Sidenote: all SSD will be more reliable than a sd card because they are less sensitive to rewrite counts
Is there a one-line command to check my SSD? I have a headless setup. When I’ve tried on the past there was more results coming back than I knew what to do with.
sudo smartctl -a /dev/sdX should do the job
Thank you, that worked!
YW 😄
Is this command to check the health of the SSD/drive in general?
Just stick to the brands with multi-year warranties.
For example? Kingston, Crucial?
Have a look here: https://www.backblaze.com/blog/ssd-edition-2023-mid-year-drive-stats-review/
I've never had a single problem with Crucial, Seagate, WDC. I've had a few issues with Seagate 990 Pros having issues, though it was later made known that Samsung had some bad batches out in the world. Hopefully that is not the case anymore.
Buy used Samsung mSata or m.2 2230 drives on fleaBay. Stick with Samsung and other well known brands with decent spec sheets and warranties, that's the cheapest way to handle durable storage on a pi. USB enclosures are like $5-7 on AliExpress or fleaBay.
Buy MLC drives if you need higher endurance (check the model no and look up the datasheet.) TLC will usually be fine for a few years, MLC will last a bit longer. If you're killing drives faster than you expect buy larger (512 instead of 256GB), blkdiscard the entire device once it's installed and then only partition 60-80% of it. Never touch the rest of the freed storage and the drive controller should be able to use those blocks for wear levelling to reduce the NAND wear rate.
Edit: One heads up, I usually buy used drives from eBay because their buyer protection is top tier, if there's anything wrong with the drive when it's delivered or when I test it it goes right back for a refund. This makes buying blind viable thanks to an easy return policy.
If you're sourcing used drives somewhere else insist on seeing SMART data before purchasing and don't buy heavily worn drives. Look at the drive model datasheet, find the warranted endurance of the drive (if it's a 512GB drive rated for 1 DWPD over 3y that means the rated endurance is ~ 0.5T * 365 * 3 or roughly ~550TB written over 3y. Pass on buying drives approaching their rated endurance, try to buy lightly used drives wherever possible and you shouldn't have problems with reliability.
whoa whoa, I would not recommend a cheap AliExpress USB enclosure at all. As others have already pointed out there's a whole ever-growing blacklist of partially incompatible enclosures that basically flake out whenever they feel like it. Worse yet, not every device is on the list so you frequently have to research and add devices yourself.
The last generic Inland m.2 enclosure I bought worked fine... for 1 hour. Then it disconnected and reconnected. I thought it was just random chance, until it happened again and again and again. Did the deep-dive research, found the chipset was partially incompatible and I had to return it.
DO NOT BUY CHEAP ENCLOSURES FOR EXTERNAL MEDIA ON RPI
My approach to this has always been to buy one enclosure and validate it, then go buy like 8 more after thorough testing. Obviously don't place an order for 10 units of an unknown tech item from AliExpress or you're looking at a bad time. Look for enclosures that use known good chipsets and there's not as much risk as you're expecting. I have something like 8 msata enclosures here that work flawlessly and another half dozen sata+nvme rtl9210b enclosures that also work well.
What about enclosures from Sabrent? That should be a reliable manufacturer, no?
Unfortunately I don't have any recommendations I can give you as each enclosure could use a different chipset. It seems that the brand does not have a good reputation for compatibility but that list is fairly old at this point. All I can say is if you find an enclosure you like, plug the model number into the raspberry pi forum and see if anybody had to add it to the quirks list.
I heard it's not so good practice to buy used drives so I will not do that for now but thanks for the second part I will consider that!
Since you mentioned MLC, maybe you have some suggestions for eg used server grade disks? Would the Rpi be able to run something like the Intel datacenter SSDs eg S3700? The power loss protection is really something I would like to have, especially in a homelab scenario. Or any other notable MLCs with larger capacities? I am having trouble finding a good list sorted by max potential storage.
Most enterprise drives are TLC these days, MLC just doesn't provide the storage density that enterprises require anymore. I only mentioned MLC because you'll occasionally find mSATA drives in the <=256GB range that use MLC. You have to check the datasheet for each model, look for endurance rated at 5DWPD or higher, those will typically be MLC or heavily over provisioned TLC. If you want enterprise drives with greater endurance than the usual 0.5 or 1 DWPD look for the over provisioned models with capacities like 400GB, 800GB, 1.6T or 3.2T. those are 512GB, 1TB, 2TB and 4TB raw capacity drives with a bunch of flash set aside for wear leveling purposes. You don't often see 300GB, 600GB, 1.2T or 2.4T drives anymore but those are often very high endurance (write intensive, 10 DWPD or so) models.
Check the datasheets for drives when you're shopping and you can get a pretty good idea of what their durability is like, I usually buy 1 DWPD drives for write occasional bulk storage and 3+ DWPD for anything with a serious write workload. You can also help the drive controller a bit by running blkdiscard against the entire device before partitioning, then only partition and use ~80% of available space. The drive controller will typically grab free unused blocks and use them for wear leveling but only if they've been marked free (TRIMmed) and never allocated after. If you can't find or can't afford high endurance drives you can usually buy a larger lower endurance drive and over provision it in this way to extend its lifespan.
(The last time MLC flash was really common was back in maybe 2014-2015, some of the older Samsung pro drives like the 850/860 pro were built using MLC. Those had legendary real world endurance, I think they'd get up to 10+PB written before actually failing. It's a shame they didn't have PLP because they would have made good budget array storage if they did.)
Legend, thank you!
Do you want reliable, or do you want cheap? You must choose 1 from that list.
If you're not planning on putting anything critical on it and you're doing backups, and you don't mind being without its use for however long it would take you to replace it if it dies, pick anything.
I've never had an SSD die on me Yet but I don't buy cheap brands though I don't buy top of the range and I usually buy at a good deal. Crucial MX has been reasonably priced in the past.
I stumbled upon the Crucial BX, do you think that one is not reliable enough? Because I think I use that in my PC atm and it runs fine. But it's not a home server so there is probably less load on this one.
Crucial is fine. It's commonly found in corporate and government workstations.
Don't have much personal experience with BX but they're probably fine. But once you have backups of anything you care about the worst that can happen is you need to restore those backups. If its running a service you can't do without then maybe a backup pi?
I have RPis running on SD cards for years with no issues so realistically you probably won't have any either but better to be prepared than not. And it also means that if you mess something up you can restore it to when it worked.
Crucial is fine but IIRC the 1TB+ variants are too good to be true (cheap) and will die quite fast. Just a note for everyone to look into the underlying technology on the particular model.
I've a number of 1GB + 2GB models. Admittedly none in the last 3 years but some are 8+ years going strong. I know of some others with no issues but maybe I've been lucky
As the Pi can boot from USB, I wonder if anyone tried to use those "SSD in pendrive form factor" devices for this, eg. the Transcend ESD310S 512GB?
Happy cake day!
Is cake day even a thing on day zero? Haha.. thanks! :)
Jeff Geerling ran some benchmarks on USB storage options in 2020, including a pen drive SSD: https://www.jeffgeerling.com/blog/2020/fastest-usb-storage-options-raspberry-pi
I've personally been using a Teamgroup 1tb from Amazon and a few KingSpec (from Aliexpress) 500gb SATA and PCIe M.2 drives for about a year and a half now without much problem or reported health loss. They are not performance winners of course but they still beat the pants off spinning rust.
YMMV of course, always keeps backups or don't keep anything you'd be super sad to lose on them.
What does YMMV mean? And yeah sure, I just want to run the OS on the SSD and a few containers but the rest on HDDs and also backups on other HDDs.
Your mileage may vary - your experience might be different for one reason or another
Samsung T7 totally worth every cent. You connect it via USB-C.
Which Raspberry Pi do you have? There are some very reasonably priced M2 hats out there that you can boot from on the Pi 5, including the Raspberry Pi branded one.
But an M.2 is usually more expensive than a normal 2.5" SSD. Is it better to boot from a M.2 HAT than from USB?
M.2 would technically be the best performance available, but realistically I don't think you'd notice the difference over a USB SSD. Maybe a few extra seconds on boot.
Okay and what about longevity of the drives? That should just depend on the number of writes, right?
I'm not sure what kind of money you want to spend? The M2 Hat is ~$14 USD and a 2242 NVME SSD can be had for ~$30-$40 USD since you don't care as much about performance.
The USB to SATA adapter is going to run ~$10 USD and the SATA SSD drives are going to start ~$20 USD are go up from there depending on size, performance, etc.
If size of storage is an issue, the SATA SSD is probably the better route. I believe the NVME would be better performance since it utilizes the bus on the Pi more fully.
I would guess that for the money, most M2 drives and SATA SSD drives are going to be similar lifespans
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| HA | Home Assistant automation software |
| ~ | High Availability |
| NAS | Network-Attached Storage |
| PCIe | Peripheral Component Interconnect Express |
| RPi | Raspberry Pi brand of SBC |
| SATA | Serial AT Attachment interface for mass storage |
| SBC | Single-Board Computer |
| SSD | Solid State Drive mass storage |
[Thread #925 for this sub, first seen 19th Aug 2024, 07:35] [FAQ] [Full list] [Contact] [Source code]
Look for log2ram on a LLM.
Could you elaborate?
Log2ram is a service which keeps your log files in RAM, avoids the constant writes to disk and really helps with SDcard longevity. Probably helps with SSDs too.
You can just Google it and check out the github page, no need for LLM accuracy lottery
Thank you very much, this helped way more than the other comment!
what it's log2ram
Log2Ram is a utility designed to extend the lifespan of storage devices, particularly SD cards in Raspberry Pi systems, by reducing write operations[1][4]. Here's an overview:
Log2Ram redirects log files from being written directly to storage (/var/log) to being written to RAM instead[1]. It creates a mount point in RAM for /var/log, so all logs are written to temporary memory (tmpfs) rather than the physical disk[2].
Extended storage life: By reducing write operations to SD cards or SSDs, Log2Ram helps prolong their lifespan[1][4].
Improved performance: Writing to RAM is faster than writing to disk, enhancing system responsiveness[4].
Transparency: Applications continue to write logs to /var/log as usual, unaware that it's now in RAM[1].
While Log2Ram offers significant benefits, it's important to note that logs may be lost in case of unexpected power loss or system crashes before synchronization occurs[3].
Citations: [1] How log2ram works. | The World's Linux Journal https://linuxfun.org/en/2021/01/01/what-log2ram-does-en/ [2] Log2Ram https://nimno.net/dev/linux/log2ram/ [3] Extend The Lifespan of Your Raspberry Pi's SD Card with log2ram https://ikarus.sg/extend-sd-card-lifespan-with-log2ram/ [4] Optimizing Performance with Raspberry Pi Log2Ram https://fleetstack.io/blog/raspberry-pi-log2ram-guide [5] Using Log2RAM on the Raspberry Pi - Pi My Life Up https://pimylifeup.com/raspberry-pi-log2ram/ [6] How To Write Log Files In RAM Using Log2ram In Linux - OSTechNix https://ostechnix.com/how-to-write-log-files-in-ram-using-log2ram-in-linux/ [7] Extend your Pi's SD Card Life with Log2Ram! - YouTube https://www.youtube.com/watch?v=MrRGa7kAhQo [8] How to extend the lifespan of memory cards on the Raspberry Pi with Log2Ram https://www.geekbitzone.com/posts/2020/log2ram/log2ram-raspberry-pi/