this post was submitted on 05 Dec 2023
958 points (98.7% liked)

linuxmemes

20401 readers
2286 users here now

I use Arch btw

Sister communities:

Community rules

  1. Follow the site-wide rules and code of conduct
  2. Be civil
  3. Post Linux-related content
  4. No recent reposts

Please report posts and comments that break these rules!

founded 1 year ago
MODERATORS
poopsmith@lemmy.world
zephyr@lemmy.world
958
usb formatting (sh.itjust.works)
submitted 8 months ago by SkySyrup@sh.itjust.works to c/linuxmemes@lemmy.world
55 comments fedilink hide all child comments
 

shamelessly stolen from nixCraft on mastodon

you are viewing a single comment's thread
view the rest of the comments
[–] MonkderZweite@feddit.ch 51 points 8 months ago (4 children)

Obligatory useless use of dd comment.

[–] stjobe@lemmy.world 8 points 8 months ago

That was an interesting read, thank you!

[–] Molten_Moron@lemmings.world 7 points 8 months ago

I haven't touched dd since I read that about a year ago, super interesting!

For people that use dd because they like the progress bar, I highly recommend pv.

[–] azertyfun@sh.itjust.works 5 points 8 months ago* (last edited 8 months ago) (1 children)

Except the proposed alternative should not be cp or pv, but dd bs=4M oflag=direct,sync status=progress.

I feel like I'm taking crazy pills with all the advice in this thread, because for USB keys you will otherwise end up instantly filling the write cache... which will block the apparent progress of the copy operation (so why even use pv since all you're doing is measuring your RAM speed and available cache size) as well as heavily slow down (even potentially partially freeze in some circumstances) the rest of your system as the kernel is running out of free pages and can't flush caches fast enough due to the slow-ass write speeds of usb keys.

* (Alternatively there is a kernel setting somewhere to disable caching globally for a block device... but in most cases caching is good, just not when you're flashing an ISO).

[–] MonkderZweite@feddit.ch 1 points 8 months ago* (last edited 8 months ago) (1 children)

so why even use pv since all you're doing is measuring your RAM speed and available cache size

This is probably why pv progress fills in a second but is only done after a few minutes. Nonetheless, shell redirect, cat, cp work fine and handle blocksize and cache dynamically.

Your worst case scenario never happened to me after years of using pv/cp for flashing sticks/overwriting/copying partitions, even with some ...risky mount settings. Honestly doesn't make much sense to me either. Again, dd isn't some sort of magical safe handle to make the process progress smoothly. Like i use to say, dd is a skalpell, not a shovel.

[–] azertyfun@sh.itjust.works 2 points 8 months ago

I mean yeah, the bits end up where they should. It's just that the speed/progress indication is near useless with pv since at the end of the copy you still need to wait for the entire write buffer to be flushed (2 GiB in my experience, which can take several minutes).

So IMO dd with at least oflag=osync,odirect is safer than cp and pv with which a newbie might forget to run sync and unplug the usb key immediately, so they'll be missing a lot of data.

Maybe some people use dd for the wrong reason, it's their problem, but the solution is to use dd bs=4M oflag=osync,odirect, not to use cp.

[–] AffineConnection@lemmy.world 1 points 8 months ago* (last edited 8 months ago)

It was less useless for that purpose when cp and cat were less I/O efficient compared to dd with the appropriate block size, which isn't as much the case now as it used to be.