Okay, I've actually figured simplifying it out, it's not that bad. Let me share my config:

First, since the PR isn't in nixos-unstable yet, I'm adding the fork as a flake input

inputs = {
  nixpkgs-lemmy.url = "github:CobaltCause/nixpkgs/lemmy-module-improvements";
};

then, in my system configuration, I add this:

# Not sure if this is required, maybe caddy auto-allows it
networking.firewall.interfaces.eth0.allowedTCPPorts = [443 80];

# Override the lemmy module with the one from the PR
disabledModules = ["services/web-apps/lemmy.nix"];
imports = [
  "${inputs.nixpkgs-lemmy}/nixos/modules/services/web-apps/lemmy.nix"
];

services.lemmy = {
  database.createLocally = true;
  database.uri = "postgres:///lemmy?host=/run/postgresql&user=lemmy";
  enable = true;
  settings = {
    hostname = "<YOUR_HOSTNAME>";
  };
  caddy.enable = true;
};

and, that's it!
However, I'm not sure if it will cleanly deploy, as you might get an error while starting.
If so, please check postgresql logs sudo journalctl -fu postgresql. The error will most likely be something like this:
[...] [10289] ERROR: permission denied: "RI_ConstraintTrigger_a_16639" is a system trigger [...]

If that happens, you need to manually run the migration until the fix is merged into Lemmy. Here's how I did it:

1. sudo su - postgres
2. psql -v ON_ERROR_STOP=1 lemmy postgres
3. (in psql) SET ROLE lemmy;
4. Paste the SQL code from here: https://github.com/LemmyNet/lemmy/issues/2784#issuecomment-1578337686

After that's done, you can exit the postgres CLI by typing \q, exit the postgres user and just simply sudo systemctl restart lemmy which should start properly now, and be accessible to the outside network.
Open it and it will give you the initial setup screen. Good luck!