Asklemmy

I see a couple ways you could do this. For what it's worth, I think Lemmy may be close enough to what you want out of the box. I run my own private instance similar to how you're laying out, although I don't "disable" registration, I do require account approval by an admin. I'm sure at some point I'll start getting spammed with registrations, but thus far it hasn't been an issue. Similarly, federation can be disabled as a setting in the admin section of Lemmy.

What you're laying out on the technical side is also absolutely possible. Lemmy has a JS client you can pull in as a dependency to manage the calls from Javascript, but it also describes the HTTP endpoint for said call, so you could make your own calls using a separate back end service (ie: after your form submits it makes the call over to your Lemmy instance to register the user).

Here's the register call specifically, which registers a new user on your instance: https://join-lemmy.org/api/classes/LemmyHttp.html#register

However, note that they would still not be "approved" (if your instance required approval to join), but you could just make another API call to complete that process: https://join-lemmy.org/api/classes/LemmyHttp.html#approveRegistrationApplication

As another option, I haven't dove into the actual code, but to the best of my knowledge Lemmy is ultimately just interacting with a Postgres database, my guess is you could likely also connect to this database and directly insert the appropriate rows to create your new user accounts. Definitely use some caution with this approach though, ensure you're using separate database users with appropriate permissions for each application. Additionally, know that if you write code to handle custom user registrations by writing straight to the DB, you'll have to be mindful of updates to Lemmy that change the DB schema for that table will (most likely) break your registration script/code.

Lastly, if you really wanted to, you could just fork Lemmy itself! The UI and API are completely separate services, there's nothing stopping you from forking the UI part of the project and applying your own custom changes to the registration flow for your own instance!