this post was submitted on 11 Feb 2024
19 points (100.0% liked)
Programming
16798 readers
227 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I haven't played around with the internals of C++ myself, but isn't that a one way thing? Wouldn't you need to be able to "unload" a query after you're done with it?
Personally I think child processes are the right approach for this. Launch a new process* for each query and it can (if you choose to go that route) dynamically load in compiled code. Exit when you're done, and the dynamically loaded code is gone. A side benefit of that is memory leaks are contained, since all memory you allocate is about to be removed anyway.
(*) On most operating systems launching new process is a bit slow, so you likely wouldn't want to do that when the query is requested. Instead you'd maintain a pool of processes that are running and ready to receive a query. That's how HTTP servers are often configured to run. The number of processes "pool" is generally limited by how much memory they need. Is it 1MB per process? 2GB?
Honestly, I wonder if you could just use an actual HTTP server for this? They can handle hundreds or even thousands of simultaneous requests. They can handle requests that complete in a fraction of a millisecond or ones that run for several hours. And they have good tools to catch/deal with code that segfaults, hits an endless loop, attempts to allocate terabytes of swap, etc. HTTP also has wonderful tools to load balance across multiple servers if you do need to scale to massive numbers of requests.
I would also seriously consider using JavaScript instead of C++. I hate JavaScript... but modern JavaScript JIT compilers are really special... they apply compiler optimisations AT RUNTIME. So a loop will compile to different machine code if it iterates three times vs three million times. The code is literally recompiled on the fly when the JIT compiler detects a tight loop. Same thing with a function that's called over and over again - it will be inlined if inlining is appropriate.
As flexible as your system sounds, I suspect runtime optimisations like that would provide real performance advantages. Well optimised C++ code is faster than JavaScript, but you're probably not always going to generate well optimised code.
JavaScript would also eliminate entire categories of security vulnerabilities. And any time you're generating code on the fly, you really need to be careful about those.
The good news is if you use a HTTP server like I suggested... then you can literally use any language you want, C++, JavaScript, Python, Rust... you can decide on a case by case basis.
I'd probably be fine with hundreds or thousands of these hanging in memory. I suspect the generated code for a single query would be in hundreds of kilobytes, maybe a megabyte. But yeah, this is one of those technical details I'd worry about.
Not sure how a HTTP server would solve the CPU bottleneck of scanning terabytes of data per query?