Skip to content

Handling forked processes with universalasync decorators #25

Description

@cheshirex

I just debugged a case where a user of a module I maintain which uses universalasync had combined it with multiprocessing and caused a hang. I figure the case is atypical, but worth bringing up.

Combining multiprocessing with async doesn't normally make sense, but I'm using universalasync to maintain a library that provides service to both sync and async code, and in this particular case, had sync code running with a ProcessPool, and the called function was in turn calling another function using async_to_sync_wraps. The result was a hang, because the fork still picks up the event loop from the parent process, but that event loop isn't valid in the subprocess.

I've got a local hack fixing this in my own use case now, but the AI-suggested fix within universalasync would be:

_LOOP_PID: int = os.getpid()

def get_event_loop() -> asyncio.AbstractEventLoop:
    global _LOOP_PID
    loop = _get_event_loop()
    current_pid = os.getpid()
    if loop.is_closed() or current_pid != _LOOP_PID:
        _LOOP_PID = current_pid
        return _create_new_event_loop()
    return loop

i.e. check the current process ID against whatever process first entered the code, and if they don't match, we're in a forked process, so get a new event loop and save the process ID in memory for the current process.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions