Using executors to unblock your shotgun calls

The Shotgun API is fast, but occasionally, it can hiccup and the user experience, particularly to tooling that has a GUI, plummets. The reason is that the Shotgun API runs synchronously, so each request "stops the world" until it is completed.

As Python developers asycio is never far from reach, but there is one small issue... QT and asyncio don't necessarily play well together straight out of the box and working with Shotgun often goes hand in hand with QT. You see, each brings its OWN event loop, so care must be taken to synchronize the loops, choosing one to drive the other. In fact, that exact approach is outlined on Pyside6's documentation for both trio and asyncio. This... well this is a pain. While it's not exactly difficult to set up, it is certainly not a great developer experience; requiring far too much understanding of the system to hop right in and use it. Seriously, go look at it before continuing.

Enter qasync.

We'll get to the code momentarily, and its python-ness will be evident quickly. However, let's look first to our original function call to bask in its simplicity:

# other stuff... now we're inside a MainWindow QWidget

    @slot
    def getProjects(self):
        self.projects.clear()
        result = self.sg.find("Project", [], ["name", "id"])

        if result and len(result):
            for i, project in enumerate(result):
                item = self.projects.addItem(project["name"])
                self.projects.setItemData(i, {"id":project["id"]})
                self.handleProjectChange(0)

This is a pretty straightforward API call/usage. When I initially wanted to take an asynchronous approach, I abandoned the approach shown by QT and instead created an event loop manager, that spun up an asyncio event loop (and then shut it down, enforcing the "1 event loop" at a time) for each API call. It worked quite well despite the unidiomatic usage of asyncio. For whatever reason, I didn't think to look for alternative, existing solutions for this problem at the moment, I coincidently came across qasync shortly thereafter.

Let's take a look at adding qasync (and asyncio) into the mix.

import asyncio
import qasync
from qasync import asyncSlot, asyncClose, QApplication

# other stuff... now we're inside a MainWindow QWidget

    @asyncSlot()
    async def getProjects(self):
        self.projects.clear()
        loop = asyncio.get_running_loop()
        result = await loop.run_in_executor(None, lambda: self.sg.find("Project", [], ["name", "id"]))

        if result and len(result):
            for i, project in enumerate(result):
                item = self.projects.addItem(project["name"])
                self.projects.setItemData(i, {"id":project["id"]})
                self.handleProjectChange(0)

# other stuff... now we're at the file entry point

if __name__ == "__main__":
    try:
        qasync.run(main())
    except asyncio.exceptions.CancelledError:
        sys.exit(0)

qasync gives us the asyncSlot and a modified QApplication while asyncio is used to take actions in the active loop. run_in_executor allows any function to run async in the event loop. In this case an anonymous lambda is used to wrap the function call.

Lastly, the main entry-point function is ran by calling it via qasync.run.

With very few changes, and in a way that feels incredibly python-y, you can bring asynchronous requests to your QT application. This I cannot emphasize enough, the code doesn't get "lost in the sauce", it remains readable and manages to bridge the QT and Python worlds in a way that manages to feel at-home in both.

Even when not using QT, asyncio's run_in_executor can bring the same benefits to Shotgun scripts or to any long running, opaque function, you might need to call.