This will be VERY technical, and if you’re reading this, you probably hit it because of a web search.
I like to document some of my technical challenges and solutions, partially for my own benefit, partially for others. I’ll do it publicly here if I had a lot of trouble finding good docs for it.
This particular implementation took me quite a lot to get working properly. The issue is that when you’re connecting a higher performance C++ app to a DB backend without using a mammoth library like Drogon or Boost, there’s not much documentation on how to establish proper C++20 coroutines that work with ASIO’s own coroutines.
Add in the fact that libpgxx is a synchrous library… and you can see where the issues arise!
The problem:
- ASIO coroutines (awaitable) use await_transform to whitelist what
you can co_await. Foreign awaitables are rejected when compiling. - libpqxx is synchronous and blocking. It cannot live directly on the ASIO executor.
- std::future is also foreign to ASIO’s await_transform.
The solution is to create an interface for your coroutines to use the DB and have the ASIO process return the data correctly to them.
It’s done in 3 layers:
-
dbPool (thread pool)
Owns n persistent pqxx::connection objects, one per worker thread.
submit() accepts a lambda, runs it on a worker, returns std::future.
The blocking happens here, isolated on worker threads where it belongs, not blocking the application. -
async_compose (this is how you bridge the gap)
Wrap the std::future in something ASIO recognizes as its own.
The initiating lambda spawns a detached thread to wait on the future, then calls self.complete() which posts resumption back onto the
ASIO executor. Coroutine resumes on the correct thread. -
db_await(ex, lambda) (the call side API)
Grabs the executor from the calling coroutine via:
auto ex = co_await asio::this_coro::executor;
Passes it to async_compose so resumption lands back on ASIO, returns an asio::awaitable which ASIO’s await_transform accepts.
Usage in any coroutine is pretty simple:
auto ex = co_await asio::this_coro::executor;
auto res = co_await db_await(ex, [](pqxx::connection& conn)
{
pqxx::work tx{conn};
return tx.exec("SELECT ...");
});
The db.h side looks like this:
dbPool& getPool();
template<typename Fn>
auto db_await(asio::any_io_executor ex, Fn&& fn)
{
using Ret = decltype(fn(std::declval<pqxx::connection&>()));
auto token = asio::use_awaitable_t<>{};
return asio::async_compose<asio::use_awaitable_t<>, void(std::exception_ptr, std::optional<Ret>)> (
[fut = getPool().submit(std::forward<Fn>(fn))]
(auto& self) mutable {
std::thread([self = std::move(self), fut = std::move(fut)]() mutable
{
try
{
auto res = fut.get();
self.complete(nullptr, std::move(res));
} catch (...) {
self.complete(std::current_exception(), std::nullopt);
}
}).detach();
},
token, ex
);
}
Which allows a pretty normal caller, like this:
asio::awaitable<bool> asyncPoolTest() {
auto ex = co_await asio::this_coro::executor;
try
{
auto res = co_await db_await(ex, [](pqxx::connection& conn)
{
pqxx::work tx{conn};
return tx.exec("SELECT 1;");
});
// co_return _res.size() etc
co_return true;
} catch (...) {
co_return false;
}
}
I’ll post updates later after I’ve done heavy hammering on this if I need to refine the connection pooling or scaling.