Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

pg_durable: Microsoft open sources in-database durable execution (https://github.com)

222 points by coffeemug about 5 hours ago | 54 comments | View on ycombinator

levkk about 5 hours ago |

2026 is the year of the Postgres queue! (DBOS[0], pgQue[1]) It's awesome that the community is contributing this and giving us the option to use it.

As an ex-app engineer though, I kind of prefer my queue logic to be in code, in Git, but maybe with the right tooling, you can change my mind. :)

[0]: https://www.dbos.dev/

[1]: https://github.com/NikolayS/pgque

steno132 37 minutes ago |

I would argue that for all but the largest tech companies you only need a single data system which is Postgres. Message brokers, analytical databases all can be built on Postgres. Unfortunately, Postgres as it's built now lacks any semblence of extensibility which makes this impossible in practice.

I would propose a rewrite of Postgres in another language like Rust, introducing a pluggable application layer on top. While ambitious in scope I think it would be helpful and even necessary.

jraedisch about 5 hours ago |

If understanding correctly, Absurd (by the Pi LLM harness devs) minimizes the pure db approach as much as possible. I only just started getting into the topic myself, though.

https://github.com/earendil-works/absurd

linuxhiker 15 minutes ago |

Hopefully they will start sponsoring PGRX now that they are so publicly using it.

kilobaud about 5 hours ago |

> When not to use it > … > The workflow mostly lives outside Postgres and spans many heterogeneous systems.

How is this project at all comparable to something like Temporal? Am I misunderstanding the limitation implied by this particular recommendation?

CharlieDigital about 2 hours ago |

A few things are not clear to me from reading through docs and examples:

    df.wait_for_schedule()
How does this call work? Is it idempotent if I call it from an application? If I run it 2x with the same parameters, does it double tick? Am I invoking this manually from a query console to only do this one time? Am I running this as part of a migration script?

For this[0]:

    -- Wait for human signal (5 minute timeout)
    ~> (df.wait_for_signal('approval', 300) |=> 'sig')

    ~> df.if(
        $$SELECT NOT ($sig::jsonb->>'timed_out')::boolean
            AND ($sig::jsonb->'data'->>'approved')::boolean$$,
Is the `timed_out` a fixed constant that is returned on timeout?

Also not immediately clear: how to handle errors/exceptions?

[0] https://github.com/microsoft/pg_durable/blob/main/examples/i...

TuringNYC about 4 hours ago |

I'm trapped on Azure at work and we're constantly waiting for Azure pg to catch up with modernity.

For example, you cant use this: https://www.paradedb.com/blog/hybrid-search-in-postgresql-th...

Also for example, you dont get ultra-wide high dimensionality vectors.

It is nice they are open sourcing pg_durable, but how about adopting table stakes I'd get with AWS?

faxmeyourcode about 5 hours ago |

This feels like the wrong solution to an age old problem solved by the DAG schedulers like Apache Airflow for a while now.

Why would I want to store my control flow in the database and not in code? It feels strange.

Not trying to dismiss the project, I'm just not getting it yet I think.

oa335 about 5 hours ago |

Can anyone explain why I would want to use this over an orchestration tool that lives outside the DB? Read through the Readme and some of the examples, I still don't get it.

joelthelion about 4 hours ago |

Isn't the database already one of the hardest piece of infras to scale? Why would you want to load it with additional long-running jobs?

rastignack about 4 hours ago |

I hope it could be used in the future to export pg_dump formated exports to s3.

One would be able to trigger maintenance jobs via simple lambda functions whose duration is capped.

mikey_p about 4 hours ago |

Is this an open sourcing of something they use internally? My first thought on durable jobs was GHA aka Azure Devops.

redmonduser about 4 hours ago |

Seems like an interesting idea to add durability and resumability to lengthy cron jobs.

cpursley about 5 hours ago |

Looks pretty good but I wonder why they didn’t build it on pgmq? If you’re on elixir I maintain a DAG package around this (based on and compatible with pgflow.dev which is TS/Deno).

https://github.com/agoodway/pgflow