Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Restartable Sequences (https://justine.lol)

253 points by grappler 6 days ago | 68 comments | View on ycombinator

GlenTheMachine 6 days ago |

If you had no idea what a restorable sequence is the takeaway is about halfway down the OP:

“This is why Linux now provides rseq() which is a much more enlightened solution. With restartable sequences, you actually can get rid of both the mutex and atomics, while the OS continues to fully abstract scheduling. The way it works is you advise the kernel whenever your program enters a critical section of code that you don't want interrupted. It's probably going to be maybe 10 assembly instructions tops. The first assembly opcode should be a move instruction that sets the rseq_cs field. The last instruction needs to be the thing that makes the modification to your global data structure. Think of it sort of like a really tiny database transaction. What makes it go fast, is that the bidirectional communication with the kernel happens via shared memory.”

khuey 6 days ago |

Maybe I'm just getting old but the "if you don't spend $20,000 on a workstation you're going to be left behind like a dinosaur" at the top of this article is a huge turn off to reading any further. And I say that as someone who owns a workstation with more cores than the author's.

senderista 6 days ago |

I'm surprised there was no reference to the librseq library, maintained by the rseq implementer:

https://github.com/compudj/librseq

This has helpers for common use cases like counters and linked lists. You shouldn't need to write assembly at all to use rseq in most applications.

Veserv 6 days ago |

Restartable windows, or more generically introspection windows, are a really useful technique you can apply in any situation where you understand or control the sources of preemption. The earliest uses of this technique in operating systems that I am aware of are ~25 years old.

The key insight is that the preempter can introspect the program counter of the code being preempted (which is now stable since it was preempted) and act accordingly. The simplest mechanism is to reset their program counter if in a critical section. The more generic mechanism is to jump them to a supplied address. This allows you to do things like hard abort and more.

You can further remove the need for the preempter to understand the preempted code by having the preempted code create a self-introspection code snippet and supplying that with the program counter at preemption. So the preempter just vectors them to their own code which knows how to interpret its own state at any preemption point.

squirrellous 6 days ago |

IIUC rseq is similar to thread-local data with the additional benefit that it scales with number of CPU cores, not threads. However if you are an application developer and is able to control all the threads in an application, then rseq isn’t that superior.

I fully agree that rseq should be more easily available to Linux developers, though.

smasher164 6 days ago |

I was having a conversation with someone recently if RSEQ would be a good primitive to build a load-link/store-conditional implementation in user-space. It gives you a critical window, though you still have to deal with spurious restarts, and provide a way for one core to abort another.

HackerThemAll 5 days ago |

The name is so misleading... The first thing I see when hearing "sequence" is the "arithmetic sequence", like 1,2,3,4. Therefore "restartable sequence" is like 1,2,3,4, 1,2,3,4, 1,2,3,4... Closer to SQL's "CREATE SEQUENCE" than "restartable sequence of assembly instructions". I could not comprehend how this can help with lock free data exchange. I've done my homework now.

yubblegum 6 days ago |

> chances are the CPU's internal mutexes aren't as good as the ones you've implemented in userspace

Anyone with an informed opinion on this statement? It's seems counter intuitive (npi).

1vuio0pswjnm7 5 days ago |

Was greenbean ever tested with NetBSD ftp

Something like

   greenbean
then

   ftp -vvd4o/dev/stdout http://127.0.0.1:8080
This is labeled as an error: "fragmented message"

Also why doesn't redbean do TLS1.3

And rusage "wall time" output seems to be wrong

Too much emphasis on Unicode for me, it's off-putting

I like consoles that do _not_ support UTF-8. At least, it should be optional

matheusmoreira 6 days ago |

This is amazing. I'll definitely use this in my projects.

NuclearPM 5 days ago |

People who say 10x programmers don’t exist have never heard of Justine.

dan_sbl 6 days ago |

[flagged]

keyle 6 days ago |

   ... bidirectional communication with the kernel happens via shared memory.
What could possibly go wrong?