Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

There's no single best way to store information (https://www.quantamagazine.org)

47 points by 7777777phil about 4 hours ago | 23 comments | View on ycombinator

akhil08agrawal 9 minutes ago |

This clicked for me in a way I didn't expect.

I've been thinking about trade-offs as "pick two of three" in the abstract, but the bookshelf example made it concrete. The insight that matters is: if you know your query patterns, you can optimize differently.

As a PM, I keep trying to build systems that work for "every case." But this article reminded me that's the wrong goal. The hash table works because it accepts the space-time trade-off. The heap works because it embraces disorder for non-priority items.

Sometimes the best system isn't the most elegant one—it's the one that matches how you'll actually use it.

Good reminder to stop over-optimizing for flexibility I'll never need.

Thanks for sharing.

bob1029 about 1 hour ago |

The best way to store information depends on how you intend to use (query) it.

The query itself represents information. If you can anticipate 100% of the ways in which you intend to query the information (no surprises), I'd argue there might be an ideal way to store it.

ronsor about 1 hour ago |

There are plenty of good enough ways:

* For lossless compression of generic data, gzip or zstd.

* For text, documentation, and information without fancy formatting, markdown, which is effectively a plain-text superset.

* For small datasets, blobs, objects, and what not, JSON.

* For larger datasets and durable storage, SQLite3.

Whenever there's text involved, use UTF-8. Whenever there's dates, use ISO8601 format (UTC timezone) or Unix timestamps.

Following these rules will keep you happy 80% of the time.

__MatrixMan__ about 2 hours ago |

There are, however, several objectively bad ways. In "Service Model" (a novel that I recommend) a certain collection of fools decides to sort bits by whether it's a 1 or a 0, ending up with a long list of 0's followed by a long list of 1's.

danans about 1 hour ago |

Pedantic, but the article is talking about the way we structure/organize information, not store it. When I think of the word store, I think of the physical medium. The way we organize the information is only partially related

eliasdejong 8 minutes ago |

See also, RUM Conjecture: https://www.codementor.io/@arpitbhayani/the-rum-conjecture-1...

Conceptually similar to CAP, but with storage trade-offs. The idea is you can only pick 2 out of 3.

notepad0x90 26 minutes ago |

would it be more accurate to say "to store using information, using information"? Since everything ultimately boils down to information, humans trying to store information is a bit recursive?

pbreit about 3 hours ago |

Postgres is close.

andix about 1 hour ago |

It's always Markdown. Markdown is the best way to store information. ;)

kittikitti about 1 hour ago |

Or it's the opposite, where the slowest possible retrieval time is the intended effect, as is the basis of many cryptographic algorithms.