Hister builds a personal search index from pages you visit, bookmarks, browser history, local files, and crawled websites. It stores extracted content with offline result previews, so information remains searchable even when the original page changes or disappears. It supports full text and semantic search, can run entirely on your own machine, and includes a web interface, command line tools, and an MCP endpoint for assistant integrations.

Website: https://hister.org/

Tiny read-only demo: https://demo.hister.org/

Ps.: It looks like our name conflicts with a registered trademark in the US. The owner of the other project has asked us to change it, so we’ll probably need to comply sooner or later.

Name suggestions are welcome! Ideally, the new name should be relatively short, sound good, and have an available .org domain.

Thanks!

I have it up on GitHub, but I don't think anyone should use my implementation.

Loosely, what I built:

* On each of my machines I have a cron job running that looks at all my web browser history (usualy it's inspecting the brower's SQLlite across firefox and chrome). If it matches my rule list: hacker news stories, certain reddits, etc. it'll grab the page, convert to markdown and drop in my Obsidian Vault incoming.

* It has a whole de-duping architecture since I might open the same page on multiple machines. Uses the CloudFlare SQLITE D1 storage for tracking the processed links.

* it'll then trigger the LLM to do some Karpathy wiki style taxonomy assignment to the articles, organize them, create an index etc.

It's then available for my "bot" stuff to do writings for me.... I will probably write more about it at some point. I'm not certain it's totally useful and not just a yak-shave on hoarding knowledge.

Ai-drafted article on this [1]

Example AI-Drafted article based on some discussions the other day on Ollma vs LLama.cpp [2]

[1] https://taude.xyz/posts/how-archivore-turns-browsing-into-a-...

[2] https://taude.xyz/posts/skip-ollama-run-llama-cpp-directly-o...

Nobody seems to remember it, even though it was a headline feature. Was removed in 2013, I think due to technical constraints.

Will definitely try this.

How do other people handle this dilemma?

Even solution I can think of involves are a great amount of extra work.

I built myself a little extension last year that tracks what information I was looking at, but focused on generating "new info" recaps for the day / week.

I realized that I open / quick view a lot of pages and close them, which is a strong signal that I don't care about that specific page, and it shouldn't be a source of "new insights" that I learnt that day (since I probably don't care about that topic).

I'd love to re-try a simpler version of that project that builds on Hister as a backend actually.

For person using resource-constrained computers where CPU, memory and storage space is limited

URLs from the local forward proxy log are extracted periodically and stored in compressed files (URL logs)

(I also store post-data)

The compression method used is old and unpopular: recursive pairing

Compression ratio is better than gzip but worse than zstd, compression/decompression speed better than zstd but worse than gzip

More recently a method was developed to search these compressed files

Size of compression utility: 42.3K static binary

Size of search utility: 102.4K static binary

No Java

Limitations include basic regex only (no back-references) and files must be line-oriented

No decompression step is needed. IME, this search is very fast. If it is slow then this means the keyword is too common: refine the search

With minor modification (insert a newline at the top of file) I can also search inside compressed tar files

As a www user with underpowered computers doing relatively small jobs, these old, unpopular methods have proven to be fast and reliable for me

When I'm searching more than just URL strings, e.g., dates, titles, etc., I reformat the data into SQL and store it in a text file

Instead of storing large SQL database files, I store the text file compressed with recusrive pairing

I can then search the compressed SQL using basic regex; the output is piped into sqlite3 to create a "results" SQL database, e.g., in memory

For me, the speed of sqlite3 in creating relatively small databases is excellent

Then I can query the "results.db" using SQL

(For file search, I use voidtools' Everything, and I'm not sure why anyone other than Microsoft would want to mix local file results and web results)

This project looks cool, though I misread the name as "Hipster" initially [sips almond milk macha coffee]

https://news.ycombinator.com/item?id=49351802

Can I add NOTES about pages? This might be a good spot to do that...? Maybe the interface can be in a web page instead of terminal?

Before Google took off there was a vibrant ecosystem of FOSS dev around search, all different little aspects of it. Then after Google people stopped fiddling with search, search became "solved" or maybe "must be coded by the big boys". Shame.

Thank you for this, looooong time coming

I also have it index my Obsidian notes, which is another little bonus for global search.

I did need to build up quite a few exclusion rules early on, but it's been hands off since.

I like the search ui. my projects become functional but never polished. https://github.com/sbeckeriv/memoir

[1] https://git.brucewillis.sexy/~tombert/fs_index I promise, safe for work, despite the URL.

Is there any site/project that works as a fully customizable personal front-end to all other SERPs?

When I search for something, I always want a link to the best Wikipedia result. This should always be in the same place and have a giant icon/picture.

Then there could be easily clickable links to the SERP pages for Google, DDG, etc. for that query.

A big link to route it to your favorite LLM.

Seems like you could have a really useful "homepage" for all searches that sat in front of all the other sites. It could be local only and would not require indexing the web. Also wouldn't be a files search thing, as Hister appears to be.

Every single web page I’ve found interesting, since the advent of the Web, I have printed to PDF and stored locally for my own personal reference.

Something like 80,000+ files - my own copy of my own Internet - indexable, searchable.

Available offline. Something to read when I am far out to sea.

There is no need to involve third parties in your Internet history - no matter how trustworthy they seem to want to appear.

Print to PDF, and you’ve got everything you need, safe and sound.

Does Hister handle this well? If not, can anyone suggest other options?

One of the most useful use cases for me, especially since I work in GTM, is keeping track of new ways to get replies from prospects, whether through cold outbound or things like SEO/GEO optimization. I read at least 2-3 articles a day on this, and it genuinely helps me figure out which ideas are worth trying because I can now keep track of everything I’ve read.

Running this would be nice but it still takes time and still there is no ROI for me.

Of course there will be people who find it useful but I am pretty much done with building knowledge bases or having todo lists.

Stuff that I need to do or remember - everything else if I forget nothing happens and it doesn’t impact my life or work.

https://www.history.co.uk/articles/did-nostradamus-really-pr...

https://stashpad.ai/

It's badly needed, and so far it's working well for me.