twitter.cat

twitter search engine & crawler.

development

requirements

• postgres db with the required tables and indexes
• manticore instance with the required indexes
• bun installed
• a server accessible from the internet

client

1. set your server url in client/js/config.js
2. cd client
3. bunx serve

this should be deployed to your preferred cdn.

server

1. cd server
2. rename .env.example to .env and fill in your credentials
3. bun install & `bun run dev

crawling

in order for you to have any tweets to search, you'll need to manage crawlers that ingest tweet data to the postgres db, along with a sync script to push the data to manticore.

these crawlers are not open-source. the production twitter.cat instance uses many crawlers running in parallel on different machines.

biases

the current crawlers have the following biases:

• none (most of the index)
• community noted tweets (small amount of the index)
• popular tweets (small amount of the index)