Wayback Tweets is moving to the command line

Some updates on the project...


It's been a while since I've been planning to develop a more comprehensive and robust application for Wayback Tweets, as the project has had a good reception. We're already reaching nearly 14k unique accesses, and I found a request for help on Reddit to use the tool, so it's time to evolve.

The Python-written prototype with the Streamlit framework will remain online, and I will only provide maintenance if necessary; it will continue to be hosted on Streamlit Cloud.

I've written a plan for the new development, but I haven't made a schedule, so everything will be done gradually, in my spare time. In conversation with the historian who made the posts on Reddit, some ideas came up, and making the tool a kind of framework for archiving tweets seemed an interesting way.

Currently, Wayback Tweets is only a viewer, but I want to make it more comprehensive, so I'm moving to the command line.

What I intend to develop:

DOWNLOAD ARCHIVED TWEETS

Retrieve all tweets in the Wayback Machine.

A file containing:

  • Wayback Machine URL
  • Original tweet URL
  • Tweet text
  • Images, if the tweet contains any
  • MIME type
  • Date it was saved on Wayback Machine

SCRAPE AND ARCHIVE

  • Download the IDs of existing tweets from X accounts
  • Insert the tweet links into the Wayback Machine to archive them

OFFLINE VIEWING

  • Markdown generator

BACKLOG

  • Implement other web archiving services (such as MementoWeb API)

For each of the items above, I will create a new release, making it easier to test the tool. After a few months, we will have a complete tweet archiving framework that allows retrieving archived tweets, downloading tweet IDs (I don't promise this functionality), uploading links to the Wayback Machine (I still need to do a lot of testing), and viewing them on a static website.

I have a lot of work ahead of me.

Cheers.


by Claromes