Publish Your Tweets

A theme in nature documentaries is that one creature’s death is new life for others.

I recently followed another dead-end Twitter link and realized anyone with a website could self-host their tweet archive and make them available however they want. I made a weekend project out of publishing mine.

They’re your words and they always were. Why not own where they live?

What if we all did?

I generated a Twitter account export in late 2022 just before I deleted it. I probably owe the ability to GDPR, and it was a convenient, simple thing to do. The archive is a ZIP file with a bunch of JavaScript, media assets, and a nice HTML page for browsing everything.

I opened the HTML page again and wondered what it would take to re-expose the internet to so many years of unnecessary thoughts.

It didn’t take much, and it was a great way to spend more time getting comfortable with TypeScript.¹

I created a new .astro page on my site and used the twitter-archive-reader package to read the ZIP file and give back an archive class with everything parsed and ready to browse—including TypeScript definitions to make it fun.

import TwitterArchive from "twitter-archive-reader"

const archive = new TwitterArchive("my-archive.zip")

await archive.ready()

You don’t need to use a library to do this, especially if you’re only after the tweets. (There’s much more in the archive!) Your ZIP file includes data/tweets.js, which is an almost a perfect JSON blob except for the very beginning:

window.YTD.tweets.part0 = [
// ...

Remove window.YTD.tweets.part0 = and you’ve got ~~a stew going~~ JSON you can import and play with.

It’s an array of objects that represent tweets. Here’s one I’ve purposefully selected because it has an image in it:

{
  "tweet": {
    "edit_info": {
      "initial": {
        "editTweetIds": ["691695276218683392"],
        "editableUntil": "2016-01-25T19:22:59.584Z",
        "editsRemaining": "5",
        "isEditEligible": true
      }
    },
    "retweeted": false,
    "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
    "entities": {
      "user_mentions": [],
      "urls": [],
      "symbols": [],
      "media": [
        {
          "expanded_url": "https://twitter.com/mattrambles/status/691695276218683392/photo/1",
          "indices": ["77", "100"],
          "url": "https://t.co/FkWk3HumW1",
          "media_url": "http://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
          "id_str": "691695275723743232",
          "id": "691695275723743232",
          "media_url_https": "https://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
          "sizes": {
            "medium": {
              "w": "303",
              "h": "305",
              "resize": "fit"
            },
            "small": {
              "w": "303",
              "h": "305",
              "resize": "fit"
            },
            "thumb": {
              "w": "150",
              "h": "150",
              "resize": "crop"
            },
            "large": {
              "w": "303",
              "h": "305",
              "resize": "fit"
            }
          },
          "type": "photo",
          "display_url": "pic.twitter.com/FkWk3HumW1"
        }
      ],
      "hashtags": []
    },
    "display_text_range": ["0", "100"],
    "favorite_count": "0",
    "id_str": "691695276218683392",
    "truncated": false,
    "retweet_count": "0",
    "id": "691695276218683392",
    "possibly_sensitive": false,
    "created_at": "Mon Jan 25 18:52:59 +0000 2016",
    "favorited": false,
    "full_text": "At what point do higher-than-normal call volumes simply become normal again? https://t.co/FkWk3HumW1",
    "lang": "en",
    "extended_entities": {
      "media": [
        {
          "expanded_url": "https://twitter.com/mattrambles/status/691695276218683392/photo/1",
          "indices": ["77", "100"],
          "url": "https://t.co/FkWk3HumW1",
          "media_url": "http://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
          "id_str": "691695275723743232",
          "id": "691695275723743232",
          "media_url_https": "https://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
          "sizes": {
            "medium": {
              "w": "303",
              "h": "305",
              "resize": "fit"
            },
            "small": {
              "w": "303",
              "h": "305",
              "resize": "fit"
            },
            "thumb": {
              "w": "150",
              "h": "150",
              "resize": "crop"
            },
            "large": {
              "w": "303",
              "h": "305",
              "resize": "fit"
            }
          },
          "type": "photo",
          "display_url": "pic.twitter.com/FkWk3HumW1"
        }
      ]
    }
  }
}

While the library was handy and simple to work with, it seemed excessive to commit the ZIP archive into my Astro project. I used the archive reader package temporarily to spit out JSON exactly how I wanted it.

Keeping a JSON blob around as a data source is better than a one-and-done HTML page because I can separate content from presentation with clean stylistic control, the ability to add stats or little features, and better odds of surviving a replatform or redesign.

The result for me is a directory of images and video attachments that came straight from the ZIP file (data/tweets_media/), along with my pared-down JSON where the same tweet looks like this:

{
  "id": "691695276218683392",
  "body": "At what point do higher-than-normal call volumes simply become normal again? ",
  "media": [
    {
      "kind": "image",
      "src": "/images/twitter/media/691695276218683392-CZllMEJUYAAFsML.png"
    }
  ],
  "created": "Mon Jan 25 18:52:59 +0000 2016",
  "type": "tweet",
  "retweets": 0,
  "favorites": 0
}

I only a cheated a little bit with some in-between cleanup tasks:

Limited image and video references to those I actually have locally.
Pulled image and video references out of the tweet content and exclusively into a simpler media array.
Un-shortened URLs with tall.
Made up my own type property (tweet, retweet, or reply) for easier filtering.
Downloaded and replaced the handful of Twitpic images I had.

I’m loving TypeScript (with somebody else’s tsconfig.json), and my cleaned-up data uses these little types:

src/types.ts

type TweetImage = {
  kind: "image"
  src: string
}

type TweetVideo = {
  kind: "video"
  src: string
  contentType: string
  gif: boolean
}

type Tweet = {
  id: string
  body: string
  media: Array<TweetImage | TweetVideo>
  created: string
  type: "tweet" | "retweet" | "reply"
  retweets: number
  favorites: number
}

The page loads up the JSON, limits the data to tweets (no replies or retweets), and dumps out an obscenely-long pile of markup:

src/pages/twitter.astro

---
import Breadcrumb from "@components/Breadcrumb.astro"
import Layout from "@layouts/Layout.astro"
import PostBody from "@components/PostBody.astro"
import PostTitle from "@components/PostTitle.astro"
import tweetData from "src/data/tweets.json"
import Tweet from "@components/Tweet.astro"
const tweets: Tweet[] = tweetData as Tweet[]
const pageTitle = "Twitter Archive"
---
<Layout title={pageTitle} description={`Matt’s 2008–2022 tweet archive.`}>
  <Breadcrumb items={[{ label: pageTitle, current: true }]} />
  <main>
    <PostTitle title={pageTitle} />
    <p class="text-xl my-12">
      Arrived in 2008, said some forgettable things, and left in 2022.
    </p>
    <div class="wrapper">
      {
        tweets
          .filter((tweet: Tweet) => {
            return tweet.type == "tweet"
          })
          .map((tweet: Tweet) => {
            return (
              <Tweet
                data={tweet}
                name="Matt Stein"
                handle="mattrambles"
                avatarUrl="/images/twitter/avatar.jpg"
              />
            )
          })
      }
    </div>
    <PostBody />
  </main>
</Layout>

The Tweet component renders each random thought:

src/components/Tweet.astro

---
import { formatDate } from "../utils/content"
import {
  ArrowPathRoundedSquareIcon,
  HeartIcon
} from "@heroicons/vue/24/outline/index.js"
export interface Props {
  data: Tweet
  name: string
  handle: string
  avatarUrl: string
}
const {
  data: {
    id,
    created,
    body,
    media,
    retweets,
    favorites
  },
  name,
  handle,
  avatarUrl
} = Astro.props
---
<div
  id={`t` + id}
  class={`tweet flex my-12 space-x-3 max-w-xl overflow-x-scroll`}
>
  <div class="avatar flex-none w-12 h-12 rounded-full overflow-hidden">
    <img src={avatarUrl} alt="" />
  </div>
  <div class="content -mt-1">
    <div class="top">
      <b>{name}</b>{" "}
      <span>
        <span class="opacity-50">@{handle}</span>
        <span class="opacity-25 hidden sm:inline"> •</span>
        <span class="opacity-50 block mb-3 sm:mb-0 sm:inline"
          >{formatDate(created)}</span
        >
      </span>
    </div>
    <div class="body">
      <div set:html={body} />
      {
        media.map((mediaItem) => {
          return (
            <>
              {mediaItem.kind == "image" && (
                <img
                  src={mediaItem.src}
                  class="block rounded-xl mt-2 shadow-sm max-w-full"
                  loading="lazy"
                />
              )}
              {mediaItem.kind == "video" && (
                <video
                  class={`rounded-xl mt-2 shadow-sm`}
                  autoplay={mediaItem.gif}
                  muted={mediaItem.gif}
                  loop={mediaItem.gif}
                  controls={!mediaItem.gif}
                >
                  <source src={mediaItem.src} type={mediaItem.contentType} />
                </video>
              )}
            </>
          )
        })
      }
    </div>
    <div class="meta mt-3 flex text-sm opacity-75">
      <div class="retweets">
        <ArrowPathRoundedSquareIcon
          class="w-4 h-4 inline-block -mt-0.5 mr-0.5"
        />
        {retweets}
      </div>
      <div class="favorites ml-4">
        <HeartIcon class="w-4 h-4 inline-block -mt-0.5 mr-0.5" />
        {favorites}
      </div>
    </div>
  </div>
</div>

There’s plenty of room here to come back and add stuff like stats, filters, civilized lazyloading, and navigation. You could surely do something cooler!

That’s the point: you can do whatever you want when it’s your data.

I spent hours making a cleaned-up example repository for you, but couldn’t manage to configure TypeScript to run it with the handful of imports I used. If anybody can help me iron it out I’d be happy to publish a working example. (Everything ran in an Astro page just fine.) ↩

Publish Your Tweets

Footnotes