Application/service that feeds tweets into spreadsheet automatically?

Digital Humanities Questions & Answers » Applications, Tools, Formats

Application/service that feeds tweets into spreadsheet automatically?

(9 posts) (7 voices)

Asked 5 years ago by Eric Johnson
Latest answer from Mark Sample
This question has a best answer.

Tags:

Eric Johnson
Member
For a project I'm working on, I'd like to be able to feed the tweetstreams of a couple of different accounts into spreadsheets so they can be sorted different ways. Does anybody know of an app or tool that does that on a live, rolling basis?

I've found a couple of descriptions of set-ups (like this one) that do pull feeds in, but as far as I can see they mostly seem to cap off at about 20 tweets and require that they be refreshed manually. Not the end of the world if that's the best that can be done, but not ideal.
Tweet this question
Posted 5 years ago Permalink
Tad Suiter
Member

Could something like ThinkUp either do that or be hacked into doing it?

Posted 5 years ago Permalink
Wayne Graham
Member
Best Answer

Short answer, I don't think you can do this without some programming.

The more complete answer is that you can actually get these updates through their Jabber/XMPP service. Essentially you would need to write a custom client that would subscribe to the account via Jabber, then write those updates to a CSV file (actually a database with a CSV view would be better).

Another tact to try is to subscribe to a twitter account with an IM client that logs the conversations, then paste that in to Excel when you want to see it. Down side is that this will only catch the tweets when the application is running.

Third option, twapper-keeper has an API that you could use to capture these tweets, then call and have a small script that converts the JSON response to CSV.

Again, a couple different ways to go...I'm thinking that option 3 gets you closest with the least amount of code written.

HTH,
Wayne

Posted 5 years ago Permalink
Dorothea Salo
Member

Yahoo Pipes might be able to do the Twitter --> RSS --> CSV conversion moderately well.

Posted 5 years ago Permalink
Julie Meloni
Admin

I've marked Wayne's answer as the best because it does speak to the various parts of the puzzle: there's an API you can access, and CSV files (or tab-delimited, or whatever) can be made. BUT, there's some stuff in between that has to happen. I wrote a blog post about it last year, but that info is outdated now (though the concept is similar). TwapperKeeper does do this (or setting up a polling/saving script to fire via cron on your own host would achieve the same thing) but still, programming in the mix.

Posted 5 years ago Permalink
Joe Gilbert
Admin

Replying to @Eric Johnson's post:

I haven't spent much time with this Windows-only tool, but it seems like a powerful Twitter archiving solution (with Excel export):
http://visitmix.com/labs/archivist-desktop/

Update: There's a web version, too! http://archivist.visitmix.com/

Posted 5 years ago Permalink
Julie Meloni
Admin

Replying to @Joe Gilbert's post:

Joe, how does this differ from what TwapperKeeper is doing? They both require setting up watches (e.g. from: [someone]) and don't have a combined feed out of the box, right?

Posted 5 years ago Permalink
Eric Johnson
Member

I'll probably try both Joe's Archivist suggestion first, and if necessary I'll toy some with Wayne's approach. I like TwapperKeeper, but they really only want you to archive individual people/organization accounts if you are that person/organization. I certainly understand the concern, but in this case I'm trying to follow a couple of institutions' accounts to look at patterns of tweeting. Thanks, everybody.

Posted 5 years ago Permalink
Mark Sample
Member

If you want to follow several different Twitter accounts, I'd try doing something with the RSS feeds for each of the accounts. You can dump each account's RSS into a Yahoo pipe, and output a combined json or xml file. You'd still have to manually dump into a spreadsheet though.

I've been using the Archivist since it was in alpha, and I've been quite happy with it as a local alternative to something like TwapperKeeper. It can only follow single accounts, but you can run multiple instances, each following a different account. The Archivist exports to CSV, so you can merge each account into a spreadsheet. Again, though, that's something you'd have to do manually.

Posted 5 years ago Permalink

RSS feed for this topic

Reply

You must log in to post.