Digital Humanities Questions & Answers » User Favorites: johnlaudun

Digital Humanities Questions & Answers » User Favorites: johnlaudun johnlaudun Digital Humanities Questions & Answers » User Favorites: johnlaudun en-US Wed, 04 May 2016 03:27:53 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://digitalhumanities.org/answers/search.php johnlaudun on "Which novels to assign for seminar in algorithmic criticism / distant reading?" http://digitalhumanities.org/answers/topic/which-novels-to-assign-for-seminar-in-algorithmic-criticism-distant-reading#post-2330 Wed, 20 May 2015 17:26:12 +0000 johnlaudun 2330@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/ullyot'>ullyot</a>'s <a href="http://digitalhumanities.org/answers/topic/which-novels-to-assign-for-seminar-in-algorithmic-criticism-distant-reading#post-2329">post</a>: Sure. Right now I have no travel plans for July. When you're ready, drop me a note. I'm going to be working on the repo some in June, and I'm making a note to add the PDF. Part of my plan is also to turn chunks of it into an iPython notebook (now Jupyter) which would allow there to be a mix of exposition (written in Markdown) with code that can be executed live. (I have a pretty complete run-down on how to set all this up on a Mac, if anyone is interested.) ullyot on "Which novels to assign for seminar in algorithmic criticism / distant reading?" http://digitalhumanities.org/answers/topic/which-novels-to-assign-for-seminar-in-algorithmic-criticism-distant-reading#post-2329 Wed, 20 May 2015 16:14:43 +0000 ullyot 2329@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/johnlaudun'>johnlaudun</a>'s <a href="http://digitalhumanities.org/answers/topic/which-novels-to-assign-for-seminar-in-algorithmic-criticism-distant-reading#post-2328">post</a>: Thanks very much, John! I'd love to review the PDF version, too, and talk to you about exercises. Perhaps in July when I start developing the course? You can reach me at ullyot[@]ucalgary[.]ca or @<a href='/profile/ullyot'>ullyot</a> on Twitter. johnlaudun on "Which novels to assign for seminar in algorithmic criticism / distant reading?" http://digitalhumanities.org/answers/topic/which-novels-to-assign-for-seminar-in-algorithmic-criticism-distant-reading#post-2328 Fri, 15 May 2015 16:39:41 +0000 johnlaudun 2328@http://digitalhumanities.org/answers/ If you want something intermediate between Codecademy's Python tutorials and Jockers' R, as well as a small text through which you can work methodically with your students, I've got a small collection of commented Python scripts in a GitHub repository. It comes with a plain text version of "The Most Dangerous Game." I can also include a PDF version I've long used, which gives you a paginated source for comparison. If you're interested, I can also add notes on the exercises/activities that I use in conjunction with the text. (Essentially, I use it as a way to get students thinking about both the advantages of working through the particularities of texts computationally -- e.g., KWiC -- as well as some of the underlying statistics behind things like topic modeling. Here's the repo: <a href="https://github.com/johnlaudun/upst" rel="nofollow">https://github.com/johnlaudun/upst</a> ullyot on "Which novels to assign for seminar in algorithmic criticism / distant reading?" http://digitalhumanities.org/answers/topic/which-novels-to-assign-for-seminar-in-algorithmic-criticism-distant-reading#post-2327 Fri, 15 May 2015 13:29:58 +0000 ullyot 2327@http://digitalhumanities.org/answers/ I’m designing the graduate seminar I’ll teach in the Department of English this fall (2015) on the subject of ‘Algorithmic Criticism,’ a title I took from the subtitle of Stephen Ramsay’s 2011 book, Reading Machines. It’s an introduction to computational text-analysis for students of literature, from word frequency to topic modelling. By the end of the course, students will be comfortable moving between close reading and distant reading, or what Matthew Jockers calls micro-, meso-, and macro-analysis. (Along with Ramsay’s book, Jockers’ 2013 study Macroanalysis and his 2014 guide to Text Analysis with R for Students of Literature will be required readings.) Students will learn and implement some programming basics using Python and R, so they can see what happens when natural-language processing and other tools parse and rearrange the words in both individual texts and larger corpora. I haven’t developed more detailed course outcomes than that. We’ll use Codecademy’s Python tutorials alongside Jockers’ book on R. So which literary texts do you assign for old-fashioned linear close readings in a course like this? They should be long enough to have a lot of words to work with, and complex enough that they contain a lot of topics. They should provide good contrasts with each other – that is, contain a lot of different words and topics – yet be close enough in time that the comparison makes sense. And they should be in the public domain, so we have texts to manipulate in whatever repository we’re drawing them from. Ben Marwick on "Topic Modeling (MALLET) with JSTOR Data For Research" http://digitalhumanities.org/answers/topic/topic-modeling-mallet-with-jstor-data-for-research#post-2142 Thu, 30 Jan 2014 19:50:12 +0000 Ben Marwick 2142@http://digitalhumanities.org/answers/ Replying to @Ben Marwick's <a href="http://digitalhumanities.org/answers/topic/topic-modeling-mallet-with-jstor-data-for-research#post-1842">post</a>: Just a short follow-up, I have now bundled my snippets into a more complete R package for working with JSTOR DFR data. It takes DFR output and does ngrams, word correlations over time, document clustering and topic modelling (with MALLET or in R, and inlcuding hot and cold topic identification): <a href="https://github.com/UW-ARCHY-textual-macroanalysis-lab/JSTORr" rel="nofollow">https://github.com/UW-ARCHY-textual-macroanalysis-lab/JSTORr</a> Scott Weingart on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1993 Sun, 28 Apr 2013 23:09:18 +0000 Scott Weingart 1993@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/johnlaudun'>johnlaudun</a>'s <a href="http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1992">post</a>: Ah, sorry that I neglected to mention the header row! What you're seeing in the first picture is actually what you're looking for (a document-document network based on shared words), but the word nodes were not removed. Preprocessing->Delete Isolates can do that. Also, if that was the bibliographic coupling, than the other algorithm will have the word-word network based on shared documents. As for getting the graph into gephi, try saving the network out as graphml, renaming the file from .xml to .graphml, and then loading it into gephi. In any case, because there are likely many words per document, you might do well to threshold above a certain value (filtering in gephi or preprocessing->extract edges above or below value in sci2) to make the network a bit more sparse. johnlaudun on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1992 Sun, 28 Apr 2013 22:49:40 +0000 johnlaudun 1992@http://digitalhumanities.org/answers/ Actually you said it quite well the first time: <blockquote> once you have a csv with titles in one column and text in the second </blockquote> But apparently my reading comprehension is not up to par! Here's a brief summary of travails: <ul> <li> Sci2 expects CSVs to have a header row. This is important. It will not process the first row in a CSV and it will lead to awkwardly named outputs. (This took me a surprisingly long time to puzzle out.) </li> <li> I was glad to figure out the format of the file, because it also means I could potentially do my own stopwording, lemmatizing, etc. The menu item here is a wonderful one-stop shop, but I don't know that I always want everything it brings to the table.</li> <li> I got the directed network, no problem, but a visualization (using GUESS because I couldn't get the Gephi hand-off to work) reveals that the projection isn't really what one would expect:</li> </ul> <a href="http://www.flickr.com/photos/johnlaudun/8691631906/" title="Screen Shot 2013-04-28 at 18.04.38 by johnlaudun, on Flickr"><img src="http://farm8.staticflickr.com/7054/8691631906_8554b3905e_z.jpg" width="640" height="424" alt="Screen Shot 2013-04-28 at 18.04.38"></a> As I note on Flickr -- if this image makes it through -- what should have been either a word-to-word projection or a text-to-text projection based on a text-to-word CSV is in face a rather large visualization in which the sixteen texts sit in a tight ball in the middle of an oort cloud of words. To be sure, I grabbed a screen capture of the Sci2 Data Manager pane, which does a lovely job of capturing at a glance what I had done: <a href="http://www.flickr.com/photos/johnlaudun/8690511907/" title="Screen Shot 2013-04-28 at 18.12.46 by johnlaudun, on Flickr"><img src="http://farm8.staticflickr.com/7055/8690511907_c97c81dcc9_z.jpg" width="501" height="150" alt="Screen Shot 2013-04-28 at 18.12.46"></a> The CSV should work: <pre> TEXT,WORDS ancelet-88,in|charenton|north|o ancelet-89,went|meet|old|man|ma ancelet-90,mom|said|use|dig|lot ancelet-91,know|jess|venabl|fat laudun-1,yeah|s|like|talk|shit| laudun-2,like|said|famili|weird </pre> I'm going to keep playing with the various options to see what else might work. It's been a rough day -- also trying to get an 8 year old to step through a book presentation. That'll drive you nuts, right there. Kevin Hawkins on "What exists as an open platform for editorial management?" http://digitalhumanities.org/answers/topic/what-exists-as-an-open-platform-for-editorial-management#post-1990 Sat, 27 Apr 2013 15:37:44 +0000 Kevin Hawkins 1990@http://digitalhumanities.org/answers/ In fact, when configuring a journal in OJS, there's an option to say that the OJS is used only for submissions and editorial management but not for actually publishing the journal. This prevents the "current issue" and "archives" links from appearing in the interface, confusing readers. See <a href="http://journal.tei-c.org/" rel="nofollow">http://journal.tei-c.org/</a> for an example of an OJS instance that is used for managing submissions but not for actually publishing issues. Here is a list of things that at one point or another struck me as covering editorial management. Many are not open-source, but I'll let you sort through for anything of interest: <ul> <li><a href="http://www.bepress.com/edikit.html">EdiKit</a> (being rebranded as "Digital Commons Publishing")</li> <li><a href="http://highwire.org/publishers/benchpress.dtl">Bench>Press</a></li> <li><a href="http://www.epress.ac.uk/">epress</a></li> <li><a href="http://www.editorialmanager.com/homepage/home.htm">Editorial Manager and Preprint Manager</a></li> <li><a href="http://www.subs-espere.org/">Espere</a></li> <li><a href="http://library.queensu.ca/webir/planning/e-journal_publishing_support.htm">http://library.queensu.ca/webir/planning/e-journal_publishing_support.htm</a></li> <li><a href="https://jscholarship.library.jhu.edu/handle/1774.2/32737?show=full">A Survey and Evaluation of Open-Source Electronic Publishing Systems</a></li> <li><a href="http://www.ejpress.com/">eJournal Press</a></li> <li><a href="http://scholarone.com/">ScholarOne</a></li> <li><a href="http://www.editorialmanager.com/homepage/home.htm">Aries Editorial Manager</a></li> </ul> Scott Weingart on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1989 Fri, 26 Apr 2013 15:23:56 +0000 Scott Weingart 1989@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/johnlaudun'>johnlaudun</a>'s <a href="http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1988">post</a>: I think maybe I didn't explain what I meant well enough the first time, sorry! If you have a csv that looks like <pre> title1, fulltext1 title2, fulltext2 title3, fulltext3 </pre> Doing the steps I described above will get you <pre> t1, w1 t1, w2 t1, w3 t2, w1 t2, w3 t3, w2 t3, w3 t3, w4 </pre> without any additional scripting. Hope it helps! johnlaudun on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1988 Fri, 26 Apr 2013 15:19:09 +0000 johnlaudun 1988@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/johnlaudun'>johnlaudun</a>'s <a href="http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1987">post</a>: What I'm working on is a script that goes from scraping out word frequencies to changing the csv to simply be the name-of-the-text, word, like this: <pre> text1, word1 text1, word2 text1, word3 </pre> I can then join those CSVs, to produce a CSV that looks like so: <pre> t1, w1 t1, w2 t1, w3 t2, w1 t2, w3 t3, w2 t3, w3 t3, w4 and so on ... </pre> I'm going to try that out in Sci2. johnlaudun on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1987 Fri, 26 Apr 2013 14:52:15 +0000 johnlaudun 1987@http://digitalhumanities.org/answers/ Replying to @Scott Weingart's <a href="http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1986">post</a>: Hey, Scott, thanks for the replies. I will try them out, but first I have to get to the CSV of the bimodal network! Any ideas on how to get there from a collection of plain texts? It's this bit that has stumped me on a couple of occasions. Scott Weingart on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1986 Fri, 26 Apr 2013 12:42:19 +0000 Scott Weingart 1986@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/johnlaudun'>johnlaudun</a>'s <a href="http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1984">post</a>: Also, on a side note, if you do the first normalization step and then "Extract Word Co-Occurrence" network, you can also get to words that co-occur frequently, as explained here: <a href="http://wiki.cns.iu.edu/pages/viewpage.action?pageId=2200066#id-514StudyingFourMajorNetSciResearchersISIData-5145WordCo-OccurrenceNetwork5145WordCo-OccurrenceNetwork" rel="nofollow">http://wiki.cns.iu.edu/pages/viewpage.action?pageId=2200066#id-514StudyingFourMajorNetSciResearchersISIData-5145WordCo-OccurrenceNetwork5145WordCo-OccurrenceNetwork</a> Scott Weingart on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1985 Fri, 26 Apr 2013 12:40:13 +0000 Scott Weingart 1985@http://digitalhumanities.org/answers/ Replying to @<a href='/profile/johnlaudun'>johnlaudun</a>'s <a href="http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1984">post</a>: This can actually be done within Sci2 itself, once you have a csv with titles in one column and text in the second. Once that's loaded into Sci2, go to Preprocessing->Topical->Stopword Text (you can select your own stopword list from that dialog). Then go to Data Preparation->Extract Directed Network, first column title, second column text, hit okay, and voila! You can then make that into texts based on words in common or words based on texts in common using either Data Prep->Extract Co-Citation Network, or Extract Bibliographic Coupling Network (depending on your need), and you've got what you're looking for. johnlaudun on "Methods to process a plain text collection into a bimodal graph..." http://digitalhumanities.org/answers/topic/methods-to-process-a-plain-text-collection-into-a-bimodal-graph#post-1984 Fri, 26 Apr 2013 12:24:20 +0000 johnlaudun 1984@http://digitalhumanities.org/answers/ I am starting a new project wherein I want to take a closer look at narratives both in terms of topics as well as in terms of morphologies. For the moment, I have narrowed my initial exploration to two collections of treasure legends, one drawn from oral sources (and transcribed by folklorists) and one drawn from materials found on on-line forums. Both collections are, purposefully as I take baby steps here, small: each is only sixteen texts. The texts range in size from 153 to 1025 words for the oral collection and 155-3081 words for the web collection. I am looking to create a bimodal graph for each of the collections which represents the relationship between the words used in each text, such that I can examine the relationship either between texts, based on words in common, or between words, based on texts in common. What I need, I think, is a Python script or some other kind of code/application which will work through each collection of plain text files and generate a CSV or network file of some kind that will let me then work in Gephi or Sci2. I would especially like it if the script would allow me to feed in a stopword list of my choosing. johnlaudun on "What exists as an open platform for editorial management?" http://digitalhumanities.org/answers/topic/what-exists-as-an-open-platform-for-editorial-management#post-1983 Fri, 26 Apr 2013 11:48:51 +0000 johnlaudun 1983@http://digitalhumanities.org/answers/ I don't know about the two WordPress solutions -- though I have been really, really impressed with all the things that can be done, and done well, with WordPress in general -- but I'd like to echo the Open Journal software and its cousins (Open Conference et al.). I've been on the reviewing side of the setup, as well as someone who reads a number of open access journals published using the OJS setup, and I have to say it is very close to painless. Easy to use. Nice outputs.