Experience to share on DH data curation, FOSS/OA advocacy? « Digital Humanities Questions & Answers

Digital Humanities Questions & Answers » Project Management & DH Professions

Experience to share on DH data curation, FOSS/OA advocacy?

(7 posts) (5 voices)

Asked 7 years ago by Bethany Nowviskie
Latest answer from Hugh Cayless

Tags:

Bethany Nowviskie
Admin
In December 2010, the Outreach Committee of the ACH (Association for Computers and the Humanities) will draft open letters to the funders of digital humanities projects, advocating for the establishment of data curation or management plans and open source / open access guidelines for grant-funded DH.

We will share drafts of the letters for public comment before circulating them to funders. In the meantime, we're hoping you'll help inform the process by sharing your ideas and examples below -- positive or negative -- of the relationship of digital humanities projects to open source, open access, and data curation or management.

We've also set up a Google Doc which can be edited anonymously: http://is.gd/i6DNw

What should we be sure to address in these letters?

Many thanks on behalf of the ACH Outreach Committee,
Bethany Nowviskie
Tweet this question
Posted 7 years ago Permalink
desmond
Member

Please do not ask them to mandate the use of any form of embedded markup for data curation. What should be archived for others to freely repurpose and reuse is not text embedded with the subjective judgements of individuals, or the programming instructions of today that will mean nothing tomorrow. The only mandate for format (if any) should be plain Unicode text.

Posted 7 years ago Permalink
Martin Holmes
Member

In response to the above, I'd like also to propose that you not ask them to exclude embedded markup. It is doubtless useful to be able to access Unicode text versions of document data, but this is trivial to retrieve from any markup system; in the case of most marked-up documents, the bulk of the actual value of the document (in terms of scholarly work invested in creating the transcription) inheres in the markup itself. I think it is good practice to provide easy access to Unicode text versions of marked-up texts as part of an application that hosts and delivers them, and if the application itself is being curated, then this functionality needs to be supported by the curation system.

Posted 7 years ago Permalink
desmond
Member

Yes it is possible but not trivial to remove embedded markup from archived documents: 1) white space between elements is regarded as markup. So dividing text with tags creates an ambiguity: does the markup white space equal white space in the text or not? i.e. are part-words divided by tags to be conjoined or not? 2) alternatives such as addition, deletion, sic and corr, variants etc when the markup is removed produces nonsense. To correct this you need to write a program and to have a parser handy. Show me a complete and working SGML parser for 12 year old texts. 3) markup added to overcome its deficiencies such as virtual joins can't just be removed without loss of important information. I never asked for it to be mandated not to use it, only that its use not be mandated.

Posted 7 years ago Permalink
Hugh Cayless
Member

First (though I'm sure you've looked at it already) this thread: http://digitalhumanities.org/answers/topic/what-does-it-mean-to-future-proof-a-dh-project has some good stuff in it (even my comment).

Second: the anti/pro embedded markup (read TEI) stuff is a rat-hole.* Avoid it. Open, well-documented (and ideally common) formats are the answer. Any old format can be rescued/reverse-engineered if it was well-documented. I've converted enough SGML (even GML!) to know it isn't a problem. A corollary to this is that if you're using a homebrewed data format, your documentation burden must be correspondingly higher. TEI is almost certainly better than any XML format I might invent for this reason.

TEI-ish XML is good in some ways, because editorial decisions/practices can be documented and bundled in the same file as the text. But there's absolutely nothing wrong with plain text. Pick the best tool for your task and move on.

The data curation task is enabled by careful thought and planning around data format decisions, good documentation of the practices used in creating the data and the recording of information about its state and how it got there.

*A rabbit hole is a distracting tangent to the discussion that might have something good as an outcome. A rat hole does not have that attraction.

Posted 7 years ago Permalink
Dorothea Salo
Member

I've put in my two cents at Book of Trogool: the Four Sons of digital curation.

Posted 7 years ago Permalink
Hugh Cayless
Member

Nice one Dorothea! Even though I was trying to put the brakes on, I'm so much a first son that I have my own ideologies about formats, and those probably shine through quite clearly :-).

Trying for a better approach: it seems to me that most of the DH-related data curation I've been involved with has revolved around migration/rescue. That is, the conversion from either an obsolete format (e.g. SGML) to a modern one (e.g. XML), or from one type of encoding to another (ISO 8859-1 to UTF-8), or extracting data from some system (perhaps a desktop db like Filemaker or Access) for insertion into another (say MySQL). That seems like a core task that isn't going away. It's made difficult when I have to ask questions or make assumptions about the intentions of the data's creators: "What does this mean?"; "How are these things related"; "What's missing?"

It's harder still when there's nobody to ask those questions of, or when there are layers of different decisions by different people in the data.

I think documentation and metadata are things that can help mitigate the pain of those who will have to deal with your data in its afterlife. Open formats and the use of open source are good in this context because they are more likely to be available and usable (or able to be modified so that they can be used, or at the very least able to be read so they can be re-created).

Update/postscript: put another way, participants in DH projects should imagine they are in a conversation with future scholars, and should explain not just what was done, but why and how.

Posted 7 years ago Permalink

RSS feed for this topic

Reply

You must log in to post.