I am working on adapting existing CVs/thesauri to the needs of our new environmental humanities portal and wanted to know if anyone had any reference/best practice to hint me to. I am particulary interested in issues related to the use of CVs in defining tags/search terms, the relationships between terms/tags and their practical use within a site/dbase and in CV scalability over time.
Many thanks
Wilko
Replying to @Wilko von Hardenberg's post:
My impression is that controlled vocabularies and thesauri are dying in the era of full-text search. Ever since library catalogs have allowed users to search for words appearing anywhere in a title (or in the whole record), few users actually make use of subject headings any more.
Similarly, information-retrieval techniques allow you to cluster similar documents automatically and label them with automatically selected (or generated) terms, further making controlled vocabularies obsolete. Developing and maintaining a controlled vocabulary or thesaurus is a lot of work; it's usually better to let users sift through a bit of noise in the search results to find what they're looking for than expect they will make use of a well-developed controlled vocabulary.
For keywords, terms, tags, and other metadata attached to digital objects, an increasing number of discovery interfaces allow for browsing and narrowing of search results by facet. So you can present terms and tags as facets.
While controlled vocabularies often have a hierarchy, I can say that every library catalog that I know of that presents LCSH terms as facets treats the whole hierarchical term as a single string. See, for example, http://mirlyn.lib.umich.edu/ .
Ethan Gruber
Member
Controlled vocabularies are far from dying, but rather entering a new era of importance within the LAM and digital humanities communities. LCSH is not the only controlled vocabulary resource, but others are related to biological taxonomies, personal, family, and corporate names, geographical places, terms that describe artifacts of cultural heritage (like Getty's Art and Architecture Thesaurus). There are a number of open controlled vocabulary services, such as geonames.org for modern places (pleiades for ancient) or viaf.org for personal and corporate names. SNAC (http://socialarchive.iath.virginia.edu/prototype.html) is another major resource in development right now. These resources link related intellectual concepts together with unique identifiers. This goes beyond codifying terms based simply on a string, but rather enables different systems built in different languages to recognize that "Alexander the Great" in the Library of Congress system is in fact the same person as Alexander Magnus or Alexander le Grand.
Full text searching is only tip of the iceberg when it comes to querying, organizing, and sorting through a collection of digital materials. Kevin is correct that facets are very useful for searching a collection, and facets derived from controlled vocabulary significantly increase the relevancy of the search. At the American Numismatic Society, there was little adherence to CV terms as the first edition of the database is about 20 years old, and thus we are engaged in an ongoing process of cleaning up our data to make faceting more useful. Terms like "denarius" and "Denarius" for coin denominations are interpreted differently by our search index. "AV" and "AU" for gold are also different. Adhering to a controlled vocabulary for your particular fields at the very start of the project will lead to better functionality immediately and improved sustainability over the long term.
I am currently working on two projects that have tight controlled vocabulary integration built into the data-entry portion of applications. The apps enable easy selection of terms from VIAF, LCSH, and geonames by catalogers and other subject specialists. The unique identifiers of these terms from their respective services are stored within the XML source document, and so updates to the terms can easily be automated. Moreover, VIAF, LCSH, and geonames have REST APIs that respond with XML (usually RDF) representations of each term, enabling querying and indexing of "broader terms" for hierarchical faceting.
The importance of controlled vocabularies in major data projects cannot be understated.
Ethan's points are well taken, so let me offer a softer version of my thesis. The Linked Data movement is making unique global identifiers for entities more important than ever, but I think the movement here is for people and institutions, not concepts. That is, we can generally agree on which people and institutions need identifiers and what counts as a single person or institution, but giving a unique identifier to a concept (which is what a controlled vocabulary does: it treats a certain concept as a string of characters) is something that is much harder to get everyone to agree on. I fully agree that any database should be as normalized as possible (by limiting the values allowed for input), and this certainly extends to assigning subjects to records in your database, which absolutely should be done using a controlled vocabulary if one exists.
However, if you have a large corpus of documents that have full text you can search, I think it's mad to give someone the task of categorizing them according to a controlled vocabulary. Better to use information retrieval techniques to cluster them automatically. Or just rely on full-text searching. This is the use case I was envisioning when responding to Wilko's inquiry.
Replying to @kevin.s.hawkins's post:
I must admit that I was not very clear in the outset. And Kevin's answer actually scared me; I asked myself: is it all meaningless?
Let me define the scope of our endeavour: we are creating a portal with a certain number of informative articles about the environmental humanities. Most of these articles have still to be written, so what we are actually looking for is to build a CV of tags that will then be used by the authors to define their entries. At the moment we have the tags defined by our first authors, but it is actually an incoherent and not very useful mass of terms. What we need/want to do now is to check these terms against existing CVs to have a bit of consistency and project-sustainability.
Any further suggestions anybody?
Ethan Gruber
Member
Since you want to describe documents with tags, Library of Congress Subject Headings may be a good place to start if you select a number of terms that are particular to environmental humanities. I'm not familiar with the field, so I can't suggest anything more specific.
Kevin is right that there is a lot of work involved in assigning relevant tags to a massive volume of documents. The question is: do you expect most people to locate documents strictly through full-text searching? Metadata (like controlled vocabulary for tags) are only as useful as the system created to take advantage of them. It's possible to build sophisticated visualization systems derived purely from facets (based on cv terms) that completely depart from traditional full-text search interfaces. If you don't plan on building an interface which relies heavily on facets, it perhaps isn't worth investing the time to create tags for each document.
I also must confess that I don't know what environmental humanities is, but I can say that I don't recommend using LCSH for a collection of documents in a particular discipline. It's so broad that you will only find a few terms that apply, and many of them will share components in a way that is only useful if you are browsing an alphabetical list of headings.
If you give users a search box, they usually type words into it rather than try browsing by a controlled vocabulary. Facets, including those for a CV, can be useful for narrowing down search results, but users rarely start with them.
Do you have a metadata librarian at your institution? I think Ethan and I are happy to continue dispensing advice, but a metadata specialist who could look at your material and discuss how users will be using it can make a better assessment than either of us.
You must log in to post.
A project of the Association for Computers and the Humanities and the Chronicle of Higher Education's ProfHacker
Icons by famfamfam
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
