Written by Anne-Marie Viola, Metadata and Cataloging Specialist
As described in last month’s post on usability testing, ICFA conducted a series of tests over two weeks in January in advance of our February 10th public launch (490 visits to date). Doing so yielded both anticipated and unexpected findings, which we categorized into three areas of concern: problems that we could address immediately, problems that we could address through instructions, and problems that we would discuss with AtoM’s developers, Artefactual Systems.
I previously explained our immediate solutions and the development of our AtoM@DO-specific Help page, leaving the “big stuff” for this post. Having drafted a formal report of our findings for Artefactual, I now feel like I have a handle on the major issues that testing identified, each of which serve as great reminders that archivists and researchers approach archival description with different expectations and that every system has a learning curve. I address four of these issues here along with proposing possible development-oriented solutions where necessary.
ARCHIVAL LINGO DOESN’T MAKE FOR USER-FRIENDLY LABELS
Out-of-the-box ICA-AtoM features a menu of options for browsing the system, which include links with labels such as “Archival descriptions,” “Authority records,” “Archival institutions,” etc. Concerned that our users might not understand what each of these represents, ICFA changed each of these to what we thought would be more user-friendly terms, specifically “Archival records,” “Names,” and “Repositories.”
To test the usability of these labels, we designed a series of tasks inviting users to determine the number of collections in AtoM@DO and browse to the records of a specific creator.
TASK 3: “You are interested in an overview of Dumbarton Oaks’ archival holdings represented in AtoM. Please tell me how you would determine the number of collections represented in AtoM.”
After having been given an opportunity to familiarize themselves with the homepage, only two participants in Week 1 were able to successfully accomplish this task in less than two minutes. In fact, only one participant clicked on the link that took them directly to the Archival descriptions page, which lists the collections. Instead, four of five participants clicked on the Browse menu option for Repositories – an alternative pathway to determining this answer as each repository record includes links to each of its collections under the heading “Holdings.” Yet two of these participants confused the number of repositories listed as the number of collections and never even saw the Archival descriptions page during this task.
This feedback highlighted the confusion users have around terms like “Repository” and how holdings are defined, which lead us to develop the following FAQs on our Help page, “What is a collection?” and “What is a repository?”.
When shown the Archival descriptions page after testing ended, one of these two participants remarked, “‘Archival records’ sounds like administrative information about the archives. Perhaps ‘Collections’ is a better term.” And, as noted in my previous post, we immediately changed this label to “Collections.” Testing Week 2 confirmed the efficacy of this decision with a 100% success rate on this same task and a reduction of average time on task by 1:26 seconds (from 2:22 to 0:56).
MISSING THE CONTEXT TREE
HOW A LACK OF UNDERSTANDING ABOUT HIERARCHICAL ARRANGEMENT AFFECTED NAVIGATION (OR HOW ARCHIVISTS’ ASSUMPTIONS CAN FAIL TO MEET USERS’ EXPECTATIONS)
ICA-AtoM is designed to provide archival description in a more robust fashion than the typical PDF finding aid. The system incorporates the ability to search and browse across collections and also represents each record level with a separate description within a navigable hierarchy that begins at the collection-level. This is something I wrote about in our system selection announcement.
And yet, it was apparent by the end of Week 1 that participants were generally unaware of this navigational functionality, which is referred to as the Context tree.
Specifically in Week 1, only one participant proactively navigated using the Context tree. One task in particular highlighted the confusion around this functionality:
TASK: “You are also interested in correspondence of the Byzantine Institute from the 1940’s. Please identify the files that meet this research interest. You may navigate however you would like.”
One participant from Week 1 noticed the Context tree during this task and verbally expressed their surprise. Another participant commented that “there’s correspondence in here, but I’m not sure how to get at it…” and then when shown the context tree after testing, added, “I had no idea that was there, that would have been super helpful.” A third participant wondered aloud, “How is the correspondence organized and can I get specifically correspondence from the 1940’s or do I have to more generally sift through files?” She then concluded, “There’s no way I’m going to figure out what’s in these boxes unless I go through a real finding aid that describes the contents of them.” At that point, the participant clicked on the link to the PDF finding aid and located the correspondence series and stated, “This is as close as I can get…”
Observing participants’ search behavior – scanning the description displayed on the right-hand side of the screen – and listening to their comments, it appears obvious that users do not expect the presence of separate child records. This speaks to a larger lack of understanding around the hierarchical nature of archival arrangement.
As an immediate in-house solution, ICFA created the FAQ on our Help page, “How do I view the files within a collection?”. It should be noted that following the change described above to help users find our Collections page, during Week 2 all three participants were using the Context tree by the second half of the test.
However, we believe a development-based solution would be more effective and speed users’ recognition of this critical functionality. Instead of the current default display of the context tree, which lists other records at the same level, we propose making the default display an expanded view with immediate child records visible upon loading any record (e.g., when loading a collection-level record, all the series titles would display).
HOW USERS’ ASSUMPTIONS LEAD THEM ASTRAY, PART I: RESTRICTIONS
WHAT’S INCLUDED IN ARCHIVAL DESCRIPTION AND AT WHAT LEVEL
One of the functional requirements specified in the design of the ICA-AtoM system is that “the application must support multi-level description with inheritance…” In this context, inheritance means that “descriptions at lower levels should inherit information from higher levels (e.g., creator).” Within ICA-AtoM, inherited fields include Creator, Reference Code, Repository, and Rights. ICFA’s testing suggests that it would be useful to expand the inheritance functionality to other information elements, such as Conditions of access and reproduction, Languages, and Finding Aids.
ICFA’s support for this proposed enhancement is based on our observations of where participants actually looked for this information. All participants had particular difficulty with a task to determine if the material they had just identified was available for research (i.e., no access restrictions). This was one of the most problematic tasks in the whole test; half the participants failed to find this information. Most participants looked for this information in the file they had identified, rather than in the collection-level record where it was actually supplied. When they did not see it in the file-level record, most assumed that there were no restrictions. One participant specifically remarked, “I don’t see any restrictions.” A few participants navigated to the repository record or the ICFA website, assuming this information was not specific to the collection.
To address this confusion, ICFA added a citation template to the AtoM homepage and devised the FAQ on our Help page, “Where can I find information about how to access a collection and if there are any restrictions?”.
HOW USERS’ ASSUMPTIONS LEAD THEM ASTRAY, PART II: DIACRITICS
WHAT’S MISSING FROM OUR SEARCH INDEX
As previously explained, ICFA’s collections include materials in foreign languages, which necessitates the use of diacritics in our cataloging. This was a known problem for AtoM 1.3’s search engine. Concerned about the impact that this would have on discoverability, we included a task to determine whether users would include diacritics when searching for terms or if there was a general assumption that the search engine would be able to find the term without them.
TASK: “Alright, now you are looking for some materials from the Collège de France.”
Only three of eight participants used the diacritic in their search. Also, because a search for “College de France” returns records with these individual terms (college, de, and france), participants incorrectly assumed that this included their desired results. To address this, ICFA devised the FAQ on our Help page, “Do I have to include accent marks and other diacritics in my search terms?” But, ideally, alternative forms of these terms without their diacritics would be indexed and therefore return the proper search results.
In conclusion, usability testing yields a simple, yet invaluable, benefit: an understanding of the inherent limitations of your system and a clear picture of your users’ expectations. Our findings not only informed our immediate next steps, but will also play a key role in deciding what enhancements we will work with Artefactual to develop down the road.