Thesis Proposal: Sense-making in an Annotated digital resource (SADR)

IFEYINWA OKOYE

April 2, 2008

RESEARCH QUESTION

How can we use agile software development and participatory design methodologies to transform Greenstone (a digital library server) into a collaborative sense-making annotated digital resource of computing research that will help learners more easily make the transition to computing researchers?

1. BACKGROUND

Becoming a novice yet proficient researcher in computer science has been a challenging time-consuming process for me. I know I would be much further along in the process if there was one place where I could see all the information on skills you need as a computer science researcher - skills such as how to properly keep a journal, how to read research papers and extract relevant information, how to do a literature review, how to give effective lab presentations, how to install and administer various computer systems and libraries, how to write a grant proposal, how to come up with a research statement, etc.

If such a place also provided a means of interacting with other computing graduate students and professors, it would have been and would still be invaluable to me. While attending SIGCSE 2007, I talked to some of my colleagues in other schools and realized almost everyone shared my pain. They also faced similar obstacles in their quest to become part of this community of computing researchers. Hanging around members of the computer science community was a good way to try to pick up things we needed to know, which was why we were at SIGCSE. But I was convinced the process could be improved. Therefore, I began to pursue the issue of how a learner can more easily make the transition to a computing researcher.

In summer 2007, I started collaborating with Dr Holz on a digital library project called Computing Research Methods: Multi-perspective Digital Library (CRM-MPDL) geared towards creating a shared language and understanding of research methods in computing. SADR will be my contribution to the CRM-MPDL project.

A free, comprehensive, collaborative sense-making annotated digital resource for computing research will provide students curious about research an engaging way to accumulate breadth knowledge about their area of interest and get in touch with members of the computing research community.

3. APPROACH:

My approach to supporting learners is to help them with sensemaking, which is based on the constructivist philosophy [5] that people learn by constructing knowledge and assimilating that knowledge into their knowledge-base. Sensemaking refers to “… how individuals and groups construct meaning when confronted by complex, sometimes contradictory information …” [7]. I hypothesize that learners will more easily identify as computing researchers if provided with scaffolding to help them in collaborative sensemaking of computing research.

One way to help learners in sensemaking of computing research is to help them understand existing computing research methods and open up communication channels amongst themselves and between themselves and research faculty. SADR will provide learners access to research faculty and the materials needed to begin doing research in computing.

SADR will also provide tools that support sensemaking. Sense-making tools support discovery and expression of how different papers, theories and methods within a specific area of research are connected, and in doing so provide a map of the research area. “A structure that maps the relationships between conversations would not only help users to search for information and documents, but also enable them to make better sense of the subject matter the digital library deals with …” [6] by helping the researcher to understand what has been done, why it was done, what theories were borne out of it and what other research it inspired.

“In order to understand what would cause someone to participate (contribute, search, edit, volunteer, comment) in SADR, we must first understand the perspective they have constructed from which they interact with the system [1].Therefore my first task in building SADR has been to figure out the contextualized information needs of computing students. To do this, I am using personas as a discount alternative to truly having a representative sample of the user population ‘in the room’ with me throughout the design and implementation process. Personas are a design technique that creates fictitious characters as stand-ins for real users of the system being designed. A persona describes a potential user and includes a list of what that user will require of, or need from, the system being designed.” [1]

I am collecting data on the correspondence, interviews and design meetings that are taking place in order to build SADR. After the first iteration through this project, I will have a record of how we obtained information about what hinders students from getting involved in research, how that information was used in creating the student personas, how the personas were used in designing the sense-making tools and interface of SADR, how the sense-making tools were used by real-users, how the users interacted with the system and among themselves and feedback from the users about the usability and usefulness of SADR.

I will build SADR from Greenstone, a well established digital library server. I will build SADR on one of the machines in AHAT, a research lab at CSUEB. The sense-making functionalities will implement a way for people reading the same paper or interested in the same topic to interact. Interaction will foster collaborative research and help researchers to better understand the flow of the work being done in that domain. Ideally, SADR would foster a more flexible set of options for becoming involved in research, similar to the structure that has been so successful in the open source software development community.

I am using the test-driven development (TDD) approach to building SADR because TDD saves time, money and effort. Greenstone is written in java so to customize Greenstone, I will use JUnit for unit testing and Cobertura for code coverage and analysis. I am using Cobertura to calibrate my code to ensure the result is not jeopardized by faulty code. I will use Hibernate as the interface to the database for SADR because I want to ensure the code is portable to other database systems. Since I will be developing SADR using agile software development method, I plan to have a text document open as I work to document whatever I am doing. I also plan to use pair programming while coding. An undergraduate student, Dave Rogers has offered to serve as my programming pair on this project. We will be developing the modules and interfaces in java, because Greenstone is written in Java.

I will publish SADR online and open up the database so that other authenticated users can add and annotate papers. Permitting people from different areas of computer research to add and annotate papers will create a richer set of papers and annotations.

Currently, I have an approval from the institutional review board at CSUEB, to use human subjects in my research.

2. METHODOLOGY

2.1 Build additional sense-making functionalities for SADR

I will build five different modules that

· support comments in a window beside the window displaying a document.

· support a personalized user page

· support real-time dialog

· support collaborative editing

· support graphical annotations (nodes and links)

Currently Greenstone does not have any of these features. It only stores and displays documents.

2.2 Customize interface to reflect result from persona analysis

Since Summer 2007, I have been collecting data on personas that typify SADR users. I also have data from a HCI class that worked on student personas for a digital library in computing. The personas have scenarios that describe how a user would want to interact with such a digital library. I will analyze these data and use the result to customize SADR’s interface.

2.3 Build and populate SADR

I will build SADR from Greenstone 3, the modules I developed from 2.1 and the interface from 2.2. I will populate SADR with materials I have gathered about computing research.

2.4 Publish SADR online to create a user community

Publishing SADR online will create a large user community. The resulting large user community will offer me a large data set for analysis.

2.5 Collect and analyze data about SADR usage

I will use task-based interviews to collect a rich data set concerning the usefulness, usability and implementation issues of the annotated digital library. I will analyze the data to see how the sense-making modules I added affect breadth knowledge acquisition.

First, subjects will be interviewed about how and what tools they currently use to find research papers and make sense of the research area that the paper belongs to (I think the students in CS 6000 will be good candidates). Second, subjects will be videotaped using SADR to look for research papers and create a map of the research area. The students will also be encouraged to vocalize their feelings (frustration etc) while using SADR. Third, subjects will be videotaped comparing their experience with SADR to the current method they use for finding and placing a research paper in the context of the research area.

Since I am using agile development as we collect usability data from the second step, we will correct the usability errors that are discovered during analysis of the data.

I will analyze the data I get from all these interviews to evaluate the impact SADR is having on the sense-making process within the user community. The analysis will also shed light on how agile software development and participatory design practices can inform the creation of an annotated digital library as the basis for a sense-making way of doing research.

2.6 Using result of analysis, modify SADR to gain better usage and publish results

The summative results of this interview will show how agile software development and participatory design practices can inform the creation of a sense-making annotated digital library. The results will also clarify how a sense-making annotated digital library can be used to aid collaborative research, encourage non-traditional researchers to get involved in research and create researchers that are aware of the ongoing changes in their area of research. I plan to present and publish the results in conferences and journals, such as SIGCSE and JDL. The results will be interesting to universities and research labs

4. DURATION:

The proposed research will be carried out during my thesis work at California State University, East Bay. I started working on this project with Dr Holz in Summer 2007. I hope to get done and defend my thesis in March 2009.

5. S IGNIFICANCE

Students at CSUEB will have access to a free comprehensive sense-making annotated digital library on computing research methods. SADR will improve access to research in computer science for non-traditional researchers (women, other minorities, people in third world countries) since they would have free access to ongoing conversation by researchers active in their interest area. The library will also foster a more flexible set of options for becoming involved in research. SADR will also be useful to computing research laboratories and schools since it will provide them a place they can send their student to get bootstrapped into computing research. Finally, it will also provide a best practice guide for merging agile software development and participatory design methods to customize an existing software artifact.

6. REFERENCES

[1] Applin, A., Holz, H., Joel, W., Okoye, I., Deibel, K. Grasser, B., Oates, B.J.,Wood, G. “A Multi-Perspective Digital Library to Facilitate Integrating Teaching Research Methods Across the Computing Curriculum,” inroads (2007), pp.184-203.

[2] Brusilovsky, P., Karagiannidis, C. and Sampson, D. (2004) ‘Layered evaluation of adaptive learning systems’, Int. J. Cont. Engineering Education and Lifelong Learning,Vol. 14, Nos. 4/5, pp.402–421.

[3] Carroll, J.M., Kellogg, W.A., & Rosson, M.B. (1991). The task-artifact cycle. In J.M. Carroll (Ed.), Designing interaction: Psychology at the human-computer interface pp.74-102. New York: Cambridge University Press.

[4] D. Schuler and A. Namioka. Participatory Design: Principles and Practices. Lawrence Erlbaum Associates, Hillsdale, NJ, 1993.

[5] Jean Piaget (1967). Logique et Connaissance scientifique, Encyclopédie de la Pléiade.

[6] Tuominen, K., Talja, S. and Savolainen, R. (2003), “Multiperspective digital libraries: the implications of constructionism for the development of digital libraries”, Journal of the American Society for Information Science and Technology, Vol. 54 No. 6, pp. 561-9.

[7] Victoria Uren, Simon Buckingham Shum, Michelle Bachler, Gangmin Li, Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation , International Journal of Human Computer Studies (2006), 64, 5, pp. 420-445.

[8] A. Cooper. The Inmates Are Running the Asylum :Why High Tech Products Drive Us Crazy and How To Restore The Sanity. Sams, Indianapolis, IN, USA, 1999.

[9] B. Dervin. From the mind’s eye of the user: The sense-making qualitative-quantitative methodology. In B. Dervin, L. Foreman-Wernet, and E. Lauterbach, editors, Sense-Making Methodology reader: Selected writings of Brenda Dervin, pages 270–292. Hampton Press, Cresskill, NJ, 2003.

[10] K. Don. Greenstone3: A modular digital library. http://www.greenstone.org/docs/greenstone3/manual.pdf.