Developing a linked, open database of social science measures

Recent troubling findings on the consequences of lacking research and reporting transparency (a.o., Levelt, 2012) elicited a strong trend towards Open Science within the social sciences. The Open Science Framework (OSF) plays a key role in this trend, by providing a collaborative online platform where users post pre-registered proposals, and open data and materials. Despite its growing user community, the OSF currently lacks functionality that leverages network effects – i.e., “gets better the more people use it” – which is vital for online communities to prosper.

We propose to generate such functionality by designing a participatory, linked, open database of social science measures, that is, of operational definitions (e.g., scales) of theoretical concepts. Currently, the practice of selecting of measures for research is flawed by lack of overview, access, and meta-knowledge. As a result, researchers often invent their own measurement instruments – some claim there are as many scales and measures as researchers (Devlin et al., 1993). A consistent central database would improve efficiency and impact for individual researchers, and knowledge accumulation for science in general.

In collaboration with the OSF, our design would allow users to come to the OSF to find measures for their concepts of interest, and to contribute new measures if existing measures are absent. The OSF’s APIs will be employed to tie measures to empirical observation and easily export scales to questionnaire tools (e.g., Qualtrics). The computational challenge is to intelligently aggregate, index and recommend uploaded measures to provide users with best-practice measurement scales, while maintaining responsiveness to newly developed scales. Database introspection may yield new insights by linking theoretical concepts, scales, and empirical observations yielding a de facto meta-analysis, possibly opening new paths for research.

For a further description of the project, please visit https://www.dropbox.com/s/zzp2yku80j0r6c9/%281%29%20AA%20proposal%202014_Hoekstra_vdVelde.pdf?dl=0

vulogo_sm