[DDI-users] [Alliance-experts] quality of metadata

Fri Apr 25 10:10:34 EDT 2014

All,

I haven't found any papers that address metadata quality in a reasonable way. My view is that quality has to focus on the things that differentiate metadata from data. Metadata are data used to describe some resources (Dublin Core definition), therefore their quality depends on how well that's done.

There are several aspects to this. The first is how complete the descriptions are, and this depends on the model or standard under consideration. If the available attributes do not form a "complete" description, the quality suffers, i.e., users will not be able to learn all that they should, even under the best circumstances.

The next issue is how well the data supplied as metadata does its job. For instance, if one is asked to describe a sample design to a survey, a statement such as "the sample was selected randomly" says only a little. More information might be needed, such as was the sample selected in one stage or more than one? What type of random selection was used: simple, systematic, stratified? If stratified, what are the strata?

Another issue is how well the attributes are "factored". This addresses the ability to break a description into fundamental units. This maximizes re-use, and it goes far in making it easier for users to supply metadata consistently across applications. Returning to the sample design example, the non-factored approach gives one element devoted to sample design as above. The user fills in what they want to. A much more factored approach (for a probability sample) might have the following:
Number of stages
Sample size
Overall probability of selection
For each stage, sample type (simple, systematic, stratified)
For each stage, probability of selection
If stratified, the strata

Factoring also applies to the way metadata are allocated to each attribute. So take this description of a datum - the takeoff time of the president's aircraft is noon on 26 April 2014. What is the universe and variable in this example? Here are some possibilities:

Universe - aircraft
Variable - president's takeoff time

Universe - president's aircraft
Variable - takeoff time

Universe - takeoffs of president's aircraft
Variable - time

If the metadata are factored one way and a user is looking for a description in another, the relevant resource might get passed over even though the various descriptions are the same when all attributes are combined. Therefore, the ability to have multiple descriptions of the same thing might be important and adds to usefulness of the metadata.

Independent of factoring, consistent use of the same definition, the same term, the same sets of categories, the same description of a sample design, and so on is vital. Reuse, comparability, and interoperability all depend on this consistency. This depends often on the ability to supply metadata by reference - a URI to some data somewhere. This is the write once - use anywhere approach. It guarantees consistency.

There are other facets as well.

I hope this helps.

Yours,
Dan

-----Original Message-----
From: alliance-experts-bounces at icpsr.umich.edu [mailto:alliance-experts-bounces at icpsr.umich.edu] On Behalf Of Mari Kleemola
Sent: Friday, April 25, 2014 5:27 AM
To: Marion Wittenberg
Cc: alliance-experts at icpsr.umich.edu; Data Documentation Initiative Users Group
Subject: Re: [Alliance-experts] quality of metadata

Hi Marion,

I agree that your question is an important one, and I'd also be interested to hear experiences and thoughts about this. (Wendy - thank you for you informative reply.)

WP7 in the Data without Boundaries project is currently studying metadata issues like standard selection and usage. My literature review revealed many studies about assessing metadata quality (none of which were DDI), but not so many about assessing the quality of a metadata standard. However, a common agreement for metadata quality seems to be "fitness for purpose".

Thanks,
Mari

--
Mari Kleemola
Information Services Manager
FSD Finnish Social Science Data Archive
33014  UNIVERSITY OF TAMPERE, FINLAND

tel. +358 40 190 1437 / www.fsd.uta.fi

On 24.4.2014 19:35, Wendy Thomas wrote:
> Hi Marion,
>
> Interesting question and rather multi-faceted. The "quality" of DDI 
> metadata can be viewed in terms of coverage and content. More has been 
> done in terms of content but in an informal way. From the beginning 
> DDI determined what was needed in terms of object coverage based on 
> norms in the social sciences and particularly social sciences archives 
> (preservation and discovery). The intent (see a great presentation by 
> Ann Green at NADDI 2014) was to cover the objects needed to inform the 
> future user regarding where the data came from and how applicable it 
> was to the intended use. This is way beyond discovery and technical 
> usability, it goes into the intent of the data and features of the 
> methodology, collection and processing which may affect its 
> suitability for use.
>
> DDI initially covered all the content of a "well constructed" codebook 
> for a social science data file. We have extended this over the years 
> to provide additional information on provenance (exactly how data was
> processed) and collection information. Currently we are working on 
> including additional sources of data, development of data capture 
> content (question development and testing, biomarkers, qualitative 
> data etc.). Over the years this has been a bit of give an take. DDI 
> has been treated as input to documents covering metadata quality as 
> well as learning from them.
>
> Currently we are working with the NSIs creating the Generic 
> Statistical Information Model (GSIM) which is informed by the metadata 
> Quality Frameworks of their organizations. We are making sure we both 
> reflect the recommended content as well as provide a means of linking 
> to these Quality Frameworks and how they impact the content and 
> quality of the metadata and data described in DDI.
>
> With all of that there has been little formal review of how well the 
> coverage of DDI reflects the recommended coverage within various data 
> communities. This should probably be formalized.
>
> As for the content of metadata expressed in DDI there has been no work 
> that I am aware of. Others may be more involved in this area than I am.
>
> Hopefully others will jump into this conversation as it is an 
> important one. Not only for users in general but for the DDI community 
> in terms of advancing the use of DDI and in informing potential users 
> of its applicability in terms of meeting the requirements of quality 
> frameworks within their own field.
>
> Wendy
>
> p.s. I recommend contacting Ann Green directly as I'm not sure if she 
> is on this list
>
>
> On Thu, Apr 24, 2014 at 5:06 AM, Marion Wittenberg 
> <marion.wittenberg at dans.knaw.nl 
> <mailto:marion.wittenberg at dans.knaw.nl>>
> wrote:
>
>     Hi,
>
>     Does anyone know if research has been doneÂ onÂ the quality ofÂ DDIÂ
>     metadata.
>     I am working on a deliverable for the EU DASISH project on metadata
>     quality and I would like to reference to these projects, if any.
>     I know of one project in the context of the DwB project (Pascal Heus
>     et al).
>     Are there more?
>
>     Best, Marion Wittenberg
>
>
>
>
>     _______________________________________________
>     alliance-experts mailing list
>     alliance-experts at icpsr.umich.edu
>     <mailto:alliance-experts at icpsr.umich.edu>
>     http://lists.icpsr.umich.edu/mailman/listinfo/alliance-experts
>
>
>
>
> --
> Wendy L. ThomasÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> Â Phone: +1 612.624.4389 Data Access Core DirectorÂ Â Â Â Â Â Â Â Â Â 
> Â Â Â Â Â  Â Fax:Â Â  +1
> 612.626.8375
> Minnesota Population CenterÂ Â Â Â Â Â Â Â Â Â Â Â  Email: wlt at umn.edu 
> <mailto:wlt at umn.edu> University of Minnesota
> 50 Willey Hall
> 225 19th Avenue South
> Minneapolis, MN 55455
>
>
> _______________________________________________
> alliance-experts mailing list
> alliance-experts at icpsr.umich.edu
> http://lists.icpsr.umich.edu/mailman/listinfo/alliance-experts
>

_______________________________________________
alliance-experts mailing list
alliance-experts at icpsr.umich.edu
http://lists.icpsr.umich.edu/mailman/listinfo/alliance-experts