<div dir="ltr"><div>Achim,</div><div>Your understanding is correct. Note that 3.2 still retains the 3.1 means of identifying missing values also. Larry's point is also valid. The extended details are more important in terms of working with and analyzing the data than it is in searching for data. Essentially the difference would be that in 2.5 and 3.2 Disco would pull from a single depiction of a value representation while in 3.2 it should pull from both value representation and missing value representation. At that point those from a missing value representation would indicate they were missing in however Disco is relaying that. This would ensure that those using the new structure ONLY could also map missing values to Disco. For the most part people don't search for missing values when searching for data :-)</div><div><br></div><div>Wendy</div></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">On Sat, Mar 16, 2019 at 5:41 AM Wackerow, Joachim <<a href="mailto:Joachim.Wackerow@gesis.org">Joachim.Wackerow@gesis.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid">
<div lang="EN-US">
<div class="gmail-m_6504989271101567846WordSection1">
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Wendy,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Yes, this all makes absolutely sense to me. Disco can indicate per code or value if it is missing or not. This seems to be sufficient to represent 2.5. and
3.1.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">The question is if it should represent 3.2 in this regard. Is the more advanced description of missing values really important for a data search.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">A related change would be some work.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">I tend to invent for Disco a section on limitations where it is mentioned that the ConceptualVariable in 3.2 and the advanced description of missing values
in 3.2 cannot be represented.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">This is my understanding from 3.2<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Variable has VariableRepresentation 0..1<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">VariableRepresentation has ValueRepresentation 0..1 (i.e. CodeRepresentation, NumericRepresentation) and MissingValuesReference 0..1<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">CodeRepresentation has CodeListReference<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">CodeListReference points to CodeList<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">CodeList has Code 0..*<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">NumericRepresentation has NumberRange 0..*<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">MissingValuesReference points to ManagedMissingValuesRepresentation<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">ManagedMissingValuesRepresentation has MissingCodeRepresentation 0..*, MissingNumericRepresentation 0..*<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">MissingCodeRepresentation has CodeListReference 0..1<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">MissingNumericRepresentation has NumberRange 0..*<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Achim<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-family:"Tahoma","sans-serif";font-size:10pt">From:</span></b><span style="font-family:"Tahoma","sans-serif";font-size:10pt"> Wendy Thomas [mailto:<a href="mailto:wlt@umn.edu" target="_blank">wlt@umn.edu</a>]
<br>
<b>Sent:</b> Freitag, 15. März 2019 15:40<br>
<b>To:</b> Wackerow, Joachim<br>
<b>Cc:</b> DDI Structural Reform Working Group.; Zapilko, Benjamin<br>
<b>Subject:</b> Re: [DDI-SRG] [disco] issues on missing values for numeric response domain and on conceptual variable<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">RE: missing values<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Achim,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">As I recall Disco was created prior to our division of substantive and sentinel values. Earlier DDI-Lifecycle used something similar to Codebook where you could designate blank as missing and specify which values were missing either by
listing in an attribute (3.1) or indicating if a value in a catVal was determined to be missing. If Disco covers that it will work with all versions of codebook and lifecycle. When we added the ability to show missing values as a separate representation we
did not remove the short hand approach so that major surgery on earlier versions was not required.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Wendy<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Fri, Mar 15, 2019 at 7:14 AM Wackerow, Joachim <<a href="mailto:Joachim.Wackerow@gesis.org" target="_blank">Joachim.Wackerow@gesis.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentColor currentColor currentColor rgb(204,204,204);margin:5pt 0cm 5pt 4.8pt;padding:0cm 0cm 0cm 6pt">
<div>
<div>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Many thanks to Dan, Larry, and Wendy for your thoughts.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">First, I would like to mention again the frame of this discussion,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Our focus is here: What can we do for Disco like it is currently?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">The whole approach of Disco is to focus on a simple subset of DDI Codebook and DDI Lifecycle for
Discovery purposes. </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">It is not a 1:1 representation of DDI Codebook or Lifecycle. It is not related to DDI 4 which is
a moving target.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">The intention is to finalize Disco not to make Disco as good or better than DDI 4.
</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Furthermore, any changes shouldn’t be extensive. This wouldn’t be affordable.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Re: ConceptualVariable</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">I have here a similar thinking as Wendy. For the purpose of Disco, i.e. for searches on specific
data, does the ConceptualVariable really add substantial value?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">I tend to leave the current Disco structure unchanged.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Re: missing variables for numeric response domain</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">It looks like an actionable item (math expression) seems to be the right way to go. My impression
is that Disco has Representation but doesn’t make a distinction between categorical and numeric representation.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Would the simple approach be that Representation has a property (i.e. missingValue) with the type
skos:Concept?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">See variable diagram of Disco:
</span><a href="https://raw.githubusercontent.com/linked-statistics/disco-spec/master/diagrams/variable.png" target="_blank"><span style="font-family:"Calibri","sans-serif";font-size:11pt">https://raw.githubusercontent.com/linked-statistics/disco-spec/master/diagrams/variable.png</span></a><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">.
</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt">Achim</span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:"Calibri","sans-serif";font-size:11pt"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-family:"Tahoma","sans-serif";font-size:10pt">From:</span></b><span style="font-family:"Tahoma","sans-serif";font-size:10pt"> Wendy Thomas [mailto:</span><a href="mailto:wlt@umn.edu" target="_blank"><span style="font-family:"Tahoma","sans-serif";font-size:10pt">wlt@umn.edu</span></a><span style="font-family:"Tahoma","sans-serif";font-size:10pt">]
<br>
<b>Sent:</b> Donnerstag, 14. März 2019 15:29<br>
<b>To:</b> Wackerow, Joachim<br>
<b>Cc:</b> DDI Structural Reform Working Group.; Zapilko, Benjamin<br>
<b>Subject:</b> Re: [DDI-SRG] [disco] issues on missing values for numeric response domain and on conceptual variable</span><u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">I think that it is not as important to have that step in the hierarchy in Disco. The purpose of Disco, at least initially, was to facilitate discovery of data and related metadata
in an RDF environment. As one can locate and link a concept to an existing variable that is what seems to be important. The value of a Represented Variable in the discovery process is the ability to track variable reuse across iterations of a study or a common
variable, such as the U.S. OMB definition of the Race variable across studies. Unless there is some discovery advantage to exposing a Conceptual Variable I don't think its expression in Disco if vital.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Wendy<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">On Thu, Mar 14, 2019 at 7:29 AM Wackerow, Joachim <<a href="mailto:Joachim.Wackerow@gesis.org" target="_blank">Joachim.Wackerow@gesis.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-width:medium medium medium 1pt;border-style:none none none solid;border-color:currentColor currentColor currentColor rgb(204,204,204);margin:5pt 0cm 5pt 4.8pt;padding:0cm 0cm 0cm 6pt">
<div>
<div>
<p class="MsoNormal">Benjamin Zapilko and I are currently reviewing the open issues of Disco. The goal is to resolve the issues and to prepare Disco finally for publication.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Now I have questions on two issues:<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">--<u></u><u></u></p>
<p class="MsoNormal">There is an issue on how to describe missing values for a numeric response domain.<u></u><u></u></p>
<p class="MsoNormal">Details and my comment see at
<a href="https://github.com/linked-statistics/disco-spec/issues/130" target="_blank">
https://github.com/linked-statistics/disco-spec/issues/130</a>. <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">My question:<u></u><u></u></p>
<p class="MsoNormal">Is there really missing something in Disco? I don’t have the impression. But maybe I misunderstood something.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">--<u></u><u></u></p>
<p class="MsoNormal">The other issue is that the conceptual variable of DDI 3.2 does not exist in Disco.
<u></u><u></u></p>
<p class="MsoNormal">The hierarchy is only Variable, RepresentedVariable, skos:Concept.<u></u><u></u></p>
<p class="MsoNormal">Details and my comment see at
<a href="https://github.com/linked-statistics/disco-spec/issues/226" target="_blank">
https://github.com/linked-statistics/disco-spec/issues/226</a>. <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">The whole approach of Disco is to focus on a simple subset of DDI Codebook and DDI Lifecycle for Discovery purposes. It is not a 1:1 representation.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">My question:<u></u><u></u></p>
<p class="MsoNormal">Is it really important to be able to search for the ConceptualVariable in addition to Variable, RepresentedVariable, and Concept.<u></u><u></u></p>
<p class="MsoNormal">Is this addition really worth it? This might result in some work for Disco.<u></u><u></u></p>
<p class="MsoNormal">Any thoughts would be helpful.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Thanks<u></u><u></u></p>
<p class="MsoNormal">Achim<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
DDI-SRG mailing list<br>
<a href="mailto:DDI-SRG@icpsr.umich.edu" target="_blank">DDI-SRG@icpsr.umich.edu</a><br>
<a href="http://lists.icpsr.umich.edu/mailman/listinfo/ddi-srg" target="_blank">http://lists.icpsr.umich.edu/mailman/listinfo/ddi-srg</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<br>
-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Wendy L. Thomas Phone: +1 612.624.4389<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Data Access Core Director Fax: +1 612.626.8375<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Minnesota Population Center Email:
<a href="mailto:wlt@umn.edu" target="_blank">wlt@umn.edu</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">University of Minnesota<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">50 Willey Hall<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">225 19th Avenue South<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Minneapolis, MN 55455<u></u><u></u></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
DDI-SRG mailing list<br>
<a href="mailto:DDI-SRG@icpsr.umich.edu" target="_blank">DDI-SRG@icpsr.umich.edu</a><br>
<a href="http://lists.icpsr.umich.edu/mailman/listinfo/ddi-srg" target="_blank">http://lists.icpsr.umich.edu/mailman/listinfo/ddi-srg</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<br>
-- <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Wendy L. Thomas Phone: +1 612.624.4389<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Data Access Core Director Fax: +1 612.626.8375<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Minnesota Population Center Email: <a href="mailto:wlt@umn.edu" target="_blank">
wlt@umn.edu</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">University of Minnesota<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">50 Willey Hall<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">225 19th Avenue South<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Minneapolis, MN 55455<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote></div><br clear="all"><br>-- <br><div class="gmail_signature" dir="ltr"><div>Wendy L. Thomas Phone: +1 612.624.4389</div><div>Data Access Core Director Fax: +1 612.626.8375</div><div>Minnesota Population Center Email: <a href="mailto:wlt@umn.edu" target="_blank">wlt@umn.edu</a></div><div>University of Minnesota</div><div>50 Willey Hall</div><div>225 19th Avenue South</div><div>Minneapolis, MN 55455</div></div>