From lyle at umich.edu Mon Apr 5 22:32:11 2021 From: lyle at umich.edu (Jared Lyle) Date: Mon, 5 Apr 2021 22:32:11 -0400 Subject: [DDI-users] Join the new DDI Slack collaborative workspace Message-ID: The DDI Alliance has a new collaborative workspace on Slack ( https://ddialliance.slack.com/) that is free to members who would like to join. We encourage you to opt-in for participant-directed conversations about DDI Alliance products, training, conferences, resources, and related topics. The DDI Alliance Slack is free and open to the public, but participants must complete the application form (https://myumi.ch/NxbDy) and be accepted before access is granted. View our Community Guidelines and How-to instructions to help you set up and use your account. Jared Lyle Executive Director, DDI Alliance ICPSR , University of Michigan lyle at umich.edu *Follow the DDI Alliance on Twitter * [image: DDI-tagline2.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210405/a4be295f/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: DDI-tagline2.png Type: image/png Size: 16865 bytes Desc: not available Url : http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210405/a4be295f/attachment.png From lyle at umich.edu Wed Apr 14 23:30:08 2021 From: lyle at umich.edu (Jared Lyle) Date: Wed, 14 Apr 2021 23:30:08 -0400 Subject: [DDI-users] DDI Executive Board nominations Message-ID: Dear DDI community: This is an election year for the DDI Executive Board . Four board seats are up for election (terms are four years). According to the Alliance Bylaws : "In election years, nominations for At-Large members will be solicited in April and a slate will be prepared by the Executive Director for discussion at the annual meeting with the election occurring in June. In the event that there are more candidates than positions, the election will be decided on the basis of those candidates getting the most votes." The responsibilities of the Executive Board include: setting overall policy and budget for the Alliance, providing strategic guidance and review of the Alliance?s activities, and overseeing the management of the financial affairs of the Alliance on behalf of the Members. I invite anyone interested in serving on or nominating another person to the Executive Board to email me directly with the nomination. The Alliance is committed to broad representation and encourages a wide and diverse pool of nominations. Sincerely, Jared Jared Lyle Executive Director, DDI Alliance ICPSR , University of Michigan lyle at umich.edu *Follow the DDI Alliance on Twitter * [image: DDI-tagline2.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210414/0a8490f1/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: DDI-tagline2.png Type: image/png Size: 16865 bytes Desc: not available Url : http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210414/0a8490f1/attachment-0001.png From alina.danciu at sciencespo.fr Tue Apr 20 06:06:46 2021 From: alina.danciu at sciencespo.fr (Alina DANCIU) Date: Tue, 20 Apr 2021 12:06:46 +0200 Subject: [DDI-users] [DDI Users] : Webinar on Introduction to Metadata for Research Data Management: A Data Documentation Initiative (DDI) Perspective Message-ID: * (Part I of the **DDI Training Webinar Series**)* *Date: *29 April 2021 *Time: *10:00 ? 11:30 EDT/16:00 ? 17:30 CEST *Register for the webinar here: https://attendee.gotowebinar.com/register/656847561327953932 * Description: This workshop introduces the definition and use of metadata in modern research data management. Metadata is a critical ingredient in applying the FAIR principles, but it is not necessarily easy to understand. The presentation will cover the definition of metadata and the ways in which it can be used in systems and will briefly introduce the DDI specifications which embody these perspectives and understandings. The DDI Alliance has for many years provided popular standards and specifications for modeling metadata and using it in data management and dissemination in the social, behavioural, economic, and health sciences, and in official statistics. Many of their approaches will apply in other domains as well. Metadata for RDM purposes can be a complex topic ? this workshop will introduce a way of understanding and addressing that complexity effectively in systems aimed at FAIR data sharing. Data Privacy Statement for participants *About DDI-CDI* Introduction: https://ddi-alliance.atlassian.net/wiki/download/attachments/860815393/Part_1_DDI-CDI_Intro_PR_1.pdf Public review page: https://ddi-alliance.atlassian.net/wiki/x/IQBPMw Complete download package: https://ddi-alliance.bitbucket.io/DDI-CDI/DDI-CDI_Public_Review_1.zip Announcement at DDI Alliance website: https://ddialliance.org/announcement/public-review-ddi-cross-domain-integration-ddi-cdi Thanks, Alina -- Alina Danciu *Centre de Donn?es Socio-Politiques* Sciences Po 27 rue Saint-Guillaume 75337 Paris cedex 07 http://cdsp.sciences-po.fr/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210420/05bdbe0c/attachment.html From wlt at umn.edu Fri Apr 23 14:13:20 2021 From: wlt at umn.edu (Wendy Thomas) Date: Fri, 23 Apr 2021 13:13:20 -0500 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook Message-ID: I had this question within another conversation recently and was wondering what people were using and if there were any new tools out there. "While I have your attention: what is the process of creating the DDI codebooks based on existing data sets? Suppose I have a labeled data set in Stata, what would I need to do to produce a DDI codebook XML file out if it? " Answer to the list. I'll compile and I'm sure others are interested. Wendy -- Wendy L. Thomas Phone: +1 612.624.4389 Data Access Core Director Fax: +1 612.626.8375 Minnesota Population Center Email: wlt at umn.edu University of Minnesota 50 Willey Hall 225 19th Avenue South Minneapolis, MN 55455 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210423/8cc54f66/attachment.html From dusa.adrian at unibuc.ro Fri Apr 23 16:46:34 2021 From: dusa.adrian at unibuc.ro (=?UTF-8?B?QWRyaWFuIER1yJlh?=) Date: Fri, 23 Apr 2021 23:46:34 +0300 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook In-Reply-To: References: Message-ID: I have an R package that can do that out of the box. The command is as simple as: convert("yourfile.dta", to = "DDI") The latest development version depends on another R package called "haven", which (in its latest development version but yet not published on CRAN) can read and write Stata style defined missing values such as .a, .b etc. In the XML codebook file, they are automatically (and consistently across all variables) replaced with negative numbers such as -1, -2 etc. Both development versions can be installed from GitHub, using: library(remotes) install_github("tidyverse/haven") install_github("dusadrian/DDIwR") I'd be happy to be of assistance in case you need any help. I hope this helps, Adrian On Fri, Apr 23, 2021 at 9:23 PM Wendy Thomas wrote: > I had this question within another conversation recently and was wondering > what people were using and if there were any new tools out there. > > "While I have your attention: what is the process of creating the DDI > codebooks based on existing data sets? Suppose I have a labeled data set in > Stata, what would I need to do to produce a DDI codebook XML file out if > it? " > > Answer to the list. I'll compile and I'm sure others are interested. > > Wendy > > -- > Wendy L. Thomas Phone: +1 612.624.4389 > Data Access Core Director Fax: +1 612.626.8375 > Minnesota Population Center Email: wlt at umn.edu > University of Minnesota > 50 Willey Hall > 225 19th Avenue South > Minneapolis, MN 55455 > _______________________________________________ > DDI-users mailing list > DDI-users at icpsr.umich.edu > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > -- Adrian Dusa University of Bucharest Romanian Social Data Archive Soseaua Panduri nr. 90-92 050663 Bucharest sector 5 Romania https://adriandusa.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210423/24fae0cd/attachment.html From pascal.heus at gmail.com Sat Apr 24 13:45:44 2021 From: pascal.heus at gmail.com (Pascal Heus) Date: Sat, 24 Apr 2021 11:45:44 -0600 Subject: [DDI-users] DDI-users Digest, Vol 185, Issue 4 In-Reply-To: References: Message-ID: <30e2eccc-a6b1-a51d-14f8-a2edc1316b89@gmail.com> Wendy: We discontinued our SlegdeHammer package a while back but still have an internal command line version which you can download at https://mtna.s3.amazonaws.com/sledgehammer/sledgehammer-cl.zip The technical user's guide is at http://goo.gl/9xukB This can produce DDI from Stata, SPSS, SAS, as well as syntax files for these stats packages or SQL. You can also use DDI as input (or CSV+DDI). Let me know if you have any question. best *P On 4/24/21 10:00, ddi-users-request at icpsr.umich.edu wrote: > Send DDI-users mailing list submissions to > ddi-users at icpsr.umich.edu > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > or, via email, send a message with subject or body 'help' to > ddi-users-request at icpsr.umich.edu > > You can reach the person managing the list at > ddi-users-owner at icpsr.umich.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of DDI-users digest..." > > > Today's Topics: > > 1. Translation tools/processes from syntax files to DDICodebook > (Wendy Thomas) > 2. Re: Translation tools/processes from syntax files to > DDICodebook (Adrian Du?a) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 23 Apr 2021 13:13:20 -0500 > From: Wendy Thomas > Subject: [DDI-users] Translation tools/processes from syntax files to > DDICodebook > To: Data Documentation Initiative Users Group > > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > I had this question within another conversation recently and was wondering > what people were using and if there were any new tools out there. > > "While I have your attention: what is the process of creating the DDI > codebooks based on existing data sets? Suppose I have a labeled data set in > Stata, what would I need to do to produce a DDI codebook XML file out if > it? " > > Answer to the list. I'll compile and I'm sure others are interested. > > Wendy > From wlt at umn.edu Sat Apr 24 15:14:25 2021 From: wlt at umn.edu (Wendy Thomas) Date: Sat, 24 Apr 2021 14:14:25 -0500 Subject: [DDI-users] DDI-users Digest, Vol 185, Issue 4 In-Reply-To: <30e2eccc-a6b1-a51d-14f8-a2edc1316b89@gmail.com> References: <30e2eccc-a6b1-a51d-14f8-a2edc1316b89@gmail.com> Message-ID: Thanks Pascal On Sat, Apr 24, 2021, 12:46 Pascal Heus wrote: > Wendy: > We discontinued our SlegdeHammer package a while back but still have an > internal command line version which you can download at > https://mtna.s3.amazonaws.com/sledgehammer/sledgehammer-cl.zip > The technical user's guide is at http://goo.gl/9xukB > This can produce DDI from Stata, SPSS, SAS, as well as syntax files for > these stats packages or SQL. You can also use DDI as input (or CSV+DDI). > Let me know if you have any question. > best > *P > > On 4/24/21 10:00, ddi-users-request at icpsr.umich.edu wrote: > > Send DDI-users mailing list submissions to > > ddi-users at icpsr.umich.edu > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > > or, via email, send a message with subject or body 'help' to > > ddi-users-request at icpsr.umich.edu > > > > You can reach the person managing the list at > > ddi-users-owner at icpsr.umich.edu > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of DDI-users digest..." > > > > > > Today's Topics: > > > > 1. Translation tools/processes from syntax files to DDICodebook > > (Wendy Thomas) > > 2. Re: Translation tools/processes from syntax files to > > DDICodebook (Adrian Du?a) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Fri, 23 Apr 2021 13:13:20 -0500 > > From: Wendy Thomas > > Subject: [DDI-users] Translation tools/processes from syntax files to > > DDICodebook > > To: Data Documentation Initiative Users Group > > > > Message-ID: > > < > CAOrpSNrJmmVUPthtS5+xM19itgyaW160z5pnKAxdsBHQxx5YgA at mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > I had this question within another conversation recently and was > wondering > > what people were using and if there were any new tools out there. > > > > "While I have your attention: what is the process of creating the DDI > > codebooks based on existing data sets? Suppose I have a labeled data set > in > > Stata, what would I need to do to produce a DDI codebook XML file out if > > it? " > > > > Answer to the list. I'll compile and I'm sure others are interested. > > > > Wendy > > > > _______________________________________________ > DDI-users mailing list > DDI-users at icpsr.umich.edu > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210424/984d2d7c/attachment.html From jl at epidata.dk Sun Apr 25 04:41:34 2021 From: jl at epidata.dk (jens lauritsen) Date: Sun, 25 Apr 2021 10:41:34 +0200 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook In-Reply-To: References: Message-ID: <53DC5D6A-3BE5-4109-9E8F-758659ADF544@epidata.dk> You may read the Stata file in Epidata software and export to DDI v3. 1 EpiData is available from www.epidata.dk You need the Manager. I will check if we implemented this in Analysis. The Manager or Analysis may also check conformity and completeness, e.g. If all missi g is documented. Best wishes Jens Lauritsen Denmark Initiator and coordinator of Epidata The DDI implementation was made in collaboration with the Danish Data Archives, now part of the National Danish Archives. Den 23. april 2021 20.13.20 CEST, Wendy Thomas skrev: >I had this question within another conversation recently and was >wondering >what people were using and if there were any new tools out there. > >"While I have your attention: what is the process of creating the DDI >codebooks based on existing data sets? Suppose I have a labeled data >set in >Stata, what would I need to do to produce a DDI codebook XML file out >if >it? " > >Answer to the list. I'll compile and I'm sure others are interested. > >Wendy > >-- >Wendy L. Thomas Phone: +1 612.624.4389 >Data Access Core Director Fax: +1 612.626.8375 >Minnesota Population Center Email: wlt at umn.edu >University of Minnesota >50 Willey Hall >225 19th Avenue South >Minneapolis, MN 55455 -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210425/864531fc/attachment.html From e.j.hrudey at vu.nl Wed Apr 28 03:41:08 2021 From: e.j.hrudey at vu.nl (Hrudey, E.J.) Date: Wed, 28 Apr 2021 07:41:08 +0000 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook In-Reply-To: References: Message-ID: Thanks for sharing your R package, Adrian, and for starting the conversation, Wendy. Re: the original question-I used NESSTAR and uploaded the data files I had (in this case SPSS, but it works with STATA too). NESSTAR automatically uploads the existing metadata from the dataset, but there is still a lot of manual work to do if the questionnaire had conditional questions/routing. NESSTAR is nice in that you can add additional information or include question text that is longer than the standard variable label, but everything is manual so it takes a lot of work. @Adrian-with your R package, is it designed solely to translate existing labeled data into a codebook or is it possible to add and modify the metadata (e.g. with NESSTAR there was the possible of providing additional information about routing)? And can is be used to convert data formats other than STATA? On a general note, after the time I spent working in NESSTAR and comparing it to Colectica, it seems to me it would be more efficient if the questions in a survey could be marked up at the level of the survey itself (to allow for capturing additional text or routing that cannot be readily included in the labels of the dataset variables). I think that?s what the Colectica Designer tool aims to do, but the problem is that while it allows you to design a survey in Designer and then export that marked-up survey to be uploaded into a survey tool, there are only a handful of survey tools that can utilize this exported marked-up survey. Unfortunately, it?s not so simple for my institution to just switch to another survey tool (we use Qualtrics), so I?m not sure how we can best create marked-up surveys that can be reused and that should make the process of creating a DDI codebook easier, without requiring researchers to create the survey two times: once in a DDI tool like Colectica and once in the survey tool itself. Has anyone else had this experience? Sincerely, Jessica From: on behalf of Adrian Du?a Reply-To: Data Documentation Initiative Users Group Date: Friday, April 23, 2021 at 10:58 PM To: Data Documentation Initiative Users Group Subject: Re: [DDI-users] Translation tools/processes from syntax files to DDICodebook I have an R package that can do that out of the box. The command is as simple as: convert("yourfile.dta", to = "DDI") The latest development version depends on another R package called "haven", which (in its latest development version but yet not published on CRAN) can read and write Stata style defined missing values such as .a, .b etc. In the XML codebook file, they are automatically (and consistently across all variables) replaced with negative numbers such as -1, -2 etc. Both development versions can be installed from GitHub, using: library(remotes) install_github("tidyverse/haven") install_github("dusadrian/DDIwR") I'd be happy to be of assistance in case you need any help. I hope this helps, Adrian On Fri, Apr 23, 2021 at 9:23 PM Wendy Thomas > wrote: I had this question within another conversation recently and was wondering what people were using and if there were any new tools out there. "While I have your attention: what is the process of creating the DDI codebooks based on existing data sets? Suppose I have a labeled data set in Stata, what would I need to do to produce a DDI codebook XML file out if it? " Answer to the list. I'll compile and I'm sure others are interested. Wendy -- Wendy L. Thomas Phone: +1 612.624.4389 Data Access Core Director Fax: +1 612.626.8375 Minnesota Population Center Email: wlt at umn.edu University of Minnesota 50 Willey Hall 225 19th Avenue South Minneapolis, MN 55455 _______________________________________________ DDI-users mailing list DDI-users at icpsr.umich.edu http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users -- Adrian Dusa University of Bucharest Romanian Social Data Archive Soseaua Panduri nr. 90-92 050663 Bucharest sector 5 Romania https://adriandusa.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210428/11878fe8/attachment.html From dusa.adrian at unibuc.ro Wed Apr 28 05:37:41 2021 From: dusa.adrian at unibuc.ro (=?UTF-8?B?QWRyaWFuIER1yJlh?=) Date: Wed, 28 Apr 2021 12:37:41 +0300 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook In-Reply-To: References: Message-ID: Hi Jessica, The package DDIwR is designed to convert to and from all major statistical software, plus DDI. It can for instance convert from SPSS to Stata and the other way round, including missing values. It is also possible to extract all the metadata from any such data file (SPSS, Stata, SAS and R) to create a DDI Codebook v2.5. But that is about all it does automatically. Otherwise, additional information can of course be added to the XML file using the already existing R package xml2. Using this package, it is possible to query individual XML nodes, or modify those nodes or even create new ones. However, this is a command line experience, I am afraid the simplicity of NESSTAR could only be matched by creating a similar Graphical User Interface for the DDIwR package. This is also possible, and not even that complicated, but I didn't yet have the time to do something like this. For the moment, if you don't mind writing a few lines of code, you can achieve everything you might imagine in R. I reckon that documenting conditional routing can hardly be automatic. This is by definition a manual task, someone (preferably the original researcher) needs to put that information somewhere. The GUI that I have in mind would allow for this kind of information to be documented, and it would also allow creating the research instrument to be exported to virtually any type of survey tool, either out of the box of the tool is already standard and established, or to be imported via a dedicated API. I just need to make some time for this, the technology is already there. Best wishes, I hope this helps, Adrian On Wed, Apr 28, 2021 at 10:48 AM Hrudey, E.J. wrote: > Thanks for sharing your R package, Adrian, and for starting the > conversation, Wendy. > > > > Re: the original question-I used NESSTAR and uploaded the data files I had > (in this case SPSS, but it works with STATA too). NESSTAR automatically > uploads the existing metadata from the dataset, but there is still a lot of > manual work to do if the questionnaire had conditional questions/routing. > NESSTAR is nice in that you can add additional information or include > question text that is longer than the standard variable label, but > everything is manual so it takes a lot of work. > > > > @Adrian-with your R package, is it designed solely to translate existing > labeled data into a codebook or is it possible to add and modify the > metadata (e.g. with NESSTAR there was the possible of providing additional > information about routing)? And can is be used to convert data formats > other than STATA? > > > > On a general note, after the time I spent working in NESSTAR and comparing > it to Colectica, it seems to me it would be more efficient if the questions > in a survey could be marked up at the level of the survey itself (to allow > for capturing additional text or routing that cannot be readily included in > the labels of the dataset variables). I think that?s what the Colectica > Designer tool aims to do, but the problem is that while it allows you to > design a survey in Designer and then export that marked-up survey to be > uploaded into a survey tool, there are only a handful of survey tools that > can utilize this exported marked-up survey. Unfortunately, it?s not so > simple for my institution to just switch to another survey tool (we use > Qualtrics), so I?m not sure how we can best create marked-up surveys that > can be reused and that should make the process of creating a DDI codebook > easier, without requiring researchers to create the survey two times: once > in a DDI tool like Colectica and once in the survey tool itself. > > Has anyone else had this experience? > > > > Sincerely, > > > > Jessica > > > > > > *From: * on behalf of Adrian Du?a < > dusa.adrian at unibuc.ro> > *Reply-To: *Data Documentation Initiative Users Group < > ddi-users at icpsr.umich.edu> > *Date: *Friday, April 23, 2021 at 10:58 PM > *To: *Data Documentation Initiative Users Group > > *Subject: *Re: [DDI-users] Translation tools/processes from syntax files > to DDICodebook > > > > I have an R package that can do that out of the box. The command is as > simple as: > > convert("yourfile.dta", to = "DDI") > > > > The latest development version depends on another R package called > "haven", which (in its latest development version but yet not published on > CRAN) can read and write Stata style defined missing values such as .a, .b > etc. > > In the XML codebook file, they are automatically (and consistently across > all variables) replaced with negative numbers such as -1, -2 etc. > > > > Both development versions can be installed from GitHub, using: > > > > library(remotes) > > install_github("tidyverse/haven") > > install_github("dusadrian/DDIwR") > > > > I'd be happy to be of assistance in case you need any help. > > > > I hope this helps, > > Adrian > > > > On Fri, Apr 23, 2021 at 9:23 PM Wendy Thomas wrote: > > I had this question within another conversation recently and was wondering > what people were using and if there were any new tools out there. > > > > "While I have your attention: what is the process of creating the DDI > codebooks based on existing data sets? Suppose I have a labeled data set in > Stata, what would I need to do to produce a DDI codebook XML file out if > it? " > > > > Answer to the list. I'll compile and I'm sure others are interested. > > > > Wendy > > > > -- > > Wendy L. Thomas Phone: +1 612.624.4389 > > Data Access Core Director Fax: +1 612.626.8375 > > Minnesota Population Center Email: wlt at umn.edu > > University of Minnesota > > 50 Willey Hall > > 225 19th Avenue South > > Minneapolis, MN 55455 > > _______________________________________________ > DDI-users mailing list > DDI-users at icpsr.umich.edu > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > > > > > -- > > Adrian Dusa > University of Bucharest > Romanian Social Data Archive > Soseaua Panduri nr. 90-92 > 050663 Bucharest sector 5 > Romania > > https://adriandusa.eu > _______________________________________________ > DDI-users mailing list > DDI-users at icpsr.umich.edu > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > -- Adrian Dusa University of Bucharest Romanian Social Data Archive Soseaua Panduri nr. 90-92 050663 Bucharest sector 5 Romania https://adriandusa.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210428/488abad8/attachment-0001.html From e.j.hrudey at vu.nl Wed Apr 28 08:24:45 2021 From: e.j.hrudey at vu.nl (Hrudey, E.J.) Date: Wed, 28 Apr 2021 12:24:45 +0000 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook In-Reply-To: References: Message-ID: Thank you for the follow-up Adrian. You?re plans sound exactly like the kind of tool that researchers would need. I?m still a bit of a noob to programming, but it?s good to know that these options are possible so if I ever get time I will also try to give it a go with the tips you?ve provided! From: on behalf of Adrian Du?a Reply-To: Data Documentation Initiative Users Group Date: Wednesday, April 28, 2021 at 11:44 AM To: Data Documentation Initiative Users Group Subject: Re: [DDI-users] Translation tools/processes from syntax files to DDICodebook Hi Jessica, The package DDIwR is designed to convert to and from all major statistical software, plus DDI. It can for instance convert from SPSS to Stata and the other way round, including missing values. It is also possible to extract all the metadata from any such data file (SPSS, Stata, SAS and R) to create a DDI Codebook v2.5. But that is about all it does automatically. Otherwise, additional information can of course be added to the XML file using the already existing R package xml2. Using this package, it is possible to query individual XML nodes, or modify those nodes or even create new ones. However, this is a command line experience, I am afraid the simplicity of NESSTAR could only be matched by creating a similar Graphical User Interface for the DDIwR package. This is also possible, and not even that complicated, but I didn't yet have the time to do something like this. For the moment, if you don't mind writing a few lines of code, you can achieve everything you might imagine in R. I reckon that documenting conditional routing can hardly be automatic. This is by definition a manual task, someone (preferably the original researcher) needs to put that information somewhere. The GUI that I have in mind would allow for this kind of information to be documented, and it would also allow creating the research instrument to be exported to virtually any type of survey tool, either out of the box of the tool is already standard and established, or to be imported via a dedicated API. I just need to make some time for this, the technology is already there. Best wishes, I hope this helps, Adrian On Wed, Apr 28, 2021 at 10:48 AM Hrudey, E.J. > wrote: Thanks for sharing your R package, Adrian, and for starting the conversation, Wendy. Re: the original question-I used NESSTAR and uploaded the data files I had (in this case SPSS, but it works with STATA too). NESSTAR automatically uploads the existing metadata from the dataset, but there is still a lot of manual work to do if the questionnaire had conditional questions/routing. NESSTAR is nice in that you can add additional information or include question text that is longer than the standard variable label, but everything is manual so it takes a lot of work. @Adrian-with your R package, is it designed solely to translate existing labeled data into a codebook or is it possible to add and modify the metadata (e.g. with NESSTAR there was the possible of providing additional information about routing)? And can is be used to convert data formats other than STATA? On a general note, after the time I spent working in NESSTAR and comparing it to Colectica, it seems to me it would be more efficient if the questions in a survey could be marked up at the level of the survey itself (to allow for capturing additional text or routing that cannot be readily included in the labels of the dataset variables). I think that?s what the Colectica Designer tool aims to do, but the problem is that while it allows you to design a survey in Designer and then export that marked-up survey to be uploaded into a survey tool, there are only a handful of survey tools that can utilize this exported marked-up survey. Unfortunately, it?s not so simple for my institution to just switch to another survey tool (we use Qualtrics), so I?m not sure how we can best create marked-up surveys that can be reused and that should make the process of creating a DDI codebook easier, without requiring researchers to create the survey two times: once in a DDI tool like Colectica and once in the survey tool itself. Has anyone else had this experience? Sincerely, Jessica From: > on behalf of Adrian Du?a > Reply-To: Data Documentation Initiative Users Group > Date: Friday, April 23, 2021 at 10:58 PM To: Data Documentation Initiative Users Group > Subject: Re: [DDI-users] Translation tools/processes from syntax files to DDICodebook I have an R package that can do that out of the box. The command is as simple as: convert("yourfile.dta", to = "DDI") The latest development version depends on another R package called "haven", which (in its latest development version but yet not published on CRAN) can read and write Stata style defined missing values such as .a, .b etc. In the XML codebook file, they are automatically (and consistently across all variables) replaced with negative numbers such as -1, -2 etc. Both development versions can be installed from GitHub, using: library(remotes) install_github("tidyverse/haven") install_github("dusadrian/DDIwR") I'd be happy to be of assistance in case you need any help. I hope this helps, Adrian On Fri, Apr 23, 2021 at 9:23 PM Wendy Thomas > wrote: I had this question within another conversation recently and was wondering what people were using and if there were any new tools out there. "While I have your attention: what is the process of creating the DDI codebooks based on existing data sets? Suppose I have a labeled data set in Stata, what would I need to do to produce a DDI codebook XML file out if it? " Answer to the list. I'll compile and I'm sure others are interested. Wendy -- Wendy L. Thomas Phone: +1 612.624.4389 Data Access Core Director Fax: +1 612.626.8375 Minnesota Population Center Email: wlt at umn.edu University of Minnesota 50 Willey Hall 225 19th Avenue South Minneapolis, MN 55455 _______________________________________________ DDI-users mailing list DDI-users at icpsr.umich.edu http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users -- Adrian Dusa University of Bucharest Romanian Social Data Archive Soseaua Panduri nr. 90-92 050663 Bucharest sector 5 Romania https://adriandusa.eu _______________________________________________ DDI-users mailing list DDI-users at icpsr.umich.edu http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users -- Adrian Dusa University of Bucharest Romanian Social Data Archive Soseaua Panduri nr. 90-92 050663 Bucharest sector 5 Romania https://adriandusa.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.icpsr.umich.edu/pipermail/ddi-users/attachments/20210428/d0815f95/attachment-0001.html From dan at colectica.com Wed Apr 28 09:35:56 2021 From: dan at colectica.com (Dan Smith) Date: Wed, 28 Apr 2021 08:35:56 -0500 Subject: [DDI-users] Translation tools/processes from syntax files to DDICodebook In-Reply-To: References: Message-ID: Hi Jessica, Colectica can automatically import the Qualtrics .qsf file to document a Qualtrics survey using DDI. You can import both your existing Qualtrics survey .qsf file and the SPSS dataset from Qualtrics. Cheers! Dan On 4/28/2021 2:41 AM, Hrudey, E.J. wrote: > Thanks for sharing your R package, Adrian, and for starting the > conversation, Wendy. > > Re: the original question-I used NESSTAR and uploaded the data files I > had (in this case SPSS, but it works with STATA too). NESSTAR > automatically uploads the existing metadata from the dataset, but there > is still a lot of manual work to do if the questionnaire had conditional > questions/routing. NESSTAR is nice in that you can add additional > information or include question text that is longer than the standard > variable label, but everything is manual so it takes a lot of work. > > @Adrian-with your R package, is it designed solely to translate existing > labeled data into a codebook or is it possible to add and modify the > metadata (e.g. with NESSTAR there was the possible of providing > additional information about routing)? And can is be used to convert > data formats other than STATA? > > On a general note, after the time I spent working in NESSTAR and > comparing it to Colectica, it seems to me it would be more efficient if > the questions in a survey could be marked up at the level of the survey > itself (to allow for capturing additional text or routing that cannot be > readily included in the labels of the dataset variables). I think that?s > what the Colectica Designer tool aims to do, but the problem is that > while it allows you to design a survey in Designer and then export that > marked-up survey to be uploaded into a survey tool, there are only a > handful of survey tools that can utilize this exported marked-up survey. > Unfortunately, it?s not so simple for my institution to just switch to > another survey tool (we use Qualtrics), so I?m not sure how we can best > create marked-up surveys that can be reused and that should make the > process of creating a DDI codebook easier, without requiring researchers > to create the survey two times: once in a DDI tool like Colectica and > once in the survey tool itself. > > Has anyone else had this experience? > > Sincerely, > > Jessica > > *From: * on behalf of Adrian Du?a > > *Reply-To: *Data Documentation Initiative Users Group > > *Date: *Friday, April 23, 2021 at 10:58 PM > *To: *Data Documentation Initiative Users Group > *Subject: *Re: [DDI-users] Translation tools/processes from syntax files > to DDICodebook > > I have an R package that can do that out of the box. The command is as > simple as: > > convert("yourfile.dta", to = "DDI") > > The latest development version depends on another R package called > "haven", which (in its latest development version but yet not published > on CRAN) can read and write Stata style defined missing values such as > .a, .b etc. > > In the XML codebook file, they are automatically (and consistently > across all variables) replaced with negative numbers such as -1, -2 etc. > > Both development versions can be installed from GitHub, using: > > library(remotes) > > install_github("tidyverse/haven") > > install_github("dusadrian/DDIwR") > > I'd be happy to be of?assistance in case you need any help. > > I hope this helps, > > Adrian > > On Fri, Apr 23, 2021 at 9:23 PM Wendy Thomas > wrote: > > I had this question within another conversation recently and was > wondering what people were using and if there were any new tools out > there. > > "While I have your attention: what is the process of creating the > DDI codebooks based on existing data sets? Suppose I have a labeled > data set in Stata, what would I need to do to produce a DDI codebook > XML file out if it? " > > Answer to the list. I'll compile and I'm sure others are interested. > > Wendy > > -- > > Wendy L. Thomas?????????????????????????? ???Phone: +1 612.624.4389 > > Data Access Core Director??????????????? ?Fax:?? +1 612.626.8375 > > Minnesota Population Center???????????? Email: wlt at umn.edu > > > University of Minnesota > > 50 Willey Hall > > 225 19th Avenue South > > Minneapolis, MN 55455 > > _______________________________________________ > DDI-users mailing list > DDI-users at icpsr.umich.edu > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > > > > -- > > Adrian Dusa > University of Bucharest > Romanian Social Data Archive > Soseaua Panduri nr. 90-92 > 050663 Bucharest sector 5 > Romania > > https://adriandusa.eu > > > _______________________________________________ > DDI-users mailing list > DDI-users at icpsr.umich.edu > http://lists.icpsr.umich.edu/mailman/listinfo/ddi-users > -- Dan Smith +1 608-213-2867 Colectica - Statistical Data Management http://www.colectica.com