Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions doc/release-notes/12614-topicClassTermURI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
Analogue to the addition of a termURI field to the keywords in the citation metadata blocks, the same idea should also be applied to the topicClassification fields:

add a new subfield topicClassTermURI to the citation metadata block

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused. In this pull request should scripts/api/data/metadatablocks/citation.tsv be updated to include a new field?

Assuming so, this release note snippet should also include a ## Upgrade instructions section at the bottom explaining how to load it and how to update Solr to include the new field. https://github.com/IQSS/dataverse/releases/tag/v6.8 is probably a decent example. Here's a screenshot:

Image

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pdurbin I have updated the documentation by adapting the former keywordTermURI migration with https://github.com/IQSS/dataverse/releases/tag/v6.3

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. From a quick look I'm wondering if all that SQL should be put into a migration script so it can be run automatically. If you're not familiar with how we use Flyway for this: https://guides.dataverse.org/en/6.11/developers/sql-upgrade-scripts.html

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it can be automated - people may have been using the existing field correctly, in which case values shouldn't be migrated.

migrate contents from topicClassVocabURI to topicClassTermURI
change DataCite export format, so that the content of topicClassTermURI goes into the property valueURI of the subject element
old:

<subject schemeURI="https://github.com/tibonto/dfgfo/44" subjectScheme="DFGFO">
Computer Science, Systems and Electrical Engineering
</subject>



new:

<subject schemeURI="https://github.com/tibonto/dfgfo/" valueURI="https://github.com/tibonto/dfgfo/44" subjectScheme="DFGFO">
Computer Science, Systems and Electrical Engineering
</subject>

in the general case:



<subject schemeURI="topicClassVocabURI" valueURI="topicClassTermURI" subjectScheme="topicClassVocab">topicClassValue</subject>


The implementation expects a new metadata field called "topicClassTermURI". The old field was called "topicClassVocabURI".
The old field still holds an URL but the schemeUrl but not the TermURI of the metadata value.
This change has been adapted to the
DdiExportUtil.java
OpenAireExportUtil.java
XmlMetadataTemplate.java

In DdiExportUtil the field "topicClassVocabURI" is overwritten with "topicClassTermURI" because this is the new vocabUri.

In OpenAireExportUtil the changes are made and tested like in the explanation above.

In XmlMetadataTemplate there is just a new getter and an addition of the new "valueURI" attribute.

You can migrate your topicClassVocabURI data containing URIs to the new topicClassTermURI field.
In case of data migration, view the affected data with the following database query:

SELECT value FROM datasetfieldvalue dfv
INNER JOIN datasetfield df ON df.id = dfv.datasetfield_id
WHERE df.datasetfieldtype_id = (SELECT id FROM datasetfieldtype WHERE name = 'topicClassVocabURI')
AND value ILIKE 'http%';


If you wish to migrate your data, a database update is then necessary:


UPDATE datasetfield df
SET datasetfieldtype_id = (SELECT id FROM datasetfieldtype WHERE name = 'topicClassTermURI')
FROM datasetfieldvalue dfv
WHERE dfv.datasetfield_id = df.id
AND df.datasetfieldtype_id = (SELECT id FROM datasetfieldtype WHERE name = 'topicClassVocabURI')
AND dfv.value ILIKE 'http%';
107 changes: 54 additions & 53 deletions scripts/api/data/metadatablocks/citation.tsv

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ public class DatasetFieldConstant implements java.io.Serializable {
public static final String topicClassValue="topicClassValue";
public static final String topicClassVocab="topicClassVocab";
public static final String topicClassVocabURI="topicClassVocabURI";
public static final String topicClassTermURI="topicClassTermURI";
public static final String descriptionText="dsDescriptionValue";
public static final String descriptionDate="dsDescriptionDate";
public static final String timePeriodCovered="timePeriodCovered"; // SEK added 6/13/2016
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -837,7 +837,7 @@ DatasetFieldConstant.keywordValue, CITATION_BLOCK_NAME, new Locale(lang),
for (HashSet<FieldDTO> foo : fieldDTO.getMultipleCompound()) {
Comment thread
pdurbin marked this conversation as resolved.
String topicClassificationValue = "";
String topicClassificationVocab = "";
String topicClassificationURI = "";
String topicClassificationTermURI = "";
for (Iterator<FieldDTO> iterator = foo.iterator(); iterator.hasNext();) {
FieldDTO next = iterator.next();
if (DatasetFieldConstant.topicClassValue.equals(next.getTypeName())) {
Expand All @@ -851,14 +851,14 @@ DatasetFieldConstant.keywordValue, CITATION_BLOCK_NAME, new Locale(lang),
if (DatasetFieldConstant.topicClassVocab.equals(next.getTypeName())) {
topicClassificationVocab = next.getSinglePrimitive();
}
if (DatasetFieldConstant.topicClassVocabURI.equals(next.getTypeName())) {
topicClassificationURI = next.getSinglePrimitive();
if (DatasetFieldConstant.topicClassTermURI.equals(next.getTypeName())) {
topicClassificationTermURI = next.getSinglePrimitive();
}
}
if (!topicClassificationValue.isEmpty()) {
xmlw.writeStartElement("topcClas");
XmlWriterUtil.writeAttribute(xmlw, "vocab", topicClassificationVocab);
XmlWriterUtil.writeAttribute(xmlw, "vocabURI", topicClassificationURI);
XmlWriterUtil.writeAttribute(xmlw, "vocabURI", topicClassificationTermURI);
if (lang != null && isCVV) {
XmlWriterUtil.writeAttribute(xmlw, "xml:lang", defaultLocale.getLanguage());
xmlw.writeCharacters(ControlledVocabularyValue.getLocaleStrValue(
Expand All @@ -875,7 +875,7 @@ DatasetFieldConstant.keywordValue, CITATION_BLOCK_NAME, new Locale(lang),
if (translatedValue != null) {
xmlw.writeStartElement("topcClas");
XmlWriterUtil.writeAttribute(xmlw, "vocab", topicClassificationVocab);
XmlWriterUtil.writeAttribute(xmlw, "vocabURI", topicClassificationURI);
XmlWriterUtil.writeAttribute(xmlw, "vocabURI", topicClassificationTermURI);
XmlWriterUtil.writeAttribute(xmlw, "xml:lang", lang);
xmlw.writeCharacters(translatedValue);
xmlw.writeEndElement(); // topcClas
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,7 @@ public static void writeSubjectsElement(XMLStreamWriter xmlw, DatasetVersionDTO
String subject = null;
String subjectScheme = null;
String schemeURI = null;
String valueURI = null;

for (Iterator<FieldDTO> iterator = fieldDTOs.iterator(); iterator.hasNext();) {
FieldDTO next = iterator.next();
Expand All @@ -507,11 +508,15 @@ public static void writeSubjectsElement(XMLStreamWriter xmlw, DatasetVersionDTO
if (DatasetFieldConstant.topicClassVocabURI.equals(next.getTypeName())) {
schemeURI = next.getSinglePrimitive();
}

if (DatasetFieldConstant.topicClassTermURI.equals(next.getTypeName())) {
valueURI = next.getSinglePrimitive();
}
}

if (StringUtils.isNotBlank(subject)) {
subject_check = writeOpenTag(xmlw, "subjects", subject_check);
writeSubjectElement(xmlw, subjectScheme, null, schemeURI, subject, language);
writeSubjectElement(xmlw, subjectScheme, valueURI, schemeURI, subject, language);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@ private void writeSubjects(XMLStreamWriter xmlw, DvObject dvObject) throws XMLSt
String topic = null;
String scheme = null;
String schemeUri = null;
String valueURI = null;

for (DatasetField subField : topicFieldValue.getChildDatasetFields()) {

Expand All @@ -401,6 +402,9 @@ private void writeSubjects(XMLStreamWriter xmlw, DvObject dvObject) throws XMLSt
case DatasetFieldConstant.topicClassVocabURI:
schemeUri = subField.getValue();
break;
case DatasetFieldConstant.topicClassTermURI:
valueURI = subField.getValue();
break;
}
}
if (StringUtils.isNotBlank(topic)) {
Expand All @@ -411,6 +415,9 @@ private void writeSubjects(XMLStreamWriter xmlw, DvObject dvObject) throws XMLSt
if (StringUtils.isNotBlank(schemeUri)) {
attributesMap.put("schemeURI", schemeUri);
}
if (StringUtils.isNotBlank(valueURI)) {
attributesMap.put("valueURI", valueURI);
}
subjectsCreated = XmlWriterUtil.writeOpenTagIfNeeded(xmlw, "subjects", subjectsCreated);
XmlWriterUtil.writeFullElementWithAttributes(xmlw, "subject", attributesMap, StringEscapeUtils.escapeXml10(topic));
}
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/propertyFiles/citation.properties
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ datasetfieldtype.topicClassification.title=Topic Classification
datasetfieldtype.topicClassValue.title=Term
datasetfieldtype.topicClassVocab.title=Controlled Vocabulary Name
datasetfieldtype.topicClassVocabURI.title=Controlled Vocabulary URL
datasetfieldtype.topicClassTermURI.title=Term URI
datasetfieldtype.publication.title=Related Publication
datasetfieldtype.publicationRelationType.title=Relation Type
datasetfieldtype.publicationCitation.title=Citation
Expand Down Expand Up @@ -110,6 +111,7 @@ datasetfieldtype.topicClassification.description=Indicates a broad, important to
datasetfieldtype.topicClassValue.description=A topic or subject term
datasetfieldtype.topicClassVocab.description=The controlled vocabulary used for the keyword term (e.g. LCSH, MeSH)
datasetfieldtype.topicClassVocabURI.description=The URL where one can access information about the term's controlled vocabulary
datasetfieldtype.topicClassTermURI.description=A URI that points to the web presence of the Topic Classification Term
datasetfieldtype.publication.description=The article or report that uses the data in the Dataset. The full list of related publications will be displayed on the metadata tab
datasetfieldtype.publicationRelationType.description=The nature of the relationship between this Dataset and the related publication
datasetfieldtype.publicationCitation.description=The full bibliographic citation for the related publication
Expand Down Expand Up @@ -190,6 +192,7 @@ datasetfieldtype.topicClassification.watermark=
datasetfieldtype.topicClassValue.watermark=
datasetfieldtype.topicClassVocab.watermark=
datasetfieldtype.topicClassVocabURI.watermark=https://
datasetfieldtype.topicClassTermURI.watermark=https://
datasetfieldtype.publication.watermark=
datasetfieldtype.publicationRelationType.watermark=
datasetfieldtype.publicationCitation.watermark=
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -300,8 +300,8 @@
"typeClass": "primitive",
"value": "TC Vocabulary"
},
"topicClassVocabURI": {
"typeName": "topicClassVocabURI",
"topicClassTermURI": {
"typeName": "topicClassTermURI",
"multiple": false,
"typeClass": "primitive",
"value": "http://www.topicClass.com/one"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -304,8 +304,8 @@
"typeClass": "primitive",
"value": "TC Vocabulary"
},
"topicClassVocabURI": {
"typeName": "topicClassVocabURI",
"topicClassTermURI": {
"typeName": "topicClassTermURI",
"multiple": false,
"typeClass": "primitive",
"value": "http://www.topicClass.com/one"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks are failing:

Image

We should fix this. Don't worry, we'll help. 😄

Expand Down
Loading