Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"override_extras": true,
"default_tags": [{
"name": "Harvested"
}],
"default_extras": {
"private":true
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
[
{
"title": "Harvesting example dataset 1",
"description": "A longer description of dataset 1",
"identifier": "https://data.some.org/catalog/datasets/1",
"issued": "2013-04-26T00:00:00Z",
"modified": "2013-04-26T00:00:00Z",
"landingPage": "",
"publisher": {
"name": "Publishing Organization for dataset 1",
"mbox": "contact@some.org"
},
"distribution": [
{
"accessURL": "",
"byteSize": "",
"description": "A longer description for the distribution",
"downloadURL": "https://data.some.org/the/actual/catalog/datasets/1/resources/1.csv",
"format": "text/csv",
"issued": "",
"license": "https://data.some.org/link/to/license",
"modified": "",
"title": "A distribution for this dataset in CSV format"
},
{
"accessURL": "",
"byteSize": "",
"description": "A longer description for the distribution",
"downloadURL": "https://data.some.org/the/actual/catalog/datasets/1/resources/2.pdf",
"format": "application/pdf",
"issued": "",
"license": "https://data.some.org/link/to/license",
"modified": "",
"title": "A distribution for this dataset in PDF format"
},
{
"accessURL": "https://data.some.org/the/actual/catalog/link/to/an/api",
"byteSize": "",
"description": "A longer description for the distribution",
"downloadURL": "",
"format": "",
"issued": "",
"license": "",
"modified": "",
"title": "A distribution for this dataset as an API"
}
],
"keyword": [
"stats",
"pollution"
],
"language": "http://publications.europa.eu/resource/authority/language/ENG"
},
{
"title": "Harvesting example dataset 2",
"description": "A longer description of dataset 2: with two distributions, a website and a PDF file. Some special characters: Łódź",
"identifier": "https://data.some.org/catalog/datasets/2",
"issued": "2012-06-21",
"modified": "2013-04-01",
"landingPage": "",
"publisher": {
"name": "Publishing Organization for dataset 2",
"mbox": "contact@another.org"
},
"distribution": [
{
"accessURL": "",
"byteSize": "",
"description": "A longer description for the distribution",
"downloadURL": "https://external.site.org/",
"format": "html",
"issued": "",
"license": "",
"modified": "",
"title": "A distribution for this dataset as an external website"
},
{
"accessURL": "",
"byteSize": "",
"description": "A longer description for the distribution",
"downloadURL": "https://data.some.org/the/actual/catalog/datasets/2/resources/2.pdf",
"format": "application/pdf",
"issued": "",
"license": "",
"modified": "",
"title": "A distribution for this dataset in PDF format"
}
],
"keyword": [
"stats",
"education",
"sloths"
],
"language": "http://publications.europa.eu/resource/authority/language/SWE"
}
]

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
{
"primaryKey": "id",
"fields": [
{
"name": "id",
"title": "Unique Identifier (Integer)",
"type": "integer"
},
{
"name": "Name",
"title": "Full Name",
"type": "string"
},
{
"name": "Date of Birth",
"title": "Date of Birth (Date)",
"type": "date",
"format": "%d/%m/%Y"
},
{
"name": "Nationality",
"title": "Nationality (String:enum)",
"type": "string",
"constraints": {
"enum": [
"Dutch",
"Portuguese",
"Greek"
]
}
},
{
"name": "Height",
"title": "How's the weather up there? (Number: required)",
"type": "number",
"constraints": {
"required": true
}
},
{
"name": "latitude",
"title": "latitude",
"type": "string",
"constraints": {
"pattern": "[0-9]{2}\\.[0-9]{6,}"
}
},
{
"name": "longitude",
"title": "longitude",
"type": "string",
"constraints": {
"pattern": "[0-9]{2}\\.[0-9]{6,}"
}
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id,Name,Date of Birth,Nationality,Height,latitude,longitude
1,Niek,01/02/1990,Dutch,1.85,52.1060481,05.2580944
2,Gil,03/04/1990,Portuguese,2.45,52.1060481,05.2580944
3,Georgios,05/06/1990,Greek,0.5,52.1060481,05.2580944
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id,Name,Date of Birth,Nationality,Height,latitude,longitude
1,Niek,01/02/1990,Spanish,1.85,52.1060481,5.2580944
2,Gil,25/25/1990,Portuguese,2.45,52.1060481,05.2580944
3,Georgios,05-06-1990,Greek,this is not a number,52.1060481,5.2
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
id,Name,Date of Birth,Nationality,missing_height_header,latitude,longitude, extra_header_1, extra_header_2
1,Niek,01/02/1990,Dutch,1.85,0,52.1060481,5.2580944
2,Gil,03/04/1990,Portuguese,2.45,15,52.1060481,5.2580944
,,,,,,,
3,Georgios,05/06/1990,Greek,0.5,2.00,52.1060481,5.2580944
1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6,7,8,9,1,2,3,4
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"skip_checks": ["extra-headers"]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
id: ckan_Overview
title: Overzicht
sidebar_label: Overzicht
---

Hier vindt u een overzicht van wat CKAN is, naast wat u met CKAN kunt bereiken.

## CKAN

Dataplatform maakt gebruik van de open source software CKAN. Dit is wereldwijd de meest gebruikte software voor open data-omgevingen.

CKAN staat voor "Comprehensive Knowledge Archive Network", een <a href="https://en.wikipedia.org/wiki/Web_application" target="_blank">web-based</a>, <a href="https://en.wikipedia.org/wiki/Open-source_software" target="_blank">open-source</a> managementsysteem voor de opslag en distributie van <a href="https://en.wikipedia.org/wiki/Open_data" target="_blank">open data</a>..

Het Dataplatform-project van Civity maakt gebruik van een CKAN-instantie als onze webgebaseerde tool voor het onderhoud van datasets, gegevensbronnen en metagegevens. U kunt onze CKAN-instantie vinden op <a href="https://ckan.dataplatform.nl/" target="_blank">ckan.dataplatform.nl</a>. De CKAN omgeving binnen het Dataplatform is uitgebreid met extra uitbreidingen zoals DCAT, DataStore (API), Data Quality controle en GeoServer.

## Taalkeuze
>De taalkeuze bevindt zich in de voettekst.

CKAN ondersteunt vele Europese talen. Momenteel is het Nederlandse CKAN Dataplatform beschikbaar in het Nederlands en Engels;

Het Zweedse Dataplatform is beschikbaar in het Zweeds (Svenska) en Engels.

## Datasets
> datasets worden beheerd op <a href="https://ckan.dataplatform.nl/" target="_blank">https://ckan.dataplatform.nl</a> en gepresenteerd op <a href="https://dataplatform.nl" target="_blank">https://dataplatform.nl</a>

Datasets bestaan uit twee hoofdaspecten:
* Een verzameling gegevensbronnen
* Metadata


## Metadata van de gegevenssets
De metadata bevat informatie over de verzamelen aan databronnen in de dataset. De metagegevens moeten informatie geven over:
* Wie de eigenaar is van de dataset
* Beschrijvende informatie over de inhoud
* Toegangsrechten

## Gegevensbronnen
Een dataset bestaat uit een verzameling gegevensbronnen; dit zijn de feitelijke bestanden of koppelingen die gegevens leveren. Uw gegevensbron kan bestaan uit één bron of uit meerdere bronnen, elke bestandsformaat en moet expliciete metagegevens bevatten.

## Metadata
> Metadata is gewoon data over data

Elke dataset bevat metadata. Voor open data volgen we de richtlijnen uit de DCAT-Standaard.

Metadata is informatie die data over andere data oplevert. Meestal betekent dit een beschrijving en context over bestaande gegevens.

Zie [Metadata](metadata_Metadata#metadata-example) voor een uitgebreid voorbeeld.
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
id: dataoverheid_LinkToDataOverheid
title: Link To Data Overheid
sidebar_label: Link To Data Overheid
---

The link to *Data Overheid* is the service that allows datasets in <a href="https://www.dataplatform.nl/" target="_blank">**Dataplatform**</a> to be synched with <a href="https://data.overheid.nl/" target="_blank">**Data Overheid**</a>.
![Dataplatform_To_Data_Overheid](assets/Dataplatform/LinkToDataOverheid/Dataplatform_to_Overheid.svg)

## How it works

This service **runs every night**, and syncronyzes all, **publicly available**, datasets in *Dataplatform*.

> Only **public datasets** in *Dataplatform* are synched to *Data Overheid*

Metadata fields **restrictions** vary in *Data Overheid* and *Dataplatform*.
In order to comply with both schemas we defined a set of **default values** that are used
to replace invalid/absent values.

Here is the list of **Default Values** and corresponding metadata fields in *Dataplatform*/*Data Overheid*:


| Dataplatform Field | Data Overheid Field | Default Value |
|---|---|---|
|owner_org (*organization*) | authority | 'http://standaarden.overheid.nl/op/terms/overig' |
|maintainer_email | contact_point_email | 'support@civity.nl' |
|maintainer | contact_point_name | 'Dataplatform' |
|(any) date field | (any) date field | Current Time in following format '%Y-%m-%dT%H:%M:%S' |
|description | description | 'No Description' |
|format | format | 'http://publications.europa.eu/resource/authority/file-type/TXT' |
|language | language | 'http://publications.europa.eu/resource/authority/language/NLD' |
|license_id | license_id | 'http://creativecommons.org/publicdomain/zero/1.0/deed.nl' |
|metadata_language | metadata_language | 'http://publications.europa.eu/resource/authority/language/NLD' |
|name | name | 'Naamloze bron' |
|notes | notes | 'Geen beschrijving' |
|publisher | publisher | 'http://standaarden.overheid.nl/op/terms/overig' |
|theme | theme | 'http://standaarden.overheid.nl/owms/terms/Bestuur' |
|title | title | 'Naamloze dataset' |

## Data Overheid
More information about ***Data Overheid*** here:
- <a href="https://data.overheid.nl/" target="_blank">***Data Overheid* Portal**</a>
- <a href="https://data.overheid.nl/data/" target="_blank">***Data Overheid* CKAN**</a>
- <a href="https://ckanext-dcatdonl.readthedocs.io/en/latest/index.html" target="_blank">**Documentation DataOverheid Extension**</a>
- <a href="https://ckanext-dcatdonl.readthedocs.io/en/latest/schema-dataset.html" target="_blank">**Dataset Metadata Schema**</a>
- <a href="https://ckanext-dcatdonl.readthedocs.io/en/latest/schema-resource.html" target="_blank">**Resource Metadata Schema**</a>
- <a href="https://github.com/dataoverheid/ckanext-dataoverheid" target="_blank">**GitHub repository for DataOverheid Extension**</a>

## Known Limitations

- Resources can't have the same name under the same dataset.
- Names are the unique identifier, can't be duplicated in the whole Data Overheid side.
- Currently not all the metadata fields are being synchronized.
Loading