OpenRefine: Difference between revisions

From The QA Company
Jump to navigation Jump to search
(Created page with "This page contains instructions to connect OpenRefine to edit the Eu Knowledge Graph. We offer a reconciliation service against the Eu Knowledge Graph that makes it easy to us...")
 
No edit summary
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page contains instructions to connect OpenRefine to edit the Eu Knowledge Graph. We offer a reconciliation service against the Eu Knowledge Graph that makes it easy to use OpenRefine.
This page contains instructions to connect OpenRefine to this Wikibase instance. We offer a reconciliation service against this Wikibase that makes it easy to use OpenRefine.


=== Requirements ===
=== Requirements ===
* an account on the Eu Knowledge Graph allowing to make edits. Request one here [mailto:CNECT-ECDORIS@ec.europa.eu CNECT-ECDORIS@ec.europa.eu].
* an account on the Wikibase instance allowing to make edits


=== Setup ===
=== Setup ===


* [https://openrefine.org/download.html Download OpenRefine] (we tested with version 3.5.2)
* [https://openrefine.org/download.html Download OpenRefine] (we tested with version 3.6.0)


* Upload your data. For example  
* Upload your data. For example  
Line 17: Line 17:
{| class="wikitable"
{| class="wikitable"
|+
|+
|entity
|Nom
|program cci
|Nationalité
|website
|Date de naissance
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463293</nowiki>
|<nowiki>Dennis Diefenbach</nowiki>
|2014GR16M1OP001
|Allemand/Italienne
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/Greece/2014GR16M1OP001</nowiki>
|<nowiki>2/15/1988</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463518</nowiki>
|<nowiki>Ali Haidar</nowiki>
|2014TC16I5CB001
|Libanais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB001</nowiki>
|<nowiki>5/10/1997</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463519</nowiki>
|<nowiki>Guo Kungpeng</nowiki>
|2014TC16I5CB002
|Chinois
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB002</nowiki>
|<nowiki>1/13/1996</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463520</nowiki>
|<nowiki>Clement Defretiere</nowiki>
|2014TC16I5CB003
|Francais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB003</nowiki>
|<nowiki>7/2/1999</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q4294076</nowiki>
|<nowiki>Lois Veni</nowiki>
|2014TC16I5CB004
|Francais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB004</nowiki>
|
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463524</nowiki>
|<nowiki>Jonathan Mallet</nowiki>
|2014TC16I5CB005
|Francais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB005</nowiki>
|
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463525</nowiki>
|2014TC16I5CB006
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB006</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463526</nowiki>
|2014TC16I5CB007
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB007</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463529</nowiki>
|2014TC16I5CB008
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB008</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463530</nowiki>
|2014TC16I5CB009
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB009</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463531</nowiki>
|2014TC16I5CB010
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB010</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463534</nowiki>
|2014TC16M4TN001
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16M4TN001</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463535</nowiki>
|2014TC16M4TN002
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16M4TN002</nowiki>
|}
|}


=== Configure the Eu Knowledge Graph as a Wikibase ===
=== Configure Open Refine for this Wikibase ===
* On the top right click on the Extensions Wikidata button and select "Select Wikibase instance"
* On the top right click on the Extensions Wikidata button and select "Select Wikibase instance"
* Add Wikibase
* Add Wikibase
Line 80: Line 53:
<syntaxhighlight lang="json">
<syntaxhighlight lang="json">
{
{
  "version":"1.0",
"version":"1.0",
  "mediawiki":{
  "mediawiki":{
      "name":"EU Knowledge Graph",
    "name":"Wikibase - The QA Company",
      "root":"https://linkedopendata.eu/wiki/",
    "root":"https://wikibase.the-qa-company.com/wiki/",
      "main_page":"https://linkedopendata.eu/wiki/Main_Page",
    "main_page":"https://wikibase.the-qa-company.com/wiki/Wikibase_-_The_QA_Company",
      "api":"https://linkedopendata.eu/w/api.php"
    "api":"https://wikibase.the-qa-company.com/w/api.php"
  },
  },
  "wikibase":{
  "wikibase":{
      "site_iri":"https://linkedopendata.eu/entity/",
    "site_iri":"https://wikibase.the-qa-company.com/entity/",
      "tag":"",
    "tag":"",
      "maxlag":5,
    "maxlag":5,
      "properties":{
    "properties":{
        "instance_of":"P35",
        "instance_of":"P5",
        "subclass_of":"P302"
        "subclass_of":"P47"
      },
    },
      "constraints":{
    "constraints":{
        "property_constraint_pid":"P861",
        "property_constraint_pid":"P58",
        "exception_to_constraint_pid":"P406",
        "exception_to_constraint_pid":"P120",
        "constraint_status_pid":"P1150",
 
        "mandatory_constraint_qid":"Q236765",
        "constraint_status_pid":"P119",
        "suggestion_constraint_qid":"Q236827",
        "mandatory_constraint_qid":"Q76",
        "distinct_values_constraint_qid":"Q196794"
        "suggestion_constraint_qid":"Q354",
      }
        "distinct_values_constraint_qid":"Q425"
  },
    }
  "reconciliation":{
  },
      "endpoint":"https://openrefine-reconciliation.linkedopendata.eu/${lang}/api"
  "reconciliation":{
  },
    "endpoint":"https://openrefine-reconciliation.wikibase.the-qa-company.com/${lang}/api"
  "editgroups":{
  },
      "url_schema":"([[:toollabs:editgroups/b/OR/${batch_id}|details]])"
  "editgroups":{
  }
    "url_schema":"([[:toollabs:editgroups/b/OR/${batch_id}|details]])"
  }
}
}
</syntaxhighlight>
</syntaxhighlight>


* Press "Add Wikibase" and the Eu Knowledge Graph will appear in the list as one options to choose.
* Press "Add Wikibase" and the Wikibase will appear in the list as one options to choose.


=== Reconcile your data ===
=== Reconcile your data ===
Line 119: Line 93:
In general follow the [https://docs.openrefine.org/manual/wikibase/reconciling instructions on OpenRefine] for these steps. Here is just an example:
In general follow the [https://docs.openrefine.org/manual/wikibase/reconciling instructions on OpenRefine] for these steps. Here is just an example:


* Choose the column "entity" and click on "Reconcile -> Use Values as Identifiers"
* Choose the column "Nom" and click on "Reconcile -> Use Values as Identifiers"
* Then the column will be completely reconcile
* Then the column will be completely reconcile


Line 128: Line 102:
* Click on "add item".  
* Click on "add item".  
* Drag and drop "entity" to the field "item"
* Drag and drop "entity" to the field "item"
* Click on "add statement" and find the property "CCI ID" (P1367)
* Click on "add statement" and find the property "birth date" (P145)
* Drag and drop the column name "program cci" to the statement item
* Drag and drop the column name "Date de naissance" to the statement item
* You can now do the same for the website column with the property "info Regio url" (P1742)
* You can now check if you have some issues with the import by checking the "Issues" tab or see a preview of what you've done.
* You can now check if you have some issues with the import by checking the "Issues" tab or see a preview of what you've done.


Line 139: Line 112:
* Upload Edits
* Upload Edits


The data you ingested is now available in the Eu Knowledge Graph!
The data you ingested is now available in the Wikibase!

Latest revision as of 20:30, 15 August 2022

This page contains instructions to connect OpenRefine to this Wikibase instance. We offer a reconciliation service against this Wikibase that makes it easy to use OpenRefine.

Requirements[edit | edit source]

  • an account on the Wikibase instance allowing to make edits

Setup[edit | edit source]

  • Upload your data. For example
    • Create Project
    • Clipboard
    • Insert the data in the Table below
    • Next
    • Create Project
Nom Nationalité Date de naissance
Dennis Diefenbach Allemand/Italienne 2/15/1988
Ali Haidar Libanais 5/10/1997
Guo Kungpeng Chinois 1/13/1996
Clement Defretiere Francais 7/2/1999
Lois Veni Francais
Jonathan Mallet Francais

Configure Open Refine for this Wikibase[edit | edit source]

  • On the top right click on the Extensions Wikidata button and select "Select Wikibase instance"
  • Add Wikibase
  • Paste
{
"version":"1.0",
  "mediawiki":{
     "name":"Wikibase - The QA Company",
     "root":"https://wikibase.the-qa-company.com/wiki/",
     "main_page":"https://wikibase.the-qa-company.com/wiki/Wikibase_-_The_QA_Company",
     "api":"https://wikibase.the-qa-company.com/w/api.php"
  },
  "wikibase":{
     "site_iri":"https://wikibase.the-qa-company.com/entity/",
     "tag":"",
     "maxlag":5,
     "properties":{
        "instance_of":"P5",
        "subclass_of":"P47"
     },
     "constraints":{
        "property_constraint_pid":"P58",
        "exception_to_constraint_pid":"P120",

        "constraint_status_pid":"P119",
        "mandatory_constraint_qid":"Q76",
        "suggestion_constraint_qid":"Q354",
        "distinct_values_constraint_qid":"Q425"
     }
  },
  "reconciliation":{
     "endpoint":"https://openrefine-reconciliation.wikibase.the-qa-company.com/${lang}/api"
  },
  "editgroups":{
     "url_schema":"([[:toollabs:editgroups/b/OR/${batch_id}|details]])"
  }
}
  • Press "Add Wikibase" and the Wikibase will appear in the list as one options to choose.

Reconcile your data[edit | edit source]

In general follow the instructions on OpenRefine for these steps. Here is just an example:

  • Choose the column "Nom" and click on "Reconcile -> Use Values as Identifiers"
  • Then the column will be completely reconcile

Model your data using the Wikibase Schema[edit | edit source]

In general follow the instructions on OpenRefine for these steps. Here is just an example:

  • Click on the schema tab
  • Click on "add item".
  • Drag and drop "entity" to the field "item"
  • Click on "add statement" and find the property "birth date" (P145)
  • Drag and drop the column name "Date de naissance" to the statement item
  • You can now check if you have some issues with the import by checking the "Issues" tab or see a preview of what you've done.

Upload your data[edit | edit source]

In general follow the instructions on OpenRefine for these steps. Here is just an example:

  • In the upper right click on Wikidata -> Upload Edits to Wikibase
  • Log in
  • Upload Edits

The data you ingested is now available in the Wikibase!

Retrieved from "https://wikibase.the-qa-company.com/w/index.php?title=OpenRefine&oldid=1512"