OpenRefine: Difference between revisions

From The QA Company
Jump to navigation Jump to search
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 6: Line 6:
=== Setup ===
=== Setup ===


* [https://openrefine.org/download.html Download OpenRefine] (we tested with version 3.5.2)
* [https://openrefine.org/download.html Download OpenRefine] (we tested with version 3.6.0)


* Upload your data. For example  
* Upload your data. For example  
Line 17: Line 17:
{| class="wikitable"
{| class="wikitable"
|+
|+
|entity
|Nom
|program cci
|Nationalité
|website
|Date de naissance
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463293</nowiki>
|<nowiki>Dennis Diefenbach</nowiki>
|2014GR16M1OP001
|Allemand/Italienne
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/Greece/2014GR16M1OP001</nowiki>
|<nowiki>2/15/1988</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463518</nowiki>
|<nowiki>Ali Haidar</nowiki>
|2014TC16I5CB001
|Libanais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB001</nowiki>
|<nowiki>5/10/1997</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463519</nowiki>
|<nowiki>Guo Kungpeng</nowiki>
|2014TC16I5CB002
|Chinois
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB002</nowiki>
|<nowiki>1/13/1996</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463520</nowiki>
|<nowiki>Clement Defretiere</nowiki>
|2014TC16I5CB003
|Francais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB003</nowiki>
|<nowiki>7/2/1999</nowiki>
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q4294076</nowiki>
|<nowiki>Lois Veni</nowiki>
|2014TC16I5CB004
|Francais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB004</nowiki>
|
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463524</nowiki>
|<nowiki>Jonathan Mallet</nowiki>
|2014TC16I5CB005
|Francais
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB005</nowiki>
|
|-
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463525</nowiki>
|2014TC16I5CB006
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB006</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463526</nowiki>
|2014TC16I5CB007
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB007</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463529</nowiki>
|2014TC16I5CB008
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB008</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463530</nowiki>
|2014TC16I5CB009
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB009</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463531</nowiki>
|2014TC16I5CB010
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16I5CB010</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463534</nowiki>
|2014TC16M4TN001
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16M4TN001</nowiki>
|-
|<nowiki>https://linkedopendata.eu/entity/Q2463535</nowiki>
|2014TC16M4TN002
|<nowiki>https://ec.europa.eu/regional_policy/EN/atlas/programmes/2014-2020/europe/2014TC16M4TN002</nowiki>
|}
|}


Line 114: Line 87:
</syntaxhighlight>
</syntaxhighlight>


* Press "Add Wikibase" and the Eu Knowledge Graph will appear in the list as one options to choose.
* Press "Add Wikibase" and the Wikibase will appear in the list as one options to choose.


=== Reconcile your data ===
=== Reconcile your data ===
Line 120: Line 93:
In general follow the [https://docs.openrefine.org/manual/wikibase/reconciling instructions on OpenRefine] for these steps. Here is just an example:
In general follow the [https://docs.openrefine.org/manual/wikibase/reconciling instructions on OpenRefine] for these steps. Here is just an example:


* Choose the column "entity" and click on "Reconcile -> Use Values as Identifiers"
* Choose the column "Nom" and click on "Reconcile -> Use Values as Identifiers"
* Then the column will be completely reconcile
* Then the column will be completely reconcile


Line 129: Line 102:
* Click on "add item".  
* Click on "add item".  
* Drag and drop "entity" to the field "item"
* Drag and drop "entity" to the field "item"
* Click on "add statement" and find the property "CCI ID" (P1367)
* Click on "add statement" and find the property "birth date" (P145)
* Drag and drop the column name "program cci" to the statement item
* Drag and drop the column name "Date de naissance" to the statement item
* You can now do the same for the website column with the property "info Regio url" (P1742)
* You can now check if you have some issues with the import by checking the "Issues" tab or see a preview of what you've done.
* You can now check if you have some issues with the import by checking the "Issues" tab or see a preview of what you've done.


Line 140: Line 112:
* Upload Edits
* Upload Edits


The data you ingested is now available in the Eu Knowledge Graph!
The data you ingested is now available in the Wikibase!

Latest revision as of 20:30, 15 August 2022

This page contains instructions to connect OpenRefine to this Wikibase instance. We offer a reconciliation service against this Wikibase that makes it easy to use OpenRefine.

Requirements[edit | edit source]

  • an account on the Wikibase instance allowing to make edits

Setup[edit | edit source]

  • Upload your data. For example
    • Create Project
    • Clipboard
    • Insert the data in the Table below
    • Next
    • Create Project
Nom Nationalité Date de naissance
Dennis Diefenbach Allemand/Italienne 2/15/1988
Ali Haidar Libanais 5/10/1997
Guo Kungpeng Chinois 1/13/1996
Clement Defretiere Francais 7/2/1999
Lois Veni Francais
Jonathan Mallet Francais

Configure Open Refine for this Wikibase[edit | edit source]

  • On the top right click on the Extensions Wikidata button and select "Select Wikibase instance"
  • Add Wikibase
  • Paste
{
"version":"1.0",
  "mediawiki":{
     "name":"Wikibase - The QA Company",
     "root":"https://wikibase.the-qa-company.com/wiki/",
     "main_page":"https://wikibase.the-qa-company.com/wiki/Wikibase_-_The_QA_Company",
     "api":"https://wikibase.the-qa-company.com/w/api.php"
  },
  "wikibase":{
     "site_iri":"https://wikibase.the-qa-company.com/entity/",
     "tag":"",
     "maxlag":5,
     "properties":{
        "instance_of":"P5",
        "subclass_of":"P47"
     },
     "constraints":{
        "property_constraint_pid":"P58",
        "exception_to_constraint_pid":"P120",

        "constraint_status_pid":"P119",
        "mandatory_constraint_qid":"Q76",
        "suggestion_constraint_qid":"Q354",
        "distinct_values_constraint_qid":"Q425"
     }
  },
  "reconciliation":{
     "endpoint":"https://openrefine-reconciliation.wikibase.the-qa-company.com/${lang}/api"
  },
  "editgroups":{
     "url_schema":"([[:toollabs:editgroups/b/OR/${batch_id}|details]])"
  }
}
  • Press "Add Wikibase" and the Wikibase will appear in the list as one options to choose.

Reconcile your data[edit | edit source]

In general follow the instructions on OpenRefine for these steps. Here is just an example:

  • Choose the column "Nom" and click on "Reconcile -> Use Values as Identifiers"
  • Then the column will be completely reconcile

Model your data using the Wikibase Schema[edit | edit source]

In general follow the instructions on OpenRefine for these steps. Here is just an example:

  • Click on the schema tab
  • Click on "add item".
  • Drag and drop "entity" to the field "item"
  • Click on "add statement" and find the property "birth date" (P145)
  • Drag and drop the column name "Date de naissance" to the statement item
  • You can now check if you have some issues with the import by checking the "Issues" tab or see a preview of what you've done.

Upload your data[edit | edit source]

In general follow the instructions on OpenRefine for these steps. Here is just an example:

  • In the upper right click on Wikidata -> Upload Edits to Wikibase
  • Log in
  • Upload Edits

The data you ingested is now available in the Wikibase!