README.md 16.5 KB
Newer Older
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
1
2
# BDTI Data Catalogue

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
3
4
5
6
1. **Canada Open Data**
2. **DBpedia**
3. **Estat**
4. **European Open Data Portal**
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
7
8
9
10
11
5. **IMF Economic Data**
6. **NASA Planetary Data System**
7. **National Centers for Environmental Information (NCEI)**
8. **National Institute of Drug Abuse (NIDA)**
9. **The CIA World Factbook**
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
12
13
14
15
16
17
18
19
20

The following table provide for each of the aforementioned sources the following information (that will be included in the catalogue):

- Name
- Description
- Types of API
- Notes
- Policy domain

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
21
### 1. Canada open data
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
Government of Canada | Canada open data allows searching open data, learning how to work with datasets, and seeing what people have done with open data across the country. | Canada open data APIs provide a wealth of government data and information – to build new applications and services (especially for Canadian people). A list of Canada open data APis is available [here](https://api.canada.ca/en/homepage).

**Notes**

Canada open data is a pilot project with many government and geospatial datasets. It could help to explore how the Government of Canada creates greater transparency, accountability, increases citizen engagement, and drives innovation and economic opportunities through open data, open information, and open dialogue. |

**Policy Domains**

- Agriculture
- Art, music, literature
- Economics and industries
- Education and training
- Government and politics
- Health and safety
- History and archaeology
- Information and communication
- Labour
- Language and linguistics
- Law
- Military
- Nature and environment
- Persons
- Processes
- Science and technology
- Society and culture
- Transport

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
52
### 2. DBpedia
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
53
54
55

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
56
| DBpedia | DBpedia aims at getting structured content from the valuable information that Wikipedia created. | SPARQL queries against endpoint or by downloading their [dumps](http://wikidata.dbpedia.org/develop/datasets?msclkid=610998fdb4fe11ecb18b5848ded87514). |
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

**Notes**

With DBpedia, you can semantically search and explore relationships and properties of Wikipedia resource. This includes links to other related datasets as well.There are around 4.58 million entities in the DBpedia dataset. 4.22 million are classified in ontology, including 1,445,000 persons,  5,000 places, 123,000 music albums,  ,000 films, 19,000 video games, 241,000 organizations, 251,000 species and 6,000 diseases.There are labels and abstracts for these entities in around 125 languages. There are 25.2 million links to images. There are 29.8 million links to external web pages.To use DBpedia one must write SPARQL queries against endpoint or by downloading their dumps.DBpedia has benefitted several enterprises, such as Apple (via Siri), Google (via Freebase and Google Knowledge Graph), and IBM (via Watson), and particularly their respective prestigious projects associated with artificial intelligence. 

**Policy Domains**

- Social questions
- Science
- Environment
- Employment and working conditions
- Economics
- Finance
- Production, technology and research
- Trade
- European Union
- Industry
- Agriculture. Forestry and fisheries
- Energy
- Transport
- Business and competition
- International relations
- Geography
- Education and communications
- Law
- International organisations
- Politics

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
85
### 3. Estat
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| Estat | Eurostat is the statistical office of the European Union, responsible for publishing high-quality Europe-wide statistics and indicators that enable comparisons between countries and regions. | There are two kind of Web Services available: [SDMX Web Services (SOAP and REST)](http://ec.europa.eu/eurostat/web/sdmx-web-services/about-this-service) and [Json and Unicode WebServices(REST)](http://ec.europa.eu/eurostat/web/json-and-unicode-web-services/about-this-service). |

**Notes**

Eurostat data contains many indicators (short-term, structural, theme-specific and others) on the EU-28 and the Eurozone, the Member States and their partners. The database of Eurostat contains always the latest version of the datasets meaning that there is no versioning on the data. The SDMX Web Services are a programmatic access to Eurostat data, with the possibility to: i) get a complete list of publicly available datasets, ii) detail the complete structure definition of a given dataset, iii) download a subset of a given dataset or a full dataset, meanwhile the JSON & UNICODE Web Services he Web Services i) provide data in JSON-stat and UNICODE formats, ii) support only Representation State Transfer (REST) protocol, iii) deliver responses in English, French and German language.

**Policy Domains**

- General and regional statistics
- Industry, trade and services
- Transport
- Economy and finance
- Agriculture and fisheries
- Environment and energy
- Population and social conditions
- International trade
- Science, technology and digital society

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
107
### 4. European Open Data Portal
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| European Open Data Portal | Catalogue of metadata describing open data published by different institutions and bodies of the European Union (e.g., Eurostat, Publication Office, the different DG Units, European Environment Agency, European Banking Authority, etc.) | Two programming interfaces are proposed to developers to access to datasets: REST API access with CKAN: http://data.europa.eu/euodp/data/api/ and the SPARQL endpoint: http://data.europa.eu/euodp/sparqlep ||

**Notes**

The European Union open data portal is, as in the previous case, a catalogue of metadata that documents the variety of open datasets across institutions and bodies of the European Union. As such, it does not properly provide itself APIs for the data it documents. Rather, it provides SPARQL queries and API REST functionalities over the catalogue of metadata it collects. The datasets documented in the EU Open data portal may be associated with specific types of APIs that have been analysed in the context of this study. To the best of our knowledge, the APIs categories available through this catalogue are mainly APIs Rest and SPARQL-based queries, as reported above. 

**Policy Domain** 

- Social questions
- Science
- Environment
- Employment and working conditions
- Economics
- Finance
- Production, technology and research
- Trade
- European Union
- Industry
- Agriculture. Forestry and fisheries
- Energy
- Transport
- Business and competition
- International relations
- Geography
- Education and communications
- Law
- International organisations
- Politics
- Agri-foodstuffs

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
141
### 5. IMF Economic Data
Wouter Maroy's avatar
Wouter Maroy committed
142

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
143
144
145
| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| | The International Monetary Fund (IMF) publishes a range of time series data on IMF lending, exchange rates and other economic and financial indicators. Manuals, guides, and other material on statistical practices at the IMF, in member countries, and of the statistical community at large are also available. | IMF economic data are available through APIs/Data Services. These allow machine-to-machine communication of data. It is possible to use Data Services to import data from databases available on http://data.imf.org/ into your data systems or applications. Developers can use Data Services to make applications with the ability to import data from the repository databases in the SDMX formats 2.0 (as ASP.NET, WCF, or RESTFUL Services), SDMX 2.1 and JSON Restful. The specifications can be accessed through the API tab located at the top of each dataset. |
Wouter Maroy's avatar
Wouter Maroy committed
146

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
147
**Notes**
Wouter Maroy's avatar
Wouter Maroy committed
148

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
149
IMF economic data are a very useful source of information that includes global financial stability reports, regional economic reports, international financial statistics, exchange rates, directions of trade, and more.
Wouter Maroy's avatar
Wouter Maroy committed
150

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
151
**Policy Domains**
Wouter Maroy's avatar
Wouter Maroy committed
152

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
153
154
155
156
- Finance
- Economy
- Trade
- Government
Wouter Maroy's avatar
Wouter Maroy committed
157

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
158
### 6. NASA Planetary Data System
Wouter Maroy's avatar
Wouter Maroy committed
159

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
160
161
162
| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| NASA Planetary Data System | NASA has recorded over 100 terabytes of space images, telemetry, models and just about everything one can imagine from all the planetary missions from the past 30 years. The data stored is within NASA's planetary data system (PDS). |  All PDS data are available free at [http://pds.nasa.gov](http://pds.nasa.gov/). But, while rich in depth and breath, the PDS databases have developed in a disparate fashion over the years with different architectures and formats for different scientific needs; thereby making acquisition of data problematic. The PDS API Challenges goal is to develop a common API to allow public access to data across all the PDS nodes. |
Wouter Maroy's avatar
Wouter Maroy committed
163

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
164
**Notes**
Wouter Maroy's avatar
Wouter Maroy committed
165

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
166
167
168
169
170
171
172
173
The Planetary Data System (PDS) is a long-term archive of digital data products returned from NASA's planetary missions, and from other kinds of flight and ground-based data acquisitions, including laboratory experiments. But it is more than just a facility - the archive is actively managed by planetary scientists to help ensure its usefulness and usability by the world wide planetary science community.Archive submissions are prepared by researchers under the guidance of PDS personnel. All products are peer-reviewed, well-documented, and easily accessible via a system of online catalogues that are organized by planetary disciplines.Archived products are available on-line. When needed, PDS provides users access to staff to help with data selection. PDS also provides a variety of tools useful in producing, obtaining and using archived data. There is no cost associated with acquiring PDS archived data or tools or in getting reasonable amounts of PDS help. All PDS archived data may be exported outside of the United States under the U.S. Government's Technology and Software Publicly Available (TSPA) classification.

**Policy Domains**

- Science
- Engineering
- Geoscience

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
174
### 7. National Centers for Environmental Information (NCEI)
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| NCEI | NCEI is the world's largest provider of weather and climate data. Land-based, marine, model, radar, weather balloon, satellite, and paleoclimate are just a few of the types of datasets available. | For API data access it is possible to use the NCEI suite of API services: **1.** [Access Data Service API](https://www.ncei.noaa.gov/support/access-data-service-api-user-documentation) **2.** [Access Search Service API](https://www.ncei.noaa.gov/support/access-search-service-api-user-documentation) **3.** [Access Order Service API](https://www.ncei.noaa.gov/support/access-order-service) **4.** [Access Support Service API](https://www.ncei.noaa.gov/support/access-support-service) |

**Notes**

The demand for high-value environmental data and information has dramatically increased in recent years. To improve its ability to meet that demand, NOAA's former three data centers—the National Climatic Data Center, the National Geophysical Data Center, and the National Oceanographic Data Center, which includes the National Coastal Data Development Center—have merged into the National Centers for Environmental Information (NCEI). NCEI is responsible for hosting and providing access to one of the most significant archives on Earth, with comprehensive oceanic, atmospheric, and geophysical data. From the depths of the ocean to the surface of the sun and from million-year-old ice core records to near real-time satellite images, NCEI is the Nation's leading authority for environmental information.NCEI continues the tradition of excellence, unmatched expertise, and trusted, authoritative data that the previous three Data Centers established. The top priority during the near future is to build on the full spectrum of atmospheric, oceanographic, coastal, and geophysical products and services that the Data Centers delivered.

**Policy Domains**

- Climate
- Agriculture
- Air quality
- Construction
- Education
- Energy
- Engineering
- Forestry
- Health
- Insurance
- Landscape design
- Livestock management
- Manufacturing
- National security,
- Recreation and tourism,
- Transportation
- Water resources management

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
204
### 8. National Institute on Drug Abuse (NIDA)
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| NIDA | The NIDA is a great resource for datasets on tobacco, alcohol, illicit drug, and prescription opioid abuse in the U.S. NIDA uses multiple sources to monitor the prevalence and trends regarding drug use in the United States. Its resources cover a variety of drug related issues, including information on drug usage, emergency room data, prevention and treatment programs, and other research findings. | The National Institute on Drug Abuse (NIDA) maintains four screening tools, each with its own API (see the list below) |

The four screening tools used by the NIDA:

- **Brief Screener for Tobacco, Alcohol, and Other Drugs (BSTAD)**
- **Screening to Brief Intervention (S2BI)**
- **Tobacco, Alcohol, Prescription Medication, and Other Substance use Tool (TAPS)**
- **NIDA Drug Screening Tool: NIDA-Modified ASSIST (NMASSIST)**

Documentation for each of the NIDA Screening Tool APIs can be found at the [NIDA API Gateway](https://api.drugabuse.gov/prod).


**Notes**

NIDA mission is to advance science on the causes and consequences of drug use and addiction and to apply that knowledge to improve individual and public health.

This involves:

- Strategically supporting and conducting basic and clinical research on drug use (including nicotine), its consequences, and the underlying neurobiological, behavioral, and social mechanisms involved.
- Ensuring the effective translation, implementation, and dissemination of scientific research findings to improve the prevention and treatment of substance use disorders and enhance public awareness of addiction as a brain disorder.

**Policy Domains**

- Drugs
- Prevention
- Medicine
- Treatment programs
- Science
- Safety

QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
238
### 9.The CIA World Factbook
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278

| **Data source name** | **Description** | **Types of API** |
| --- | --- | --- |
| The CIA World | The World Factbook is a reference resource produced by the Central Intelligence Agency (CIA). It provides information on the history, people and society, government, economy, energy, geography, communications, transportation, military, and transnational issues for 267 world entities. | The CIA The World Factbook provides the following APIs: **1.** [The CIA World Factbook - the Datahub](https://datahub.io/dataset/cia-world-factbook) original CIA World Factbook packaged **2.** [DBpedia](http://dbpedia.org/) provides semantic data based on Wikipedia articles. You could download it or to use SPARQL endpoint **3.** [Wikimedia Downloads](https://dumps.wikimedia.org/) helps you to get copy of whole Wikipedia and to parse country pages **4.** [For Developers](http://data.worldbank.org/developers) WorldBank project with country data available as datasets and via API **5.** [The World Factbook](https://github.com/factbook) github account with factbook data |

**Notes**

The World Factbook is prepared by the Central Intelligence Agency for the use of US Government officials, and the style, format, coverage, and content are designed to meet their specific requirements. Information is provided by:

- Antarctic Information Program (National Science Foundation)
- Armed Forces Medical Intelligence Center (Department of Defense)
- Bureau of the Census (Department of Commerce)
- Bureau of Labor Statistics (Department of Labor)
- Council of Managers of National Antarctic Programs
- Defense Intelligence Agency (Department of Defense)
- Department of Energy
- Department of State
- Fish and Wildlife Service (Department of the Interior)
- Maritime Administration (Department of Transportation)
- National Geospatial-Intelligence Agency (Department of Defense)
- Naval Facilities Engineering Command (Department of Defense)
- Office of Insular Affairs (Department of the Interior)
- Office of Naval Intelligence (Department of Defense)
- Oil & Gas Journal
- United States Board on Geographic Names (Department of the Interior)
- United States Transportation Command (Department of Defense)

The Factbook is in the public domain. Accordingly, it may be copied freely without the permission of the Central Intelligence Agency (CIA).The Reference tab of the CIA The World Factbook includes a variety of world, regional, country, ocean, and time zone maps, flags of the world, and a Country Comparison function that ranks the country information and data in more than   Factbook fields. 

**Policy Domains**

- Geography
- People
- Government
- Transport
- Economy
- History
- Society
- Energy
- Military
QuintenVandenhoeck's avatar
QuintenVandenhoeck committed
279
- Communication