#FlashHacks: Extractives Industry
Comments
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

 
£
%
123
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
PROPOSE A MISSION#FlashHacks
2
What kinds of data are we looking for? http://turbot.opencorporates.com/docs/frequently_asked_questions#what_kinds_of_dataProposed by
3
#Description: What is the dataset about?URLCountry or JurisdictionBot DifficultyNameOrganisationEmail address / twitterComments
4
Australian Authorised Deposit Taking Institutionshttp://www.apra.gov.au/adi/pages/adilist.aspxaueasyJohn LehmannSimple list on the web page
5
Australian Companies Registryhttps://connectonline.asic.gov.au/aumediumJohn Lehmann
Search form combined with company name from above list will give details of the company
6
Mineral Exploration Licences
http://www.minerals.dmitre.sa.gov.au/licensing_and_regulation/tenement_information#holders
au_samedium
7
Mining Tenement Application Advertising (Mining Notices)https://emits.dmp.wa.gov.au/emits/advert/index.xhtmlau_wahard
8
Drilling Activity - Alberta, Canadahttp://www.aer.ca/data-and-publications/statistical-reports/st49CanadamediumXavier RileyOpenCorporates
Daily reports of drilling activity in text format
9
Well List for DE/SD Commingling Process Declarations - Alberta, Canadahttp://www.aer.ca/data-and-publications/activity-and-data/comminglingCanadahardXavier RileyOpenCorporates
Last link on the page. Excel file with 13,226 lines. Company name only - no jurisdiction.
10
Confidential Well List - Alberta, Canada
http://www.aer.ca/data-and-publications/activity-and-data/confidential-well-list
CanadaeasyXavier RileyOpenCorporates
The AER provides listings of confidential and confidential below (CB) wells in the province of Alberta. Updated daily, these listings include: well locations, licence numbers, licensee operator codes, company names, well confidential types, confidential depths for CB wells and the confidential release dates.
11
Facility Approval Daily Listings - Alberta, Canadahttp://www.aer.ca/data-and-publications/activity-and-data/st97CanadaeasyXavier RileyOpenCorporates
Provides a summary of the Surface Facility Licences issued for the day. The list includes new facilities and amendments to existing facilities covered under Directive 056.
12
Licensee Agent Codes – ST 104 - Alberta, Canadahttp://www.aer.ca/data-and-publications/activity-and-data/reference-codesCanadamediumXavier RileyOpenCorporates
List of codes, company names, abbreviations, addresses, phone number and agent details for all licenced companies in Alberta, Canada. Can be used in conjuction with other datasets published by the Alberta Energy Regulator
13
Czech Republic Regulated institutions and registered financial market entities lists,https://www.cnb.cz/cnb/jerrs_encs_czhardMichal Cichra
14
DECC - Licence Data - by Company Group and Block
https://www.og.decc.gov.uk/information/licence_reports/databycompanyandblock.html
gbeasy
Includes some hierarchy information. Original licenses as PDF include geographic bounds as OS references: https://www.og.decc.gov.uk/web_files/recent_licences/oglicences.htm. See bottom of https://www.gov.uk/oil-and-gas-licence-data for full list of data
15
ISDA.org ISDA August 2012 DF Protocol adherents
http://www2.isda.org/functional-areas/protocol-management/protocol-adherence/8?csv=true
globaleasyAlex Skene
Key data for derivatives trading & reporting required to adhere to CFTC Dodd-Frank regulations
16
ISDA.org ISDA March 2012 DF Protocol adherents
http://www2.isda.org/functional-areas/protocol-management/protocol-adherence/12?csv=true
globaleasyAlex Skene
Key data for derivatives trading & reporting required to adhere to CFTC Dodd-Frank regulations
17
Further lists of ISDA protocol adherents, eg for EMIR, FATCA regulations
http://www2.isda.org/functional-areas/protocol-management/open-protocols/
globaleasyAlex Skene
data available in csv format by appending ?csv=true to any list of adherents
18
National Futures Association Swap Dealers Directory
https://www.nfa.futures.org/NFA-swaps-information/regulatory-info-sd-and-msp/SD-MSP-registry.HTML
globaleasyAlex Skene
listing of swap dealers and major swap participants along with their respective CFTC registration statuses. XML & csv versions also available
19
Financial institutions of Hungary
http://www.mnb.hu/Statisztika/informaciok-adatszolgaltatoknak/rendeletekallasfoglalasok/mnbhu_statisztika_eloirasok/informacioszolg_hu
hueasyGergely ImrehTaipei Hackerspace@imrehg
Excel sheet ("A pénzügyi vállalatok tételes listája")
20
EITI reports datahttp://eiti.org/countries
i.e. Azerbaijan, Nigeria, Peru + many more
Anton RühlingOpenOil
i.e. Company Payments to Governments
21
Japanese banks, and branches of foreign banks in Japanhttp://www.fsa.go.jp/menkyo/menkyoj/ginkou.xlsjpeasyNicolas Raoul@Nicolas_Raoul
First part is Japanese banks, second part is branches of foreign banks in Japan
22
Companies with licenses to perform financial services in Japanhttp://www.fsa.go.jp/menkyo/menkyo.htmljpeasyNicolas Raoul@Nicolas_Raoul
Many Excel files for different kinds of financial licenses
23
Kenya Commercial Banks
https://www.centralbank.go.ke/index.php/bank-supervision/commercial-banks-mortgage-finance-institutions
kemediumBen Parker
Easier than the CMA listing of Kenya investment banks
24
Investment Advisors in Puerto Ricohttp://www.ocif.gobierno.pr/documents/cons/IA.pdfprhardStefan Oderbolz@odi
PDF with a table in a strange format. All other PDFs from http://www.ocif.gobierno.pr/concesionariosbusqueda_eng.htm could be scraped, this is a particularly hard one.
25
Companies listed on the Istanbul Stock Exchange
http://www.cmb.gov.tr/indexcont.aspx?action=showpage&showmenu=yes&menuid=7&pid=0&subid=3&submenuheader=0
treasy
26
Financial institutions of Taiwan
http://www.cbc.gov.tw/lp.asp?ctNode=495&CtUnit=210&BaseDSD=7&mp=2
tweasyGergely ImrehTaipei Hackerspace@imrehg
27
Ukraine legal entity registerhttp://data.gov.ua/passport/73cfe78e-89ef-4f06-b3ab-eb5f16aea237uahardIvan Yanixml+zip, large file
28
Applications for Authority to Import and/or Export Natural Gas to/from the United States
https://app.fossil.energy.gov/app/fergas/DocketOrderList.goUnited StatesmediumXavier RileyOpenCorporates
Linked from http://energy.gov/fe/services/natural-gas-regulation
29
Aggregate national, state, and plant-level electricity generation statisticshttp://api.eia.gov/bulk/ELEC.zipUnited StateseasyXavier RileyOpenCorporates
Has an 'operator' field with company names for every power plany in the US. Linked from http://api.eia.gov/bulk/manifest.txt
30
Plant-level location and capacity information for all natural gas processing plants in the United States.
http://www.eia.gov/cfapps/ngqs/ngqs.cfm?f_report=RP9&f_sortby=&f_items=&f_year_start=&f_year_end=&f_show_compid=&f_fullscreen
United StateseasyXavier RileyOpenCorporates
31
Refinery Capacity Data by Individual Refinery as of January 1, 2014http://www.eia.gov/petroleum/refinerycapacity/refcap14.xlsUnited StatesmediumXavier RileyOpenCorporates
Linked from http://www.eia.gov/petroleum/data.cfm#refining
32
Oil & Gas Leases Issued - US Department of the Interior (Except Alaska)http://www.blm.gov/landandresourcesreports/rptapp/menu.cfm?appCd=2United StateshardXavier RileyOpenCorporates
Complicated query tool but very good data.
33
Mining Claim Summary - Alaska Case Retrieval Enterprise System (ACRES)http://sdms.ak.blm.gov/acres/acres_menuus_alhardXavier RileyOpenCorporates
34
State Chartered Banks and Trust Companieshttps://dbf.georgia.gov/banking-information-and-searchesus_gaeasyMatthew L Daniel@mdaniel
There is an Excel file filled with all the data, and a note saying it is refeshed nightly: https://dbfweb.dbf.state.ga.us/InternetData/Banks.zip
35
Illinois Department of Financial and Professional Regulationhttp://www.idfpr.com/banks/agency/licenseinfo.aspus_ilmediumMatthew L Daniel@mdaniel
Data trapped behind search form; markup is not nice
36
Kentucky Department of Financial Instutionshttps://dfiweb.ky.gov/dfipublic/default.aspxus_kymediumMatthew L Daniel@mdaniel
Data trapped behind search form; site is Microsoft asp.net which is normally unfriendly to scrapers; search results markup is not especially nice
37
Montana Division of Banking and Financial Instutionshttp://www.banking.mt.gov/licensees.aspxus_mtmediumMatthew L Daniel@mdaniel
Data trapped in PDF docs; data is 2 months old
38
Oil & Gas Lease Auction History
http://www.land.nd.gov/minerals/mineralapps/auctions/auctionhistorysale.aspx
us_ndeasy
39
Current Active Drilling Rig Listhttps://www.dmr.nd.gov/oilgas/riglist.aspus_ndeasy
40
North Dakota Oil and Gas Division - Daily Activity Report Indexhttps://www.dmr.nd.gov/oilgas/dailyindex.aspus_ndhard
41
New Hampshire - State Chartered Banks, Credit Unions and Trust Companieshttp://www.nh.gov/banking/bank-cu-trust/us_nheasyMatthew L Daniel@mdaniel
Excel sheets; unclear what the data refresh policy is
42
Rhode Island - Banks and Financial Institutionshttp://www.dbr.state.ri.us/divisions/banking/banks.phpus_rimediumMatthew L Daniel@mdaniel
Data trapped in PDF docs; unclear what the data refresh policy is
43
Rhode Island - Credit Unionshttp://www.dbr.state.ri.us/divisions/banking/credit.phpus_rimediumMatthew L Daniel@mdaniel
Data trapped in PDF docs; unclear what the data refresh policy is
44
Washington State - Commercial Banks and Savings Inst.http://dfi.wa.gov/banks/commercial_banks.htmus_wamediumMatthew L Daniel@mdaniel
Regrettably the markup on the page is not friendly for extraction, but on the bright note it is only one page (no pagination or search forms)
45
Washington State - Credit Unionshttp://www.dfi.wa.gov/cu/cucontacts.htmus_wamediumMatthew L Daniel@mdaniel
Regrettably the markup on the page is not friendly for extraction, but on the bright note it is only one page (no pagination or search forms)
46
Wisconsin Department of Financial Institutionshttps://www.wdfi.org/fi/banks/licensee_lists/us_wimediumMatthew L Daniel@mdaniel
Data trapped behind search form; markup is not nice
47
Compile a list of Mexican Financial Institutions
http://www.banxico.org.mx/ServiciosFinancieros/autorizaciones/publicacionesInter.xhtml?idTipoDisposicion=1&idDisposicion=0
48
Anguilla - Commercial Registry - Financial Services Directoryhttp://www.commercialregistry.ai/company-management.htmlAnguillaeasyCelyn Birkinshaw
Link to data set on Gov of Anguilla website - Registrar of Companies
49
Anguilla - Commercial Registry - Financial Services Directory - Overseas Agentshttp://www.commercialregistry.ai/directory-overseas-agent.htmlAnguillaeasyCelyn Birkinshaw
Link to data set on Gov of Anguilla website - Registrar of Companies - Overseas Agents
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...
 
 
 
Datasets