Survey on the design of a competition for energy disaggregation algorithms
Please help us to design a competition for energy disaggregation algorithms. The aim of the NILM competition is to enable the NILM community (both academics and companies) to directly compare the disaggregation accuracy of different NILM algorithms.

You can do this survey anonymously. You don't have to answer every question. This survey should take about 15 minutes to complete. The main benefit of participating in this survey is that you get to mould the NILM competition to suit your needs! Please note that, for your responses to be recorded, you need to click "submit" at the very end of the survey. So if you run out of time while filling in this survey then please click "next" until you get to the end and then click "submit"!

Here is a brief overview of the current proposal for the NILM competition:

• The competition will provide training data and testing data from multiple homes.
• Competitors will train and test their algorithms on their own machines and upload the disaggregated estimates to the competition website for comparison with the secret ground-truth data.
• Multiple performance metrics will be computed by the competition website. Metrics will be grouped by use-case.
• Competitors can be anonymous if they wish.
• Each competitor can submit data from multiple NILM algorithms. Each algorithm will be displayed separately in the results.
• Training and testing data will be provided in five pre-prepared temporal resolutions: 1 Hz, every 10 seconds, minutely, half-hourly, and hourly. To prevent cheating, the different sample rates will be taken from different time periods (but from the same set of houses). E.g. the 1Hz data will be from January to mid-March for 5 homes; the 10-secondly data will be from mid-March to May for the same 5 homes, etc. Each NILM algorithm can compete at any combination of temporal resolutions.
• Regarding high frequency NILM (>1Hz): we would love to extend the competition in the future to include high frequency (kHz or MHz) data. So, if you are a high-frequency NILM researcher or service provider then please still consider filling in this survey, because hopefully we will collect a high frequency dataset in the future for the competition.
• Competitors must declare whether their algorithm was trained on *just* the data provided by the competition; or whether additional training data was used. Competitors can submit each of their algorithms to one or both of these tracks.
• The training data will include the ground-truth appliance power demand from individual appliances as well as the whole-home aggregate power demand.
• The testing data will only provide the whole-house aggregate (and maybe some metadata about each house).
• The software which runs the competition will be open source.

If you plan to submit to multiple tracks of the competition (e.g. multiple sample rates) but you only have time to submit this survey once then please submit responses which are relevant to your "main" algorithm. Alternatively, you're more than welcome to submit the survey multiple times; e.g. once where you fill out the majority of the survey and then additional times where you just fill out the parts of the survey which vary between your different planned submissions.

This survey is composed of the following sections: General features of the competition; training; features & data; disaggregation categories; sample rate; final section.

The full draft proposal for the NILM competition is available on Google Docs: https://docs.google.com/document/d/1CGoiNNkcnAFpo7Lci0Dv7AIliK3xz7Rjq_mBBh0WwrQ/edit?usp=sharing

If you have any queries then please contact jack.kelly@imperial.ac.uk

What is your affiliation?
Are you happy to use your real name (or your company's real name) in the competition?
The name you provide will be published in the disaggregation results tables
Would you like the competition to run continuously or yearly?
Which "disaggregation tracks" would you be interested in competing?
For 2017, the competition will probably be focused on disaggregation of domestic electricity demand. But, in the future, it might be possible to expand the competition to include "tracks" for the disaggregation of water or natural gas; and to include tracks for disaggregating commercial (office) buildings or industrial buildings; and possibly a purely unsupervised track. Please indicate which tracks you might be interested in. You can tick multiple boxes.
Kaggle or custom-built platform?
Would you be happy to submit an executable program instead of data?
The current proposal for the competition is that each team would run their disaggregation software on their own machines. But that involves downloading and uploading a lot of data (many gigabytes) and means that we can only use data which is licensed for public release. Some machine learning competitions ask teams to submit executable code. This code is then run on the competition's servers. This means that teams don't have to transfer lots of data over the internet; and also means that the competition can use more data (for example, some utility companies have private data that they might allow the competition to use, as long as the data was not released). Would you be happy to provide an executable program instead of running the program on your own machine? You'd still *train* your algorithms on your own machine.
Next
Never submit passwords through Google Forms.
This content is neither created nor endorsed by Google. Report Abuse - Terms of Service - Additional Terms