OEDI SI/Scenarios/Data Imputation Scenario for SMART-DS systems

Data Imputation Scenario for SMART-DS systems​ Summary

• Date Created: 2024/05/10
• Objective: Provide estimate for missing streaming measurement data.
• Use Case: Data Preprocessing
• Methodology
• Inputs
• Outputs
• Configuration
• Webinars
Data preprocessing.pdf

Docker Container

Download the docker container at:

Run Locally

http://localhost:8080/edit_scenario?Data Imputation Scenario for SMART-DS systems

    Run Locally

Input Data

Download input data at:

Output Data

Download output data at:

References

Back to Data Preprocessing

Denoising Autoencoders (DAEs) are unsupervised Deep learning algorithms.

Encoder encodes training into an encoding on a higher or lower dimensional hyperplane.

Decoder decodes the encoding to reconstruct the original data.

The application imputes on streaming data when missingness is detected.

Input from sensor federate.

At each time step, if the sensor data has no missing data then the data imputation federate output/publication is the same as sensor federate publication. If there is missing data point(s) then the data imputation federate will publish an estimate for the missing value.

To run data imputation scenario with smartds small system: docker run --rm openenergydatainitiative/oedisi-singlecontainer-demo:0.1.0 data_imputation smartds_small

To run data imputation scenario with smartds large system: docker run --rm openenergydatainitiative/oedisi-singlecontainer-demo:0.1.0 data_imputation smartds_large

docker pull openenergydatainitiative/oedisi-singlecontainer-demo:0.1.0

All relevant data is within the container file system.

Output data is contained within the container file system. There are two ways to access it, a) using the -it flag while running "docker run" or b) by using single container cli (https://github.com/openEDI/oedi-si-single-container) to setup, run and gather results.