Azure Data Factory for Data Ingestion to Azure Data Lake. Azure Databricks to transform and aggregate data. Azure Analysis Services for Analytics Models. Power BI Reports or Dashboards for Data Visualization.
Ingest Sensors/Devices events to Event Hub/IoT Hub. Process Real-time events using Stream Analytics. Raw data into Azure Cosmos DB and Azure Data Lake. Real-time metrics to Power BI for Real-time reporting.
Electronic Point of Sales(EPOS) machine collects stores customer sales data for each retail store and this sales data is stored in their regional operational database systems. For each retail store, a csv files is generated at midnight and placed in an on-premise shared location.
Azure Data Factory is used to Ingest data from on-premise locations to Data Lake. Data from these files in Data Lake are transformed and aggregated using Azure Databricks and loaded into Data Lake. Data will be finally loaded into Azure Synapse Analytics using Azure Databricks.
Ingest reference data from on-premise systems using Azure Data Factory to Data Lake. Process and transform reference data using Azure Databricks.
Applying aggregations and model data in Azure. Building Retail Analytical models for Reporting. Aggregate and Historical reporting using Power BI.
Wind Turbines produces energy and power. Energy and Power related 20+ events generates real-time. Turbine related 100+ events(temperature, pressure, torque etc.). These events are pushed to Azure in real-time.
Solar Panels produces energy and power. Energy and Power related 20+ events generates real-time. Solar Panel related 25+ events(temperature, air etc.). These events are pushed to Azure in real-time.
Endpoint to receive events from Turbines/Panels in Azure. Process streaming data using real-time analytics engine. Aggregations and transformations of streams. Process raw events in Data Lake and aggregated data in DW.
Ingest reference data from on-premise systems. Process and transform reference data using Azure Databricks.
Applying aggregations and model data in Azure. Building Energy Analytical models for Reporting. Aggregate and Historical reporting using Power BI.
Applying aggregations and model data in Azure. Building Energy Analytical models for Reporting. Real-Time Turbine and Grid Monitoring using Power BI.