ddiruscio/logseq
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

commit.

Browse Source
This commit is contained in:
ddiruscio
2025-06-05 10:18:13 +02:00
parent 9b881bf058
commit b2116c22df
6304 changed files with 464 additions and 708461 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pages/Teaching/20_21/SE4IOT/IoT Cloud and Machine Learning The Building Blocks.md
-29
View File
@@ -1,29 +0,0 @@
#Highlights
---
# **IoT Cloud and Machine Learning The Building Blocks**
Date: 2021-01-05
Time: 22:03
URL: (https://www.opensourceforu.com/2020/10/iot-cloud-and-machine-learning-the-building-blocks/) [[Obsidian-Highlights]]
Tags: #machinelearning #IOT #dataprocessing
---
# Main building blocks of an ML infrastructure
Given below are the tools and frameworks needed for an Apache Spark based ML infrastructure.
- _Spark 2:_ Comes up with all the necessary tools from the Spark ecosystem — Hadoop, Mlib, etc. You can get more details from https://spark.apache.org/docs/latest/.
- _Hadoop:_ You can either install Spark2 with Hadoop or Hadoop as standalone.
- _Python3/Scala/Java:_ This depends on what language you prefer to write ML programs
- PostgreSQL/MongoDB: Install this if you have to store data into traditional databases other than Hadoop HDFS for future use and reference.
- _MLib/Tensorflow/Keras/Scikit learn_: Choose from these ML libraries based on your needs.
- _Data analytics tools:_ Choose these based on your needs.
# Common ML use cases in IoT
Given below are a few common use cases based on the data received from devices (strictly based on my experience and may differ in your business case).
- _Analysis of data patterns for a specific period_: As an example, if data comes from a temperature sensor, then the pattern of temperature data for a location where the device is installed for a day can be analysed for that specific period.
- _Data missing/changes in duration/changes in pattern, etc:_ It is important to understand the missing data or changes in frequency because immediate action is required, or it can lead to potential errors in our analytics/forecasts.
- _Inactivities or other ambiguities in data flow:_ These errors in data processing must be avoided.
- _Difference between the forecasted and real data:_ This may lead to a correction in data models and trainings.
- _User and location behaviour from device to device:_ Data for each device may differ if user and location behaviour contribute some points to the data.
- _Frequency of maintenance and root causes for that:_ This may be specific to location, usage, transaction volume, etc.