page-type:: [[TEACHING/SE4IOT]] - # Slides - Da slide 15 di [Bononi e Felice - IoT Sensor Data Processing 1.pdf](file:///C:/Users/david/Zotero/storage/WUYV8GM5/Bononi%20e%20Felice%20-%20IoT%20Sensor%20Data%20Processing.pdf) - Slides [Bononi e Felice - IoT Sensor Data Management.pdf](file:///C:/Users/david/Zotero/storage/4Z3ESYTU/Bononi%20e%20Felice%20-%20IoT%20Sensor%20Data%20Management.pdf) - # Installation 1.x According to [Downloads (influxdata.com)](https://portal.influxdata.com/downloads/) ``` sudo apt-get install influxdb sudo service influxdb start ``` - # Play with 1.x (on WSL) ``` influx // this will start the client show databases create database myfirstdatabase use myfirstdatabase show measurements ``` We get empty measurements. They are typically collected by agents like Telegraf,etc. We can manually insert measurements as follows: ``` insert cpu,host=node1 value=10 ``` Now we can show the recently added measurement with ``` show measurements ``` We can get the content of the *cpu* measurement as follows ``` select * from cpu ``` We can drop measurements with the following command ``` drop measurement cpu ``` Let's insert some measurements: ``` insert cpu,host=node1 value=10 insert cpu,host=node2 value=15 insert cpu,host=node3 value=22 ``` Now we can do again ``` select * from cpu ``` We can see all the series as follows: ``` show series ``` We can do some filtering as follows: ``` select * from cpu where host='node2' ``` We can also do filtering based on time, e.g. I want to retrieve the data related to the last 5 minutes ``` select * from cpu where time >= now() - 5m ``` ``` select * from cpu where time >= now() - 5m and host='node3' ``` And what happens if I do as follows? ``` select * from cpu where time >= now() - 1m ``` - # Installation 2.x - Remember to stop influxdb 1.x - ```sudo service influxdb start sudo service influxdb stop ``` - ``` wget https://dl.influxdata.com/influxdb/releases/influxdb2\_2.0.3\_amd64.deb sudo dpkg -i influxdb2\_2.0.3\_amd64.deb influxdb ``` The web dashboard is available at [localhost:8086](http://localhost:8086/) - - # Play with 2.x (in [docker] - First of all you have to install it according to [Install InfluxDB | InfluxDB OSS v2 Documentation (influxdata.com)](https://docs.influxdata.com/influxdb/v2/install/?t=Docker) and run it - ``` docker run --name influxdb -p 8086:8086 influxdb:2.7.4 ``` - ![image.png](../../../../assets/image_1701374013220_0.png) - 5LW7B-5ySei2kPoLtPIwBFS0KfK-uAcnxYE04BoJhalwdzQjvRz5zi7bELcTNw_WDkfCUkJiDd-wvIIDvnYHjQ== - # Set up a configuration profile [NOT NEEDED HERE] ``` influx config create -n default \ -u http://localhost:8086 \ -o univaq \ -t 5LW7B-5ySei2kPoLtPIwBFS0KfK-uAcnxYE04BoJhalwdzQjvRz5zi7bELcTNw_WDkfCUkJiDd-wvIIDvnYHjQ== \ -a ``` The token given above needs to be retrieved from [Tokens | Load Data | -- | InfluxDB](http://localhost:8086/orgs/fc77876718e82aaf/load-data/tokens) To list the created configuration (**from within the docker container**) ``` influx config list ``` The bucket given above is selected from [Buckets | Load Data | -- | InfluxDB](http://localhost:8086/orgs/fc77876718e82aaf/load-data/buckets) **Insert measaurements via NodeRed from MQTT** Publish to `/nodered/powerconsumption` ``` { "device":"device1", "host":"NodeRed", "power":"373i" } ``` - # Insert measurements via Telegraf from MQTT [Install Telegraf | Telegraf Documentation (influxdata.com)](https://docs.influxdata.com/telegraf/v1/install/#download) ``` # influxdata-archive_compat.key GPG Fingerprint: 9D539D90D3328DC7D6C8D3B9D8FF8E1F7DF8B07E wget -q https://repos.influxdata.com/influxdata-archive_compat.key echo '393e8779c89ac8d958f81f942f9ad7fb82a25e133faddaf92e15b16e6ac9ce4c influxdata-archive_compat.key' | sha256sum -c && cat influxdata-archive_compat.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg > /dev/null echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive_compat.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list sudo apt-get update && sudo apt-get install telegraf ``` - ![image.png](../../../../assets/image_1732472841676_0.png) - Telegraf Token ``` export INFLUX_TOKEN=YQ9t9PhYSeA6Ug0i44BInA-53gYAvq6hwa4_fVPI7MrXhUOugIFMl_o0_ib5Ulhr80-nFVtOmvr-QZWZtxt--Q== ``` - Run the Telegraf agent ``` telegraf --config http://localhost:8086/api/v2/telegrafs/0e03dad63d8aa000 ``` - Publish this to the topic `sensors/powerconsumption` ``` powerconsumption,device=device2,host=MyHost power=375i ``` There will be a Telegraf agent that will take such data and sends it to InfluxDB. To this end, Telegraf should be started (even on WSL) by using the following configuration file *telegraf_mqtt_to_influxdb2* ``` [[outputs.influxdb_v2]] ## The URLs of the InfluxDB cluster nodes. ## ## Multiple URLs can be specified for a single cluster, only ONE of the ## urls will be written to each interval. ## urls exp: http://127.0.0.1:8086 urls = ["http://localhost:8086"] ## Token for authentication. token = "WdpITesY4yOqb0dBy7n2WQgWSHPRP66O9mAzlEn7vjYBaCrnn7-bJEiKU5M0XqnU4Amf6So1XunJKKfyt9KGAA==" ## Organization is the name of the organization you wish to write to; must exist. organization = "univaq" ## Destination bucket to write into. bucket = "se4iot" [[inputs.mqtt_consumer]] ## Broker URLs for the MQTT server or cluster. To connect to multiple ## clusters or standalone servers, use a seperate plugin instance. ## example: servers = ["tcp://localhost:1883"] ## servers = ["ssl://localhost:1883"] ## servers = ["ws://localhost:1883"] servers = ["tcp://127.0.0.1:1883"] ## Topics that will be subscribed to. topics = [ "sensors/#", ] ## The message topic will be stored in a tag specified by this value. If set ## to the empty string no topic tag will be created. # topic_tag = "topic" ## QoS policy for messages ## 0 = at most once ## 1 = at least once ## 2 = exactly once ## ## When using a QoS of 1 or 2, you should enable persistent_session to allow ## resuming unacknowledged messages. # qos = 0 ## Connection timeout for initial connection in seconds # connection_timeout = "30s" ## Maximum messages to read from the broker that have not been written by an ## output. For best throughput set based on the number of metrics within ## each message and the size of the output's metric_batch_size. ## ## For example, if each message from the queue contains 10 metrics and the ## output metric_batch_size is 1000, setting this to 100 will ensure that a ## full batch is collected and the write is triggered immediately without ## waiting until the next flush_interval. # max_undelivered_messages = 1000 ## Persistent session disables clearing of the client session on connection. ## In order for this option to work you must also set client_id to identify ## the client. To receive messages that arrived while the client is offline, ## also set the qos option to 1 or 2 and don't forget to also set the QoS when ## publishing. # persistent_session = false ## If unset, a random client ID will be generated. # client_id = "" ## Username and password to connect MQTT server. # username = "telegraf" # password = "metricsmetricsmetricsmetrics" ## Optional TLS Config # tls_ca = "/etc/telegraf/ca.pem" # tls_cert = "/etc/telegraf/cert.pem" # tls_key = "/etc/telegraf/key.pem" ## Use TLS but skip chain & host verification # insecure_skip_verify = false ## Data format to consume. ## Each data format has its own unique set of configuration options, read ## more about them here: ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md data_format = "influx" ``` Similar things can be done via command line as follows ```sh export INFLUX_TOKEN=Ga8nVsXP4FAe5_M1a5j7uCa4zO_u_M9oUsO8wUSWh_wPbR3clc9ZTv420Li9adOVCPl1tGvn6hLHfI5gP7Lm5A== influx write -b se4iot -o univaq -p s 'powerconsumption,device=device2,host=CommandLine power=675i' ``` - sensors/powerconsumption - # Insert measurements via Java ```java package it.univaq.disim.se4iot.influxdbexample; import java.time.Instant; import com.influxdb.annotations.Column; import com.influxdb.annotations.Measurement; import com.influxdb.client.InfluxDBClient; import com.influxdb.client.InfluxDBClientFactory; import com.influxdb.client.WriteApi; import com.influxdb.client.domain.WritePrecision; import com.influxdb.client.write.Point; import com.influxdb.query.FluxTable; public class InfluxDB2Example { public static void main(final String[] args) { // You can generate a Token from the "Tokens Tab" in the UI String token = "5LW7B-5ySei2kPoLtPIwBFS0KfK-uAcnxYE04BoJhalwdzQjvRz5zi7bELcTNw_WDkfCUkJiDd-wvIIDvnYHjQ=="; String bucket = "se4iot"; String org = "univaq"; InfluxDBClient client = InfluxDBClientFactory.create("http://localhost:8086", token.toCharArray()); Point point = Point .measurement("powerconsumption") .addTag("device", "device3") .addField("power", 300) .time(Instant.now(), WritePrecision.NS); try (WriteApi writeApi = client.getWriteApi()) { writeApi.writePoint(bucket, org, point); } } } ``` In a dedicated Maven project with the following pom.xml ```xml 4.0.0 it.univaq.disim.se4iot.influxdbexample it.univaq.disim.se4iot.influxdbexample 0.0.1-SNAPSHOT com.influxdb influxdb-client-java 1.8.0 src maven-compiler-plugin 3.8.0 12 ``` - # Some tutorials [(14) InfluxDB Tutorial - Complete Guide to getting started with InfluxDB - YouTube](https://www.youtube.com/watch?v=Vq4cDIdz_M8&ab_channel=DevOpsJourney) [(14) Introduction to InfluxDB 2 & Flux | Rawkode Live - YouTube](https://www.youtube.com/watch?v=LfWVZHuCCnE&ab_channel=DavidMcKay) [(14) Node-Red: inseriamo dati su InfluxDB - YouTube](https://www.youtube.com/watch?v=4Xeb3cOwrbs&ab_channel=L%27angolodelBIT) - - To remove all the data from the simple bucket - influx delete --bucket se4iot --org univaq --start 1970-01-01T00:00:00Z --stop 2100-01-01T00:00:00Z