GCP IoT Core

In this tutorial, we will deploy a simple open-source application that collects metrics and send them to AWS IoT Core for further processing. All code for this blog post can be found at:
​https://github.com/synpse-hq/metrics-nats-example-app - Sample metrics application https://github.com/synpse-hq/gcp-iot-core-example - GCP IoT Core example
Technologies used
1.​Synpse - manage devices and deploy applications to them
2.​NATs - a lightweight message broker that can run on-prem
3.​Google IoT Core - message broker between all devices and GCP
Steps:
1.Create GCP IoT core
2.Configure data flow to forward results into Google storage account
3.Create GCP device for Synpse
4.Demo Synpse application from 3 microservices -Metrics demo, Nats messaging, AWS IoT python forwarder containers
GCP IoT Core
GCP Core works by creating a "device registry". So this is one of the first steps we have to do.
Configure GCP IoT Core create registry
Create pubsub topic for registry
gcloud pubsub topics create synpse-events
Create GCP IoT registry:
gcloud iot registries create synpse-registry --region=us-central1 --enable-http-config --enable-mqtt-config --state-pubsub-topic projects/{project_id}/topics/synpse-events
Create storage event
gsutil mb -l us-central1 -b on gs://synpse-events
Create dataflow job
gcloud dataflow jobs run ps-to-avro-synpse-events --gcs-location gs://dataflow-templates-us-central1/latest/Cloud_PubSub_to_Avro --region us-central1 --staging-location gs://synpse-events/temp --parameters inputTopic=projects/iot-hub-xxxxxx/topics/synpse-events,outputDirectory=gs://synpse-events/events,avroTempDirectory=gs://synpse-events/avro-temp
Data flow would look as bellow:
DataflowCreate a device
gcloud iot devices create synpse --region=us-central1 --registry=synpse-registry
Generate certificate for our device
openssl ecparam -genkey -name prime256v1 -noout -out ec_private.pem
openssl ec -in ec_private.pem -pubout -out ec_public.pem
Add public key to GCP IoT Core
gcloud iot devices credentials create --region=us-central1 --registry=synpse-registry --device=synpse --path=ec_public.pem --type=es256
Download google root CA
curl  https://pki.goog/roots.pem -o roots.pem
(Optional) Test application
python gateway/gcp.py --device_id synpse --private_key_file ./ec_private.pem --cloud_region=us-central1 --registry_id synpse-registry --project_id iot-hub-326815 --algorithm ES256 --message_type state
Deploy an application
Deploy Synpse application. Modify application yaml with your thing endpoint.
Create certificate secret
synpse secret create gcp-cert -f ec_private.pem
synpse secret create gcp-root -f roots.pem
Deploy the application:
synpse deploy -f synpse-gcp-example.yaml
where synpse-gcp-example.yaml is
name: GCP-IoT-Hub
description: Google Cloud IoT Core Synpse example
scheduling:
  type: Conditional
  selectors:
    gcp: iot
spec:
  containers:
    - name: nats
      image: nats
      restartPolicy: {}
    - name: metrics
      image: quay.io/synpse/metrics-nats-example-app
      restartPolicy: {}
    - name: gcp-iot
      image: quay.io/synpse/gcp-iot-hub-example
      command: /server/gcp.py
      args:
        - --device_id=synpse
        - --private_key_file=/server/ec_private.pem
        - --cloud_region=us-central1
        - --registry_id=synpse-registry 
        - --project_id=iot-hub-326815 
        - --algorithm=ES256 
        - --message_type=state
        - --ca_certs=/server/roots.pem
      secrets:
      - name: gcp-cert
        filepath: /server/ec_private.pem
      - name: gcp-root
        filepath: /server/roots.pem
      env:
        - name: NATS_HOSTNAME
          value: nats
      restartPolicy: {}
Once running, you should see application running and data coming into the GCP storage account blob.
Last modified 1yr ago