GCP IoT Core

In this tutorial, we will deploy a simple open-source application that collects metrics and send them to AWS IoT Core for further processing. All code for this blog post can be found at:

https://github.com/synpse-hq/metrics-nats-example-app - Sample metrics application https://github.com/synpse-hq/gcp-iot-core-example - GCP IoT Core example

Technologies used

  1. Synpse - manage devices and deploy applications to them

  2. NATs - a lightweight message broker that can run on-prem

  3. Google IoT Core - message broker between all devices and GCP


  1. Create GCP IoT core

  2. Configure data flow to forward results into Google storage account

  3. Create GCP device for Synpse

  4. Demo Synpse application from 3 microservices -Metrics demo, Nats messaging, AWS IoT python forwarder containers

GCP IoT Core

GCP Core works by creating a "device registry". So this is one of the first steps we have to do.

Configure GCP IoT Core create registry

Create pubsub topic for registry

gcloud pubsub topics create synpse-events

Create GCP IoT registry:

gcloud iot registries create synpse-registry --region=us-central1 --enable-http-config --enable-mqtt-config --state-pubsub-topic projects/{project_id}/topics/synpse-events

Create storage event

gsutil mb -l us-central1 -b on gs://synpse-events

Create dataflow job

gcloud dataflow jobs run ps-to-avro-synpse-events --gcs-location gs://dataflow-templates-us-central1/latest/Cloud_PubSub_to_Avro --region us-central1 --staging-location gs://synpse-events/temp --parameters inputTopic=projects/iot-hub-xxxxxx/topics/synpse-events,outputDirectory=gs://synpse-events/events,avroTempDirectory=gs://synpse-events/avro-temp

Data flow would look as bellow:

gcloud iot devices create synpse --region=us-central1 --registry=synpse-registry

Generate certificate for our device

openssl ecparam -genkey -name prime256v1 -noout -out ec_private.pem
openssl ec -in ec_private.pem -pubout -out ec_public.pem

Add public key to GCP IoT Core

gcloud iot devices credentials create --region=us-central1 --registry=synpse-registry --device=synpse --path=ec_public.pem --type=es256

Download google root CA

curl  https://pki.goog/roots.pem -o roots.pem

(Optional) Test application

python gateway/gcp.py --device_id synpse --private_key_file ./ec_private.pem --cloud_region=us-central1 --registry_id synpse-registry --project_id iot-hub-326815 --algorithm ES256 --message_type state

Deploy an application

Deploy Synpse application. Modify application yaml with your thing endpoint.

Create certificate secret

synpse secret create gcp-cert -f ec_private.pem
synpse secret create gcp-root -f roots.pem

Deploy the application:

synpse deploy -f synpse-gcp-example.yaml

where synpse-gcp-example.yaml is

name: GCP-IoT-Hub
description: Google Cloud IoT Core Synpse example
  type: Conditional
    gcp: iot
    - name: nats
      image: nats
      restartPolicy: {}
    - name: metrics
      image: quay.io/synpse/metrics-nats-example-app
      restartPolicy: {}
    - name: gcp-iot
      image: quay.io/synpse/gcp-iot-hub-example
      command: /server/gcp.py
        - --device_id=synpse
        - --private_key_file=/server/ec_private.pem
        - --cloud_region=us-central1
        - --registry_id=synpse-registry 
        - --project_id=iot-hub-326815 
        - --algorithm=ES256 
        - --message_type=state
        - --ca_certs=/server/roots.pem
      - name: gcp-cert
        filepath: /server/ec_private.pem
      - name: gcp-root
        filepath: /server/roots.pem
        - name: NATS_HOSTNAME
          value: nats
      restartPolicy: {}

Once running, you should see application running and data coming into the GCP storage account blob.

Last updated