GCP Part 2 - Sending Data to BigQuery

This chapter describes sending data from deviceWISE to Google Cloud's BigQuery.

In this section you need to create a dataset and will later send data from deviceWISE to BigQuery.

  1. Open the BigQuery web UI in the Google's Cloud Console.
    You will view the connected project at the left-hand side of the screen.
  2. Click on the project.
  3. Click CREATE DATASET.
  4. Enter Dataset ID. For example: Factory_Line2_Dataset.
  5. Click Create dataset.
    The dataset gets created.
  6. Click on the created dataset.
  7. Click Create table.
  8. Enter Table name.
  9. Click Add field.
  10. Enter the name of the fields (RPM - INTEGER, Tag - STRING, Timestamp - TIMESTAMP).
  11. Click Create table.

This section provides a step-by-step guide to create the logic, to send the deviceWISE data to BigQuery. To create a trigger, do the following:

  1. Right-click on Projects and select New.
  2. Enter a Name and Description.
  3. Right-click on the new project and click Start.

    The project state changes to Started.
  4. Double-click on the project you just started, to open the project on a new tab.
  5. Click New, to start defining a new Trigger.

    The Trigger screen appears. The yellow box contains the event types, the blue box contains the actions, and the green box is the canvas. To define the trigger do the following:
    1. Enter Name of the trigger.
    2. Click Trigger Event Type drop-down and select On-Demand as the event type.
    3. Expand Google Cloud action group and drag & drop the BigQuery - InsertAll action onto the canvas.
    4. Drag & drop the end execution block, located above the Trigger action group.
    5. Connect the Start, to the BigQuery - InsertAll action block.Connect the BigQuery - InsertAll action block to the End Execution.
    6. Connect the grey arrow to the End Execution.
    7. Double-click the BigQuery - InsertAll action block to edit the action.
    8. In the BigQuery - InsertAll window, you will notice all the fields are populated from the dataset, that you created from the previous section.
    9. Enter the values for all the Input variables. Map the Data Point Timestamp to the $EVENT_TIME macro.
    10. Save and close the Trigger editor.
  6. Right-click on the trigger and click Start.

    The trigger is started. You will notice 1 in the success count indicating the row is added to the BigQuery.
  7. Open the BigQuery web UI in the Google's Cloud Console and go to the project you created.
  8. In the Query editor, enter the following query: Select * from Factory_Line2_Dataset.Machine1.
  9. Click Run.
    The rows with the values you entered in the Workbench is displayed.