CI/CD of cloud functions using Typescript and Cloudbuild

Photo by Luís Perdigão on Unsplash

In this story, we are going to create a background cloud function for pubsub google using typescript. We will be using cloud build for continuous deployment.

This will be a push-based background function that will be automatically executed once there is a message available to consume. However, you can easily modify the approach best suited to your needs. Since this is a push-based function we will not be using any request/response, since we receive only the payload. The approach is similar to other types of cloud functions, only the code logic will change everything other remains the same.

So let’s create a typescript project. Given below the tsconfig.json according to my set up,

{
  "compilerOptions": {
    "target": "es6",                          
    "module": "commonjs",                     
    "moduleResolution":"node",
    "lib": ["es6"],                           
    "allowJs": true,                             
    "outDir": "build",                        
    "rootDir": "src",                         
    "strict": true,                           
    "noImplicitAny": true,                    
    "esModuleInterop": true,                  
    "resolveJsonModule": true,                
    "skipLibCheck": true,                     
    "forceConsistentCasingInFileNames": true  
  }
}

Given below the folder structure that we will be using,

In the src folder let’s create a index.ts file. This will be the entry point to our project and we will export a function here, which will be used by the cloud function to execute,

import axios from "axios";
const webHookSubscriber = async(event:any,context:any)=>{
console.log("Received Message");
 const message = event.data
    ? Buffer.from(event.data, 'base64').toString()
    : 'Hello, World';
try{
await axios.post(process.env.WEBHOOK_URL,message);
}catch(err){
console.log(err);
}
}
export {
 webHookSubscriber
}

This is a simple application, which accepts a pubsub message and forwards that message to some other URL that is specified via an environmental variable (WEBHOOK_URL). For testing purposes, you can use a `Ngrok` url or a nice service like pipedream.

Note: Google also has a Typescript boilerplate setup readily available here. It’s pretty easy to set up and configure if you need a quick start solution. However in this story I have not used that. It’s because I need to avoid the coupling with that package so that with minimal change I can move across different cloud solutions.

Now let’s deploy this little function using cloud build. For those who are new to cloud build, google’s cloud build is a platform to run your ci/cd workloads. You can specify the configuration and Google does the heavy lifting for you.

So let’s write a simple cloud build configuration. Note that you can also create a cloud function directly from the console and attach a repository to it. But it’s not that flexible when you want to run some custom workflows. Also, you can create a custom cloud function from the console and still follow along with this article. No issues.

Here I will be creating the workflow as two separate steps. One will be initially used to deploy basic configuration and code, and the rest will be only supporting ci/cd. So let’s build the workflow.

Given below the initial cloud build file,

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'install']
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'run','build']
- name: 'bash'
  args: [cp,'package.json','build']
- name: 'bash'
  args: [cp,'package-lock.json','build']
- name: gcr.io/cloud-builders/gcloud
  args: [ functions, deploy, YOUR_APP_NAME,--source,build,--trigger-topic,YOUR_PUBSUB_TOPIC,--entry-point,webHookSubscriber,--runtime,nodejs10,--ingress-settings,internal-only,--retry,--max-instance,'2',--set-env-vars,WEBHOOK_URL=yourwebhookurl]

“ For simplicity the build step only executes tsc command. ”

Here we have a five-step cloud build configuration. The first 2 steps are installing the npm packages and run your build step. In our case building the typescript files. In the third and fourth step copy the package and package-lock file build folder. Which will be later used to install necessary dependencies. The last step actually deploys the code. Given below the parameter reference. For a complete reference head over to here.

--source: It indicates the directory at which the code is residing.

--trigger-topic: This indicates the corresponding trigger topic that we will be using. Note that --trigger-topic param only valid if the function triggers based on pubsub topic. Check the available trigger topic in the reference link attached above.

--entry-point: Indicates the entry point to the code. ie The exported function name that used by the cloud function for execution. You can export many functions but only that indicated by entry point param is used by Google.

--runtime: Here we use nodejs10 runtime which is stable. Google also offers a nodejs 12 runtime. But it is in the Beta stage.

--ingress-settings: Ingress settings indicate incoming traffic sources. By default, it has access to all. But we will be limiting it to only internal traffic.

--retry: We will be asking the function to retry itself on errors. Since we have added try catch the function will not retry since all the errors will be caught. So make sure you modify according to your need.

--max-instances: Max instances specify the number of instances that will be running. This will be based on the load that you assume. Please load-test and determine the need.

--set-env-vars: This will set environment variables that will be available to the function while executing. Note that this option will remove all the existing environmental variables and reset it. So if you want to remove/update env vars use the corresponding option available in the google reference doc attached above and at the end of this story. Note that you have to provide the value as KEY=VALUE syntax. Also if you have multiple environmental variables you have to use this option multiple times. We will be using these arguments only in the initial deployment. Later we will remove most of these options and create a clean build file.

So let’s deploy this using gcloud ,

gcloud builds submit --config cloudbuild.yaml

This will take approximately 2 to 3 minutes. If everything goes well we can see a successful output.

Now let’s modify the cloud build configuration like the one given below,

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'install']
- name: 'gcr.io/cloud-builders/npm'
  args: [ 'run','build']
- name: 'bash'
  args: [cp,'package.json','build']
- name: 'bash'
  args: [cp,'package-lock.json','build']
- name: gcr.io/cloud-builders/gcloud
  args: [ functions, deploy, YOUR_APP_NAME,--source,build,--trigger-topic,YOUR_PUBSUB_TOPIC,--entry-point,webHookSubscriber]

Now we have a clean project directory. Now head over to the cloud build page and add a push trigger to master and select the cloud build.YAML file from the configuration option.

Note that if you only have tsc command as your build keep in mind that it will not copy any other static files which are not recognised by the typescript compiler. So you may want to handle that in additional build steps.

Now push the code and head over to build history. We can see the build is up and running. Once it is completed head over to the cloud function section and we can see the new version deployed there.

If you like my work and want to support it, buy me a cup of coffee!

Refernces:

gcloud functions deploy | Cloud SDK Documentation | Google Cloud

Deploying Cloud Functions | Cloud Functions Documentation

Building Cloud Functions Images | Cloud Functions Documentation