Create a continuous integration pipeline using Jenkins and GitLab/Github

Executive Summary :
There is always many solutions to a problem but whichever is straight forward and accurate always should be approached first. In earlier days the builds or applications used to be deployed on dedicated server or dedicated infrastructures were required to host applications or solutions and they were also too much costly. But in today's time CLOUD is the solution of those problems.

In the same context, CI/CD pipeline come into the picture where with just one click you can build/test/deploy your solution without any lengthy process to be followed. That's what this blog is all about, it will guide you through how you can create the Pipeline to automate build/test/deploy stages using Jenkins, GitLab and Virtualisation (It can be a machine, docker, kubernetes - cluster).


Assumptions and Pre-requisites :

• You have GitLab in your local server with an account or you must be holding account in public Github.
• Jenkins must be installed on your local machine or on any VM which you can use.
• You must be holding the account for Jenkins
• You must be having basic knowledge of Jenkins.

High level steps to create simple Jenkins CI/CD pipeline by design : 

Step 1 : Login to Jenkins

Login to Jenkins with specified credentials. If you're logging into Jenkins for the first time then you will be prompted to install some default set of plugins, it may take a few mins and restart Jenkins. Just for the information, Jenkins has rich set of plugins available to support wide range of platforms and jobs to accomplish. Once the plugins gets installed successfully, you will be redirected to the home page.

Step 2 : Create and Build Sample Project : 

There will be create new jobs link to get started, once you click on the link, it will prompt a page where you require to enter the title of the project name and select the project type. Please see below image.

Here you can name your project/job and select Free Style Project and press OK, this will create a new Job/Project in your Jenkins. Once Job is created you can configure the job with different parameters as shown in below image.

As mentioned in the snaps, there are many section you can configure but for now you can select the build section and click on the Execute Shell for the demo job execution. Further clicking on that will open a text area where you can write Shell command as mentioned below in snap. 

Now clicking on apply and save will save this configuration and you will be able to build this job and see the results in console output. With this configuration, this will print the message on the console as you have added the Shell command *echo*.

Step 3 : Build a GitLab Project :

Now that you know the basics of using Jenkins, let's move to the "real world" project. You can use simple java + maven project by forking the ready project from here and add the same in your own repository where you can modify it. 

Further you can copy the SSH url of the project and keep it somewhere saved. Now you can go back to the Jenkins dashboard  and select the job/project ('MavenPipeline') which you have created to console the message. Click on Configure, enter necessary details in General section and move to the SCM - Source Code Management  section and select Git option to configure the same.  As you can see you can define the branch as well for which you want to trigger the build while user commit on that branch. Along with that you can also define the Repository Browser which should be set ideally default 'Auto' but you can also select specific option from it e.g. GitLab, GitList etc.

Well, you might be thinking how the repo access is open for the Jenkins so answer to this is you must get the ssh key of the Jenkins root user and set the same in the GitLab/Github profile. To do the same follow the below steps :

• Open terminal
• Type sudo passwd jenkins (Only if you don't remember the password of jenkins user) 
• Enter new password and confirm password => the same password store somewhere in text file
• Type su jenkins press enter 
• Enter your jenkins user password
• Type ssh-keygen 
• It will ask you to write the ssh key in id_rsa file in your system => Press enter
• It will ask  you to enter passphrase => Press enter
• Your ssh key will get generated for that particular Jenkins user
• Now Type cat ~/.ssh/id_rsa.pub  this will console the public ssh key you recently generated 
• Copy/Paste that key to the GitLab/Github => account settings => SSH Keys
• And your good to go with this!

This project is based on Maven so before you (Jenkins) build it, it needs Maven to be installed as plugin in Jenkins. To do so you can click on the Jenkins logo to move to Jenkins dashboard then Manage Jenkins => Global Tool Configuration. Further, you can move to the Maven section in the same page and click on Maven  installations and hit Apply-Save.

Further, move on the job (MavenPipeline) configuration and scroll up to the Build section => Add Build step => Select Invoke Top level Maven Targets => select Maven as version from the dropdown and set the Goals to 'clean package' to clean the build before generating new, click Apply-Save.

Now head back to the project list/jobs page in Jenkins page  => select the 'MavenPipeline' => click on Build Now. With this Jenkins will Clone your repository and build it through Maven. You can verify it once the build phase is completed you can review the console output to check the same.

Step 4 : Create a continuous integration pipeline for GitLab Project

The next step is to connect the both main entities of the pipeline i) GitLab/Github ii) Jenkins. Once this connection is well established and tested properly, your single commit to particular branch in your GitLab/Github will trigger the build generation step automatically in Jenkins and that's what the Pipeline is all about. 

• Visit your GitLab/Github repo settings page
• For GitLab => head back to project settings => Integrations

• For Github => head back to project settings => Side Menu Webhooks

• Both will offer you almost same page having 
• For GitLab i) URL field, ii) Secret  key, iii) On which event you want to trigger the build 
• For Github => Add WebHook => Fill almost the same details as above
• Save the configurations 
• In our case below should be the configurations
• For GitLab 
• URL  should be http://IP_WHERE_JENKINS_SETUP:PORT/project/FREESTYLE_PROJECT_NAME/ in our case it should be  http://IP_WHERE_JENKINS_SETUP:8080/project/MavenPipeline/
• The secret token is for the success-full hand-shaking between Jenkins and GitLab so to generate the same below are the steps : 
• GoTo Jenkins project/job => Configure => Build Triggers => select Build when a change is pushed to GitLab. GitLab webhook URL: http://localhost:8080/project/MavenPipeline => Advanced options => Secret token => Generate 
• This will generate a token which you can please store somewhere in text file
• Paste the same secret token to the Secret token field
• Tick the events on which you want to trigger the build and enable SSL
• Save configurations and Add WebHook

• For Github
• Enter Payload URL the same as above (But make sure here, the IP you are using should be public to let the WebHook be able to access your Jenkins)
• Content Type : application/json
• The same secret key that you generated above
• Events on which you want to trigger the build

• For Gihub  you are good to go
• For GitLab : now as final configuration => GoTo Jenkins => MavenPipeline Job configuration => Build Triggers => configure the events from here too
• Build when a change is pushed to GitLab. GitLab webhook URL: http://localhost:8080/project/MavenPipeline

Your pipeline is now all configured, what remains is just to test it and further check while you commit and push anything from either GitLab or Github should trigger the build in Jenkins.

As part of testing I have created separated branch 'dev', modified and committed the ReadMe file as mentioned in snap and it triggered the job in Jenkins to build the particular branch with the latest committed state.

That's all about the Pipeline!! Happy Pipelining. . 

To Be continued. .  Stay tuned for the next part of this Pipeline with Node Project!!