Run your first, private Large Language Model (LLM) on Google Cloud Platform
What are Large Language Models (LLMs)? You want to build a private LLM-based assistant to generate the financial report summary. Although Large…
Read morePlease dive in the third part of a blog series based on a project delivered for one of our clients. Please click part I, part II to read the previous parts.
In the second phase of the CI/CD process, artifacts with service-executable files are built and saved to the container file system. In the third phase, the job uploads them to the artifact registry.
Each job is launched in a new container. In order for the job commenced in phase three to read these files, you must use the mechanism for saving the artifacts resulting from the job, which is provided by GitLab. Otherwise, the job in phase three won't have access to the artifacts that were created in phase two jobs.
An example job definition in the .gitlab-ci.yml
file:
build-service1:
stage: build
image:
name: registry.gitlab.com/foobar/deadbeef/docker-base:latest
# Upload artifacts produced by each job to GitLab server.
artifacts:
paths:
- bin/
when: on_success
# Tell GitLab to remove artifacts automatically after 2 weeks.
expire_in: 2 weeks
script:
- make build
The artifacts
section indicates in which directory (paths)
artifacts are created that should be saved (uploaded to the GitLab server). We use the expire_in
keyword to define how long the artifacts are to be stored by GitLab. Setting the lifetime of the artifacts is good practice because it prevents the consumption of too much disk space on the server.
After completing the job, files from the specified directory will be sent to the GitLab server. You will be able to download them using a web browser.
Each job from the next stage, before being executed will download from the GitLab server artifacts produced by the jobs from the previous stage. This happens automatically and you don't need to modify the definition of these jobs in the .gitlab-ci.yml
file.
In order to speed up the process of creating and developing our pipeline, we want to be able to run individual CI/CD jobs manually, on a laptop/computer. Thanks to this, we can check for potential errors faster. Running a job via CI/CD will always take longer than running it manually on a laptop.
GitLab runner sets specific environment variables when it starts the job. One of them, CI_PROJECT_DIR,
is used in our scripts. When we run the job manually, this variable will not be set. You have to set its value yourself. In Dockerfile
we introduced an additional check that ensures that the necessary variable is set. If it is not set, the corresponding error message is displayed.
Example of Dockerfile
:
FROM centos:7
ARG CI_PROJECT_DIR=INVALID-ONE-ONE-ONE
RUN if [ "${CI_PROJECT_DIR}" = "INVALID-ONE-ONE-ONE" ]; then echo "ERROR! Set CI_PROJECT_DIR variable. If you are invoking 'docker build', then use '--build-arg CI_PROJECT_DIR=/SoMe/PaTh'."; exit 1; fi
ENV CI_PROJECT_DIR=${CI_PROJECT_DIR}
RUN bash -xe "${CI_PROJECT_DIR}/foobar/prepare-docker-base-image.sh"
The above Dockerfile
is used to build the image of the container in the job in the first stage of the pipeline.
An example of running docker build
with setting an environment variable:
doker build -t cdocker-base-local-$(date +%Y%m%d-%H%M%S) --build-arg
CI_PROJECT_DIR=/builds/foobar/something/project1 -f dir1/Dockerfile .
If we forget to set the required variable with --build-arg
, we will see the following error message:
% docker build -t cdocker-base-local-$(date +%Y%m%d-%H%M%S) -f Dockerfile .
Sending build context to Docker daemon 90.11 kB
Step 1/5 : FROM centos:7
---> 5e35e350aded
Step 2/5 : ARG CI_PROJECT_DIR=INVALID-ONE-ONE-ONE
---> Using cache
---> 0e858c9a7a5f
Step 3/5 : RUN if [ "${CI_PROJECT_DIR}" = "INVALID-ONE-ONE-ONE" ]; then echo "ERROR! Set CI_PROJECT_DIR variable. If you are invoking 'docker build', then use '--build-arg CI_PROJECT_DIR=/SoMe/PaTh'."; exit 1; fi
---> Running in 2ccf334c80b1
ERROR! Set CI_PROJECT_DIR variable. If you are invoking 'docker build', then use '--build-arg CI_PROJECT_DIR=/SoMe/PaTh'.
The command '/bin/sh -c if [ "${CI_PROJECT_DIR}" = "INVALID-ONE-ONE-ONE" ]; then echo "ERROR! Set CI_PROJECT_DIR variable. If you are invoking 'docker build', then use '--build-arg CI_PROJECT_DIR=/SoMe/PaTh'."; exit 1; fi' returned a non-zero code: 1
In CI/CD jobs and scripts we rely on the status code returned by the application to detect the occurrence of an error. The first stage of our pipeline uses, amongst others, Puppet to install the necessary tools. Even if errors appear during puppet apply
(e.g. you cannot install the package or download a file from the Internet), the command will still end with a code of 0 indicating success. This is well-known,and developers have their explanations for such behavior of the application. Fortunately, you can use this additional command line option: --detailed-exitcodes
.
Example:
puppet apply --detailed-exitcodes --modulepath="${CI_PROJECT_DIR}" -e
"include foobar::prepare"
This option causes puppet apply
to return an exit code calculated in a different way than the default one. Code 0 means that there were no changes or errors. Code 2 indicates that changes have been made. All other codes listed in the documentation mean that some errors have occurred.
This is the code that performs puppet apply
and checks its result. It's used in the first job in our pipeline:
set +e
puppet apply --detailed-exitcodes --modulepath="${CI_PROJECT_DIR}" -e "include foobar::prepare"
retcode="$?"
set -e
echo "Puppet finished with return code '${retcode}'."
if [[ "${retcode}" != "0" ]] && [[ "${retcode}" != "2" ]]; then
echo "Exiting because of Puppet's return code."
exit 1
fi
This is the sample .gitlab-ci.yml
file:
image: centos:7
stages:
- prepare
- build
- deploy
before_script:
# Workaround for "Could not set the value of environment variable 'GITLAB_USER_NAME': could not convert string to current locale" problem.
# https://gitlab.com/gitlab-org/gitlab-foss/issues/38698
- export GITLAB_USER_NAME=$(echo $GITLAB_USER_LOGIN)
build-docker-base-image:
stage: prepare
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/cicd/Dockerfile-base-image --destination ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG} --build-arg CI_PROJECT_DIR=${CI_PROJECT_DIR}
build-albatross:
stage: build
image:
name: registry.gitlab.com/acmecorp1/foobarproject/docker-base-image:latest
# Upload artifacts produced by each job to GitLab server.
artifacts:
paths:
- artifacts/
when: on_success
# Tell GitLab to remove artifacts automatically after 2 weeks.
expire_in: 2 weeks
script:
- bash -xe ./cicd/build-albatross.sh
build-alien:
stage: build
image:
name: registry.gitlab.com/acmecorp1/foobarproject/docker-base-image:latest
# Upload artifacts produced by each job to GitLab server.
artifacts:
paths:
- artifacts/
when: on_success
# Tell GitLab to remove artifacts automatically after 2 weeks.
expire_in: 2 weeks
script:
- bash -xe ./cicd/build-alien.sh
# [...]
build-waterbird:
stage: build
image:
name: registry.gitlab.com/acmecorp1/foobarproject/docker-base-image:latest
# Upload artifacts produced by each job to GitLab server.
artifacts:
paths:
- artifacts/
when: on_success
# Tell GitLab to remove artifacts automatically after 2 weeks.
expire_in: 2 weeks
script:
- bash -xe ./cicd/build-waterbird.sh
upload-artifacts:
stage: deploy
image:
name: registry.gitlab.com/acmecorp1/foobarproject/docker-base-image:latest
script:
- bash -xe ./cicd/upload-artifacts.sh
Thanks to the implementation of automated CI/CD based on GitLab and Kubernetes, we were able to accelerate the process of developing, testing and implementing applications by the client. Developers receive feedback much faster (daily, instead of once a week). We have reduced operating costs by using cloud services. The Kubernetes cluster is self-scalable and, if not used, the cost of its operation is small.
Our client is very satisfied with the work we've done so far. The next stage of the project is automatic end-to-end testing of the application. For this we will use Terraform and Ansible, and of course Google Cloud services.
What are Large Language Models (LLMs)? You want to build a private LLM-based assistant to generate the financial report summary. Although Large…
Read moreThe 8th edition of the Big Data Tech Summit is already over, and we would like to thank all of the attendees for joining us this year. It was a real…
Read moreMachine learning is becoming increasingly popular in many industries, from finance to marketing to healthcare. But let's face it, that doesn't mean ML…
Read moreFounded by former Spotify data engineers in 2014, GetInData consists of a team of experienced and passionate Big Data veterans with proven track of…
Read moreOn June 16, 2021, the Polish Insurance Association published the Cloud computing standard for the insurance industry. It is a set of rules for the…
Read moreApache NiFi is a popular, big data processing engine with graphical Web UI that provides non-programmers the ability to swiftly and codelessly create…
Read moreTogether, we will select the best Big Data solutions for your organization and build a project that will have a real impact on your organization.
What did you find most impressive about GetInData?