One-click Automation for Full Data Backup on GitLab

The official backup solution provided by GitLab is relatively loose, requiring separate maintenance of data, configurations, and certain SSH-related data, which is not very convenient.

As a result, I have recently completed a project that achieves a one-time backup of all dependent data, archives them by date, allows you to define the backup storage cycle for local and remote locations, and supports pushing the latest backup status through WeChat Work (Enterprise WeChat), making it convenient for all of you to dynamically monitor the backup status of GitLab.

Project Address:

https://github.com/galtjay/gitlab-fully-backup

Project Introduction:

1.Business Data: This involves using the gitlab-backup create command to achieve a comprehensive backup of GitLab's overall data.

2.Configuration Files: All GitLab configuration files are backed up using the gitlab-ctl backup-etc command.

3.User Uploaded Public Key Data: Backup of the authorized_keys for Git, preserving user-uploaded public key data.

4.Backup of /etc/ssh Data: When restoring or scaling Rails nodes, it's necessary to overlay several key strings from /etc/ssh to ensure proper functioning.

The project is designed for GitLab deployments using the Omnibus method, including GitLab instances with PostgreSQL and MinIO clusters.

The program is based on Python 3. If you are using an older Linux distribution, you can follow the steps below to install Python 3.

yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make
wget https://www.python.org/ftp/python/3.9.17/Python-3.9.17.tar.xz
tar -Jxvf Python-3.9.17.tar.xz
cd Python-3.9.17/
./configure  prefix=/usr/local/python3 
make && make install
/usr/local/python3/bin/python3   -V

Configuration File: config.txt, defining the Enterprise WeChat webhook URL, the backup bucket for MINIO, and the storage cycle.

[WECHAT]
WEBHOOK_URL = https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=fffffff-aaaaaaa-kkkkkk-eeeeee

[MINIO]
MINIO_ENDPOINT = 123.123.123.123:9000
MINIO_ACCESS_KEY = gitlab
MINIO_SECRET_KEY = yoursecretkey
GITLAB_BUCKET_NAME = gitlab-backup

[BACKUP]
#Local storage days
LOCAL_STORE_DAYS= 3
#Remote storage days
MINIO_STORE_DAYS= 14

Installing Dependencies: /usr/local/python3/bin/pip3 install -r requirements.txt

How to Run: /usr/local/python3/bin/python3 main.py # It can also be run using a virtual environment.

Scheduled Task: 0 23 * * * cd /path/to/gitlab-fully-backup && /usr/local/python3/bin/python3 main.py