feat: Configurar Dist-Git como Lookaside Cache y corregir bootstrap
- Configurar rol dist_git como Lookaside Cache únicamente (sin git-daemon). - Actualizar estructura de caché a 'rpms/pkg/file/algo/hash/file' compatible con upstream. - Añadir scripts bin/sync-sources.bash y bin/sync-all-sources.bash para sincronizar fuentes desde CentOS Stream. - Eliminar bin/import-srpm.bash obsoleto. - Actualizar script mxos-get-sources.bash en kojid para usar la nueva estructura de URL. - Corregir lógica de 'add-host' en koji_bootstrap para manejar reintentos limpiamente. - Actualizar documentación (README.rst, docs/instalación.rst) reflejando los cambios. - Correcciones menores en rutas de configuración y nombres de grupos en deploy.yaml.
This commit is contained in:
parent
3aec0d2e01
commit
fa97cba322
GPG key ID:
AB2A9F49384FE324
13 changed files with 282 additions and 216 deletions
28
README.rst
28
README.rst
|
|
@ -146,19 +146,27 @@ Uso del Cliente (CLI):
|
|||
|
||||
Gestión de Fuentes (Dist-Git)
|
||||
=============================
|
||||
El entorno incluye un servidor Dist-Git (`cs10-koji-04.mx-os`) para gestionar los repositorios de código fuente y el caché de archivos grandes (lookaside cache).
|
||||
El entorno incluye un servidor Dist-Git (`cs10-koji-04.mx-os`) configurado exclusivamente como **Lookaside Cache** para almacenar archivos grandes (tarballs). Los repositorios Git con los archivos SPEC y parches se alojan externamente en `https://git.mx-os.mx`.
|
||||
|
||||
Importar un Paquete (SRPM):
|
||||
Hemos incluido una utilidad para importar SRPMs fácilmente al Dist-Git. Esto inicializa el repositorio git y sube los fuentes.
|
||||
Sincronización de Fuentes:
|
||||
Para que Koji pueda construir paquetes, los archivos fuentes grandes deben estar presentes en el Lookaside Cache local. Hemos incluido scripts para sincronizar estos archivos desde los repositorios upstream (CentOS Stream).
|
||||
|
||||
**Sincronizar un paquete individual:**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# Desde tu máquina local (requiere acceso root/sudo para algunas operaciones del script o ejecutarlo en el servidor dist-git)
|
||||
# Se recomienda copiar el script y el SRPM al servidor dist-git o ejecutarlo allí directamente por ahora.
|
||||
ssh root@cs10-koji-04.mx-os
|
||||
/var/lib/dist-git/import-srpm.bash paquete.src.rpm
|
||||
# Uso: ./bin/sync-sources.bash <paquete> [rama]
|
||||
# Ejemplo para bash en la rama c10s:
|
||||
./bin/sync-sources.bash bash c10s
|
||||
|
||||
*Nota: El script `bin/import-srpm.bash` está diseñado para facilitar esta tarea.*
|
||||
**Sincronizar todos los paquetes:**
|
||||
Este script obtiene la lista de todos los repositorios en la organización `RPMs` de `git.mx-os.mx` y los sincroniza uno por uno.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./bin/sync-all-sources.bash
|
||||
|
||||
*Nota: Estos scripts se ejecutan desde tu máquina local y se conectan al servidor dist-git vía SSH para realizar la descarga.*
|
||||
|
||||
Ejemplos de Comandos `koji`:
|
||||
Interfaz Web:
|
||||
|
|
@ -168,11 +176,11 @@ Ejemplos de Comandos `koji`:
|
|||
Usa el comando `koji taskinfo <task_id>` para ver el estado de una tarea específica. También puedes usar la interfaz web.
|
||||
|
||||
Hacer una Compilación:
|
||||
Para compilar un paquete que ya reside en tu Dist-Git:
|
||||
Para compilar un paquete, referencia el repositorio Git externo:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
koji build mxos0-build "git+https://cs10-koji-04.mx-os/RPMs/bash.git?#c10s"
|
||||
koji build mxos0-build "git+https://git.mx-os.mx/RPMs/bash.git?#c10s"
|
||||
|
||||
Listar Tareas:
|
||||
`koji list-tasks` para ver tareas activas.
|
||||
|
|
|
|||
|
|
@ -2,13 +2,16 @@
|
|||
set -e
|
||||
|
||||
distgit_host="cs10-koji-04.mx-os"
|
||||
repo_name="test-repo.git"
|
||||
repo_path="/var/lib/dist-git/git/RPMs/${repo_name}"
|
||||
cache_file="test-file.txt"
|
||||
cache_path="/var/lib/dist-git/cache/${cache_file}"
|
||||
pkg_name="test-pkg"
|
||||
hash_algo="sha512"
|
||||
dummy_hash="dummyhash"
|
||||
# Path: /var/lib/dist-git/cache/rpms/pkg/file/algo/hash/file
|
||||
cache_rel_path="rpms/${pkg_name}/${cache_file}/${hash_algo}/${dummy_hash}/${cache_file}"
|
||||
cache_full_path="/var/lib/dist-git/cache/${cache_rel_path}"
|
||||
test_content="Hello Dist-Git"
|
||||
|
||||
echo "Testing Dist-Git functionality on ${distgit_host}..."
|
||||
echo "Testing Dist-Git Lookaside Cache functionality on ${distgit_host}..."
|
||||
|
||||
# Ensure we can connect via SSH first
|
||||
if ! ssh -q -o BatchMode=yes -o StrictHostKeyChecking=no root@${distgit_host} exit; then
|
||||
|
|
@ -17,42 +20,26 @@ if ! ssh -q -o BatchMode=yes -o StrictHostKeyChecking=no root@${distgit_host} ex
|
|||
fi
|
||||
|
||||
# 1. Setup Test Environment on Server
|
||||
echo "Setting up test repository and cache file on server..."
|
||||
echo "Setting up test cache file on server..."
|
||||
ssh -o StrictHostKeyChecking=no root@${distgit_host} "
|
||||
# Create git repo
|
||||
if [ -d ${repo_path} ]; then rm -rf ${repo_path}; fi
|
||||
mkdir -p ${repo_path}
|
||||
git init --bare ${repo_path}
|
||||
# Allow anonymous push for testing purposes if needed, though we only test clone here
|
||||
git -C ${repo_path} config http.receivepack true
|
||||
chown -R git:git ${repo_path}
|
||||
|
||||
# Create cache file
|
||||
echo '${test_content}' > ${cache_path}
|
||||
chown git:git ${cache_path}
|
||||
chmod 644 ${cache_path}
|
||||
# Create cache file structure
|
||||
mkdir -p \$(dirname ${cache_full_path})
|
||||
echo '${test_content}' > ${cache_full_path}
|
||||
chown -R git:git /var/lib/dist-git/cache/rpms
|
||||
chmod 644 ${cache_full_path}
|
||||
# Ensure dirs are executable
|
||||
find /var/lib/dist-git/cache -type d -exec chmod 755 {} +
|
||||
"
|
||||
|
||||
# 2. Test Git HTTP Clone
|
||||
echo "Testing Git HTTP Clone..."
|
||||
tmp_dir=$(mktemp -d)
|
||||
cleanup() {
|
||||
rm -rf "${tmp_dir}"
|
||||
echo "Cleaning up server..."
|
||||
ssh -o StrictHostKeyChecking=no root@${distgit_host} "rm -rf ${repo_path} ${cache_path}"
|
||||
ssh -o StrictHostKeyChecking=no root@${distgit_host} "rm -rf /var/lib/dist-git/cache/rpms/${pkg_name}"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
if git clone -q http://${distgit_host}/RPMs/${repo_name} "${tmp_dir}/clone"; then
|
||||
echo "[PASS] Git clone successful."
|
||||
else
|
||||
echo "[FAIL] Git clone failed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 3. Test Lookaside Cache Download
|
||||
# 2. Test Lookaside Cache Download
|
||||
echo "Testing Lookaside Cache Download..."
|
||||
downloaded_content=$(curl -s -f http://${distgit_host}/sources/${cache_file})
|
||||
downloaded_content=$(curl -s -f https://${distgit_host}/sources/${cache_rel_path})
|
||||
|
||||
if [[ "${downloaded_content}" == "${test_content}" ]]; then
|
||||
echo "[PASS] Lookaside cache download successful."
|
||||
|
|
@ -63,4 +50,4 @@ else
|
|||
exit 1
|
||||
fi
|
||||
|
||||
echo "All tests passed!"
|
||||
echo "All tests passed!"
|
||||
|
|
|
|||
|
|
@ -1,118 +0,0 @@
|
|||
#!/usr/bin/bash
|
||||
set -e
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <SRPM_URL_OR_FILE>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
srpm_input="$1"
|
||||
temp_dir=$(mktemp -d)
|
||||
dist_git_base="/var/lib/dist-git"
|
||||
git_repos="${dist_git_base}/git/RPMs"
|
||||
cache_dir="${dist_git_base}/cache"
|
||||
|
||||
# Ensure we are running as root or can sudo
|
||||
if [ "$EUID" -ne 0 ]; then
|
||||
echo "Please run as root"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "${temp_dir}"
|
||||
|
||||
# 1. Get SRPM
|
||||
if [[ "$srpm_input" == http* ]] || [[ "$srpm_input" == ftp* ]]; then
|
||||
echo "Downloading ${srpm_input}..."
|
||||
wget -q "$srpm_input"
|
||||
srpm_file=$(basename "$srpm_input")
|
||||
else
|
||||
cp "$srpm_input" .
|
||||
srpm_file=$(basename "$srpm_input")
|
||||
fi
|
||||
|
||||
# 2. Get Package Name
|
||||
pkg_name=$(rpm -qp --queryformat "%{NAME}" "${srpm_file}")
|
||||
echo "Package: ${pkg_name}"
|
||||
|
||||
# 3. Extract SRPM
|
||||
echo "Extracting SRPM..."
|
||||
mkdir -p extract
|
||||
cd extract
|
||||
rpm2cpio "../${srpm_file}" | cpio -idmv --quiet
|
||||
|
||||
# 4. Prepare Dist-Git Repo
|
||||
repo_dir="${git_repos}/${pkg_name}.git"
|
||||
echo "Initializing Git Repo at ${repo_dir}..."
|
||||
|
||||
if [ ! -d "${repo_dir}" ]; then
|
||||
mkdir -p "${repo_dir}"
|
||||
git init --bare "${repo_dir}"
|
||||
chown -R git:git "${repo_dir}"
|
||||
# Safe directory config should be global/system, but we can set it per repo to be sure? No, system config handles it.
|
||||
fi
|
||||
|
||||
# 5. Process Files
|
||||
# Create a temporary git worktree to commit changes
|
||||
work_dir="${temp_dir}/git-work"
|
||||
mkdir -p "${work_dir}"
|
||||
git clone "${repo_dir}" "${work_dir}"
|
||||
|
||||
# Move files to Work Dir
|
||||
mv *.spec "${work_dir}/"
|
||||
# Identify sources (archives) vs others.
|
||||
# Simple heuristic: .tar.gz, .tgz, .tar.bz2, .tbz, .zip, .xz are sources.
|
||||
# Everything else (patches, configs) goes to git.
|
||||
# BUT we need to be precise. Parsing the SPEC file is hard.
|
||||
# The "sources" file format is: SHA512 (filename) = hash
|
||||
# We will treat typical archive extensions as Lookaside Cache candidates.
|
||||
|
||||
touch "${work_dir}/sources"
|
||||
touch "${work_dir}/.gitignore"
|
||||
|
||||
for f in *; do
|
||||
if [[ "$f" == *.spec ]]; then continue; fi # Already moved
|
||||
|
||||
# Check if it looks like an archive
|
||||
if [[ "$f" =~ \.(tar\.|tgz|tbz|zip|xz|bz2|gz) ]]; then
|
||||
echo "Processing source: $f"
|
||||
# Calculate SHA512
|
||||
sum=$(sha512sum "$f" | awk '{print $1}')
|
||||
|
||||
# Add to sources file (Dist-Git format)
|
||||
echo "SHA512 ($f) = $sum" >> "${work_dir}/sources"
|
||||
|
||||
# Add to .gitignore
|
||||
echo "$f" >> "${work_dir}/.gitignore"
|
||||
|
||||
# Move to Lookaside Cache
|
||||
# Structure: package/filename/checksum/filename
|
||||
cache_path="${cache_dir}/${pkg_name}/$f/$sum"
|
||||
mkdir -p "${cache_path}"
|
||||
mv "$f" "${cache_path}/$f"
|
||||
|
||||
# Fix permissions
|
||||
chown -R git:git "${cache_dir}/${pkg_name}"
|
||||
chmod 644 "${cache_path}/$f"
|
||||
# Ensure directories are executable
|
||||
find "${cache_dir}/${pkg_name}" -type d -exec chmod 755 {} +
|
||||
else
|
||||
echo "Adding to git: $f"
|
||||
mv "$f" "${work_dir}/"
|
||||
fi
|
||||
done
|
||||
|
||||
# 6. Commit and Push
|
||||
cd "${work_dir}"
|
||||
git add .
|
||||
git config user.name "Dist-Git Importer"
|
||||
git config user.email "importer@mx-os.mx"
|
||||
git commit -m "Import ${srpm_file}"
|
||||
git push origin master
|
||||
|
||||
# 7. Fix ownership of the bare repo
|
||||
chown -R git:git "${repo_dir}"
|
||||
|
||||
# Cleanup
|
||||
rm -rf "${temp_dir}"
|
||||
|
||||
echo "Import of ${pkg_name} complete."
|
||||
62
bin/sync-all-sources.bash
Executable file
62
bin/sync-all-sources.bash
Executable file
|
|
@ -0,0 +1,62 @@
|
|||
#!/usr/bin/bash
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
FORGEJO_URL="https://git.mx-os.mx"
|
||||
ORG="RPMs"
|
||||
DIST_GIT_HOST="root@cs10-koji-04.mx-os"
|
||||
BRANCH="c10s"
|
||||
|
||||
# Check for jq
|
||||
if ! command -v jq &> /dev/null; then
|
||||
echo "Error: jq is required but not installed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Fetching repository list from ${FORGEJO_URL}/orgs/${ORG}..."
|
||||
|
||||
# Fetch all repos (paginated)
|
||||
REPOS=""
|
||||
page=1
|
||||
while true; do
|
||||
response=$(curl -s "${FORGEJO_URL}/api/v1/orgs/${ORG}/repos?page=${page}&limit=100")
|
||||
|
||||
# Validation: Check if response is valid JSON array
|
||||
if ! echo "$response" | jq -e . >/dev/null 2>&1; then
|
||||
echo "Error fetching repositories: Invalid response."
|
||||
echo "$response"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check for empty array (end of pagination)
|
||||
count=$(echo "$response" | jq 'length')
|
||||
if [ "$count" -eq 0 ]; then
|
||||
break
|
||||
fi
|
||||
|
||||
# Append names
|
||||
batch=$(echo "$response" | jq -r '.[].name')
|
||||
REPOS="${REPOS} ${batch}"
|
||||
|
||||
((page++))
|
||||
done
|
||||
|
||||
echo "Found repositories. Starting sync..."
|
||||
|
||||
for repo in $REPOS; do
|
||||
echo "----------------------------------------------------------------"
|
||||
echo "Syncing ${repo}..."
|
||||
|
||||
# Execute sync-sources.bash remotely
|
||||
# We pass the local script content to bash -s on remote
|
||||
if ssh -o BatchMode=yes -o StrictHostKeyChecking=no "${DIST_GIT_HOST}" "bash -s" < bin/sync-sources.bash "${repo}" "${BRANCH}"; then
|
||||
echo "[OK] ${repo}"
|
||||
else
|
||||
echo "[FAIL] ${repo}"
|
||||
# Optional: continue even if one fails?
|
||||
# continue
|
||||
fi
|
||||
done
|
||||
|
||||
echo "----------------------------------------------------------------"
|
||||
echo "Sync complete."
|
||||
136
bin/sync-sources.bash
Executable file
136
bin/sync-sources.bash
Executable file
|
|
@ -0,0 +1,136 @@
|
|||
#!/usr/bin/bash
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
DIST_GIT_CACHE_DIR="/var/lib/dist-git/cache"
|
||||
GIT_BASE_URL="https://git.mx-os.mx/RPMs"
|
||||
UPSTREAM_CACHE_URL="https://sources.stream.centos.org/sources/rpms"
|
||||
DEFAULT_BRANCH="c10s"
|
||||
|
||||
if [ -z "$1" ]; then
|
||||
echo "Usage: $0 <PACKAGE_NAME> [BRANCH]"
|
||||
echo "Example: $0 bash c10s"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PKG="$1"
|
||||
BRANCH="${2:-$DEFAULT_BRANCH}"
|
||||
|
||||
SOURCES_URL="${GIT_BASE_URL}/${PKG}/raw/branch/${BRANCH}/sources"
|
||||
TEMP_DIR=$(mktemp -d)
|
||||
|
||||
# Ensure we are running as root to write to cache
|
||||
if [ "$EUID" -ne 0 ]; then
|
||||
echo "Please run as root."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cleanup() {
|
||||
rm -rf "${TEMP_DIR}"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
echo "Processing package: ${PKG} (Branch: ${BRANCH})"
|
||||
echo "Fetching sources file from: ${SOURCES_URL}"
|
||||
|
||||
# Download sources file
|
||||
if ! wget -q -O "${TEMP_DIR}/sources" "${SOURCES_URL}"; then
|
||||
echo "Error: Could not fetch sources file. Check package name, branch, or network."
|
||||
# It might be 404 if no sources file exists (some meta-packages). This is not fatal, just nothing to do.
|
||||
echo "Assuming no sources or repo not found."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ ! -s "${TEMP_DIR}/sources" ]; then
|
||||
echo "Sources file is empty. Nothing to import."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Process sources file
|
||||
# Format: SHA512 (filename) = hash
|
||||
while read -r line; do
|
||||
# Skip empty lines
|
||||
[[ -z "$line" ]] && continue
|
||||
|
||||
# Parse line
|
||||
# Expecting: SHA512 (filename) = hash
|
||||
# We can use regex or simple awk/cut
|
||||
hash_type=$(echo "$line" | awk '{print $1}')
|
||||
filename_part=$(echo "$line" | cut -d'(' -f2 | cut -d')' -f1)
|
||||
hash=$(echo "$line" | awk '{print $4}')
|
||||
|
||||
if [ "$hash_type" != "SHA512" ]; then
|
||||
echo "Warning: Unsupported hash type $hash_type for $filename_part. Skipping."
|
||||
continue
|
||||
fi
|
||||
|
||||
echo "Found source: ${filename_part} (Hash: ${hash})"
|
||||
|
||||
# Format 1: Fedora/CentOS Stream 9+ (includes algorithm)
|
||||
# URL: upstream/pkg/filename/algo/hash/filename
|
||||
# Note: Algo is usually lowercase in URL
|
||||
algo_lower=$(echo "$hash_type" | tr '[:upper:]' '[:lower:]')
|
||||
DOWNLOAD_URL_1="${UPSTREAM_CACHE_URL}/${PKG}/${filename_part}/${algo_lower}/${hash}/${filename_part}"
|
||||
|
||||
# Format 2: Older CentOS (no algorithm)
|
||||
# URL: upstream/pkg/filename/hash/filename
|
||||
DOWNLOAD_URL_2="${UPSTREAM_CACHE_URL}/${PKG}/${filename_part}/${hash}/${filename_part}"
|
||||
|
||||
# Define Local Path (Matching CentOS Stream Structure: rpms/pkg/file/algo/hash/file)
|
||||
LOCAL_PATH="${DIST_GIT_CACHE_DIR}/rpms/${PKG}/${filename_part}/${algo_lower}/${hash}/${filename_part}"
|
||||
|
||||
if [ -f "${LOCAL_PATH}" ]; then
|
||||
echo " - File already exists in local cache. Verifying..."
|
||||
current_hash=$(sha512sum "${LOCAL_PATH}" | awk '{print $1}')
|
||||
if [ "$current_hash" == "$hash" ]; then
|
||||
echo " - Hash matches. Skipping download."
|
||||
continue
|
||||
else
|
||||
echo " - Hash Mismatch! Redownloading."
|
||||
rm -f "${LOCAL_PATH}"
|
||||
fi
|
||||
fi
|
||||
|
||||
DOWNLOAD_SUCCESS=0
|
||||
|
||||
echo " - Attempting download from: ${DOWNLOAD_URL_1}"
|
||||
if wget -O "${TEMP_DIR}/${filename_part}" "${DOWNLOAD_URL_1}"; then
|
||||
echo " - Download successful."
|
||||
DOWNLOAD_SUCCESS=1
|
||||
else
|
||||
echo " - Format 1 failed. Trying Format 2: ${DOWNLOAD_URL_2}"
|
||||
if wget -O "${TEMP_DIR}/${filename_part}" "${DOWNLOAD_URL_2}"; then
|
||||
echo " - Download successful."
|
||||
DOWNLOAD_SUCCESS=1
|
||||
else
|
||||
echo " - Error: Failed to download file from both formats."
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Verify Hash
|
||||
echo "Calculating hash for: ${TEMP_DIR}/${filename_part}"
|
||||
downloaded_hash=$(sha512sum "${TEMP_DIR}/${filename_part}" | awk '{print $1}')
|
||||
|
||||
if [ "$downloaded_hash" != "$hash" ]; then
|
||||
echo " - Error: Downloaded file hash mismatch!"
|
||||
echo " Expected: $hash"
|
||||
echo " Got: $downloaded_hash"
|
||||
rm -f "${TEMP_DIR}/${filename_part}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Move to Cache
|
||||
mkdir -p "$(dirname "${LOCAL_PATH}")"
|
||||
mv "${TEMP_DIR}/${filename_part}" "${LOCAL_PATH}"
|
||||
|
||||
# Set Permissions
|
||||
chown -R git:git "${DIST_GIT_CACHE_DIR}/rpms"
|
||||
chmod 644 "${LOCAL_PATH}"
|
||||
find "${DIST_GIT_CACHE_DIR}/rpms" -type d -exec chmod 755 {} +
|
||||
|
||||
echo " - Successfully imported."
|
||||
|
||||
done < "${TEMP_DIR}/sources"
|
||||
|
||||
echo "Done."
|
||||
|
|
@ -82,7 +82,7 @@
|
|||
tags: kojid
|
||||
|
||||
- name: "Deploy: Dist-Git Server"
|
||||
hosts: distgit
|
||||
hosts: dist-git
|
||||
vars_files:
|
||||
- vars/default.yaml
|
||||
roles:
|
||||
|
|
|
|||
|
|
@ -97,14 +97,14 @@ Configura Apache para Koji Web:
|
|||
|
||||
Fase 5: Configuración de Dist-Git
|
||||
=================================
|
||||
El sistema Dist-Git gestiona los repositorios Git para las fuentes de los paquetes RPM.
|
||||
El servidor Dist-Git (`cs10-koji-04.mx-os`) está configurado como un **Lookaside Cache** para gestionar los archivos fuente grandes (tarballs) necesarios para las compilaciones de Koji. **No aloja repositorios Git**; estos se gestionan externamente.
|
||||
|
||||
Instalación y Configuración:
|
||||
Instala `git`, `git-daemon`, `httpd` y `mod_ssl`.
|
||||
Configura `httpd` para servir los repositorios Git a través de HTTPS, utilizando certificados SSL generados por la CA de Koji.
|
||||
Asegúrate de que se fuerce el tráfico HTTPS mediante redirecciones.
|
||||
Instala `httpd` y `mod_ssl`.
|
||||
Configura `httpd` para servir el Lookaside Cache a través de HTTPS. El alias `/sources` apunta al directorio de caché (`/var/lib/dist-git/cache`).
|
||||
Asegúrate de que se fuerce el tráfico HTTPS mediante redirecciones y que **la navegación de directorios esté deshabilitada** para la coherencia con el comportamiento upstream de CentOS Stream.
|
||||
Directorio y Permisos:
|
||||
Crea la estructura de directorios para los repositorios Git y la caché de fuentes (lookaside cache).
|
||||
Crea la estructura de directorios para la caché de fuentes (lookaside cache) en `/var/lib/dist-git/cache`.
|
||||
Asegura que los contextos de SELinux (`httpd_sys_content_t`) sean persistentes mediante `semanage fcontext`.
|
||||
|
||||
Fase 6: Configuración de Kojid (Constructor) y Kojira (Administrador de Repos)
|
||||
|
|
@ -151,7 +151,7 @@ Configuración del Firewall:
|
|||
* Koji Hub: HTTP (80/tcp), HTTPS (443/tcp).
|
||||
* Servidor PostgreSQL: PostgreSQL (5432/tcp) desde el Koji Hub.
|
||||
* Servidor NFS: NFS (servicios nfs, rpc-bind, mountd).
|
||||
* Dist-Git: HTTP (80/tcp), HTTPS (443/tcp), Git (9418/tcp).
|
||||
* Dist-Git: HTTP (80/tcp), HTTPS (443/tcp).
|
||||
|
||||
Fase 8: Arranque del Entorno de Compilación de Koji
|
||||
===================================================
|
||||
|
|
|
|||
|
|
@ -1,9 +1,7 @@
|
|||
---
|
||||
- name: Install Git and HTTPD
|
||||
- name: Install HTTPD and Utils
|
||||
ansible.builtin.dnf:
|
||||
name:
|
||||
- git
|
||||
- git-daemon
|
||||
- httpd
|
||||
- mod_ssl
|
||||
- policycoreutils-python-utils
|
||||
|
|
@ -11,6 +9,7 @@
|
|||
- rpm-build
|
||||
- rpmdevtools
|
||||
- wget
|
||||
- git # Kept for utility, though not serving repos
|
||||
state: present
|
||||
|
||||
- name: Create dist-git group
|
||||
|
|
@ -40,10 +39,6 @@
|
|||
- "{{ dist_git_repos_dir }}/RPMs"
|
||||
- "{{ dist_git_cache_dir }}"
|
||||
|
||||
- name: Configure git safe.directory system-wide
|
||||
ansible.builtin.command: git config --system --add safe.directory "*"
|
||||
changed_when: false
|
||||
|
||||
- name: Set SELinux boolean for HTTPD CGI support
|
||||
ansible.builtin.command: setsebool -P httpd_enable_cgi 1
|
||||
changed_when: false
|
||||
|
|
@ -104,22 +99,3 @@
|
|||
name: httpd
|
||||
state: started
|
||||
enabled: true
|
||||
|
||||
- name: Deploy custom git-daemon systemd service
|
||||
ansible.builtin.template:
|
||||
src: git-daemon.service.j2
|
||||
dest: /etc/systemd/system/git-daemon.service
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
notify: Reload systemd daemon
|
||||
|
||||
- name: Reload systemd daemon to recognize new git-daemon service file
|
||||
ansible.builtin.systemd:
|
||||
daemon_reload: true
|
||||
|
||||
- name: Ensure git-daemon.service is enabled and started
|
||||
ansible.builtin.systemd_service:
|
||||
name: git-daemon.service
|
||||
state: started
|
||||
enabled: true
|
||||
|
|
|
|||
|
|
@ -16,25 +16,9 @@
|
|||
SSLCertificateFile /etc/pki/dist-git/dist-git.crt
|
||||
SSLCertificateKeyFile /etc/pki/dist-git/dist-git.key
|
||||
|
||||
# Git Smart HTTP for RPMs
|
||||
# Maps /RPMs/pkg.git to {{ dist_git_repos_dir }}/RPMs/pkg.git
|
||||
SetEnvIf Request_URI "^/RPMs/" GIT_PROJECT_ROOT={{ dist_git_repos_dir }}/RPMs
|
||||
SetEnv GIT_HTTP_EXPORT_ALL
|
||||
ScriptAlias /RPMs/ /usr/libexec/git-core/git-http-backend/
|
||||
|
||||
<Directory "/usr/libexec/git-core">
|
||||
Options +ExecCGI
|
||||
Require all granted
|
||||
</Directory>
|
||||
|
||||
<Directory "{{ dist_git_repos_dir }}">
|
||||
Require all granted
|
||||
</Directory>
|
||||
|
||||
# Lookaside Cache
|
||||
Alias /sources {{ dist_git_cache_dir }}
|
||||
<Directory "{{ dist_git_cache_dir }}">
|
||||
Options +Indexes
|
||||
Require all granted
|
||||
</Directory>
|
||||
</VirtualHost>
|
||||
|
|
|
|||
|
|
@ -10,11 +10,38 @@
|
|||
changed_when: "'Host added' in koji_add_host_result.stdout"
|
||||
failed_when: "'is already in the database' not in koji_add_host_result.stderr and koji_add_host_result.rc != 0"
|
||||
rescue:
|
||||
- name: Convert user to host in DB if it exists as user
|
||||
- name: Get user ID of the non-host user
|
||||
ansible.builtin.command: >
|
||||
sudo -u postgres psql {{ koji_db_name }} -tAc
|
||||
"SELECT id FROM users WHERE name = '{{ item }}' AND usertype = 0;"
|
||||
register: non_host_user_id
|
||||
delegate_to: "{{ koji_db_host }}"
|
||||
changed_when: false
|
||||
|
||||
- name: Delete sessions for non-host user
|
||||
ansible.builtin.command: >
|
||||
sudo -u postgres psql {{ koji_db_name }} -c
|
||||
"UPDATE users SET usertype = 1 WHERE name = '{{ item }}' AND usertype = 0;"
|
||||
"DELETE FROM sessions WHERE user_id = {{ non_host_user_id.stdout | trim }};"
|
||||
when:
|
||||
- non_host_user_id.stdout | trim | length > 0
|
||||
- "'already exists and it is not a host' in koji_add_host_result.stderr"
|
||||
delegate_to: "{{ koji_db_host }}"
|
||||
changed_when: true
|
||||
|
||||
- name: Delete non-host user
|
||||
ansible.builtin.command: >
|
||||
sudo -u postgres psql {{ koji_db_name }} -c
|
||||
"DELETE FROM users WHERE id = {{ non_host_user_id.stdout | trim }};"
|
||||
when:
|
||||
- non_host_user_id.stdout | trim | length > 0
|
||||
- "'already exists and it is not a host' in koji_add_host_result.stderr"
|
||||
delegate_to: "{{ koji_db_host }}"
|
||||
changed_when: true
|
||||
|
||||
- name: Retry adding Koji builder host
|
||||
ansible.builtin.command: >
|
||||
sudo -u {{ koji_admin_user }} koji
|
||||
--config={{ koji_hub_admin_user_koji_dir }}/config
|
||||
add-host {{ item }} x86_64
|
||||
when:
|
||||
- "'already exists and it is not a host' in koji_add_host_result.stderr"
|
||||
changed_when: true
|
||||
delegate_to: "{{ koji_db_host }}"
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
koji_hub_plugin_dir: /usr/lib/koji_hub/plugins
|
||||
koji_hub_config_dir: /etc/koji_hub/hub.conf.d
|
||||
koji_hub_koji_web_config_dir: /etc/kojiweb/
|
||||
koji_hub_plugin_dir: /usr/lib/koji-hub-plugins
|
||||
koji_hub_config_dir: /etc/koji-hub/hub.conf.d
|
||||
koji_hub_koji_web_config_dir: /etc/kojiweb
|
||||
koji_hub_httpd_ssl_config_file: /etc/httpd/conf.d/ssl.conf
|
||||
koji_hub_httpd_config_file: /etc/httpd/conf.d/kojihub.conf
|
||||
koji_hub_koji_client_config_file: /etc/koji.conf.d/main.conf
|
||||
|
|
|
|||
|
|
@ -30,7 +30,11 @@ while read -r alg filename_part eq hash; do
|
|||
# Check if there's a trailing slash in SURL, remove it if so.
|
||||
SURL=${SURL%/}
|
||||
|
||||
url="$SURL/$PKGNAME/$filename/$hash/$filename"
|
||||
# Algo to lowercase
|
||||
alg_lower=$(echo "$alg" | tr '[:upper:]' '[:lower:]')
|
||||
|
||||
# URL: SURL/rpms/PKGNAME/FILENAME/ALGO/HASH/FILENAME
|
||||
url="$SURL/rpms/$PKGNAME/$filename/$alg_lower/$hash/$filename"
|
||||
|
||||
echo "Downloading $filename from $url"
|
||||
curl -L --fail --retry 5 -o "$filename" "$url"
|
||||
|
|
|
|||
|
|
@ -102,4 +102,4 @@ koji_cleanup_script_path: /usr/local/bin/koji_cleanup.sh
|
|||
koji_cleanup_service_path: /etc/systemd/system/koji-cleanup.service
|
||||
koji_cleanup_timer_path: /etc/systemd/system/koji-cleanup.timer
|
||||
koji_callbacks_script_path: /usr/local/bin/koji_callbacks.py
|
||||
koji_callbacks_config_path: /etc/koji_hub/hub.conf.d/callbacks.conf
|
||||
koji_callbacks_config_path: /etc/koji-hub/hub.conf.d/callbacks.conf
|
||||
|
|
|
|||
Loading…
Reference in a new issue