Linux Kernel Live Patching Engineer

Прямой работодатель  Sardina Systems ( sardinasystems.com )
Миддл
Информационные технологии • Разработка • OpenStack • SaaS/PaaS
18 ноября
Удаленная работа
Опыт работы любой
Работодатель  Sardina Systems
Описание вакансии

Sardina Systems is an OpenInfra software platform company based on OpenStack and Kubernetes and is looking for a Linux Kernel Live Patching Engineer to join our team.

Position summary

Build and operate a live kernel patching feed for AlmaLinux 10. You own the full technical chain, from CVE triage to signed, tested RPMs published in a public repository. This is a hands-on kernel engineering role, not a management or release-coordination position.

Core responsibilities

Patch engineering

- Track AlmaLinux security advisories (ALSA) and upstream kernel CVEs.

- Triage CVEs against the published scope: critical, remotely exploitable or actively exploited vulnerabilities.

- Backport security fixes to the supported AlmaLinux 10 kernels, resolving conflicts across version differences.

- Generate live patch modules with klp-build. Initially, publish individual CVE patches; evaluate cumulative/atomic replace strategies as the pipeline matures.

- Identify CVEs that cannot be live patched (e.g. __init changes, data structure layout changes) and record them in the coverage-gap log.

Build and distribution pipeline

- Reproduce AlmaLinux kernel builds exactly (same source, config and compiler), e.g. in mock with the AlmaLinux buildroot, so modules match the running kernel.

- Maintain the automated pipeline: CVE scan, build plan, klp-build, RPM packaging, signing, test gate, createrepo_c publication.

- Plan and execute the move of support to new AlmaLinux kernel releases as they ship.

- Verify the exact client-side commands work on stock AlmaLinux 10 (dnf kpatch auto plus kpatch-patch-* packages).

Testing and release gate

- Run the mandatory pre-publication gate on every patch:

- load and unload on the target kernel in QEMU;

- run kernel livepatch selftests;

- for high-severity CVEs: apply and revert under load on a FishOS test cluster with running VMs and Ceph I/O.

- Document which tests were passed for each release.

- A patch that passes the gate is cleared for publication. Do not publish a patch that has failed the gate.

Release and withdrawal

- Publish approved patches with release notes listing the CVEs covered.

- Publish the source for every released patch alongside its RPM.

- If a published patch misbehaves: pull it from the repository, publish an advisory, and give users revert instructions.

- Maintain the runbook and coverage-gap log.

Secure Boot

- Manage the Sardina module-signing key and RPM signing key.

- Document and verify mokutil key enrolment for AlmaLinux users.

- Work with the Ops team to automate key enrolment on FishOS.

Required skills

- Linux kernel development: strong C, with hands-on experience backporting kernel patches or debugging kernel modules. Screening will test this directly.

- Live patching: practical experience with klp-build or kpatch, or demonstrated ability to learn them quickly from source.

- RPM and the RHEL/AlmaLinux ecosystem: kernel packaging (RPM/SRPM), mock, dnf, createrepo_c and repository metadata.

- Automation: Python or shell; building and maintaining CI-style pipelines.

- Signing: GPG key management and RPM signing; kernel module signature verification and MOK enrolment.

- English: working proficiency.

Preferred skills

- Spoken Russian and/or Mandarin, in addition to English. This is to work closely with Russian- and Mandarin-speaking team members.

- QEMU-based VM validation or canary testing.

- Upstream kernel contributions or involvement in the live patching community.

- Koji or similar build orchestration systems.

- Ceph I/O testing or storage-heavy workloads.

Key traits

- Boundary awareness: Willing to say plainly "this CVE needs a reboot" and "this kernel is out of scope" rather than forcing a risky patch

- Release discipline: Understands that a bad patch is the main reputational risk and will not skip the test gate under pressure

- Self-directed: Comfortable owning an end-to-end pipeline with minimal supervision

- Confidentiality: handles embargoed vulnerabilities and unreleased plans with discretion.

Working context

- Scope: AlmaLinux 10 only, starting with one kernel version.

- Infrastructure: runs separately from the existing Koji-based OpenStack build infrastructure, but follows the same automation approach.

- Primary tooling: klp-build, kpatch/kpatch-dnf, mock, createrepo_c, GPG, systemd timers, QEMU and a FishOS test cluster.

- Release authority: Routine patches are approved by the automated gate; no per-patch sign-off is required. A named release authority reviews weekly summaries, approves exceptions (e.g., a patch that cannot pass the full gate for a documented reason), and handles emergency withdrawals.

- Backup: one Ops team member is cross-trained on the test gate, publication and withdrawal steps early in the project.

Success Criteria

First milestone

- klp-build confirmed working on a real Alma 10 kernel

- End-to-end pipeline producing a signed, tested patch for one CVE class on one kernel version

- Test gate passed on at least two consecutive builds

- Coverage-gap log and runbook exist

Second milestone

- Pipeline operating reliably with the test gate on every build

- Secure Boot enrollment automated on FishOS

- Sources and tooling published under the appropriate license

- Runbook and coverage-gap log maintained and current


Специализация
Информационные технологииРазработкаOpenStack
Отрасль и сфера применения
SaaS/PaaS
Уровень должности
Миддл
Загрузка формы отклика...