# Hog Dynamic CI
```{warning} Outdated Documentation!
    This documentation version is out of date. Please check the [latest version 2026.1](https://hog.readthedocs.io/en/latest/).
```

```{warning} Hog Dynamic CI is still under development.
```

A novel feature of the Gitlab Continuous Integration allows to generate dynamically a YAML file for the Gitlab-CI. Exploiting this feature Hog is now able to create a CI configuration for each project inside the `Top` directory automatically.

To use the dynamic CI simply include the `hog-dynamic.yml` in your `.gitlab-ci.yml` file.

```yaml
  include:
    - project: 'hog-cern/Hog'
      file: '/hog-dynamic.yml'
      ref: 'v0.2.1'
```

Remember to add the correct path in the reference, relative to your GitLab server provider. 

## The Dynamic CI pipeline

The dynamic CI pipeline is activated for each _non-Draft_ merge request commit and it is divided into a main pipeline and a child pipeline.

The *main pipeline* is composed by the following stages:

1. *Merge*: checks that all the required Hog environmental variables are set up and that the source branch is not outdated with respect to the target branch. If it is, the pipeline fails and asks the user to update the source branch.
2. *Generate*: generates dynamically the YAML configuration file for the child pipeline.
3. *Triggers*: starts the child pipeline using the dynamically-generated YAML file.
4. *Collect*: Collects all the artefacts from the child pipeline and, if activated, creates the Doxygen documentation. If EOS is used, it copies the implementation outputs and the Doxygen documentation to the EOS repository.

<img style="float: middle;" width="700" src="../figures/main.png">

The *child pipeline* simulates, synthesises and implements the chosen HDL projects. It is composed by the following stages:

1. *Generation_and_simulation*: launches the workflow for the chosen Hog projects.
2. *Collect*: Collects all the artefacts of the previous stage.

<img style="float: middle;" width="700" src="../figures/child.png">

If the __HOG_CHECK_PROJVER__ variable is set to 1, the child pipeline will contain only jobs relative to the projects that have been modified with respect to the target branch.

## Configuring the projects

By default the dynamic CI will create a YAML configuration for each project inside the `Top` folder with a full implementation and simulation. To configure the CI, a `ci.conf` file should be created for each project in the `Top/project/` directory. 

This file uses the `.ini` syntax to configure the CI jobs. Hog will read this file to generate dynamically the YAML running the pipeline. 

An example `ci.conf` looks as follow.

```conf
#NO_VER_CHECK
[generate_project]
allow_failure=true
[generate_project:variables]
HOG_ONLY_SYNTH=0
MY_VAR=3
[simulate_project]
[my_user_job]
```

The first optional line `#NO_VER_CHECK` signals Hog to not check the version of the project. In this way, the CI is always run for the specific project, independently of whether it has been modified with respect to the target branch. 

The `[generate_project]` section creates the actual `generate_project` job in the YAML. In this section you can specify all the standard GitLab CI settings, to modify your job. For example, here we want that this job is allowed to fail using the `allow_failure=true` line.

Job variables must be specified in the subsection `[generate_project:variables]`. There is no need to specify again the `PROJECT_NAME`, since this is automatically parsed by Hog. In the example, we specified a Hog specific CI variable (`HOG_ONLY_SYNTH`) and also a user variable `MY_VAR`, which may be used by the developers. 

Similarly, the developers can set-up the simulation using the `[simulate_project]` section, or even a custom user job in the `[my_user_job]` section, where `my_user_job` is the name of the custom user CI job. For more info, please read the next subsection. 

To exclude totally a project from the CI, the developer just have to create an empty `ci.conf` file in the project folder. 

### User jobs in the dynamic CI. 
Custom user jobs must be specified in the `hog-ci-users.yml` file, in the main directory of your repository, following the instructions in the [user stages section](04b-User-Stages). Jobs that will be project-specific, must be declared using the standard template syntax.

```yaml
.my_user_job: &my_user_job
    <<: *only-default # Default Hog variables. Mandatory.
    <<: *tag-hog # To run on your private machine, use tag-sr for the shared-machines running the Hog Docker containers.
    stage: user_pre # The stage when the job runs 
    script:
        - my_script # the script to run 
    allow_failure: false # extra options of GitLab
```

Naturally, you can also define global jobs, following the GitLab-CI syntax. 


## Known Issues

:::{admonition} Child pipeline status do not appear in the main pipeline page
:class: warning

To access the child pipeline status you have to click on the status button in the first column of the Gitlab pipeline page, under `CI/CD > Pipelines`. Then you have to click on the *Downstream box* to show the child pipeline.

<img style="float: middle;" width="700" src="../figures/pipeline_page.png">

<img style="float: middle;" width="700" src="../figures/downstream.png">
:::

:::{admonition} If child pipeline passes after a retry, the main pipeline remains in the fail status
:class: warning

If a child pipeline fails, the main pipeline will fail subsequently, and even if a restarted child pipeline succeed, the main pipeline would stay in a failed status. This can lead to cases where a Gitlab 504 error would make your pipeline job fail.
:::