## Hog Continuous Integration

Hog Continuous Integration (CI) makes use of the [GitLab CI/CD](https://docs.gitlab.com/ee/ci/) and [GitHub Actions](https://docs.github.com/en/actions) workflows. The main features of Hog CI are:

- Control that merging branches are up-to-date with targets
- Create and builds Vivado projects
- Generate FPGA binary and report files with embedded git commit SHA
- Automatically generate VHDL code documentation using _doxygen_
- If configured, store IP generated files and implementation project results in a user-defined EOS folder
- Automatically tag the GitLab repository and, if enabled, create a release with the binary files and auto-produced changelog

Three pipelines are employed, triggered by the following actions:

- **Merge/Pull Request Pipeline**: triggered by each commit into a _non-Draft_ merge/pull request branch.
- **Main Pipeline**: triggered by each commit into the main branch.
- **Tag Pipeline**: triggered by the creation of a new official tag (by default starting with `v*`).

:::{note}
New commits into an already opened _non-Draft_ merge request, will interrupt the running pipeline, before starting a new one, unless the pipeline is already at the *Collect* stage.
:::

### Merge/Pull Request Pipeline
The *Merge/Pull Request* pipeline simulates, synthesises and implements the chosen HDL projects. If specified, it stores the resulting outputs to an external disk space (EOS or local drive) and creates the doxygen documentation.

The stages of the Merge/Pull Request pipeline are the following:

1. *Check*: checks that the source branch is not outdated with respect to the target branch. If it is, the pipeline fails and asks the user to update the source branch.
2. *Generation And Simulation*: the workflow of all the projects is launched. Important files produced during the workflow (binary files, reports, etc.) are saved as artifacts of these jobs.
3. *Collect*: Collects all the artifacts from the previous stages and, if enabled, produces the Doxygen documentation and copies the implementation outputs and the Doxygen documentation to the EOS repository (optionally). Finally a Job is run to check that all the projects meet the timing, if that is not the case the job fails (but not the pipeline).

During the *Generation And Simulation* step, Hog-CI performs the following steps:

- Creates a Vivado/Quartus project for each project specified in the `.gitlab-ci.yml` file or in the `.github/workflows` folder. It checks also that the Hog submodule in the repository is the same as the one specified in the CI configuration file and that the required Hog environmental variables are set up. Finally, for Vivado projects, it checks the syntax of the HDL codes, before moving to the next stage.
- Optionally handles the IPs, if the `HOG_IP_PATH` environment variable is set, as explained in the [GitLab environment variable](03-GitLab-CI/01-setup-CI.md#environment-variables)  and [GitHub secrets and inputs](04-GitHub.md#secrets-and-inputs)sections.
- Synthesises the projects. An option to run only up to the synthesis stage is available.
- Implements the projects and creates the bitstreams. An option to disable the bitstream creation is available. During this stage the implementation timing results and project version are also written in the merge request page.

If any of the projects fails timing, Hog stores the full `Projects` folder under the `DebugProjects` name and adds it to the artifacts. This is done to help you debug without having to re run the workflow locally, so these files are not collected in the `collect-artifact` stage.

The *Generation and Simulation* stage is the only one actually running on a private runner. All other stages in the Merge/Pull Request pipelines will instead run on shared-runners, using the Docker container provided by Hog.

Hog also supports a dynamically-generated CI with GitLab CI/CD. For more details, refer to the [Dynamic CI Section](03-GitLab-CI/05-Dynamic-CI.md).

Users can also add custom jobs to the MR pipeline with GitLab CI/CD, before and after the *Generation* stage, and after the *Collect* stage. For more details, see the [Custom User CI Stages section](03-GitLab-CI/04-User-Stages.md).

```{figure} figures/mr-pipeline.png
An example of a Merge Request pipeline on GitLab CI/CD: [link](https://gitlab.com/hog-cern/hog-examples/-/pipelines/1342176947).
```

```{figure} figures/pr-pipeline.png
An example of a Pull Request pipeline on GitHub Actions: [link](https://github.com/Hog-CERN/TestFirmware/actions/runs/9642630289).
```

To reduce the computing resources and time required by the CI, we introduced the ability to retrieve the build artifacts of previously run pipelines in the same Merge Request. This feature is available at the moment only for GitLab CI/CD and it is enabled by default.

When a new pipeline is triggered in a Merge Request, Hog-CI checks if there are any previous pipelines in the same Merge Request that have successfully built the same projects as the current pipeline, and that no additional changes have been made to the selected project. If such pipelines exist, Hog-CI retrieves the build artifacts from those pipelines and uses them to skip the build process for the corresponding projects in the current pipeline.


### Main Pipeline
The *Main* pipeline consists only of one stage (*Merge*), which tags the repository according to the name of the source branch. Assuming the latest tag was *vA.B.C*, the pipeline will

* increase A, if the source branch name starts with `major_version/`
* increase B, if the source branch name starts with `minor_version/`
* increase C, in the other cases.

This workflow can be customised in the `Top/repo.conf` file, as explained in the [versioning section](02-Hog-Versioning.md).

```{figure} figures/main-pipeline-gl.png
An example of a Main pipeline on GitLab CI/CD: [link](https://gitlab.com/hog-cern/hog-examples/-/pipelines/1342216583).
```

```{figure} figures/main-pipeline-gh.png
An example of a Main pipeline on GitHub Actions: [link](https://github.com/Hog-CERN/TestFirmware/actions/runs/9642888908).
```

### Tag Pipeline
The *Tag* pipeline, is triggered every time a new tag is created. The stages for this pipeline on GitLab are:

1. *Release*: Retrieves the artifacts from the Merge Request pipeline and create the GitLab release.
2. *Archive*: Copies the output products of the CI (binaries, reports, and documentation), to a path specified by the user, creating a sub-folder with the name of the new tag.
3. *Badge*: Creates the GitLab badges for the chosen projects.

On GitHub Actions, the *Tag* pipeline stages are:

1. *Get Artifacts*: Retrieves the artifacts from the Pull Request pipeline and create the GitHub release.
2. *Archive Binary Files*: Copies the output products of the CI (binaries, reports, and documentation), to a path specified by the user, creating a sub-folder with the name of the new tag.

```{figure} figures/tag-pipeline-gl.png
An example of a Tag pipeline on GitLab CI/CD: [link](https://gitlab.com/hog-cern/hog-examples/-/pipelines/1342220844).
```

```{figure} figures/tag-pipeline-gh.png
An example of a Tag pipeline on GitHub Actions: [link](https://github.com/Hog-CERN/TestFirmware/actions/runs/9642908652).
```