Setting up a Virtual Machines for firmware implementation¶

To run the Hog CI, you need a dedicated Virtual Machine, since the available shared runners are not ideal to execute heavy software like Vivado or Quartus.

Setting up a dedicated Virtual Machine¶

In this section, you can find more information on how to set up your private Gitlab runner. Instructions are provided assuming you have access to the CERN computing resources. If this is not the case, you can still use Hog assuming that you have access to a machine running CentOS 7 (or another Linux flavour supported by your preferred HDL IDE) set up as a Gitlab runner.

In this case, you can ignore the next section and jump directly to Install Gitlab runner

Create a CERN Openstack Virtual Machine¶

Openstack is a cloud operating system that controls large pools of compute, storage, and networking resources through a data-centre, managed and provisioned through APIs with common authentication mechanisms. Openstack provides you with a dashboard from which you can manage VM instances.

More information on Openstack can be found in the Openstack Dashboard Documentation.

To create a new VM, you have to connect to the CERN Openstack dashboard and create a new instance.

Openstack instances come with different flavours, meaning that you can allocate only a fixed amount of each resource to each VM.

Before creating an new instance you can add a new disk that you can use to install the needed tools. To do this go under Volumes -> Volumes on the left navigation bar. Once the Volumes summary appears you can click on + Create Volume and follow the instructions therein. We recommend having at least a 40GB HD

Once you have obtained a custom flavour and a dedicated disk, you can create a new instance. Navigate to Compute > Instances, once you get to the instances summary click on Launch Instance. Fill in the required information in the form that will appear. Under the Source tab select an updated CC7 image: this will generate a VM running under CentOS 7. Select the custom flavour in the Flavor tab. Generate a new key pair and save the private key, this will be needed later to access your VM.

Once a new instance is running (note it might take a few minutes to be generated) attach the Volume you created to the VM. This can be done through the drop down menu on the right side of the instance summary by clicking on attach volume.

You can now connect to your machine through ssh.

Note

Your machine is not fully public yet (reference the Openstack manual for this). This means your VM will be accessible only from the CERN domain. If you are not on the CERN domain, connect to a CERN public machine (lxplus) and then to your machine.

  ssh -i private-key.pem <machine_ip_or_name>

Once you are logged into your machine, change the root password:

  sudo password root

Please follow the IT recommendation when choosing a new password.

Configure the Openstack VM¶

Login to lxplus.cern.ch

ssh -XY user@lxplus.cern.ch

Clone the Hog VM setup repository, in any folder inside your AFS area.

git clone https://gitlab.cern.ch/hog/vm-setup.git

Note

EOS is not accessible from the Virtual Machine, so don’t clone the repository there.

Now login into your VM.

  ssh -XY user@<machine_ip_or_name>

Go to the just cloned vm-setup folder, and switch to root permission.

sudo su

Now install the required dependencies,

./install_dependecies.sh

Then format and mount the volume,

./volume-setup.sh

Finally, install the required software (Vivado, QuestaSim, Quartus) in the attached volume (/mnt/vd).

Follow the instructions to install them from the correspondent vendor website.

Warning

You are the one responsible for correctly licensing the software.

Once this is done, we can install our gitlab-runner. First, we need to find the gitlab-runner registering token.

Open your Gitlab repository on a web browser, and go to Settings->CI/CD->Runners. Copy the registration token.

Go back to your virtual machine, and launch the install_runner.sh script, without options to see the Usage description.

 Hog - Install gitlab runner
 ---------------------------
 Install the Gitlab Runner on a linux machine

 Usage: ./install_runner.sh -u <user_name> -g <user_group> -t <gitlab_token> [OPTIONS]
 Options: --tags: <list of tags for this VM>, default: hog,vivado
          --url: <Gitlab url>. Default: https://gitlab.cern.ch

Required arguments for the script are:

the user that will run the Hog-CI. If you work at CERN, we suggest to use a service account. More info here);
The user group. At CERN, this corresponds to your experiment group (e.g. for ATLAS is zp);
The Gitlab registering token just obtained.

As you can see, it is possible to specify the tags for the runner and the gitlab_url, to create a runner for other Gitlab’s locations (e.g. gitlab.com).

Note

By default, Hog requires the vivado and hog tags to run its CI. However, it is possible to change the required tags, individually for each CI job. This allows the users to direct the jobs to the desired VMs.

Launch now the script with the right options. This will create the home folder for your service account on the VM, and register the gitlab runner. The gitlab runner configuration will be saved on /etc/gitlab-runner/config.toml, and will look like this,

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Hog runner on tutorial-hog"
  output_limit = 20000
  url = "https://gitlab.cern.ch"
  token = "<my_token>"
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Here you can change the concurrent value, to set the maximum numbers of gitlab CI jobs this machine will run in parallel.

If you go now back to your gitlab repository webpage, under Settings->CI/CD->Runners, you should see your new runner under the Other available runners list.

The Virtual Machine is now ready to run the Hog-CI.

Manually register runner¶

This paragraph assumes you already installed a gitlab runner on your machine together with all the tools needed to work with your code. Besides the IDEs (Xilinx Vivado, Intel Quartus Prime, MentorGraphics Modelsim, …), the typical list of tools you may want to install are:

krb5-user: needed for kerberos authentication (kinit, kdestroy, …)
xrootd client: may be needed for copying files to/from eos (xrdcp, xrdfs, …)
eos client: may be needed for accessing external files on eos (eos, eosmount, …)
openafs client: may be needed for accessing external files on afs
uhal from ipbus: may be needed for IPbus usage

Once all the tools are installed and working correctly on your machine you have to make the new gitlab runner available to your repository. To register your new machine as a runner please oper a browser and navigate to your repository. Go to Settings -> CI/CD and expand the Runners tab. Copy the registration token reported in the Specific Runners column (left).

To register your new machine as a gitlab runner run the following command, substituting the placeholder with their value:

  gitlab-runner register \
    --non-interactive \
    --url <RUNNER_URL> \
    --registration-token <REGISTRATION_TOKEN> \
    --executor "shell" \
    --description "HOG runner on <PC-NAME>" \
    --tag-list "hog" \
    --output-limit 20000 \
    --run-untagged="true" \
    --locked="false"

<RUNNER_URL> is the runner url as reported under Specific Runners column close to the registration token you copied earlier <REGISTRATION_TOKEN> is the registration token you copied earlier. <PC-NAME> is the name of your machine (Any name will do this allows to distinguish runners on different machines)

For Windows machines open a command prompt and go to the folder containing the gitlab-runner executable before invoking the command.

Once the runner is registered it should appear in the Specific Runners column of your repository. Open your repository, go to Settings -> CI/CD and expand the Runners tab and make sure your new runner islisted as active in the Specific Runners column.

The Hog CI relies on a bash shell. If you are not using a system for which this is the default (e.g. Windows) please set it as default. Navigate to the gitlab-runner installation folder and open the config.toml file with your favourite text editor. Change the shell field in the runners section to bash

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Hog runner on Geronimo-Stilton"
  url = "https://gilab.myawsomedomain.edu"
  token = "wasd_wasd_wasd"
  executor = "shell"
  shell = "bash"
  builds_dir="/c/gitlab-runner/builds/"
  cache_dir="/c/gitlab-runner/cache/"

For Windows machines please also make sure builds_dir and cache_dir are set to a correct path. Typically you want them to point to a subfolder of the gitlab runner installation folder, e.g. in the example above the installation folder is C:\gitlab-runner\

Now that the runner is correctly set up you can start the runner

gitlab-runner start

For Windows machines please note the gitlab runner for Windows fails to run a bash shell even if this shall should be available. To avoid this issue please install a Git bash. Once this is available you can open a command prompt with admin priviledge (type cmd in the Windows search tab, right click, run as administrator) and run the gitlab sunner in Git bash:

"C:\Program Files\Git\bin\bash.exe" -c "/c/gitlab-runner/gitlabrunner.exe --config /c/gitlab-runner/config.toml --working-directory /c/gitlab-runner/ --service gitlab-runner --syslog"

if the service fails to start with error FATA[0000] Failed to start GitLab Runner: The service did not start due to a logon failure. also have a look at the relative page in the gitlab documentation

Setting up a Virtual Machines for firmware implementation

Contents

Setting up a Virtual Machines for firmware implementation¶

Setting up a dedicated Virtual Machine¶

Create a CERN Openstack Virtual Machine¶

Configure the Openstack VM¶

Manually register runner¶