HEP/ATLAS Computing Fundamentals

What you need to know

Throughout this workshop we will assuming previous knowledge in Unix, Python, C++ and compiling to a basic level. Luckily these topics are heavily documented already and there is a lot of material available so if you feel you are not very confident with any of these prerequisites please read the links below and work through the examples before you arrive.

It is not necessary for you to be an expert but it would be beneficial to have some experience and to feel comfortable working with them.

Pre-requisite Checklist

• Unix and the Terminal : Download this file shell-lesson-data.zip ,create a directory named “Desktop” on your home area in the UChicago Analysis Facility How to submit, and upload there the file you downloaded. Then Review and understand the contents of the Software Carpentry Tutorial. (Don’t worry if you don’t see directories like: “Aplicacions”, “Download”, “Movies”, “Pictures”, etc while going through this tutorial, you just need the “Desktop” directory that you just created.)
• Essential C++ : Ensure you understand the concepts in the “Bare Minimum” checklist.
• Building Code : Know the difference between compiling and linking.
• Python : Review and understand the contents of the Software Carpentry Tutorial.
• GitLab : The service used by CERN is GitLab. This one’s easy, just make sure you can log in.

Unix and Shells

In Unix operating systems like Linux or OSX, the shell (“the terminal”) is a program that interprets commands and acts as an intermediary between the user and the inner workings of the operating system. It can help you navigate your files and directories. You can create (touch and mkdir), copy (cp) and delete (rm) files and repositories. Even more powerful things are possible such as working remotely with ssh, the transfer of files with scp and shell scripting tasks you do regularly.

What are those commands?

If you are wondering what any of the commands are in the above paragraph, then you can always use the shell to learn more by using the man command. Read more about the ssh command by trying the following

man ssh

More importantly, if you are wondering how any of these commands work, then it is imperative that you spend a day reviewing how the shell works prior to the bootcamp.

Software Carpentry provides two very nice in-depth tutorials on the shell

• Unix and the Shell : All participants should review this tutorial and ensure they are comfortable with its contents prior to the bootcamp.

A few other very good go-to resources that are good to work through

• History of Unix and the Shell : The backstory of all this
• tutorialspoint.com : A good introduction to the shell and the basic utilities it has
• Shell Scripting : How to write code in the “bash” programming language

Please take some time before the workshop to read through these.

The bare minimum

Though the essential aspects and “bare minimum” is very subjective, a few of the most basic concepts that you should be comfortable with are the following.

• Navigating files and directories (pwd,ls and cd)
• Create, copy and delete files and directories (mkdir, touch, mv, cp and rm)
• Executing shell scripts (source script.sh)
• Environmental Variables (echo $ENVVAR)

You can use this as the “bare minimum” checklist for this bootcamp and you will be expected to be familiar with these concepts throughout the bootcamp.

Essential C++

Throughout your studies and career in particle physics, you will find a large amount of your code and software will be written in C++. Within ATLAS for example, the software is very C++ based and many analysis frameworks are constructed using C++. The problem with teaching C++ in the context of research is that it is such a broad programming language that it becomes very difficult to adequately explain “the essential” things that you need to know. It would be very worth your while to enroll in a formal university level introduction to C++ course and this will pay dividends in your research. Fortunately, there is a large amount of C++ material available and hopefully you can gain some basic understanding of C++ essentials if you are not familiar already. And for the purposes of this bootcamp, a certain amount of requisite knowledge is necessary, as highlighted below. A few good references are :

• cplusplus.com : The go-to reference for c++ with a solid tutorial. All participants should review this tutorial and ensure they are comfortable with its contents. If you have questions about whether a particular concept is important or frequently used in HEP, please ask!
• learncpp : A more long and drawn out tutorial. It presents the same thins as cplusplus.com, but from a different perspective.
• tutorialspoint.com tutorial : Another very good reference with tutorials
• W3School tutorial : W3 Schools covers more than just c++ including HTML, Java, etc. as well

The bare minimum

Though the essential aspects and “bare minimum” is very subjective, a few of the most basic concepts that you should be comfortable with are the following.

• Loops : https://www.tutorialspoint.com/cplusplus/cpp_loop_types
• Vectors : https://www.tutorialspoint.com/cpp_standard_library/vector
• Pointers : https://www.tutorialspoint.com/cplusplus/cpp_pointers
• Function : https://www.tutorialspoint.com/cplusplus/cpp_functions
• Classes : https://www.tutorialspoint.com/cplusplus/cpp_classes_objects
• Namespaces : https://www.tutorialspoint.com/cplusplus/cpp_namespaces
• Inheritance/Polymorphism : https://www.tutorialspoint.com/cplusplus/cpp_inheritance

You can use this as the “bare minimum” checklist for this bootcamp and you will be expected to be familiar with these concepts throughout the bootcamp.

The Build Process

One of the computing bootcamp days will cover build systems, specifically CMake. To many individuals, compilation of C++ source code into machine executables is simply “something you do” with the g++ or gcc commands. However, there are a number of details and understanding them (at least the language to start) will make things much more clear. A few very nice tutorials that you should review and understand can be found here.

• Tutorial 1 : Very complete including a bit of history as well.
• Tutorial 2 : A bit more succinct.
• Tutorial 3 : Only the concepts.

The bare minimum

Though the essential aspects and “bare minimum” is very subjective, a few of the most basic concepts that you should be comfortable with are the following.

• Source Code
• Headers
• Compilation
• Linking
• Libraries
• Preprocessor Flags (-D options)

You can use this as the “bare minimum” checklist for this bootcamp and you will be expected to be familiar with these concepts throughout the bootcamp.

How to submit

scp file_to_submit <your_user_name>@login.af.uchicago.edu:/home/<your_user_name>/Desktop
example:
scp shell-lesson-data.zip joe123@login.af.uchicago.edu:/home/joe123/Desktop

Python

Python is a popular programming language that runs on an interpreter system. This means that unlike C++, you do not need to explicitly compile the code prior to running it. Instead the code can be executed as soon as it is written. This makes it very appropriate for writing scripts and testing things quickly. Python was designed for readability, and has some similarities to the English language with influence from mathematics (e.g. sets). Furthermore, it is possible to link C++ compiled libraries with python code, allowing you to form an “interface” between the two, thereby taking advantage of the best of both worlds.

It is rare to not come across Python in some form when working in particle physics and data analysis, particularly when using advanced techniques in machine learning. In the ATLAS experiment, python is used in many places, most notably as a mechanism by which we “steer” our C++ code. This is done in the form of jobOptions or a steering macro. However, in this tutorial, we will not explicitly be using jobOptions to execute our code. This is something that is rather left for the ATLAS software tutorial.

Software Carpentry provides a very nice introductory tutorial to python - Link to Tutorial. All participants should review this tutorial and ensure they are comfortable with its contents prior to the bootcamp.

Beyond this, a few additional resources that are very useful

• Python3 : Main point of entry for all things python3
• Python3 Tutorial : Included linked from the page above. This is the official tutorial.
• W3School Python Tutorial : Like C++, W3Schools has their own tutorial. A different perspective.
• TutorialsPoint Python Tutorial : … and another perspective.

GitLab

You may be familiar with Git (if not, you will learn about it from Software Carpentries) and even have a GitHub account. Many other equivalent services such as this, that serve as places where one can house remote repositories exist (e.g. bitbucket and atlassian) and the one that is used at CERN is GitLab.

Log in to GitLab

Confirm that your credentials work properly by logging into GitLab on your web browser. For this, just use the same username and password that you use for everything else CERN-related.

Once you do this, explore around the site a bit. Fill in your profile information and upload a picture to serve as your avatar.