.. _Bee cluster:

Bee cluster
===========

.. _Bee overview:

Overview
--------

Bee is a university cluster available to ICP-affiliated personnel.
Software can be loaded via Spack.

All of the storage available on the cluster is handled with a BeeGFS file system.
In total, we provide :qty:`144 <TB>` of storage shared by all users.
Note that each user is not given a specified amount of storage on the cluster and
this can cause issues when it is expended.
See `cluster status page <http://bee/ganglia/>`__ for real-time stats.

In total, the cluster consists of 768 compute cores split across 45 nodes
using a :qty:`40 <Gbps>` InfiniBand.

+-----------+--------+-------+------------------+----------------+-----------------------------+
| Type      | Nodes  | Cores | RAM              | GPU            | Node list                   |
+===========+========+=======+==================+================+=============================+
| Standard  | 42     | 2x8   | :qty:`128 <GB>`  | 1x Titan Black | ``compute-[x]-[y]``         |
+-----------+--------+-------+------------------+----------------+-----------------------------+
| Big       | 2      | 4x10  | :qty:`512 <GB>`  |                | ``compute-big-[x]-[y]``     |
+-----------+--------+-------+------------------+----------------+-----------------------------+
| GPU       | 1      | 2x8   | :qty:`128 <GB>`  | 4x Tesla K20C  | ``compute-quadgpu-[x]-[y]`` |
+-----------+--------+-------+------------------+----------------+-----------------------------+

This cluster is managed by the ICP. If there are any issues,
please submit an issue to the ICP cluster team.
The cluster was acquired via a DFG grant in 2014 [#grant_Bee_cluster_2014]_.

.. _Bee partitions:

Partitions and nodes
--------------------

Contrary to most clusters, compute resources on Bee are not segmented in partitions.
Instead, one has to select resources from a node list.

Standard nodes
^^^^^^^^^^^^^^
There are 42 standard nodes equipped with 2 :cpu-intel-sheet2:`Intel Dual Xeon E5-2630v3
<83356/intel-xeon-processor-e52630-v3-20m-cache-2-40-ghz>`
(8 cores, 18 threads, :qty:`2.4 <GHz>`, :qty:`85 <W>`), :qty:`128 <GB>` of RAM, InfiniBand (:qty:`40 <Gbps>`) interconnect.
Each node has 1 `NVIDIA Titan Black <https://web.archive.org/web/20230321010915/https://www.nvidia.com/en-us/geforce/graphics-cards/geforce-gtx-titan-black/specifications/>`__ (:qty:`6 <GB>` GDDR5).
All standard nodes follow the naming convention ``compute-[x]-[y]`` where they can
take on values ``compute-0-1 ... compute-0-17`` and ``compute-1-0 ... compute-1-23``.

Big nodes
^^^^^^^^^
There are 2 big nodes equipped with 2 :cpu-intel-sheet2:`Intel Quad Xeon E5-4640v2
<75288/intel-xeon-processor-e54640-v2-20m-cache-2-20-ghz>`
(10 cores, 20 threads, :qty:`2.2 <GHz>`, :qty:`95 <W>`), :qty:`512 <GB>` of RAM, InfiniBand (:qty:`40 <Gbps>`) interconnect.
There are no GPUs.
All big nodes follow the naming convention ``compute-big-[x]-[y]`` where
they may take on values ``compute-big-0-0`` and ``compute-big-0-1``.

GPU node
^^^^^^^^
There is 1 quad-GPU node equipped with 2 :cpu-intel-sheet2:`Intel Dual Xeon E5-2630v3
<83356/intel-xeon-processor-e52630-v3-20m-cache-2-40-ghz>`
(8 cores, 18 threads, :qty:`2.4 <GHz>`, :qty:`85 <W>`), :qty:`128 <GB>` of RAM, InfiniBand (:qty:`40 <Gbps>`) interconnect.
The node has 4 NVIDIA Tesla K20C (:qty:`5 <GB>` GDDR5).
The node name is ``compute-quadgpu-1-0``.

.. _Bee access:

Access
------

Frank Huber is in charge of creating user accounts on Bee.
All ICP students and staff members are entitled to accessing Bee.
For external personnel, Christian Holm must personally approve access.
To log into Bee, refer to :ref:`Logging into clusters`.
See also :doc:`/_pages/_appendix/using_bee` for building software and submitting jobs.

.. _Bee obligations:

Obligations
-----------

The only obligation is to acknowledge funding from the DFG.
Since the project was awarded before 2017, there is no obligation
to cite the grant id. However, if you would like the publication
to appear in grant search results, here is the complete acknowledgment:

    We acknowledge funding from the Deutsche Forschungsgemeinschaft
    (DFG, German Research Foundation) through the Compute Cluster
    grant no. :dfg-gepris:`261833929`.

.. _Bee publications:

Publications
------------

* :cite:`wagner21a`: ICP simulation with 512 cores on Bee

____

.. [#grant_Bee_cluster_2014] DFG grant :dfg-gepris:`261833929` for Bee.

.. [#DFG_acknowledgement] See paragraph 15.1 of `DFG-Vordruck 2.18 - 01/20
   <https://www.dfg.de/formulare/2_18/v/dfg_2_18_de_v0120.pdf>`__
   (Besondere Verwendungsrichtlinien für Forschungsgroßgeräte
   nach Art. 91b Abs. 1 S. 1 GG i.V.m. AV-FGH).